summaryrefslogtreecommitdiffstats
path: root/database/sqlite/sqlite_aclk_chart.h
AgeCommit message (Collapse)Author
2022-09-27Remove Chart/Dim based communication (#13650)Timotej S
Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
2022-06-01Schedule retention message calculation to a worker thread (#13039)Stelios Fragkakis
* Move aclk_update_retention to the proper header file * Do a scan but avoid going through all the dimensions if we have too much to delete -- do not generate a retention message in that case * Schedule the retention calculation to a worker * Adjust messages in the access log * Fix compilation errors with --disable-cloud
2022-05-18Defer the dimension payload check to the ACLK sync thread (#12951)Stelios Fragkakis
Defer payload check to the aclk sync thread
2022-05-17Adjust the dimension liveness status check (#12933)Stelios Fragkakis
* Mark a chart to be exposed only if dimension is created or metadata changes * Add a calculate liveness for the dimension for collected to non collected (live -> stale) and vice versa * queue_dimension_to_aclk will have the rrdset and either 0 or last collected time If 0 then it will be marked as live else it will be marked as stale and last collected time will be sent to the cloud * Add an extra parameter to indicate if the payload check should be done in the database or it has been done already * Queue dimension sets dimension liveness and queues the exact payload to store in the database * Fix compilation error when --disable-cloud is specified
2022-05-03Improve agent cloud chart synchronization (#12655)Stelios Fragkakis
* Try to queue dimension always when: Trying to clean obsolete charts If chart has been sent and liveness apparently changed * delay rotation and skip chart check if not send to cloud * No need to CLEAR flag during database rotation Do not clear chart ACLK status for dimension requests * Change payload_sent to return timestamp of submitted message * Clear the dimension ACLK flag if we are processing all the charts again * Check if dimension is already queued to ACLK and ignore it If queue fails then reset it to retry Already try to queue the dimension * Improve dimension cleanup during the retention message calculation * Change queue_dimension_to_aclk to return void * If no time range for this dimension then assume it is deleted * Start streaming for inactive nodes * Remove dead code * Correctly report hostname in the access log * Schedule a dimension deletion without trying to submit a message immediately * Enable dimension cleanup -- also delete dimension if not found in the dbengine files Free hostname
2022-05-02Reduce alert events sent to the cloud. (#12544)Emmanuel Vasilakis
* filter * update filter * queue removed directly * more * logging * cleanup * cleanup 2 * cleanup 3 * finalize instead of reset
2022-04-01Fix issue with charts not properly synchronized with the cloud (#12451)Stelios Fragkakis
* Add function to check a specific chart * If a chart is not obsoleted, check if the liveness needs to be updated * Calculate liveness based on a (constant * update_every) for each dimension * Scan all dimensions when the retention message is constructed and update liveness if needed * If initial state, set to computed live * Set computed live state to dimension * Add a maximum dimension cleanup on startup to prevent message flood * Schedule chart updates if charts streaming is enabled * Adjust live state for dimension * The query executed will have a valid dimension uuid only if memory mode is dbengine
2022-03-09Adds more info to aclk-state API call (#12231)Timotej S
2022-03-08Adjust cloud dimension update frequency (#12284)Stelios Fragkakis
* Queue a chart immediately to the cloud * Do not inform the cloud immediately if a dimension stopped collecting use MAX(obsoletion time, 1.5 * update_every) * Notify cloud immediately on dimension deletion * Add debug messages * Do not schedule an update if we are shutting down
2021-11-11insert into aclk_alert instead of queuing (#11769)Emmanuel Vasilakis
2021-11-09Store and submit dimension delete messages for new cloud architecture (#11765)Stelios Fragkakis
* Enhance the dimension delete table and adjust the trigger to include chart_id and host_id * Add the aclk_process_dimension_deletion function * Change variable chart_name in aclk_upd_dimension_event (it is st->id from st.type dot st.id) * Process dimension deletion when retention updates are sent * Do not send charts if we don't have dimensions * Add check for uuid_parse return code
2021-11-03Improve the ACLK sync process for the new cloud architecture (#11744)Stelios Fragkakis
* Move retention code to the charts * Log information about node registration and updates * Prevent deadlock if aclk_database_enq_cmd locks for a node * Improve message (indicate that it comes from alerts). This will be improved in a followup PR * Disable parts that can't be used if the new cloud env is not available * Set dimension FLAG if message has been queued * Queue messages using the correct protocol enabled * Cleanup unused functions Rename functions that queue charts and dimensions Improve the generic chart payload add function Add a counter for pending charts/dimension payloads to avoid polling the db Delay the retention update message until we are done with the updates Fix full resync command to handle sequence_id = 0 correctly Disable functions not needed when the new cloud env functionality is not compiled * Add chart_payload count and retry count Output information or error message if we fail to queue chart/dimension PUSH commands Only try to queue commands if we have chart_payload_count>0 Remove the event loop shutdown opcode handle * Improve detection of shutdown (check netdata_exit) * Adjusting info messages
2021-09-21Add chart message support for ACLK new architecture (#11447)Stelios Fragkakis