diff options
| author | Costa Tsaousis <costa@netdata.cloud> | 2022-11-20 23:47:53 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2022-11-20 23:47:53 +0200 |
| commit | 284f6f3aa4f36cefad2601c490510621496c2b53 (patch) | |
| tree | 97a7d55627ef7477f431c53a20d0e6f1f738a419 /daemon/service.c | |
| parent | 2d02484954f68bf7e3015cb649e2f10a9f3c5c95 (diff) | |
streaming compression, query planner and replication fixes (#14023)
* streaming compression, query planner and replication fixes
* remove journal v2 stats from global statistics
* disable sql for checking past sql UUIDs
* single threaded replication
* final replication thread using dictionaries and JudyL for sorting the pending requests
* do not timeout the sending socket when there are pending replication requests
* streaming receiver using read() instead of fread()
* remove FILE * from streaming - now using posix read() and write()
* increase timeouts to 10 minutes
* apply sender timeout only when there are metrics that are supposed to be streamed
* error handling in replication
* remove retries on socket read timeout; better error messages
* take into account inbound traffic too to detect that a connection is stale
* remove race conditions from replication thread
* make sure deleted entries are marked as executed, so that even if deletion fails, they will not be executed
* 2 minutes timeout to retry streaming to a parent that already has this node
* remove unecessary condition check
* fix compilation warnings
* include judy in replication
* wrappers to handle retries for SSL_read and SSL_write
* compressed bytes read monitoring
* recursive locks on replication to make it faster during flush or cleanup
* replication completion chart at the receiver side
* simplified recursive mutex
* simplified recursive mutex again
Diffstat (limited to 'daemon/service.c')
| -rw-r--r-- | daemon/service.c | 11 |
1 files changed, 7 insertions, 4 deletions
diff --git a/daemon/service.c b/daemon/service.c index be1aad8164..28fdc84756 100644 --- a/daemon/service.c +++ b/daemon/service.c @@ -156,17 +156,20 @@ static void svc_rrdhost_cleanup_obsolete_charts(RRDHOST *host) { static void svc_rrdset_check_obsoletion(RRDHOST *host) { worker_is_busy(WORKER_JOB_CHILD_CHART_OBSOLETION_CHECK); + time_t now = now_realtime_sec(); time_t last_entry_t; RRDSET *st; rrdset_foreach_read(st, host) { + if(!rrdset_flag_check(st, RRDSET_FLAG_RECEIVER_REPLICATION_FINISHED)) + continue; + last_entry_t = rrdset_last_entry_t(st); - if(last_entry_t && last_entry_t < host->senders_connect_time && host->senders_connect_time - + TIME_TO_RUN_OBSOLETIONS_ON_CHILD_CONNECT + ITERATIONS_TO_RUN_OBSOLETIONS_ON_CHILD_CONNECT * st->update_every - < now_realtime_sec()) + if(last_entry_t && last_entry_t < host->senders_connect_time && + host->senders_connect_time + TIME_TO_RUN_OBSOLETIONS_ON_CHILD_CONNECT + ITERATIONS_TO_RUN_OBSOLETIONS_ON_CHILD_CONNECT * st->update_every + < now) rrdset_is_obsolete(st); - } rrdset_foreach_done(st); } |