Use one thread for ACLK synchonization (#14281)

* Remove aclk sync threads

* Disable functions if compiled with --disable-cloud

* Allocate and reuse buffer when scanning hosts
Tune transactions when writing metadata
Error checking when executing db_execute (it is already within a loop with retries)

* Schedule host context load in parallel
Child connection will be delayed if context load is not complete
Event loop cleanup

* Delay retention check if context is not loaded
Remove context load check from regular metadata host scan

* Improve checks to check finished threads

* Cleanup warnings when compiling with --disable-cloud

* Clean chart labels that were created before our current maximum retention

* Fix sql statement

* Remove structures members that of no use
Remove buffer allocations when not needed

* Fix compilation error

* Don't check for service running when not from a worker

* Code cleanup if agent is compiled with --disable-cloud
Setup ACLK tables in the database if needed
Submit node status update messages to the cloud

* Fix compilation warning when --disable-cloud is specified

* Address codacy issues

* Remove empty file -- has already been moved under contexts

* Use enum instead of numbers

* Use UUID_STR_LEN

* Add newline at the end of file

* Release node_id to prevent memory leak under certain cases

* Add queries in defines

* Ignore rc from transaction start -- if there is an active transaction, we will use it (same with commit) should further improve in a future PR

* Remove commented out code

* If host is null (it should not be) do not allocate config (coverity reports Resource leak)

* Do garbage collection when contexts is initialized

* Handle the case when config is not yet available for a host

4 files changed, 4 insertions, 5 deletions

diff --git a/daemon/event_loop.c b/daemon/event_loop.c
index 6f09cd654a..5fd02377eb 100644
--- a/daemon/event_loop.c
+++ b/daemon/event_loop.c
@@ -49,6 +49,7 @@ void register_libuv_worker_jobs() {
     worker_register_job_name(UV_EVENT_DBENGINE_SHUTDOWN, "dbengine shutdown");
 
     // metadata
+    worker_register_job_name(UV_EVENT_HOST_CONTEXT_LOAD, "metadata load host context");
     worker_register_job_name(UV_EVENT_METADATA_STORE, "metadata store host");
     worker_register_job_name(UV_EVENT_METADATA_CLEANUP, "metadata cleanup");
 
diff --git a/daemon/event_loop.h b/daemon/event_loop.h
index 0d3cc0d07c..1ff1c2c1cb 100644
--- a/daemon/event_loop.h
+++ b/daemon/event_loop.h
@@ -41,6 +41,7 @@ enum event_loop_job {
     UV_EVENT_DBENGINE_SHUTDOWN,
 
     // metadata
+    UV_EVENT_HOST_CONTEXT_LOAD,
     UV_EVENT_METADATA_STORE,
     UV_EVENT_METADATA_CLEANUP,
 
diff --git a/daemon/main.c b/daemon/main.c
index 5afc100e31..85d2996ef9 100644
--- a/daemon/main.c
+++ b/daemon/main.c
@@ -348,6 +348,7 @@ void netdata_cleanup_and_exit(int ret) {
             | ABILITY_WEB_REQUESTS
             | ABILITY_STREAMING_CONNECTIONS
             | SERVICE_ACLK
+            | SERVICE_ACLKSYNC
             );
 
     delta_shutdown_time("stop replication, exporters, ML training, health and web servers threads");
@@ -388,11 +389,6 @@ void netdata_cleanup_and_exit(int ret) {
 
     metadata_sync_shutdown_prepare();
 
-#ifdef ENABLE_ACLK
-    delta_shutdown_time("signal aclk sync to stop");
-    aclk_sync_exit_all();
-#endif
-
     delta_shutdown_time("stop aclk threads");
 
     timeout = !service_wait_exit(
diff --git a/daemon/main.h b/daemon/main.h
index 8704d60977..7e659e939a 100644
--- a/daemon/main.h
+++ b/daemon/main.h
@@ -43,6 +43,7 @@ typedef enum {
     SERVICE_CONTEXT               = (1 << 12),
     SERVICE_ANALYTICS             = (1 << 13),
     SERVICE_EXPORTERS             = (1 << 14),
+    SERVICE_ACLKSYNC              = (1 << 15)
 } SERVICE_TYPE;
 
 typedef enum {