RRD structures managed by dictionaries (#13646)

* rrdset - in progress

* rrdset optimal constructor; rrdset conflict

* rrdset final touches

* re-organization of rrdset object members

* prevent use-after-free

* dictionary dfe supports also counting of iterations

* rrddim managed by dictionary

* rrd.h cleanup

* DICTIONARY_ITEM now is referencing actual dictionary items in the code

* removed rrdset linked list

* Revert "removed rrdset linked list"

This reverts commit 690d6a588b4b99619c2c5e10f84e8f868ae6def5.

* removed rrdset linked list

* added comments

* Switch chart uuid to static allocation in rrdset
Remove unused functions

* rrdset_archive() and friends...

* always create rrdfamily

* enable ml_free_dimension

* rrddim_foreach done with dfe

* most custom rrddim loops replaced with rrddim_foreach

* removed accesses to rrddim->dimensions

* removed locks that are no longer needed

* rrdsetvar is now managed by the dictionary

* set rrdset is rrdsetvar, fixes https://github.com/netdata/netdata/pull/13646#issuecomment-1242574853

* conflict callback of rrdsetvar now properly checks if it has to reset the variable

* dictionary registered callbacks accept as first parameter the DICTIONARY_ITEM

* dictionary dfe now uses internal counter to report; avoided excess variables defined with dfe

* dictionary walkthrough callbacks get dictionary acquired items

* dictionary reference counters that can be dupped from zero

* added advanced functions for get and del

* rrdvar managed by dictionaries

* thread safety for rrdsetvar

* faster rrdvar initialization

* rrdvar string lengths should match in all add, del, get functions

* rrdvar internals hidden from the rest of the world

* rrdvar is now acquired throughout netdata

* hide the internal structures of rrdsetvar

* rrdsetvar is now acquired through out netdata

* rrddimvar managed by dictionary; rrddimvar linked list removed; rrddimvar structures hidden from the rest of netdata

* better error handling

* dont create variables if not initialized for health

* dont create variables if not initialized for health again

* rrdfamily is now managed by dictionaries; references of it are acquired dictionary items

* type checking on acquired objects

* rrdcalc renaming of functions

* type checking for rrdfamily_acquired

* rrdcalc managed by dictionaries

* rrdcalc double free fix

* host rrdvars is always needed

* attempt to fix deadlock 1

* attempt to fix deadlock 2

* Remove unused variable

* attempt to fix deadlock 3

* snprintfz

* rrdcalc index in rrdset fix

* Stop storing active charts and computing chart hashes

* Remove store active chart function

* Remove compute chart hash function

* Remove sql_store_chart_hash function

* Remove store_active_dimension function

* dictionary delayed destruction

* formatting and cleanup

* zero dictionary base on rrdsetvar

* added internal error to log delayed destructions of dictionaries

* typo in rrddimvar

* added debugging info to dictionary

* debug info

* fix for rrdcalc keys being empty

* remove forgotten unlock

* remove deadlock

* Switch to metadata version 5 and drop
  chart_hash
  chart_hash_map
  chart_active
  dimension_active
  v_chart_hash

* SQL cosmetic changes

* do not busy wait while destroying a referenced dictionary

* remove deadlock

* code cleanup; re-organization;

* fast cleanup and flushing of dictionaries

* number formatting fixes

* do not delete configured alerts when archiving a chart

* rrddim obsolete linked list management outside dictionaries

* removed duplicate contexts call

* fix crash when rrdfamily is not initialized

* dont keep rrddimvar referenced

* properly cleanup rrdvar

* removed some locks

* Do not attempt to cleanup chart_hash / chart_hash_map

* rrdcalctemplate managed by dictionary

* register callbacks on the right dictionary

* removed some more locks

* rrdcalc secondary index replaced with linked-list; rrdcalc labels updates are now executed by health thread

* when looking up for an alarm look using both chart id and chart name

* host initialization a bit more modular

* init rrdlabels on host update

* preparation for dictionary views

* improved comment

* unused variables without internal checks

* service threads isolation and worker info

* more worker info in service thread

* thread cancelability debugging with internal checks

* strings data races addressed; fixes https://github.com/netdata/netdata/issues/13647

* dictionary modularization

* Remove unused SQL statement definition

* unit-tested thread safety of dictionaries; removed data race conditions on dictionaries and strings; dictionaries now can detect if the caller is holds a write lock and automatically all the calls become their unsafe versions; all direct calls to unsafe version is eliminated

* remove worker_is_idle() from the exit of service functions, because we lose the lock time between loops

* rewritten dictionary to have 2 separate locks, one for indexing and another for traversal

* Update collectors/cgroups.plugin/sys_fs_cgroup.c

Co-authored-by: Vladimir Kobal <vlad@prokk.net>

* Update collectors/cgroups.plugin/sys_fs_cgroup.c

Co-authored-by: Vladimir Kobal <vlad@prokk.net>

* Update collectors/proc.plugin/proc_net_dev.c

Co-authored-by: Vladimir Kobal <vlad@prokk.net>

* fix memory leak in rrdset cache_dir

* minor dictionary changes

* dont use index locks in single threaded

* obsolete dict option

* rrddim options and flags separation; rrdset_done() optimization to keep array of reference pointers to rrddim;

* fix jump on uninitialized value in dictionary; remove double free of cache_dir

* addressed codacy findings

* removed debugging code

* use the private refcount on dictionaries

* make dictionary item desctructors work on dictionary destruction; strictier control on dictionary API; proper cleanup sequence on rrddim;

* more dictionary statistics

* global statistics about dictionary operations, memory, items, callbacks

* dictionary support for views - missing the public API

* removed warning about unused parameter

* chart and context name for cloud

* chart and context name for cloud, again

* dictionary statistics fixed; first implementation of dictionary views - not currently used

* only the master can globally delete an item

* context needs netdata prefix

* fix context and chart it of spins

* fix for host variables when health is not enabled

* run garbage collector on item insert too

* Fix info message; remove extra "using"

* update dict unittest for new placement of garbage collector

* we need RRDHOST->rrdvars for maintaining custom host variables

* Health initialization needs the host->host_uuid

* split STRING to its own files; no code changes other than that

* initialize health unconditionally

* unit tests do not pollute the global scope with their variables

* Skip initialization when creating archived hosts on startup. When a child connects it will initialize properly

Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
Co-authored-by: Vladimir Kobal <vlad@prokk.net>

25 files changed, 145 insertions, 152 deletions

diff --git a/collectors/cgroups.plugin/sys_fs_cgroup.c b/collectors/cgroups.plugin/sys_fs_cgroup.c
index b214b3731b..74aedf2b52 100644
--- a/collectors/cgroups.plugin/sys_fs_cgroup.c
+++ b/collectors/cgroups.plugin/sys_fs_cgroup.c
@@ -857,16 +857,16 @@ struct cgroup {
     char *filename_cpu_cfs_quota;
     unsigned long long cpu_cfs_quota;
 
-    RRDSETVAR *chart_var_cpu_limit;
+    const RRDSETVAR_ACQUIRED *chart_var_cpu_limit;
     NETDATA_DOUBLE prev_cpu_usage;
 
     char *filename_memory_limit;
     unsigned long long memory_limit;
-    RRDSETVAR *chart_var_memory_limit;
+    const RRDSETVAR_ACQUIRED *chart_var_memory_limit;
 
     char *filename_memoryswap_limit;
     unsigned long long memoryswap_limit;
-    RRDSETVAR *chart_var_memoryswap_limit;
+    const RRDSETVAR_ACQUIRED *chart_var_memoryswap_limit;
 
     // services
     RRDDIM *rd_cpu;
@@ -3708,10 +3708,10 @@ cpu_limits2_err:
     }
 }
 
-static inline int update_memory_limits(char **filename, RRDSETVAR **chart_var, unsigned long long *value, const char *chart_var_name, struct cgroup *cg) {
+static inline int update_memory_limits(char **filename, const RRDSETVAR_ACQUIRED **chart_var, unsigned long long *value, const char *chart_var_name, struct cgroup *cg) {
     if(*filename) {
         if(unlikely(!*chart_var)) {
-            *chart_var = rrdsetvar_custom_chart_variable_create(cg->st_mem_usage, chart_var_name);
+            *chart_var = rrdsetvar_custom_chart_variable_add_and_acquire(cg->st_mem_usage, chart_var_name);
             if(!*chart_var) {
                 error("Cannot create cgroup %s chart variable '%s'. Will not update its limit anymore.", cg->id, chart_var_name);
                 freez(*filename);
@@ -3727,7 +3727,7 @@ static inline int update_memory_limits(char **filename, RRDSETVAR **chart_var, u
                     *filename = NULL;
                 }
                 else {
-                    rrdsetvar_custom_chart_variable_set(*chart_var, (NETDATA_DOUBLE)(*value / (1024 * 1024)));
+                    rrdsetvar_custom_chart_variable_set(cg->st_mem_usage, *chart_var, (NETDATA_DOUBLE)(*value / (1024 * 1024)));
                     return 1;
                 }
             } else {
@@ -3742,11 +3742,11 @@ static inline int update_memory_limits(char **filename, RRDSETVAR **chart_var, u
                 char *s = "max\n\0";
                 if(strcmp(s, buffer) == 0){
                     *value = UINT64_MAX;
-                    rrdsetvar_custom_chart_variable_set(*chart_var, (NETDATA_DOUBLE)(*value / (1024 * 1024)));
+                    rrdsetvar_custom_chart_variable_set(cg->st_mem_usage, *chart_var, (NETDATA_DOUBLE)(*value / (1024 * 1024)));
                     return 1;
                 }
                 *value = str2ull(buffer);
-                rrdsetvar_custom_chart_variable_set(*chart_var, (NETDATA_DOUBLE)(*value / (1024 * 1024)));
+                rrdsetvar_custom_chart_variable_set(cg->st_mem_usage, *chart_var, (NETDATA_DOUBLE)(*value / (1024 * 1024)));
                 return 1;
             }
         }
@@ -3843,7 +3843,7 @@ void update_cgroup_charts(int update_every) {
                 }
 
                 if(unlikely(!cg->chart_var_cpu_limit)) {
-                    cg->chart_var_cpu_limit = rrdsetvar_custom_chart_variable_create(cg->st_cpu, "cpu_limit");
+                    cg->chart_var_cpu_limit = rrdsetvar_custom_chart_variable_add_and_acquire(cg->st_cpu, "cpu_limit");
                     if(!cg->chart_var_cpu_limit) {
                         error("Cannot create cgroup %s chart variable 'cpu_limit'. Will not update its limit anymore.", cg->id);
                         if(cg->filename_cpuset_cpus) freez(cg->filename_cpuset_cpus);
@@ -3868,8 +3868,6 @@ void update_cgroup_charts(int update_every) {
                             value = (NETDATA_DOUBLE)cg->cpuset_cpus * 100;
                     }
                     if(likely(value)) {
-                        rrdsetvar_custom_chart_variable_set(cg->chart_var_cpu_limit, value);
-
                         if(unlikely(!cg->st_cpu_limit)) {
                             snprintfz(title, CHART_TITLE_MAX, "CPU Usage within the limits");
 
@@ -3909,14 +3907,15 @@ void update_cgroup_charts(int update_every) {
 
                         cg->prev_cpu_usage = cpu_usage;
 
+                        rrdsetvar_custom_chart_variable_set(cg->st_cpu, cg->chart_var_cpu_limit, value);
                         rrdset_done(cg->st_cpu_limit);
                     }
                     else {
-                        rrdsetvar_custom_chart_variable_set(cg->chart_var_cpu_limit, NAN);
                         if(unlikely(cg->st_cpu_limit)) {
                             rrdset_is_obsolete(cg->st_cpu_limit);
                             cg->st_cpu_limit = NULL;
                         }
+                        rrdsetvar_custom_chart_variable_set(cg->st_cpu, cg->chart_var_cpu_limit, NAN);
                     }
                 }
             }
diff --git a/collectors/cgroups.plugin/tests/test_doubles.c b/collectors/cgroups.plugin/tests/test_doubles.c
index 7f44416255..9aa748725c 100644
--- a/collectors/cgroups.plugin/tests/test_doubles.c
+++ b/collectors/cgroups.plugin/tests/test_doubles.c
@@ -146,14 +146,6 @@ void netdev_rename_device_del(const char *host_device)
     UNUSED(host_device);
 }
 
-void sql_store_chart_label(uuid_t *chart_uuid, int source_type, char *label, char *value)
-{
-    UNUSED(chart_uuid);
-    UNUSED(source_type);
-    UNUSED(label);
-    UNUSED(value);
-}
-
 void rrdcalc_update_rrdlabels(RRDSET *st) {
     (void)st;
 }
diff --git a/collectors/cups.plugin/cups_plugin.c b/collectors/cups.plugin/cups_plugin.c
index 77bd3e0ed8..9a200c31d1 100644
--- a/collectors/cups.plugin/cups_plugin.c
+++ b/collectors/cups.plugin/cups_plugin.c
@@ -137,10 +137,7 @@ getIntegerOption(
   return ((int)intvalue);
 }
 
-static int reset_job_metrics(const char *name, void *entry, void *data) {
-    (void)name;
-    (void)data;
-
+static int reset_job_metrics(const DICTIONARY_ITEM *item __maybe_unused, void *entry, void *data __maybe_unused) {
     struct job_metrics *jm = (struct job_metrics *)entry;
 
     jm->is_collected = 0;
@@ -175,8 +172,8 @@ struct job_metrics *get_job_metrics(char *dest) {
     return jm;
 }
 
-int collect_job_metrics(const char *name, void *entry, void *data) {
-    (void)data;
+int collect_job_metrics(const DICTIONARY_ITEM *item, void *entry, void *data __maybe_unused) {
+    const char *name = dictionary_acquired_item_name(item);
 
     struct job_metrics *jm = (struct job_metrics *)entry;
 
@@ -205,7 +202,7 @@ int collect_job_metrics(const char *name, void *entry, void *data) {
         printf("DIMENSION pending '' absolute 1 1\n");
         printf("DIMENSION held '' absolute 1 1\n");
         printf("DIMENSION processing '' absolute 1 1\n");
-        dictionary_del_having_write_lock(dict_dest_job_metrics, name);
+        dictionary_del(dict_dest_job_metrics, name);
     }
 
     return 0;
@@ -243,7 +240,7 @@ int main(int argc, char **argv) {
 
     errno = 0;
 
-    dict_dest_job_metrics = dictionary_create(DICTIONARY_FLAG_SINGLE_THREADED);
+    dict_dest_job_metrics = dictionary_create(DICT_OPTION_SINGLE_THREADED);
 
     // ------------------------------------------------------------------------
     // the main loop
diff --git a/collectors/diskspace.plugin/plugin_diskspace.c b/collectors/diskspace.plugin/plugin_diskspace.c
index 5bdf8bc61f..7381e1ec93 100644
--- a/collectors/diskspace.plugin/plugin_diskspace.c
+++ b/collectors/diskspace.plugin/plugin_diskspace.c
@@ -95,8 +95,8 @@ int mount_point_cleanup(const char *name, void *entry, int slow) {
     return 0;
 }
 
-int mount_point_cleanup_cb(const char *name, void *entry, void *data) {
-    UNUSED(data);
+int mount_point_cleanup_cb(const DICTIONARY_ITEM *item, void *entry, void *data __maybe_unused) {
+    const char *name = dictionary_acquired_item_name(item);
 
     return mount_point_cleanup(name, (struct mount_point_metadata *)entry, 0);
 }
@@ -330,7 +330,7 @@ static inline void do_disk_space_stats(struct mountinfo *mi, int update_every) {
                 , SIMPLE_PATTERN_EXACT
         );
 
-        dict_mountpoints = dictionary_create(DICTIONARY_FLAG_SINGLE_THREADED);
+        dict_mountpoints = dictionary_create(DICT_OPTION_SINGLE_THREADED);
     }
 
     struct mount_point_metadata *m = dictionary_get(dict_mountpoints, mi->mount_point);
diff --git a/collectors/plugins.d/pluginsd_parser.c b/collectors/plugins.d/pluginsd_parser.c
index 3dffde5acc..ce9250d568 100644
--- a/collectors/plugins.d/pluginsd_parser.c
+++ b/collectors/plugins.d/pluginsd_parser.c
@@ -101,15 +101,19 @@ PARSER_RC pluginsd_variable_action(void *user, RRDHOST *host, RRDSET *st, char *
     UNUSED(user);
 
     if (global) {
-        RRDVAR *rv = rrdvar_custom_host_variable_create(host, name);
-        if (rv)
-            rrdvar_custom_host_variable_set(host, rv, value);
+        const RRDVAR_ACQUIRED *rva = rrdvar_custom_host_variable_add_and_acquire(host, name);
+        if (rva) {
+            rrdvar_custom_host_variable_set(host, rva, value);
+            rrdvar_custom_host_variable_release(host, rva);
+        }
         else
             error("cannot find/create HOST VARIABLE '%s' on host '%s'", name, rrdhost_hostname(host));
     } else {
-        RRDSETVAR *rs = rrdsetvar_custom_chart_variable_create(st, name);
-        if (rs)
-            rrdsetvar_custom_chart_variable_set(rs, value);
+        const RRDSETVAR_ACQUIRED *rsa = rrdsetvar_custom_chart_variable_add_and_acquire(st, name);
+        if (rsa) {
+            rrdsetvar_custom_chart_variable_set(st, rsa, value);
+            rrdsetvar_custom_chart_variable_release(st, rsa);
+        }
         else
             error("cannot find/create CHART VARIABLE '%s' on host '%s', chart '%s'", name, rrdhost_hostname(host), rrdset_id(st));
     }
@@ -127,7 +131,7 @@ PARSER_RC pluginsd_dimension_action(void *user, RRDSET *st, char *id, char *name
     RRDDIM *rd = rrddim_add(st, id, name, multiplier, divisor, algorithm_type);
     int unhide_dimension = 1;
 
-    rrddim_flag_clear(rd, RRDDIM_FLAG_DONT_DETECT_RESETS_OR_OVERFLOWS);
+    rrddim_option_clear(rd, RRDDIM_OPTION_DONT_DETECT_RESETS_OR_OVERFLOWS);
     if (options && *options) {
         if (strstr(options, "obsolete") != NULL)
             rrddim_is_obsolete(st, rd);
@@ -137,20 +141,20 @@ PARSER_RC pluginsd_dimension_action(void *user, RRDSET *st, char *id, char *name
         unhide_dimension = !strstr(options, "hidden");
 
         if (strstr(options, "noreset") != NULL)
-            rrddim_flag_set(rd, RRDDIM_FLAG_DONT_DETECT_RESETS_OR_OVERFLOWS);
+            rrddim_option_set(rd, RRDDIM_OPTION_DONT_DETECT_RESETS_OR_OVERFLOWS);
         if (strstr(options, "nooverflow") != NULL)
-            rrddim_flag_set(rd, RRDDIM_FLAG_DONT_DETECT_RESETS_OR_OVERFLOWS);
+            rrddim_option_set(rd, RRDDIM_OPTION_DONT_DETECT_RESETS_OR_OVERFLOWS);
     } else
         rrddim_isnot_obsolete(st, rd);
 
     if (likely(unhide_dimension)) {
-        rrddim_flag_clear(rd, RRDDIM_FLAG_HIDDEN);
+        rrddim_option_clear(rd, RRDDIM_OPTION_HIDDEN);
         if (rrddim_flag_check(rd, RRDDIM_FLAG_META_HIDDEN)) {
             (void)sql_set_dimension_option(&rd->metric_uuid, NULL);
             rrddim_flag_clear(rd, RRDDIM_FLAG_META_HIDDEN);
         }
     } else {
-        rrddim_flag_set(rd, RRDDIM_FLAG_HIDDEN);
+        rrddim_option_set(rd, RRDDIM_OPTION_HIDDEN);
         if (!rrddim_flag_check(rd, RRDDIM_FLAG_META_HIDDEN)) {
            (void)sql_set_dimension_option(&rd->metric_uuid, "hidden");
             rrddim_flag_set(rd, RRDDIM_FLAG_META_HIDDEN);
diff --git a/collectors/proc.plugin/ipc.c b/collectors/proc.plugin/ipc.c
index e114a05a41..e0863f6d8b 100644
--- a/collectors/proc.plugin/ipc.c
+++ b/collectors/proc.plugin/ipc.c
@@ -281,7 +281,7 @@ int do_ipc(int update_every, usec_t dt) {
     static int read_limits_next = -1;
     static struct ipc_limits limits;
     static struct ipc_status status;
-    static RRDVAR *arrays_max = NULL, *semaphores_max = NULL;
+    static const RRDVAR_ACQUIRED *arrays_max = NULL, *semaphores_max = NULL;
     static RRDSET *st_semaphores = NULL, *st_arrays = NULL;
     static RRDDIM *rd_semaphores = NULL, *rd_arrays = NULL;
     static char *msg_filename = NULL;
@@ -352,8 +352,8 @@ int do_ipc(int update_every, usec_t dt) {
             }
 
             // variables
-            semaphores_max = rrdvar_custom_host_variable_create(localhost, "ipc_semaphores_max");
-            arrays_max     = rrdvar_custom_host_variable_create(localhost, "ipc_semaphores_arrays_max");
+            semaphores_max = rrdvar_custom_host_variable_add_and_acquire(localhost, "ipc_semaphores_max");
+            arrays_max     = rrdvar_custom_host_variable_add_and_acquire(localhost, "ipc_semaphores_arrays_max");
         }
 
         struct stat stbuf;
@@ -483,11 +483,7 @@ int do_ipc(int update_every, usec_t dt) {
             rrdset_done(st_msq_messages);
             rrdset_done(st_msq_bytes);
 
-            long long dimensions_num = 0;
-            RRDDIM *rd;
-            rrdset_rdlock(st_msq_messages);
-            rrddim_foreach_read(rd, st_msq_messages) dimensions_num++;
-            rrdset_unlock(st_msq_messages);
+            long long dimensions_num = rrdset_number_of_dimensions(st_msq_messages);
 
             if(unlikely(dimensions_num > dimensions_limit)) {
                 info("Message queue statistics has been disabled");
diff --git a/collectors/proc.plugin/proc_diskstats.c b/collectors/proc.plugin/proc_diskstats.c
index 945ca25216..554bc4f897 100644
--- a/collectors/proc.plugin/proc_diskstats.c
+++ b/collectors/proc.plugin/proc_diskstats.c
@@ -874,8 +874,6 @@ static void add_labels_to_disk(struct disk *d, RRDSET *st) {
             rrdlabels_add(st->rrdlabels, "device_type", "virtual", RRDLABEL_SRC_AUTO);
             break;
     }
-
-    rrdcalc_update_rrdlabels(st);
 }
 
 int do_proc_diskstats(int update_every, usec_t dt) {
diff --git a/collectors/proc.plugin/proc_interrupts.c b/collectors/proc.plugin/proc_interrupts.c
index db2a20b5df..17c287425d 100644
--- a/collectors/proc.plugin/proc_interrupts.c
+++ b/collectors/proc.plugin/proc_interrupts.c
@@ -175,7 +175,7 @@ int do_proc_interrupts(int update_every, usec_t dt) {
             // calls of this function.
             if(unlikely(!irr->rd || strncmp(rrddim_name(irr->rd), irr->name, MAX_INTERRUPT_NAME) != 0)) {
                 irr->rd = rrddim_add(st_system_interrupts, irr->id, irr->name, 1, 1, RRD_ALGORITHM_INCREMENTAL);
-                rrddim_set_name(st_system_interrupts, irr->rd, irr->name);
+                rrddim_reset_name(st_system_interrupts, irr->rd, irr->name);
 
                 // also reset per cpu RRDDIMs to avoid repeating strncmp() in the per core loop
                 if(likely(do_per_core != CONFIG_BOOLEAN_NO)) {
@@ -237,7 +237,7 @@ int do_proc_interrupts(int update_every, usec_t dt) {
                 if(irr->used && (do_per_core == CONFIG_BOOLEAN_YES || irr->cpu[c].value)) {
                     if(unlikely(!irr->cpu[c].rd)) {
                         irr->cpu[c].rd = rrddim_add(core_st[c], irr->id, irr->name, 1, 1, RRD_ALGORITHM_INCREMENTAL);
-                        rrddim_set_name(core_st[c], irr->cpu[c].rd, irr->name);
+                        rrddim_reset_name(core_st[c], irr->cpu[c].rd, irr->name);
                     }
 
                     rrddim_set_by_pointer(core_st[c], irr->cpu[c].rd, irr->cpu[c].value);
diff --git a/collectors/proc.plugin/proc_loadavg.c b/collectors/proc.plugin/proc_loadavg.c
index 8b78ecc9ef..14415ef65b 100644
--- a/collectors/proc.plugin/proc_loadavg.c
+++ b/collectors/proc.plugin/proc_loadavg.c
@@ -99,7 +99,7 @@ int do_proc_loadavg(int update_every, usec_t dt) {
     if(likely(do_all_processes)) {
         static RRDSET *processes_chart = NULL;
         static RRDDIM *rd_active = NULL;
-        static RRDSETVAR *rd_pidmax;
+        static const RRDSETVAR_ACQUIRED *rd_pidmax;
 
         if(unlikely(!processes_chart)) {
             processes_chart = rrdset_create_localhost(
@@ -118,12 +118,12 @@ int do_proc_loadavg(int update_every, usec_t dt) {
             );
 
             rd_active = rrddim_add(processes_chart, "active", NULL, 1, 1, RRD_ALGORITHM_ABSOLUTE);
-            rd_pidmax = rrdsetvar_custom_chart_variable_create(processes_chart, "pidmax");
+            rd_pidmax = rrdsetvar_custom_chart_variable_add_and_acquire(processes_chart, "pidmax");
         }
         else rrdset_next(processes_chart);
 
         rrddim_set_by_pointer(processes_chart, rd_active, active_processes);
-        rrdsetvar_custom_chart_variable_set(rd_pidmax, max_processes);
+        rrdsetvar_custom_chart_variable_set(processes_chart, rd_pidmax, max_processes);
         rrdset_done(processes_chart);
     }
 
diff --git a/collectors/proc.plugin/proc_mdstat.c b/collectors/proc.plugin/proc_mdstat.c
index 77b75d3139..2bfb6913f3 100644
--- a/collectors/proc.plugin/proc_mdstat.c
+++ b/collectors/proc.plugin/proc_mdstat.c
@@ -80,7 +80,6 @@ static inline void make_chart_obsolete(char *name, const char *id_modifier)
 static void add_labels_to_mdstat(struct raid *raid, RRDSET *st) {
     rrdlabels_add(st->rrdlabels, "device", raid->name, RRDLABEL_SRC_AUTO);
     rrdlabels_add(st->rrdlabels, "raid_level", raid->level, RRDLABEL_SRC_AUTO);
-    rrdcalc_update_rrdlabels(st);
 }
 
 int do_proc_mdstat(int update_every, usec_t dt)
diff --git a/collectors/proc.plugin/proc_net_dev.c b/collectors/proc.plugin/proc_net_dev.c
index 79572f4420..701cc00ea7 100644
--- a/collectors/proc.plugin/proc_net_dev.c
+++ b/collectors/proc.plugin/proc_net_dev.c
@@ -188,7 +188,7 @@ static struct netdev {
     RRDDIM *rd_mtu;
 
     char *filename_speed;
-    RRDSETVAR *chart_var_speed;
+    const RRDSETVAR_ACQUIRED *chart_var_speed;
 
     char *filename_duplex;
     char *filename_operstate;
@@ -967,7 +967,8 @@ int do_proc_net_dev(int update_every, usec_t dt) {
             // update the interface speed
             if(d->filename_speed) {
                 if(unlikely(!d->chart_var_speed)) {
-                    d->chart_var_speed = rrdsetvar_custom_chart_variable_create(d->st_bandwidth, "nic_speed_max");
+                    d->chart_var_speed =
+                        rrdsetvar_custom_chart_variable_add_and_acquire(d->st_bandwidth, "nic_speed_max");
                     if(!d->chart_var_speed) {
                         error("Cannot create interface %s chart variable 'nic_speed_max'. Will not update its speed anymore.", d->name);
                         freez(d->filename_speed);
@@ -990,8 +991,6 @@ int do_proc_net_dev(int update_every, usec_t dt) {
                         d->filename_speed = NULL;
                     }
                     else {
-                        rrdsetvar_custom_chart_variable_set(d->chart_var_speed, (NETDATA_DOUBLE) d->speed * KILOBITS_IN_A_MEGABIT);
-
                         if(d->do_speed != CONFIG_BOOLEAN_NO) {
                             if(unlikely(!d->st_speed)) {
                                 d->st_speed = rrdset_create_localhost(
@@ -1020,6 +1019,8 @@ int do_proc_net_dev(int update_every, usec_t dt) {
                             rrddim_set_by_pointer(d->st_speed, d->rd_speed, (collected_number)d->speed * KILOBITS_IN_A_MEGABIT);
                             rrdset_done(d->st_speed);
                         }
+
+                        rrdsetvar_custom_chart_variable_set(d->st_bandwidth, d->chart_var_speed, (NETDATA_DOUBLE) d->speed * KILOBITS_IN_A_MEGABIT);
                     }
                 }
             }
diff --git a/collectors/proc.plugin/proc_net_snmp.c b/collectors/proc.plugin/proc_net_snmp.c
index b03a6ac74d..d667a06df8 100644
--- a/collectors/proc.plugin/proc_net_snmp.c
+++ b/collectors/proc.plugin/proc_net_snmp.c
@@ -104,7 +104,7 @@ int do_proc_net_snmp(int update_every, usec_t dt) {
              *arl_udp = NULL,
              *arl_udplite = NULL;
 
-    static RRDVAR *tcp_max_connections_var = NULL;
+    static const RRDVAR_ACQUIRED *tcp_max_connections_var = NULL;
 
     if(unlikely(!arl_ip)) {
         do_ip_packets       = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp", "ipv4 packets", CONFIG_BOOLEAN_AUTO);
@@ -216,7 +216,7 @@ int do_proc_net_snmp(int update_every, usec_t dt) {
         arl_expect(arl_udplite, "InCsumErrors", &snmp_root.udplite_InCsumErrors);
         arl_expect(arl_udplite, "IgnoredMulti", &snmp_root.udplite_IgnoredMulti);
 
-        tcp_max_connections_var = rrdvar_custom_host_variable_create(localhost, "tcp_max_connections");
+        tcp_max_connections_var = rrdvar_custom_host_variable_add_and_acquire(localhost, "tcp_max_connections");
     }
 
     if(unlikely(!ff)) {
diff --git a/collectors/proc.plugin/proc_net_sockstat.c b/collectors/proc.plugin/proc_net_sockstat.c
index 994cbad7b6..db75bee1ab 100644
--- a/collectors/proc.plugin/proc_net_sockstat.c
+++ b/collectors/proc.plugin/proc_net_sockstat.c
@@ -27,14 +27,14 @@ static struct proc_net_sockstat {
 
 static int read_tcp_mem(void) {
     static char *filename = NULL;
-    static RRDVAR *tcp_mem_low_threshold = NULL,
+    static const RRDVAR_ACQUIRED *tcp_mem_low_threshold = NULL,
                   *tcp_mem_pressure_threshold = NULL,
                   *tcp_mem_high_threshold = NULL;
 
     if(unlikely(!tcp_mem_low_threshold)) {
-        tcp_mem_low_threshold      = rrdvar_custom_host_variable_create(localhost, "tcp_mem_low");
-        tcp_mem_pressure_threshold = rrdvar_custom_host_variable_create(localhost, "tcp_mem_pressure");
-        tcp_mem_high_threshold     = rrdvar_custom_host_variable_create(localhost, "tcp_mem_high");
+        tcp_mem_low_threshold      = rrdvar_custom_host_variable_add_and_acquire(localhost, "tcp_mem_low");
+        tcp_mem_pressure_threshold = rrdvar_custom_host_variable_add_and_acquire(localhost, "tcp_mem_pressure");
+        tcp_mem_high_threshold     = rrdvar_custom_host_variable_add_and_acquire(localhost, "tcp_mem_high");
     }
 
     if(unlikely(!filename)) {
@@ -69,7 +69,7 @@ static int read_tcp_mem(void) {
 
 static kernel_uint_t read_tcp_max_orphans(void) {
     static char *filename = NULL;
-    static RRDVAR *tcp_max_orphans_var = NULL;
+    static const RRDVAR_ACQUIRED *tcp_max_orphans_var = NULL;
 
     if(unlikely(!filename)) {
         char buffer[FILENAME_MAX + 1];
@@ -81,7 +81,7 @@ static kernel_uint_t read_tcp_max_orphans(void) {
     if(read_single_number_file(filename, &tcp_max_orphans) == 0) {
 
         if(unlikely(!tcp_max_orphans_var))
-            tcp_max_orphans_var = rrdvar_custom_host_variable_create(localhost, "tcp_max_orphans");
+            tcp_max_orphans_var = rrdvar_custom_host_variable_add_and_acquire(localhost, "tcp_max_orphans");
 
         rrdvar_custom_host_variable_set(localhost, tcp_max_orphans_var, tcp_max_orphans);
         return  tcp_max_orphans;
diff --git a/collectors/proc.plugin/proc_net_stat_conntrack.c b/collectors/proc.plugin/proc_net_stat_conntrack.c
index 642e33f8e9..3182a745a6 100644
--- a/collectors/proc.plugin/proc_net_stat_conntrack.c
+++ b/collectors/proc.plugin/proc_net_stat_conntrack.c
@@ -12,7 +12,7 @@ int do_proc_net_stat_conntrack(int update_every, usec_t dt) {
     static usec_t get_max_every = 10 * USEC_PER_SEC, usec_since_last_max = 0;
     static int read_full = 1;
     static char *nf_conntrack_filename, *nf_conntrack_count_filename, *nf_conntrack_max_filename;
-    static RRDVAR *rrdvar_max = NULL;
+    static const RRDVAR_ACQUIRED *rrdvar_max = NULL;
 
     unsigned long long aentries = 0, asearched = 0, afound = 0, anew = 0, ainvalid = 0, aignore = 0, adelete = 0, adelete_list = 0,
             ainsert = 0, ainsert_failed = 0, adrop = 0, aearly_drop = 0, aicmp_error = 0, aexpect_new = 0, aexpect_create = 0, aexpect_delete = 0, asearch_restart = 0;
@@ -50,7 +50,7 @@ int do_proc_net_stat_conntrack(int update_every, usec_t dt) {
         if(!do_sockets && !read_full)
             return 1;
 
-        rrdvar_max = rrdvar_custom_host_variable_creat