Deduplicate all netdata strings (#13570)

* rrdfamily

* rrddim

* rrdset plugin and module names

* rrdset units

* rrdset type

* rrdset family

* rrdset title

* rrdset title more

* rrdset context

* rrdcalctemplate context and removal of context hash from rrdset

* strings statistics

* rrdset name

* rearranged members of rrdset

* eliminate rrdset name hash; rrdcalc chart converted to STRING

* rrdset id, eliminated rrdset hash

* rrdcalc, alarm_entry, alert_config and some of rrdcalctemplate

* rrdcalctemplate

* rrdvar

* eval_variable

* rrddimvar and rrdsetvar

* rrdhost hostname, os and tags

* fix master commits

* added thread cache; implemented string_dup without locks

* faster thread cache

* rrdset and rrddim now use dictionaries for indexing

* rrdhost now uses dictionary

* rrdfamily now uses DICTIONARY

* rrdvar using dictionary instead of AVL

* allocate the right size to rrdvar flag members

* rrdhost remaining char * members to STRING *

* better error handling on indexing

* strings now use a read/write lock to allow parallel searches to the index

* removed AVL support from dictionaries; implemented STRING with native Judy calls

* string releases should be negative

* only 31 bits are allowed for enum flags

* proper locking on strings

* string threading unittest and fixes

* fix lgtm finding

* fixed naming

* stream chart/dimension definitions at the beginning of a streaming session

* thread stack variable is undefined on thread cancel

* rrdcontext garbage collect per host on startup

* worker control in garbage collection

* relaxed deletion of rrdmetrics

* type checking on dictfe

* netdata chart to monitor rrdcontext triggers

* Group chart label updates

* rrdcontext better handling of collected rrdsets

* rrdpush incremental transmition of definitions should use as much buffer as possible

* require 1MB per chart

* empty the sender buffer before enabling metrics streaming

* fill up to 50% of buffer

* reset signaling metrics sending

* use the shared variable for status

* use separate host flag for enabling streaming of metrics

* make sure the flag is clear

* add logging for streaming

* add logging for streaming on buffer overflow

* circular_buffer proper sizing

* removed obsolete logs

* do not execute worker jobs if not necessary

* better messages about compression disabling

* proper use of flags and updating rrdset last access time every time the obsoletion flag is flipped

* monitor stream sender used buffer ratio

* Update exporting unit tests

* no need to compare label value with strcmp

* streaming send workers now monitor bandwidth

* workers now use strings

* streaming receiver monitors incoming bandwidth

* parser shift of worker ids

* minor fixes

* Group chart label updates

* Populate context with dimensions that have data

* Fix chart id

* better shift of parser worker ids

* fix for streaming compression

* properly count received bytes

* ensure LZ4 compression ring buffer does not wrap prematurely

* do not stream empty charts; do not process empty instances in rrdcontext

* need_to_send_chart_definition() does not need an rrdset lock any more

* rrdcontext objects are collected, after data have been written to the db

* better logging of RRDCONTEXT transitions

* always set all variables needed by the worker utilization charts

* implemented double linked list for most objects; eliminated alarm indexes from rrdhost; and many more fixes

* lockless strings design - string_dup() and string_freez() are totally lockless when they dont need to touch Judy - only Judy is protected with a read/write lock

* STRING code re-organization for clarity

* thread_cache improvements; double numbers precision on worker threads

* STRING_ENTRY now shadown STRING, so no duplicate definition is required; string_length() renamed to string_strlen() to follow the paradigm of all other functions, STRING internal statistics are now only compiled with NETDATA_INTERNAL_CHECKS

* rrdhost index by hostname now cleans up; aclk queries of archieved hosts do not index hosts

* Add index to speed up database context searches

* Removed last_updated optimization (was also buggy after latest merge with master)

Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
Co-authored-by: Vladimir Kobal <vlad@prokk.net>

12 files changed, 122 insertions, 123 deletions

diff --git a/web/api/badges/web_buffer_svg.c b/web/api/badges/web_buffer_svg.c
index 00b4ad650a..c4d9ced5f0 100644
--- a/web/api/badges/web_buffer_svg.c
+++ b/web/api/badges/web_buffer_svg.c
@@ -1020,19 +1020,19 @@ int web_client_api_request_v1_badge(RRDHOST *host, struct web_client *w, char *u
             label = dim;
         }
         else
-            label = st->name;
+            label = rrdset_name(st);
     }
     if(!units) {
         if(alarm) {
             if(rc->units)
-                units = rc->units;
+                units = rrdcalc_units(rc);
             else
                 units = "";
         }
         else if(options & RRDR_OPTION_PERCENTAGE)
             units = "%";
         else
-            units = st->units;
+            units = rrdset_units(st);
     }
 
     debug(D_WEB_CLIENT, "%llu: API command 'badge.svg' for chart '%s', alarm '%s', dimensions '%s', after '%lld', before '%lld', points '%d', group '%d', options '0x%08x'"
diff --git a/web/api/exporters/shell/allmetrics_shell.c b/web/api/exporters/shell/allmetrics_shell.c
index 615aab43cd..184f44b188 100644
--- a/web/api/exporters/shell/allmetrics_shell.c
+++ b/web/api/exporters/shell/allmetrics_shell.c
@@ -30,14 +30,14 @@ void rrd_stats_api_v1_charts_allmetrics_shell(RRDHOST *host, const char *filter_
     // for each chart
     RRDSET *st;
     rrdset_foreach_read(st, host) {
-        if (filter && !simple_pattern_matches(filter, st->name))
+        if (filter && !simple_pattern_matches(filter, rrdset_name(st)))
             continue;
 
         NETDATA_DOUBLE total = 0.0;
         char chart[SHELL_ELEMENT_MAX + 1];
-        shell_name_copy(chart, st->name?st->name:st->id, SHELL_ELEMENT_MAX);
+        shell_name_copy(chart, st->name?rrdset_name(st):rrdset_id(st), SHELL_ELEMENT_MAX);
 
-        buffer_sprintf(wb, "\n# chart: %s (name: %s)\n", st->id, st->name);
+        buffer_sprintf(wb, "\n# chart: %s (name: %s)\n", rrdset_id(st), rrdset_name(st));
         if(rrdset_is_available_for_viewers(st)) {
             rrdset_rdlock(st);
 
@@ -46,23 +46,23 @@ void rrd_stats_api_v1_charts_allmetrics_shell(RRDHOST *host, const char *filter_
             rrddim_foreach_read(rd, st) {
                 if(rd->collections_counter && !rrddim_flag_check(rd, RRDDIM_FLAG_OBSOLETE)) {
                     char dimension[SHELL_ELEMENT_MAX + 1];
-                    shell_name_copy(dimension, rd->name?rd->name:rd->id, SHELL_ELEMENT_MAX);
+                    shell_name_copy(dimension, rd->name?rrddim_name(rd):rrddim_id(rd), SHELL_ELEMENT_MAX);
 
                     NETDATA_DOUBLE n = rd->last_stored_value;
 
                     if(isnan(n) || isinf(n))
-                        buffer_sprintf(wb, "NETDATA_%s_%s=\"\"      # %s\n", chart, dimension, st->units);
+                        buffer_sprintf(wb, "NETDATA_%s_%s=\"\"      # %s\n", chart, dimension, rrdset_units(st));
                     else {
                         if(rd->multiplier < 0 || rd->divisor < 0) n = -n;
                         n = roundndd(n);
                         if(!rrddim_flag_check(rd, RRDDIM_FLAG_HIDDEN)) total += n;
-                        buffer_sprintf(wb, "NETDATA_%s_%s=\"" NETDATA_DOUBLE_FORMAT_ZERO "\"      # %s\n", chart, dimension, n, st->units);
+                        buffer_sprintf(wb, "NETDATA_%s_%s=\"" NETDATA_DOUBLE_FORMAT_ZERO "\"      # %s\n", chart, dimension, n, rrdset_units(st));
                     }
                 }
             }
 
             total = roundndd(total);
-            buffer_sprintf(wb, "NETDATA_%s_VISIBLETOTAL=\"" NETDATA_DOUBLE_FORMAT_ZERO "\"      # %s\n", chart, total, st->units);
+            buffer_sprintf(wb, "NETDATA_%s_VISIBLETOTAL=\"" NETDATA_DOUBLE_FORMAT_ZERO "\"      # %s\n", chart, total, rrdset_units(st));
             rrdset_unlock(st);
         }
     }
@@ -70,22 +70,22 @@ void rrd_stats_api_v1_charts_allmetrics_shell(RRDHOST *host, const char *filter_
     buffer_strcat(wb, "\n# NETDATA ALARMS RUNNING\n");
 
     RRDCALC *rc;
-    for(rc = host->alarms; rc ;rc = rc->next) {
+    foreach_rrdcalc_in_rrdhost(host, rc) {
         if(!rc->rrdset) continue;
 
         char chart[SHELL_ELEMENT_MAX + 1];
-        shell_name_copy(chart, rc->rrdset->name?rc->rrdset->name:rc->rrdset->id, SHELL_ELEMENT_MAX);
+        shell_name_copy(chart, rc->rrdset->name?rrdset_name(rc->rrdset):rrdset_id(rc->rrdset), SHELL_ELEMENT_MAX);
 
         char alarm[SHELL_ELEMENT_MAX + 1];
-        shell_name_copy(alarm, rc->name, SHELL_ELEMENT_MAX);
+        shell_name_copy(alarm, rrdcalc_name(rc), SHELL_ELEMENT_MAX);
 
         NETDATA_DOUBLE n = rc->value;
 
         if(isnan(n) || isinf(n))
-            buffer_sprintf(wb, "NETDATA_ALARM_%s_%s_VALUE=\"\"      # %s\n", chart, alarm, rc->units);
+            buffer_sprintf(wb, "NETDATA_ALARM_%s_%s_VALUE=\"\"      # %s\n", chart, alarm, rrdcalc_units(rc));
         else {
             n = roundndd(n);
-            buffer_sprintf(wb, "NETDATA_ALARM_%s_%s_VALUE=\"" NETDATA_DOUBLE_FORMAT_ZERO "\"      # %s\n", chart, alarm, n, rc->units);
+            buffer_sprintf(wb, "NETDATA_ALARM_%s_%s_VALUE=\"" NETDATA_DOUBLE_FORMAT_ZERO "\"      # %s\n", chart, alarm, n, rrdcalc_units(rc));
         }
 
         buffer_sprintf(wb, "NETDATA_ALARM_%s_%s_STATUS=\"%s\"\n", chart, alarm, rrdcalc_status2string(rc->status));
@@ -110,7 +110,7 @@ void rrd_stats_api_v1_charts_allmetrics_json(RRDHOST *host, const char *filter_s
     // for each chart
     RRDSET *st;
     rrdset_foreach_read(st, host) {
-        if (filter && !(simple_pattern_matches(filter, st->id) || simple_pattern_matches(filter, st->name)))
+        if (filter && !(simple_pattern_matches(filter, rrdset_id(st)) || simple_pattern_matches(filter, rrdset_name(st))))
             continue;
 
         if(rrdset_is_available_for_viewers(st)) {
@@ -127,11 +127,11 @@ void rrd_stats_api_v1_charts_allmetrics_json(RRDHOST *host, const char *filter_s
                 "\t\t\"last_updated\": %"PRId64",\n"
                 "\t\t\"dimensions\": {",
                 chart_counter ? "," : "",
-                st->id,
-                st->name,
-                st->family,
-                st->context,
-                st->units,
+                rrdset_id(st),
+                rrdset_name(st),
+                rrdset_family(st),
+                rrdset_context(st),
+                rrdset_units(st),
                 (int64_t)rrdset_last_entry_t_nolock(st));
 
             chart_counter++;
@@ -148,8 +148,8 @@ void rrd_stats_api_v1_charts_allmetrics_json(RRDHOST *host, const char *filter_s
                         "\t\t\t\t\"name\": \"%s\",\n"
                         "\t\t\t\t\"value\": ",
                         dimension_counter ? "," : "",
-                        rd->id,
-                        rd->name);
+                        rrddim_id(rd),
+                        rrddim_name(rd));
 
                     if(isnan(rd->last_stored_value))
                         buffer_strcat(wb, "null");
diff --git a/web/api/formatters/charts2json.c b/web/api/formatters/charts2json.c
index 4325b65301..73e4247f4e 100644
--- a/web/api/formatters/charts2json.c
+++ b/web/api/formatters/charts2json.c
@@ -57,11 +57,11 @@ void charts2json(RRDHOST *host, BUFFER *wb, int skip_volatile, int show_archived
                        ",\n\t\"memory_mode\": \"%s\""
                        ",\n\t\"custom_info\": \"%s\""
                        ",\n\t\"charts\": {"
-                   , host->hostname
-                   , host->program_version
+                   , rrdhost_hostname(host)
+                   , rrdhost_program_version(host)
                    , get_release_channel()
-                   , host->os
-                   , host->timezone
+                   , rrdhost_os(host)
+                   , rrdhost_timezone(host)
                    , host->rrd_update_every
                    , host->rrd_history_entries
                    , rrd_memory_mode_name(host->rrd_memory_mode)
@@ -74,7 +74,7 @@ void charts2json(RRDHOST *host, BUFFER *wb, int skip_volatile, int show_archived
         if ((!show_archived && rrdset_is_available_for_viewers(st)) || (show_archived && rrdset_is_archived(st))) {
             if(c) buffer_strcat(wb, ",");
             buffer_strcat(wb, "\n\t\t\"");
-            buffer_strcat(wb, st->id);
+            buffer_strcat(wb, rrdset_id(st));
             buffer_strcat(wb, "\": ");
             rrdset2json(st, wb, &dimensions, &memory, skip_volatile);
 
@@ -84,7 +84,7 @@ void charts2json(RRDHOST *host, BUFFER *wb, int skip_volatile, int show_archived
     }
 
     RRDCALC *rc;
-    for(rc = host->alarms; rc ; rc = rc->next) {
+    foreach_rrdcalc_in_rrdhost(host, rc) {
         if(rc->rrdset)
             alarms++;
     }
@@ -117,7 +117,7 @@ void charts2json(RRDHOST *host, BUFFER *wb, int skip_volatile, int show_archived
                                  "\n\t\t\t\"hostname\": \"%s\""
                                  "\n\t\t}"
                                , (found > 0) ? "," : ""
-                               , h->hostname
+                               , rrdhost_hostname(h)
                 );
 
                 found++;
@@ -131,7 +131,7 @@ void charts2json(RRDHOST *host, BUFFER *wb, int skip_volatile, int show_archived
                        , "\n\t\t{"
                          "\n\t\t\t\"hostname\": \"%s\""
                          "\n\t\t}"
-                       , host->hostname
+                       , rrdhost_hostname(host)
         );
     }
 
@@ -141,8 +141,8 @@ void charts2json(RRDHOST *host, BUFFER *wb, int skip_volatile, int show_archived
 // generate collectors list for the api/v1/info call
 
 struct collector {
-    char *plugin;
-    char *module;
+    const char *plugin;
+    const char *module;
 };
 
 struct array_printer {
@@ -176,8 +176,8 @@ void chartcollectors2json(RRDHOST *host, BUFFER *wb) {
     rrdset_foreach_read(st, host) {
         if (rrdset_is_available_for_viewers(st)) {
             struct collector col = {
-                    .plugin = st->plugin_name ? st->plugin_name : "",
-                    .module = st->module_name ? st->module_name : ""
+                    .plugin = rrdset_plugin_name(st),
+                    .module = rrdset_module_name(st)
             };
             sprintf(name, "%s:%s", col.plugin, col.module);
             dictionary_set(dict, name, &col, sizeof(struct collector));
diff --git a/web/api/formatters/csv/csv.c b/web/api/formatters/csv/csv.c
index 6d87ca3748..14dad0cb8a 100644
--- a/web/api/formatters/csv/csv.c
+++ b/web/api/formatters/csv/csv.c
@@ -23,7 +23,7 @@ void rrdr2csv(RRDR *r, BUFFER *wb, uint32_t format, RRDR_OPTIONS options, const
         }
         buffer_strcat(wb, separator);
         if(options & RRDR_OPTION_LABEL_QUOTES) buffer_strcat(wb, "\"");
-        buffer_strcat(wb, d->name);
+        buffer_strcat(wb, rrddim_name(d));
         if(options & RRDR_OPTION_LABEL_QUOTES) buffer_strcat(wb, "\"");
         i++;
     }
diff --git a/web/api/formatters/json/json.c b/web/api/formatters/json/json.c
index 6f07b9aa43..ae1e55b7a9 100644
--- a/web/api/formatters/json/json.c
+++ b/web/api/formatters/json/json.c
@@ -121,7 +121,7 @@ void rrdr2json(RRDR *r, BUFFER *wb, RRDR_OPTIONS options, int datatable,  struct
         if(unlikely((options & RRDR_OPTION_NONZERO) && !(r->od[c] & RRDR_DIMENSION_NONZERO))) continue;
 
         buffer_fast_strcat(wb, pre_label, pre_label_len);
-        buffer_strcat(wb, rd->name);
+        buffer_strcat(wb, rrddim_name(rd));
 //        buffer_strcat(wb, ".");
 //        buffer_strcat(wb, rd->rrdset->name);
         buffer_fast_strcat(wb, post_label, post_label_len);
@@ -253,7 +253,7 @@ void rrdr2json(RRDR *r, BUFFER *wb, RRDR_OPTIONS options, int datatable,  struct
             buffer_fast_strcat(wb, pre_value, pre_value_len);
 
             if(unlikely( options & RRDR_OPTION_OBJECTSROWS ))
-                buffer_sprintf(wb, "%s%s%s: ", kq, rd->name, kq);
+                buffer_sprintf(wb, "%s%s%s: ", kq, rrddim_name(rd), kq);
 
             if(co[c] & RRDR_VALUE_EMPTY && !(options & RRDR_OPTION_INTERNAL_AR)) {
                 if(unlikely(options & RRDR_OPTION_NULL2ZERO))
diff --git a/web/api/formatters/json_wrapper.c b/web/api/formatters/json_wrapper.c
index 04cace2fb3..a07ee17446 100644
--- a/web/api/formatters/json_wrapper.c
+++ b/web/api/formatters/json_wrapper.c
@@ -79,8 +79,8 @@ void rrdr_json_wrapper_begin(RRDR *r, BUFFER *wb, uint32_t format, RRDR_OPTIONS
                        "   %sgroup%s: %s%s%s,\n"
                        "   %soptions%s: %s"
                    , kq, kq
-                   , kq, kq, sq, context_mode && temp_rd?r->st->context:r->st->id, sq
-                   , kq, kq, sq, context_mode && temp_rd?r->st->context:r->st->name, sq
+                   , kq, kq, sq, context_mode && temp_rd?rrdset_context(r->st):rrdset_id(r->st), sq
+                   , kq, kq, sq, context_mode && temp_rd?rrdset_context(r->st):rrdset_name(r->st), sq
                    , kq, kq, r->update_every
                    , kq, kq, r->st->update_every
                    , kq, kq, (uint32_t) (context_param_list ? context_param_list->first_entry_t : rrdset_first_entry_t_nolock(r->st))
@@ -103,13 +103,13 @@ void rrdr_json_wrapper_begin(RRDR *r, BUFFER *wb, uint32_t format, RRDR_OPTIONS
 
         if(i) buffer_strcat(wb, ", ");
         buffer_strcat(wb, sq);
-        buffer_strcat(wb, rd->name);
+        buffer_strcat(wb, rrddim_name(rd));
         buffer_strcat(wb, sq);
         i++;
     }
     if(!i) {
 #ifdef NETDATA_INTERNAL_CHECKS
-        error("RRDR is empty for %s (RRDR has %d dimensions, options is 0x%08x)", r->st->id, r->d, options);
+        error("RRDR is empty for %s (RRDR has %d dimensions, options is 0x%08x)", rrdset_id(r->st), r->d, options);
 #endif
         rows = 0;
         buffer_strcat(wb, sq);
@@ -127,7 +127,7 @@ void rrdr_json_wrapper_begin(RRDR *r, BUFFER *wb, uint32_t format, RRDR_OPTIONS
 
         if(i) buffer_strcat(wb, ", ");
         buffer_strcat(wb, sq);
-        buffer_strcat(wb, rd->id);
+        buffer_strcat(wb, rrddim_id(rd));
         buffer_strcat(wb, sq);
         i++;
     }
@@ -149,8 +149,8 @@ void rrdr_json_wrapper_begin(RRDR *r, BUFFER *wb, uint32_t format, RRDR_OPTIONS
 
         DICTIONARY *dict = dictionary_create(DICTIONARY_FLAG_SINGLE_THREADED);
         for (i = 0, rd = temp_rd ? temp_rd : r->st->dimensions; rd; rd = rd->next) {
-            snprintfz(name, RRD_ID_LENGTH_MAX * 2, "%s:%s", rd->id, rd->name);
-            int len = snprintfz(output, RRD_ID_LENGTH_MAX * 2 + 7, "[\"%s\",\"%s\"]", rd->id, rd->name);
+            snprintfz(name, RRD_ID_LENGTH_MAX * 2, "%s:%s", rrddim_id(rd), rrddim_name(rd));
+            int len = snprintfz(output, RRD_ID_LENGTH_MAX * 2 + 7, "[\"%s\",\"%s\"]", rrddim_id(rd), rrddim_name(rd));
             dictionary_set(dict, name, output, len+1);
         }
         dictionary_walkthrough_read(dict, value_list_output, &co);
@@ -160,8 +160,8 @@ void rrdr_json_wrapper_begin(RRDR *r, BUFFER *wb, uint32_t format, RRDR_OPTIONS
         buffer_sprintf(wb, "],\n   %sfull_chart_list%s: [", kq, kq);
         dict = dictionary_create(DICTIONARY_FLAG_SINGLE_THREADED);
         for (i = 0, rd = temp_rd ? temp_rd : r->st->dimensions; rd; rd = rd->next) {
-            int len = snprintfz(output, RRD_ID_LENGTH_MAX * 2 + 7, "[\"%s\",\"%s\"]", rd->rrdset->id, rd->rrdset->name);
-            snprintfz(name, RRD_ID_LENGTH_MAX * 2, "%s:%s", rd->rrdset->id, rd->rrdset->name);
+            int len = snprintfz(output, RRD_ID_LENGTH_MAX * 2 + 7, "[\"%s\",\"%s\"]", rrdset_id(rd->rrdset), rrdset_name(rd->rrdset));
+            snprintfz(name, RRD_ID_LENGTH_MAX * 2, "%s:%s", rrdset_id(rd->rrdset), rrdset_name(rd->rrdset));
             dictionary_set(dict, name, output, len + 1);
         }
 
@@ -198,7 +198,7 @@ void rrdr_json_wrapper_begin(RRDR *r, BUFFER *wb, uint32_t format, RRDR_OPTIONS
             if (i)
                 buffer_strcat(wb, ", ");
             buffer_strcat(wb, sq);
-            buffer_strcat(wb, rd->rrdset->id);
+            buffer_strcat(wb, rrdset_id(rd->rrdset));
             buffer_strcat(wb, sq);
             i++;
         }
diff --git a/web/api/formatters/rrd2json.c b/web/api/formatters/rrd2json.c
index 7aa478d95a..b18abe59cb 100644
--- a/web/api/formatters/rrd2json.c
+++ b/web/api/formatters/rrd2json.c
@@ -8,13 +8,15 @@ static inline void free_single_rrdrim(ONEWAYALLOC *owa, RRDDIM *temp_rd, int arc
     if (unlikely(!temp_rd))
         return;
 
-    onewayalloc_freez(owa, (char *)temp_rd->id);
+    string_freez(temp_rd->id);
+    string_freez(temp_rd->name);
 
     if (unlikely(archive_mode)) {
         temp_rd->rrdset->counter--;
         if (!temp_rd->rrdset->counter) {
-            onewayalloc_freez(owa, (char *)temp_rd->rrdset->name);
-            onewayalloc_freez(owa, temp_rd->rrdset->context);
+            string_freez(temp_rd->rrdset->id);
+            string_freez(temp_rd->rrdset->name);
+            string_freez(temp_rd->rrdset->context);
             onewayalloc_freez(owa, temp_rd->rrdset);
         }
     }
@@ -111,8 +113,8 @@ void build_context_param_list(ONEWAYALLOC *owa, struct context_param **param_lis
 
     rrddim_foreach_read(rd1, st) {
         RRDDIM *rd = onewayalloc_memdupz(owa, rd1, sizeof(RRDDIM));
-        rd->id = onewayalloc_strdupz(owa, rd1->id);
-        rd->name = onewayalloc_strdupz(owa, rd1->name);
+        rd->id = string_dup(rd1->id);
+        rd->name = string_dup(rd1->name);
         for(int tier = 0; tier < storage_tiers ;tier++) {
             if(rd1->tiers[tier])
                 rd->tiers[tier] = onewayalloc_memdupz(owa, rd1->tiers[tier], sizeof(*rd->tiers[tier]));
diff --git a/web/api/formatters/rrdset2json.c b/web/api/formatters/rrdset2json.c
index de8d87bae5..9e81389fdf 100644
--- a/web/api/formatters/rrdset2json.c
+++ b/web/api/formatters/rrdset2json.c
@@ -45,18 +45,18 @@ void rrdset2json(RRDSET *st, BUFFER *wb, size_t *dimensions_count, size_t *memor
         "\t\t\t\"units\": \"%s\",\n"
         "\t\t\t\"data_url\": \"/api/v1/data?chart=%s\",\n"
         "\t\t\t\"chart_type\": \"%s\",\n",
-        st->id,
-        st->name,
-        st->type,
-        st->family,
-        st->context,
-        st->title,
-        st->name,
+        rrdset_id(st),
+        rrdset_name(st),
+        rrdset_type(st),
+        rrdset_family(st),
+        rrdset_context(st),
+        rrdset_title(st),
+        rrdset_name(st),
         st->priority,
-        st->plugin_name ? st->plugin_name : "",
-        st->module_name ? st->module_name : "",
-        st->units,
-        st->name,
+        rrdset_plugin_name(st),
+        rrdset_module_name(st),
+        rrdset_units(st),
+        rrdset_name(st),
         rrdset_type_name(st->chart_type));
 
     if (likely(!skip_volatile))
@@ -98,9 +98,9 @@ void rrdset2json(RRDSET *st, BUFFER *wb, size_t *dimensions_count, size_t *memor
             buffer_strcat(wb, ",\n\t\t\t\t\"");
         else
             buffer_strcat(wb, "\t\t\t\t\"");
-        buffer_strcat_jsonescape(wb, rd->id);
+        buffer_strcat_jsonescape(wb, rrddim_id(rd));
         buffer_strcat(wb, "\": { \"name\": \"");
-        buffer_strcat_jsonescape(wb, rd->name);
+        buffer_strcat_jsonescape(wb, rrddim_name(rd));
         buffer_strcat(wb, "\" }");
 
         dimensions++;
@@ -121,7 +121,7 @@ void rrdset2json(RRDSET *st, BUFFER *wb, size_t *dimensions_count, size_t *memor
         buffer_strcat(wb, ",\n\t\t\t\"alarms\": {\n");
         size_t alarms = 0;
         RRDCALC *rc;
-        for (rc = st->alarms; rc; rc = rc->rrdset_next) {
+        foreach_rrdcalc_in_rrdset(st, rc) {
             buffer_sprintf(
                 wb,
                 "%s"
@@ -131,7 +131,7 @@ void rrdset2json(RRDSET *st, BUFFER *wb, size_t *dimensions_count, size_t *memor
                 "\t\t\t\t\t\"units\": \"%s\",\n"
                 "\t\t\t\t\t\"update_every\": %d\n"
                 "\t\t\t\t}",
-                (alarms) ? ",\n" : "", rc->name, rc->id, rrdcalc_status2string(rc->status), rc->units,
+                (alarms) ? ",\n" : "", rrdcalc_name(rc), rc->id, rrdcalc_status2string(rc->status), rrdcalc_units(rc),
                 rc->update_every);
 
             alarms++;
diff --git a/web/api/queries/query.c b/web/api/queries/query.c
index d776f6d11c..300a4429ce 100644
--- a/web/api/queries/query.c
+++ b/web/api/queries/query.c
@@ -665,11 +665,11 @@ static void rrdr_disable_not_selected_dimensions(RRDR *r, RRDR_OPTIONS options,
     RRDDIM *temp_rd = context_param_list ? context_param_list->rd : NULL;
     int should_lock = (!context_param_list || !(context_param_list->flags & CONTEXT_FLAGS_ARCHIVE));
 
+    if(unlikely(!dims || !*dims || (dims[0] == '*' && dims[1] == '\0'))) return;
+
     if (should_lock)
         rrdset_check_rdlock(r->st);
 
-    if(unlikely(!dims || !*dims || (dims[0] == '*' && dims[1] == '\0'))) return;
-
     int match_ids = 0, match_names = 0;
 
     if(unlikely(options & RRDR_OPTION_MATCH_IDS))
@@ -685,8 +685,8 @@ static void rrdr_disable_not_selected_dimensions(RRDR *r, RRDR_OPTIONS options,
     RRDDIM *d;
     long c, dims_selected = 0, dims_not_hidden_not_zero = 0;
     for(c = 0, d = temp_rd?temp_rd:r->st->dimensions; d ;c++, d = d->next) {
-        if(    (match_ids   && simple_pattern_matches(pattern, d->id))
-               || (match_names && simple_pattern_matches(pattern, d->name))
+        if(       (match_ids   && simple_pattern_matches(pattern, rrddim_id(d)))
+               || (match_names && simple_pattern_matches(pattern, rrddim_name(d)))
                 ) {
             r->od[c] |= RRDR_DIMENSION_SELECTED;
             if(unlikely(r->od[c] & RRDR_DIMENSION_HIDDEN)) r->od[c] &= ~RRDR_DIMENSION_HIDDEN;
@@ -738,11 +738,11 @@ static inline long rrdr_line_init(RRDR *r, time_t t, long rrdr_line) {
 
     internal_error(rrdr_line >= r->n,
                    "QUERY: requested to step above RRDR size for chart '%s'",
-                   r->st->name);
+                   rrdset_name(r->st));
 
     internal_error(r->t[rrdr_line] != 0 && r->t[rrdr_line] != t,
                    "QUERY: overwriting the timestamp of RRDR line %zu from %zu to %zu, of chart '%s'",
-                   (size_t)rrdr_line, (size_t)r->t[rrdr_line], (size_t)t, r->st->name);
+                   (size_t)rrdr_line, (size_t)r->t[rrdr_line], (size_t)t, rrdset_name(r->st));
 
     // save the time
     r->t[rrdr_line] = t;
@@ -768,7 +768,7 @@ static int rrddim_find_best_tier_for_timeframe(RRDDIM *rd, time_t after_wanted,
             internal_error(true, "QUERY: NULL dimension - invalid params to tier calculation");
         else
             internal_error(true, "QUERY: chart '%s' dimension '%s' invalid params to tier calculation",
-                           (rd->rrdset)?rd->rrdset->name:"unknown", rd->name);
+                           (rd->rrdset)?rrdset_name(rd->rrdset):"unknown", rrddim_name(rd));
 
         return 0;
     }
@@ -782,7 +782,7 @@ static int rrddim_find_best_tier_for_timeframe(RRDDIM *rd, time_t after_wanted,
     for(int tier = 0; tier < storage_tiers ; tier++) {
         if(unlikely(!rd->tiers[tier])) {
             internal_error(true, "QUERY: tier %d of chart '%s' dimension '%s' not initialized",
-                           tier, rd->rrdset->name, rd->name);
+                           tier, rrdset_name(rd->rrdset), rrddim_name(rd));
     //        buffer_free(wb);
             return 0;
         }
@@ -799,7 +799,7 @@ static int rrddim_find_best_tier_for_timeframe(RRDDIM *rd, time_t after_wanted,
         int update_every = (int)rd->tiers[tier]->tier_grouping * (int)rd->update_every;
         if(unlikely(update_every == 0)) {
             internal_error(true, "QUERY: update_every of tier %d for chart '%s' dimension '%s' is zero. tg = %d, ue = %d",
-                           tier, rd->rrdset->name, rd->name, rd->tiers[tier]->tier_grouping, rd->update_every);
+                           tier, rrdset_name(rd->rrdset), rrddim_name(rd), rd->tiers[tier]->tier_grouping, rd->update_every);
     //        buffer_free(wb);
             return 0;
         }
@@ -851,7 +851,7 @@ static int rrdset_find_natural_update_every_for_timeframe(RRDSET *st, time_t aft
     if(!st->dimensions->tiers[best_tier]) {
         internal_error(
             true,
-            "QUERY: tier %d on chart '%s', is not initialized", best_tier, st->name);
+            "QUERY: tier %d on chart '%s', is not initialized", best_tier, rrdset_name(st));
     }
     else {
         ret = (int)st->dimensions->tiers[best_tier]->tier_grouping * (int)st->update_every;
@@ -859,7 +859,7 @@ static int rrdset_find_natural_update_every_for_timeframe(RRDSET *st, time_t aft
             internal_error(
                 true,
                 "QUERY: update_every calculated to be zero on chart '%s', tier_grouping %d, update_every %d",
-                st->name, st->dimensions->tiers[best_tier]->tier_grouping, st->update_every);
+                rrdset_name(st), st->dimensions->tiers[best_tier]->tier_grouping, st->update_every);
 
             ret = st->update_every;
         }
@@ -1259,7 +1259,7 @@ static inline void rrd2rrdr_do_dimension(
             // check if the db is giving us zero duration points
             if(unlikely(new_point.start_time == new_point.end_time)) {
                 internal_error(true, "QUERY: next_metric(%s, %s) returned point %zu start time %ld, end time %ld, that are both equal",
-                               rd->rrdset->name, rd->name, new_point.id, new_point.start_time, new_point.end_time);
+                               rrdset_name(rd->rrdset), rrddim_name(rd), new_point.id, new_point.start_time, new_point.end_time);
 
                 new_point.start_time = new_point.end_time - ((time_t)ops.tier_ptr->tier_grouping * (time_t)ops.rd->update_every);
             }
@@ -1267,7 +1267,7 @@ static inline void rrd2rrdr_do_dimension(
             // check if the db is advancing the query
             if(unlikely(new_point.end_time <= last1_point.end_time)) {
                 internal_error(true, "QUERY: next_metric(%s, %s) returned point %zu from %ld time %ld, before the last point %zu end time %ld, now is %ld to %ld",
-                               rd->rrdset->name, rd->name, new_point.id, new_point.start_time, new_point.end_time,
+                               rrdset_name(rd->rrdset), rrddim_name(rd), new_point.id, new_point.start_time, new_point.end_time,
                                last1_point.id, last1_point.end_time, now_start_time, now_end_time);
 
                 count_same_end_time++;
@@ -1295,7 +1295,7 @@ static inline void rrd2rrdr_do_dimension(
                     // we only log if this is not point 1
                     internal_error(new_point.end_time < after_wanted && new_point.id > 1,
                                    "QUERY: next_metric(%s, %s) returned point %zu from %ld time %ld, which is entirely before our current timeframe %ld to %ld (and before the entire query, after %ld, before %ld)",
-                                   rd->rrdset->name, rd->name,
+                                   rrdset_name(rd->rrdset), rrddim_name(rd),
                                    new_point.id, new_point.start_time, new_point.end_time,
                                    now_start_time, now_end_time,
                                    after_wanted, before_wanted);
@@ -1339,7 +1339,7 @@ static inline void rrd2rrdr_do_dimension(
                 internal_error(current_point.id > 0 && last1_point.id == 0 && current_point.end_time > after_wanted && current_point.end_time > now_end_time,
                                "QUERY: on '%s', dim '%s', after %ld, before %ld, view update every %ld, query granularity %ld,"
                                " interpolating point %zu (from %ld to %ld) at %ld, but we could really favor by having last_point1 in this query.",
-                               rd->rrdset->name, rd->name, after_wanted, before_wanted, ops.view_update_every, ops.query_granularity,
+                               rrdset_name(rd->rrdset), rrddim_name(rd), after_wanted, before_wanted, ops.view_update_every, ops.query_granularity,
                                current_point.id, current_point.start_time, current_point.end_time, now_end_time);
             }
             else if(likely(now_end_time <= last1_point.end_time)) {
@@ -1350,7 +1350,7 @@ static inline void rrd2rrdr_do_dimension(
                 internal_error(current_point.id > 0 && last2_point.id == 0 && current_point.end_time > after_wanted && current_point.end_time > now_end_time,
                                "QUERY: on '%s', dim '%s', after %ld, before %ld, view update every %ld, query granularity %ld,"
                                " interpolating point %zu (from %ld to %ld) at %ld, but we could really favor by having last_point2 in this query.",
-                               rd->rrdset->name, rd->name, after_wanted, before_wanted, ops.view_update_every, ops.query_granularity,
+                               rrdset_name(rd->rrdset), rrddim_name(rd), after_wanted, before_wanted, ops.view_update_every, ops.query_granularity,
                                current_point.id, current_point.start_time, current_point.end_time, now_end_time);
             }
             else {
@@ -1425,7 +1425,7 @@ static inline void rrd2rrdr_do_dimension(
 
     internal_error((long)points_added != points_wanted,
                    "QUERY: query on %s/%s requested %zu points, but RRDR added %zu (%zu db points read).",
-                   r->st->name, rd->name, (size_t)points_wanted, (size_t)points_added, ops.db_total_points_read);
+                   rrdset_name(r->st), rrddim_name(rd), (size_t)points_wanted, (size_t)points_added, ops.db_total_points_read);
 }
 
 // ----------------------------------------------------------------------------
@@ -1518,7 +1518,7 @@ static void rrd2rrdr_log_request_response_metadata(RRDR *r
          //"slot (after: %zu, before: %zu, delta: %zu), "
          "points (got: %ld, want: %ld, req: %ld, db: %ld), "
          "%s"
-         , r->st->name
+         , rrdset_name(r->st)
          , r->st->update_every
 
          // grouping
@@ -1733,7 +1733,7 @@ RRDR *rrd2rrdr(
             rrdset_unlock(st);
 
             if(first_entry_t == 0 || last_entry_t == 0) {
-                internal_error(true, "QUERY: chart without data detected on '%s'", st->name);
+                internal_error(true, "QUERY: chart without data detected on '%s'", rrdset_name(st));
                 query_debug_log_free();
                 return NULL;
             }
@@ -1932,13 +1932,13 @@ RRDR *rrd2rrdr(
     RRDR *r = rrdr_create(owa, st, points_wanted, context_param_list);
     if(unlikely(!r)) {
         internal_error(true, "QUERY: cannot create RRDR for %s, after=%u, before=%u, duration=%u, points=%ld",
-                       st->id, (uint32_t)after_wanted, (uint32_t)before_wanted, (uint32_t)duration, points_wanted);
+                       rrdset_id(st), (uint32_t)after_wanted, (uint32_t)before_wanted, (uint32_t)duration, points_wanted);
         return NULL;
     }
 
     if(unlikely(!r->d || !points_wanted)) {
         internal_error(true, "QUERY: returning empty RRDR (no dimensions in RRDSET) for %s, after=%u, before=%u, duration=%zu, points=%ld",
-                       st->id, (uint32_t)after_wanted, (uint32_t)before_wanted, (size_t)duration, points_wanted);
+                       rrdset_id(st), (uint32_t)after_wanted, (uint32_t)before_wanted, (size_t)duration, points_wanted);
         return r;
     }