Deduplicate all netdata strings (#13570)

* rrdfamily

* rrddim

* rrdset plugin and module names

* rrdset units

* rrdset type

* rrdset family

* rrdset title

* rrdset title more

* rrdset context

* rrdcalctemplate context and removal of context hash from rrdset

* strings statistics

* rrdset name

* rearranged members of rrdset

* eliminate rrdset name hash; rrdcalc chart converted to STRING

* rrdset id, eliminated rrdset hash

* rrdcalc, alarm_entry, alert_config and some of rrdcalctemplate

* rrdcalctemplate

* rrdvar

* eval_variable

* rrddimvar and rrdsetvar

* rrdhost hostname, os and tags

* fix master commits

* added thread cache; implemented string_dup without locks

* faster thread cache

* rrdset and rrddim now use dictionaries for indexing

* rrdhost now uses dictionary

* rrdfamily now uses DICTIONARY

* rrdvar using dictionary instead of AVL

* allocate the right size to rrdvar flag members

* rrdhost remaining char * members to STRING *

* better error handling on indexing

* strings now use a read/write lock to allow parallel searches to the index

* removed AVL support from dictionaries; implemented STRING with native Judy calls

* string releases should be negative

* only 31 bits are allowed for enum flags

* proper locking on strings

* string threading unittest and fixes

* fix lgtm finding

* fixed naming

* stream chart/dimension definitions at the beginning of a streaming session

* thread stack variable is undefined on thread cancel

* rrdcontext garbage collect per host on startup

* worker control in garbage collection

* relaxed deletion of rrdmetrics

* type checking on dictfe

* netdata chart to monitor rrdcontext triggers

* Group chart label updates

* rrdcontext better handling of collected rrdsets

* rrdpush incremental transmition of definitions should use as much buffer as possible

* require 1MB per chart

* empty the sender buffer before enabling metrics streaming

* fill up to 50% of buffer

* reset signaling metrics sending

* use the shared variable for status

* use separate host flag for enabling streaming of metrics

* make sure the flag is clear

* add logging for streaming

* add logging for streaming on buffer overflow

* circular_buffer proper sizing

* removed obsolete logs

* do not execute worker jobs if not necessary

* better messages about compression disabling

* proper use of flags and updating rrdset last access time every time the obsoletion flag is flipped

* monitor stream sender used buffer ratio

* Update exporting unit tests

* no need to compare label value with strcmp

* streaming send workers now monitor bandwidth

* workers now use strings

* streaming receiver monitors incoming bandwidth

* parser shift of worker ids

* minor fixes

* Group chart label updates

* Populate context with dimensions that have data

* Fix chart id

* better shift of parser worker ids

* fix for streaming compression

* properly count received bytes

* ensure LZ4 compression ring buffer does not wrap prematurely

* do not stream empty charts; do not process empty instances in rrdcontext

* need_to_send_chart_definition() does not need an rrdset lock any more

* rrdcontext objects are collected, after data have been written to the db

* better logging of RRDCONTEXT transitions

* always set all variables needed by the worker utilization charts

* implemented double linked list for most objects; eliminated alarm indexes from rrdhost; and many more fixes

* lockless strings design - string_dup() and string_freez() are totally lockless when they dont need to touch Judy - only Judy is protected with a read/write lock

* STRING code re-organization for clarity

* thread_cache improvements; double numbers precision on worker threads

* STRING_ENTRY now shadown STRING, so no duplicate definition is required; string_length() renamed to string_strlen() to follow the paradigm of all other functions, STRING internal statistics are now only compiled with NETDATA_INTERNAL_CHECKS

* rrdhost index by hostname now cleans up; aclk queries of archieved hosts do not index hosts

* Add index to speed up database context searches

* Removed last_updated optimization (was also buggy after latest merge with master)

Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
Co-authored-by: Vladimir Kobal <vlad@prokk.net>

11 files changed, 159 insertions, 148 deletions

diff --git a/exporting/check_filters.c b/exporting/check_filters.c
index 726fd02a1a..b07ec07b17 100644
--- a/exporting/check_filters.c
+++ b/exporting/check_filters.c
@@ -25,7 +25,7 @@ int rrdhost_is_exportable(struct instance *instance, RRDHOST *host)
     RRDHOST_FLAGS *flags = &host->exporting_flags[instance->index];
 
     if (unlikely((*flags & (RRDHOST_FLAG_EXPORTING_SEND | RRDHOST_FLAG_EXPORTING_DONT_SEND)) == 0)) {
-        char *host_name = (host == localhost) ? "localhost" : host->hostname;
+        const char *host_name = (host == localhost) ? "localhost" : rrdhost_hostname(host);
 
         if (!instance->config.hosts_pattern || simple_pattern_matches(instance->config.hosts_pattern, host_name)) {
             *flags |= RRDHOST_FLAG_EXPORTING_SEND;
@@ -69,22 +69,22 @@ int rrdset_is_exportable(struct instance *instance, RRDSET *st)
 
     if(unlikely(!(*flags & RRDSET_FLAG_EXPORTING_SEND))) {
         // we have not checked this chart
-        if(simple_pattern_matches(instance->config.charts_pattern, st->id) || simple_pattern_matches(instance->config.charts_pattern, st->name))
+        if(simple_pattern_matches(instance->config.charts_pattern, rrdset_id(st)) || simple_pattern_matches(instance->config.charts_pattern, rrdset_name(st)))
             *flags |= RRDSET_FLAG_EXPORTING_SEND;
         else {
             *flags |= RRDSET_FLAG_EXPORTING_IGNORE;
-            debug(D_EXPORTING, "EXPORTING: not sending chart '%s' of host '%s', because it is disabled for exporting.", st->id, host->hostname);
+            debug(D_EXPORTING, "EXPORTING: not sending chart '%s' of host '%s', because it is disabled for exporting.", rrdset_id(st), rrdhost_hostname(host));
             return 0;
         }
     }
 
     if(unlikely(!rrdset_is_available_for_exporting_and_alarms(st))) {
-        debug(D_EXPORTING, "EXPORTING: not sending chart '%s' of host '%s', because it is not available for exporting.", st->id, host->hostname);
+        debug(D_EXPORTING, "EXPORTING: not sending chart '%s' of host '%s', because it is not available for exporting.", rrdset_id(st), rrdhost_hostname(host));
         return 0;
     }
 
     if(unlikely(st->rrd_memory_mode == RRD_MEMORY_MODE_NONE && !(EXPORTING_OPTIONS_DATA_SOURCE(instance->config.options) == EXPORTING_SOURCE_DATA_AS_COLLECTED))) {
-        debug(D_EXPORTING, "EXPORTING: not sending chart '%s' of host '%s' because its memory mode is '%s' and the exporting engine requires database access.", st->id, host->hostname, rrd_memory_mode_name(host->rrd_memory_mode));
+        debug(D_EXPORTING, "EXPORTING: not sending chart '%s' of host '%s' because its memory mode is '%s' and the exporting engine requires database access.", rrdset_id(st), rrdhost_hostname(host), rrd_memory_mode_name(host->rrd_memory_mode));
         return 0;
     }
 
diff --git a/exporting/graphite/graphite.c b/exporting/graphite/graphite.c
index 8ca094b3b7..6375ec2094 100644
--- a/exporting/graphite/graphite.c
+++ b/exporting/graphite/graphite.c
@@ -123,24 +123,24 @@ int format_dimension_collected_graphite_plaintext(struct instance *instance, RRD
     char chart_name[RRD_ID_LENGTH_MAX + 1];
     exporting_name_copy(
         chart_name,
-        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && st->name) ? st->name : st->id,
+        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && st->name) ? rrdset_name(st) : rrdset_id(st),
         RRD_ID_LENGTH_MAX);
 
     char dimension_name[RRD_ID_LENGTH_MAX + 1];
     exporting_name_copy(
         dimension_name,
-        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && rd->name) ? rd->name : rd->id,
+        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && rd->name) ? rrddim_name(rd) : rrddim_id(rd),
         RRD_ID_LENGTH_MAX);
 
     buffer_sprintf(
         instance->buffer,
         "%s.%s.%s.%s%s%s%s " COLLECTED_NUMBER_FORMAT " %llu\n",
         instance->config.prefix,
-        (host == localhost) ? instance->config.hostname : host->hostname,
+        (host == localhost) ? instance->config.hostname : rrdhost_hostname(host),
         chart_name,
         dimension_name,
         (host->tags) ? ";" : "",
-        (host->tags) ? host->tags : "",
+        (host->tags) ? rrdhost_tags(host) : "",
         (instance->labels_buffer) ? buffer_tostring(instance->labels_buffer) : "",
         rd->last_collected_value,
         (unsigned long long)rd->last_collected_time.tv_sec);
@@ -163,13 +163,13 @@ int format_dimension_stored_graphite_plaintext(struct instance *instance, RRDDIM
     char chart_name[RRD_ID_LENGTH_MAX + 1];
     exporting_name_copy(
         chart_name,
-        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && st->name) ? st->name : st->id,
+        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && st->name) ? rrdset_name(st) : rrdset_id(st),
         RRD_ID_LENGTH_MAX);
 
     char dimension_name[RRD_ID_LENGTH_MAX + 1];
     exporting_name_copy(
         dimension_name,
-        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && rd->name) ? rd->name : rd->id,
+        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && rd->name) ? rrddim_name(rd) : rrddim_id(rd),
         RRD_ID_LENGTH_MAX);
 
     time_t last_t;
@@ -182,11 +182,11 @@ int format_dimension_stored_graphite_plaintext(struct instance *instance, RRDDIM
         instance->buffer,
         "%s.%s.%s.%s%s%s%s " NETDATA_DOUBLE_FORMAT " %llu\n",
         instance->config.prefix,
-        (host == localhost) ? instance->config.hostname : host->hostname,
+        (host == localhost) ? instance->config.hostname : rrdhost_hostname(host),
         chart_name,
         dimension_name,
         (host->tags) ? ";" : "",
-        (host->tags) ? host->tags : "",
+        (host->tags) ? rrdhost_tags(host) : "",
         (instance->labels_buffer) ? buffer_tostring(instance->labels_buffer) : "",
         value,
         (unsigned long long)last_t);
diff --git a/exporting/json/json.c b/exporting/json/json.c
index 45a8c9d9fe..627b8208e7 100644
--- a/exporting/json/json.c
+++ b/exporting/json/json.c
@@ -145,7 +145,7 @@ int format_dimension_collected_json_plaintext(struct instance *instance, RRDDIM
     RRDSET *st = rd->rrdset;
     RRDHOST *host = st->rrdhost;
 
-    const char *tags_pre = "", *tags_post = "", *tags = host->tags;
+    const char *tags_pre = "", *tags_post = "", *tags = rrdhost_tags(host);
     if (!tags)
         tags = "";
 
@@ -187,21 +187,20 @@ int format_dimension_collected_json_plaintext(struct instance *instance, RRDDIM
         "\"timestamp\":%llu}",
 
         instance->config.prefix,
-        (host == localhost) ? instance->config.hostname : host->hostname,
+        (host == localhost) ? instance->config.hostname : rrdhost_hostname(host),
         tags_pre,
         tags,
         tags_post,
         instance->labels_buffer ? buffer_tostring(instance->labels_buffer) : "",
 
-        st->id,
-        st->name,
-        st->family,
-        st->context,
-        st->type,
-        st->units,
-
-        rd->id,
-        rd->name,
+        rrdset_id(st),
+        rrdset_name(st),
+        rrdset_family(st),
+        rrdset_context(st),
+        rrdset_type(st),
+        rrdset_units(st),
+        rrddim_id(rd),
+        rrddim_name(rd),
         rd->last_collected_value,
 
         (unsigned long long)rd->last_collected_time.tv_sec);
@@ -231,7 +230,7 @@ int format_dimension_stored_json_plaintext(struct instance *instance, RRDDIM *rd
     if(isnan(value))
         return 0;
 
-    const char *tags_pre = "", *tags_post = "", *tags = host->tags;
+    const char *tags_pre = "", *tags_post = "", *tags = rrdhost_tags(host);
     if (!tags)
         tags = "";
 
@@ -272,21 +271,20 @@ int format_dimension_stored_json_plaintext(struct instance *instance, RRDDIM *rd
         "\"timestamp\": %llu}",
 
         instance->config.prefix,
-        (host == localhost) ? instance->config.hostname : host->hostname,
+        (host == localhost) ? instance->config.hostname : rrdhost_hostname(host),
         tags_pre,
         tags,
         tags_post,
         instance->labels_buffer ? buffer_tostring(instance->labels_buffer) : "",
 
-        st->id,
-        st->name,
-        st->family,
-        st->context,
-        st->type,
-        st->units,
-
-        rd->id,
-        rd->name,
+        rrdset_id(st),
+        rrdset_name(st),
+        rrdset_family(st),
+        rrdset_context(st),
+        rrdset_type(st),
+        rrdset_units(st),
+        rrddim_id(rd),
+        rrddim_name(rd),
         value,
 
         (unsigned long long)last_t);
diff --git a/exporting/opentsdb/opentsdb.c b/exporting/opentsdb/opentsdb.c
index 282de2e6b9..ed56e3d253 100644
--- a/exporting/opentsdb/opentsdb.c
+++ b/exporting/opentsdb/opentsdb.c
@@ -177,13 +177,13 @@ int format_dimension_collected_opentsdb_telnet(struct instance *instance, RRDDIM
     char chart_name[RRD_ID_LENGTH_MAX + 1];
     exporting_name_copy(
         chart_name,
-        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && st->name) ? st->name : st->id,
+        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && st->name) ? rrdset_name(st) : rrdset_id(st),
         RRD_ID_LENGTH_MAX);
 
     char dimension_name[RRD_ID_LENGTH_MAX + 1];
     exporting_name_copy(
         dimension_name,
-        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && rd->name) ? rd->name : rd->id,
+        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && rd->name) ? rrddim_name(rd) : rrddim_id(rd),
         RRD_ID_LENGTH_MAX);
 
     buffer_sprintf(
@@ -194,9 +194,9 @@ int format_dimension_collected_opentsdb_telnet(struct instance *instance, RRDDIM
         dimension_name,
         (unsigned long long)rd->last_collected_time.tv_sec,
         rd->last_collected_value,
-        (host == localhost) ? instance->config.hostname : host->hostname,
+        (host == localhost) ? instance->config.hostname : rrdhost_hostname(host),
         (host->tags) ? " " : "",
-        (host->tags) ? host->tags : "",
+        (host->tags) ? rrdhost_tags(host) : "",
         (instance->labels_buffer) ? buffer_tostring(instance->labels_buffer) : "");
 
     return 0;
@@ -217,13 +217,13 @@ int format_dimension_stored_opentsdb_telnet(struct instance *instance, RRDDIM *r
     char chart_name[RRD_ID_LENGTH_MAX + 1];
     exporting_name_copy(
         chart_name,
-        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && st->name) ? st->name : st->id,
+        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && st->name) ? rrdset_name(st) : rrdset_id(st),
         RRD_ID_LENGTH_MAX);
 
     char dimension_name[RRD_ID_LENGTH_MAX + 1];
     exporting_name_copy(
         dimension_name,
-        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && rd->name) ? rd->name : rd->id,
+        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && rd->name) ? rrddim_name(rd) : rrddim_id(rd),
         RRD_ID_LENGTH_MAX);
 
     time_t last_t;
@@ -240,9 +240,9 @@ int format_dimension_stored_opentsdb_telnet(struct instance *instance, RRDDIM *r
         dimension_name,
         (unsigned long long)last_t,
         value,
-        (host == localhost) ? instance->config.hostname : host->hostname,
+        (host == localhost) ? instance->config.hostname : rrdhost_hostname(host),
         (host->tags) ? " " : "",
-        (host->tags) ? host->tags : "",
+        (host->tags) ? rrdhost_tags(host) : "",
         (instance->labels_buffer) ? buffer_tostring(instance->labels_buffer) : "");
 
     return 0;
@@ -309,13 +309,13 @@ int format_dimension_collected_opentsdb_http(struct instance *instance, RRDDIM *
     char chart_name[RRD_ID_LENGTH_MAX + 1];
     exporting_name_copy(
         chart_name,
-        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && st->name) ? st->name : st->id,
+        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && st->name) ? rrdset_name(st) : rrdset_id(st),
         RRD_ID_LENGTH_MAX);
 
     char dimension_name[RRD_ID_LENGTH_MAX + 1];
     exporting_name_copy(
         dimension_name,
-        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && rd->name) ? rd->name : rd->id,
+        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && rd->name) ? rrddim_name(rd) : rrddim_id(rd),
         RRD_ID_LENGTH_MAX);
 
     if (buffer_strlen((BUFFER *)instance->buffer) > 2)
@@ -336,9 +336,9 @@ int format_dimension_collected_opentsdb_http(struct instance *instance, RRDDIM *
         dimension_name,
         (unsigned long long)rd->last_collected_time.tv_sec,
         rd->last_collected_value,
-        (host == localhost) ? instance->config.hostname : host->hostname,
+        (host == localhost) ? instance->config.hostname : rrdhost_hostname(host),
         (host->tags) ? " " : "",
-        (host->tags) ? host->tags : "",
+        (host->tags) ? rrdhost_tags(host) : "",
         instance->labels_buffer ? buffer_tostring(instance->labels_buffer) : "");
 
     return 0;
@@ -359,13 +359,13 @@ int format_dimension_stored_opentsdb_http(struct instance *instance, RRDDIM *rd)
     char chart_name[RRD_ID_LENGTH_MAX + 1];
     exporting_name_copy(
         chart_name,
-        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && st->name) ? st->name : st->id,
+        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && st->name) ? rrdset_name(st) : rrdset_id(st),
         RRD_ID_LENGTH_MAX);
 
     char dimension_name[RRD_ID_LENGTH_MAX + 1];
     exporting_name_copy(
         dimension_name,
-        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && rd->name) ? rd->name : rd->id,
+        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && rd->name) ? rrddim_name(rd) : rrddim_id(rd),
         RRD_ID_LENGTH_MAX);
 
     time_t last_t;
@@ -392,9 +392,9 @@ int format_dimension_stored_opentsdb_http(struct instance *instance, RRDDIM *rd)
         dimension_name,
         (unsigned long long)last_t,
         value,
-        (host == localhost) ? instance->config.hostname : host->hostname,
+        (host == localhost) ? instance->config.hostname : rrdhost_hostname(host),
         (host->tags) ? " " : "",
-        (host->tags) ? host->tags : "",
+        (host->tags) ? rrdhost_tags(host) : "",
         instance->labels_buffer ? buffer_tostring(instance->labels_buffer) : "");
 
     return 0;
diff --git a/exporting/process_data.c b/exporting/process_data.c
index d5138b787b..e1e939c51d 100644
--- a/exporting/process_data.c
+++ b/exporting/process_data.c
@@ -110,9 +110,9 @@ NETDATA_DOUBLE exporting_calculate_value_from_stored_data(
         debug(
             D_EXPORTING,
             "EXPORTING: %s.%s.%s: aligned timeframe %lu to %lu is outside the chart's database range %lu to %lu",
-            host->hostname,
-            st->id,
-            rd->id,
+            rrdhost_hostname(host),
+            rrdset_id(st),
+            rrddim_id(rd),
             (unsigned long)after,
             (unsigned long)before,
             (unsigned long)first_t,
@@ -142,9 +142,9 @@ NETDATA_DOUBLE exporting_calculate_value_from_stored_data(
         debug(
             D_EXPORTING,
             "EXPORTING: %s.%s.%s: no values stored in database for range %lu to %lu",
-            host->hostname,
-            st->id,
-            rd->id,
+            rrdhost_hostname(host),
+            rrdset_id(st),
+            rrddim_id(rd),
             (unsigned long)after,
             (unsigned long)before);
         return NAN;
diff --git a/exporting/prometheus/prometheus.c b/exporting/prometheus/prometheus.c
index 7d632164f6..560022b72f 100644
--- a/exporting/prometheus/prometheus.c
+++ b/exporting/prometheus/prometheus.c
@@ -9,9 +9,9 @@
 
 static int is_matches_rrdset(struct instance *instance, RRDSET *st, SIMPLE_PATTERN *filter) {
     if (instance->config.options & EXPORTING_OPTION_SEND_NAMES) {
-        return simple_pattern_matches(filter, st->name);
+        return simple_pattern_matches(filter, rrdset_name(st));
     }
-    return simple_pattern_matches(filter, st->id);
+    return simple_pattern_matches(filter, rrdset_id(st));
 }
 
 /**
@@ -48,8 +48,8 @@ inline int can_send_rrdset(struct instance *instance, RRDSET *st, SIMPLE_PATTERN
             debug(
                 D_EXPORTING,
                 "EXPORTING: not sending chart '%s' of host '%s', because it is disabled for exporting.",
-                st->id,
-                host->hostname);
+                rrdset_id(st),
+                rrdhost_hostname(host));
             return 0;
         }
     }
@@ -58,8 +58,8 @@ inline int can_send_rrdset(struct instance *instance, RRDSET *st, SIMPLE_PATTERN
         debug(
             D_EXPORTING,
             "EXPORTING: not sending chart '%s' of host '%s', because it is not available for exporting.",
-            st->id,
-            host->hostname);
+            rrdset_id(st),
+            rrdhost_hostname(host));
         return 0;
     }
 
@@ -69,8 +69,8 @@ inline int can_send_rrdset(struct instance *instance, RRDSET *st, SIMPLE_PATTERN
         debug(
             D_EXPORTING,
             "EXPORTING: not sending chart '%s' of host '%s' because its memory mode is '%s' and the exporting connector requires database access.",
-            st->id,
-            host->hostname,
+            rrdset_id(st),
+            rrdhost_hostname(host),
             rrd_memory_mode_name(host->rrd_memory_mode));
         return 0;
     }
@@ -349,8 +349,9 @@ struct host_variables_callback_options {
  * @param data callback options.
  * @return Returns 1 if the chart can be sent, 0 otherwise.
  */
-static int print_host_variables(RRDVAR *rv, void *data)
-{
+static int print_host_variables(const char *name __maybe_unused, void *rv_ptr, void *data) {
+    RRDVAR *rv = rv_ptr;
+
     struct host_variables_callback_options *opts = data;
 
     if (rv->options & (RRDVAR_OPTION_CUSTOM_HOST_VAR | RRDVAR_OPTION_CUSTOM_CHART_VAR)) {
@@ -366,7 +367,7 @@ static int print_host_variables(RRDVAR *rv, void *data)
         if (isnan(value) || isinf(value)) {
             if (opts->output_options & PROMETHEUS_OUTPUT_HELP)
                 buffer_sprintf(
-                    opts->wb, "# COMMENT variable \"%s\" is %s. Skipped.\n", rv->name, (isnan(value)) ? "NAN" : "INF");
+                    opts->wb, "# COMMENT variable \"%s\" is %s. Skipped.\n", rrdvar_name(rv), (isnan(value)) ? "NAN" : "INF");
 
             return 0;
         }
@@ -378,7 +379,7 @@ static int print_host_variables(RRDVAR *rv, void *data)
             label_post = "}";
         }
 
-        prometheus_name_copy(opts->name, rv->name, sizeof(opts->name));
+        prometheus_name_copy(opts->name, rrdvar_name(rv), sizeof(opts->name));
 
         if (opts->output_options & PROMETHEUS_OUTPUT_TIMESTAMPS)
             buffer_sprintf(
@@ -445,17 +446,17 @@ static void generate_as_collected_prom_help(BUFFER *wb, struct gen_parameters *p
         wb,
         "%s: chart \"%s\", context \"%s\", family \"%s\", dimension \"%s\", value * ",
         p->suffix,
-        (p->output_options & PROMETHEUS_OUTPUT_NAMES && p->st->name) ? p->st->name : p->st->id,
-        p->st->context,
-        p->st->family,
-        (p->output_options & PROMETHEUS_OUTPUT_NAMES && p->rd->name) ? p->rd->name : p->rd->id);
+        (p->output_options & PROMETHEUS_OUTPUT_NAMES && p->st->name) ? rrdset_name(p->st) : rrdset_id(p->st),
+        rrdset_context(p->st),
+        rrdset_family(p->st),
+        (p->output_options & PROMETHEUS_OUTPUT_NAMES && p->rd->name) ? rrddim_name(p->rd) : rrddim_id(p->rd));
 
     if (prometheus_collector)
         buffer_sprintf(wb, "1 / 1");
     else
         buffer_sprintf(wb, COLLECTED_NUMBER_FORMAT " / " COLLECTED_NUMBER_FORMAT, p->rd->multiplier, p->rd->divisor);
 
-    buffer_sprintf(wb, " %s %s (%s)\n", p->relation, p->st->units, p->type);
+    buffer_sprintf(wb, " %s %s (%s)\n", p->relation, rrdset_units(p->st), p->type);
 }
 
 /**
@@ -521,7 +522,7 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(
     rrdhost_rdlock(host);
 
     char hostname[PROMETHEUS_ELEMENT_MAX + 1];
-    prometheus_label_copy(hostname, host->hostname, PROMETHEUS_ELEMENT_MAX);
+    prometheus_label_copy(hostname, rrdhost_hostname(host), PROMETHEUS_ELEMENT_MAX);
 
     format_host_labels_prometheus(instance, host);
 
@@ -529,8 +530,8 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(
         wb,
         "netdata_info{instance=\"%s\",application=\"%s\",version=\"%s\"",
         hostname,
-        host->program_name,
-        host->program_version);
+        rrdhost_program_name(host),
+        rrdhost_program_version(host));
 
     if (instance->labels_buffer && *buffer_tostring(instance->labels_buffer)) {
         buffer_sprintf(wb, ",%s", buffer_tostring(instance->labels_buffer));
@@ -551,15 +552,19 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(
 
     // send custom variables set for the host
     if (output_options & PROMETHEUS_OUTPUT_VARIABLES) {
-        struct host_variables_callback_options opts = { .host = host,
-                                                        .wb = wb,
-                                                        .labels = (labels[0] == ',') ? &labels[1] : labels,
-                                                        .exporting_options = exporting_options,
-                                                        .output_options = output_options,
-                                                        .prefix = prefix,
-                                                        .now = now_realtime_sec(),
-                                                        .host_header_printed = 0 };
-        foreach_host_variable_callback(host, print_host_variables, &opts);
+
+        struct host_variables_callback_options opts = {
+            .host = host,
+            .wb = wb,
+            .labels = (labels[0] == ',') ? &labels[1] : labels,
+            .exporting_options = exporting_options,
+            .output_options = output_options,
+            .prefix = prefix,
+            .now = now_realtime_sec(),
+            .host_header_printed = 0
+        };
+
+        rrdvar_walkthrough_read(host->rrdvar_root_index, print_host_variables, &opts);
     }
 
     // for each chart
@@ -575,10 +580,9 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(
             char family[PROMETHEUS_ELEMENT_MAX + 1];
             char units[PROMETHEUS_ELEMENT_MAX + 1] = "";
 
-            prometheus_label_copy(
-                chart, (output_options & PROMETHEUS_OUTPUT_NAMES && st->name) ? st->name : st->id, PROMETHEUS_ELEMENT_MAX);
-            prometheus_label_copy(family, st->family, PROMETHEUS_ELEMENT_MAX);
-            prometheus_name_copy(context, st->context, PROMETHEUS_ELEMENT_MAX);
+            prometheus_label_copy(chart, (output_options & PROMETHEUS_OUTPUT_NAMES && st->name) ? rrdset_name(st) : rrdset_id(st), PROMETHEUS_ELEMENT_MAX);
+            prometheus_label_copy(family, rrdset_family(st), PROMETHEUS_ELEMENT_MAX);
+            prometheus_name_copy(context, rrdset_context(st), PROMETHEUS_ELEMENT_MAX);
 
             int as_collected = (EXPORTING_OPTIONS_DATA_SOURCE(exporting_options) == EXPORTING_SOURCE_DATA_AS_COLLECTED);
             int homogeneous = 1;
@@ -590,13 +594,13 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(
                 if (rrdset_flag_check(st, RRDSET_FLAG_HETEROGENEOUS))
                     homogeneous = 0;
 
-                if (st->module_name && !strcmp(st->module_name, "prometheus"))
+                if (!strcmp(rrdset_module_name(st), "prometheus"))
                     prometheus_collector = 1;
             } else {
                 if (EXPORTING_OPTIONS_DATA_SOURCE(exporting_options) == EXPORTING_SOURCE_DATA_AVERAGE &&
                     !(output_options & PROMETHEUS_OUTPUT_HIDEUNITS))
                     prometheus_units_copy(
-                        units, st->units, PROMETHEUS_ELEMENT_MAX, output_options & PROMETHEUS_OUTPUT_OLDUNITS);
+                        units, rrdset_units(st), PROMETHEUS_ELEMENT_MAX, output_options & PROMETHEUS_OUTPUT_OLDUNITS);
             }
 
             if (unlikely(output_options & PROMETHEUS_OUTPUT_HELP))
@@ -604,10 +608,10 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(
                     wb,
                     "\n# COMMENT %s chart \"%s\", context \"%s\", family \"%s\", units \"%s\"\n",
                     (homogeneous) ? "homogeneous" : "heterogeneous",
-                    (output_options & PROMETHEUS_OUTPUT_NAMES && st->name) ? st->name : st->id,
-                    st->context,
-                    st->family,
-                    st->units);
+                    (output_options & PROMETHEUS_OUTPUT_NAMES && st->name) ? rrdset_name(st) : rrdset_id(st),
+                    rrdset_context(st),
+                    rrdset_family(st),
+                    rrdset_units(st));
 
             // for each dimension
             RRDDIM *rd;
@@ -651,7 +655,7 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(
 
                             prometheus_label_copy(
                                 dimension,
-                                (output_options & PROMETHEUS_OUTPUT_NAMES && rd->name) ? rd->name : rd->id,
+                                (output_options & PROMETHEUS_OUTPUT_NAMES && rd->name) ? rrddim_name(rd) : rrddim_id(rd),
                                 PROMETHEUS_ELEMENT_MAX);
 
                             if (unlikely(output_options & PROMETHEUS_OUTPUT_HELP))
@@ -667,7 +671,7 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(
 
                             prometheus_name_copy(
                                 dimension,
-                                (output_options & PROMETHEUS_OUTPUT_NAMES && rd->name) ? rd->name : rd->id,
+                                (output_options & PROMETHEUS_OUTPUT_NAMES && rd->name) ? rrddim_name(rd) : rrddim_id(rd),
                                 PROMETHEUS_ELEMENT_MAX);
 
                             if (unlikely(output_options & PROMETHEUS_OUTPUT_HELP))
@@ -694,7 +698,7 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(
 
                             prometheus_label_copy(
                                 dimension,
-                                (output_options & PROMETHEUS_OUTPUT_NAMES && rd->name) ? rd->name : rd->id,
+                                (output_options & PROMETHEUS_OUTPUT_NAMES && rd->name) ? rrddim_name(rd) : rrddim_id(rd),
                                 PROMETHEUS_ELEMENT_MAX);
 
                             if (unlikely(output_options & PROMETHEUS_OUTPUT_HELP))
@@ -705,8 +709,8 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(
                                     context,
                                     units,
                                     suffix,
-                                    (output_options & PROMETHEUS_OUTPUT_NAMES && rd->name) ? rd->name : rd->id,
-                                    st->units,
+                                    (output_options & PROMETHEUS_OUTPUT_NAMES && rd->name) ? rrddim_name(rd) : rrddim_id(rd),
+                                    rrdset_units(st),
                                     (unsigned long long)first_time,
                                     (unsigned long long)last_time);
 
@@ -809,7 +813,7 @@ static inline time_t prometheus_preparation(
         buffer_sprintf(
             wb,
             "# COMMENT netdata \"%s\" to %sprometheus \"%s\", source \"%s\", last seen %lu %s, time range %lu to %lu\n\n",
-            host->hostname,
+            rrdhost_hostname(host),
             (first_seen) ? "FIRST SEEN " : "",
             server,
             mode,
diff --git a/exporting/prometheus/remote_write/remote_write.c b/exporting/prometheus/remote_write/remote_write.c
index 03feb2c08b..29215283e8 100644
--- a/exporting/prometheus/remote_write/remote_write.c
+++ b/exporting/prometheus/remote_write/remote_write.c
@@ -171,12 +171,12 @@ int format_host_prometheus_remote_write(struct instance *instance, RRDHOST *host
     char hostname[PROMETHEUS_ELEMENT_MAX + 1];
     prometheus_label_copy(
         hostname,
-        (host == localhost) ? instance->config.hostname : host->hostname,
+        (host == localhost) ? instance->config.hostname : rrdhost_hostname(host),
         PROMETHEUS_ELEMENT_MAX);
 
     add_host_info(
         connector_specific_data->write_request,
-        "netdata_info", hostname, host->program_name, host->program_version, now_realtime_usec() / USEC_PER_MS);
+        "netdata_info", hostname, rrdhost_program_name(host), rrdhost_program_version(host), now_realtime_usec() / USEC_PER_MS);
     
     if (unlikely(sending_labels_configured(instance))) {
         struct format_remote_write_label_callback tmp = {
@@ -200,10 +200,10 @@ int format_chart_prometheus_remote_write(struct instance *instance, RRDSET *st)
 {
     prometheus_label_copy(
         chart,
-        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && st->name) ? st->name : st->id,
+        (instance->config.options & EXPORTING_OPTION_SEND_NAMES && st->name) ? rrdset_name(st) : rrdset_id(st),
         PROMETHEUS_ELEMENT_MAX);
-    prometheus_label_copy(family, st->family, PROMETHEUS_ELEMENT_MAX);
-    prometheus_name_copy(context, st->context, PROMETHEUS_ELEMENT_MAX);
+    prometheus_label_copy(family, rrdset_family(st), PROMETHEUS_ELEMENT_MAX);
+    prometheus_name_copy(context, rrdset_context(st), PROMETHEUS_ELEMENT_MAX);
 
     as_collected = (EXPORTING_OPTIONS_DATA_SOURCE(instance->config.options) == EXPORTING_SOURCE_DATA_AS_COLLECTED);
     homogeneous = 1;
@@ -215,7 +215,7 @@ int format_chart_prometheus_remote_write(struct instance *instance, RRDSET *st)
             homogeneous = 0;
     } else {
         if (EXPORTING_OPTIONS_DATA_SOURCE(instance->config.options) == EXPORTING_SOURCE_DATA_AVERAGE)
-            prometheus_units_copy(units, st->units, PROMETHEUS_ELEMENT_MAX, 0);
+            prometheus_units_copy(units, rrdset_units(st), PROMETHEUS_ELEMENT_MAX, 0);
     }
 
     return 0;
@@ -249,8 +249,8 @@ int format_dimension_prometheus_remote_write(struct instance *instance, RRDDIM *
                     D_EXPORTING,
                     "EXPORTING: not sending dimension '%s' of chart '%s' from host '%s', "
                     "its last data collection (%lu) is not within our timeframe (%lu to %lu)",
-                    rd->id, rd->rrdset->id,
-                    (host == localhost) ? instance->config.hostname : host->hostname,
+                    rrddim_id(rd), rrdset_id(rd->rrdset),
+                    (host == localhost) ? instance->config.hostname : rrdhost_hostname(host),
                     (unsigned long)rd->last_collected_time.tv_sec,
                     (unsigned long)instance->after,
                     (unsigned long)instance->before);
@@ -263,14 +263,14 @@ int format_dimension_prometheus_remote_write(struct instance *instance, RRDDIM *
 
                 prometheus_label_copy(
                     dimension,
-                    (instance->config.options &