modularized all source code (#4391)

* modularized all external plugins

* added README.md in plugins

* fixed title

* fixed typo

* relative link to external plugins

* external plugins configuration README

* added plugins link

* remove plugins link

* plugin names are links

* added links to external plugins

* removed unecessary spacing

* list to table

* added language

* fixed typo

* list to table on internal plugins

* added more documentation to internal plugins

* moved python, node, and bash code and configs into the external plugins

* added statsd README

* fix bug with corrupting config.h every 2nd compilation

* moved all config files together with their code

* more documentation

* diskspace info

* fixed broken links in apps.plugin

* added backends docs

* updated plugins readme

* move nc-backend.sh to backends

* created daemon directory

* moved all code outside src/

* fixed readme identation

* renamed plugins.d.plugin to plugins.d

* updated readme

* removed linux- from linux plugins

* updated readme

* updated readme

* updated readme

* updated readme

* updated readme

* updated readme

* fixed README.md links

* fixed netdata tree links

* updated codacy, codeclimate and lgtm excluded paths

* update CMakeLists.txt

* updated automake options at top directory

* libnetdata slit into directories

* updated READMEs

* updated READMEs

* updated ARL docs

* updated ARL docs

* moved /plugins to /collectors

* moved all external plugins outside plugins.d

* updated codacy, codeclimate, lgtm

* updated README

* updated url

* updated readme

* updated readme

* updated readme

* updated readme

* moved api and web into webserver

* web/api web/gui web/server

* modularized webserver

* removed web/gui/version.txt

4 files changed, 1127 insertions, 0 deletions

diff --git a/collectors/plugins.d/Makefile.am b/collectors/plugins.d/Makefile.am
new file mode 100644
index 0000000000..59250a997e
--- /dev/null
+++ b/collectors/plugins.d/Makefile.am
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: GPL-3.0-or-later
+
+AUTOMAKE_OPTIONS = subdir-objects
+MAINTAINERCLEANFILES = $(srcdir)/Makefile.in
+
+SUBDIRS = \
+    $(NULL)
+
+dist_noinst_DATA = \
+    README.md \
+    $(NULL)
diff --git a/collectors/plugins.d/README.md b/collectors/plugins.d/README.md
new file mode 100644
index 0000000000..a3ed8c5d27
--- /dev/null
+++ b/collectors/plugins.d/README.md
@@ -0,0 +1,347 @@
+# Netdata External Plugins
+
+`plugins.d` is the netdata internal plugin that collects metrics
+from external processes, thus allowing netdata to use **external plugins**.
+
+## Provided External Plugins
+
+plugin|language|O/S|description
+:---:|:---:|:---:|:---
+[apps.plugin](../apps.plugin/)|`C`|linux, freebsd|monitors the whole process tree on Linux and FreeBSD and breaks down system resource usage by **process**, **user** and **user group**.
+[charts.d.plugin](../charts.d.plugin/)|`BASH`|all|a **plugin orchestrator** for data collection modules written in `BASH` v4+.
+[fping.plugin](../fping.plugin/)|`C`|all|measures network latency, jitter and packet loss between the monitored node and any number of remote network end points.
+[freeipmi.plugin](../freeipmi.plugin/)|`C`|linux|collects metrics from enterprise hardware sensors, on Linux servers.
+[node.d.plugin](../node.d.plugin/)|`node.js`|all|a **plugin orchestrator** for data collection modules written in `node.js`.
+[python.d.plugin](../python.d.plugin/)|`python`|all|a **plugin orchestrator** for data collection modules written in `python` v2 or v3 (both are supported).
+
+
+## Motivation
+
+This plugin allows netdata to use **external plugins** for data collection:
+
+1. external data collection plugins may be written in any computer language.
+2. external data collection plugins may use O/S capabilities or `setuid` to
+   run with escalated privileges (compared to the netdata daemon).
+   The communication between the external plugin and netdata is unidirectional
+   (from the plugin to netdata), so that netdata cannot manipulate an external
+   plugin running with escalated privileges.
+
+## Operation
+
+Each of the external plugins is expected to run forever.
+Netdata will start it when it starts and stop it when it exits.
+
+If the external plugin exits or crashes, netdata will log an error.
+If the external plugin exits or crashes without pushing metrics to netdata,
+netdata will not start it again.
+
+The `stdout` of external plugins is connected to netdata to receive metrics,
+with the API defined below.
+
+The `stderr` of external plugins is connected to netdata `error.log`.
+
+## Configuration
+
+This plugin is configured via `netdata.conf`, section `[plugins]`.
+At this section there a list of all the plugins found at the system it runs
+with a boolean setting to enable them or not. 
+
+Example:
+
+```
+[plugins]
+	# enable running new plugins = yes
+	# check for new plugins every = 60
+	
+	# charts.d = yes
+	# fping = yes
+	# node.d = yes
+	# python.d = yes
+```
+
+The setting `enable running new plugins` changes the default behavior for all external plugins.
+So if set to `no`, only the plugins that are explicitly set to `yes` will be run.
+
+The setting `check for new plugins every` controls the time the directory `/usr/libexec/netdata/plugins.d`
+will be rescanned for new plugins. So, new plugins can give added anytime. 
+
+For each of the external plugins enabled, another `netdata.conf` section
+is created, in the form of `[plugin:NAME]`, where `NAME` is the name of the external plugin.
+This section allows controlling the update frequency of the plugin and provide
+additional command line arguments to it.
+
+For example, for `apps.plugin` the following section is available:
+
+```
+[plugin:apps]
+	# update every = 1
+	# command options = 
+```
+
+- `update every` controls the granularity of the external plugin.
+- `command options` allows giving additional command line options to the plugin.
+
+
+## External Plugins API
+
+Any program that can print a few values to its standard output can become a netdata external plugin.
+
+There are 7 lines netdata parses. lines starting with:
+
+- `CHART` - create or update a chart
+- `DIMENSION` - add or update a dimension to the chart just created
+- `BEGIN` - initialize data collection for a chart
+- `SET` - set the value of a dimension for the initialized chart
+- `END` - complete data collection for the initialized chart
+- `FLUSH` - ignore the last collected values
+- `DISABLE` - disable this plugin
+
+a single program can produce any number of charts with any number of dimensions each.
+
+Charts can be added any time (not just the beginning).
+
+### command line parameters
+
+The plugin **MUST** accept just **one** parameter: **the number of seconds it is
+expected to update the values for its charts**. The value passed by netdata
+to the plugin is controlled via its configuration file (so there is no need
+for the plugin to handle this configuration option).
+
+The external plugin can overwrite the update frequency. For example, the server may
+request per second updates, but the plugin may ignore it and update its charts
+every 5 seconds.
+
+### environment variables
+
+There are a few environment variables that are set by `netdata` and are
+available for the plugin to use.
+
+variable|description
+:------:|:----------
+`NETDATA_USER_CONFIG_DIR`|The directory where all netdata related user configuration should be stored. If the plugin requires custom user configuration, this is the place the user has saved it (normally under `/etc/netdata`).
+`NETDATA_STOCK_CONFIG_DIR`|The directory where all netdata related stock configuration should be stored. If the plugin is shipped with configuration files, this is the place they can be found (normally under `/usr/lib/netdata/conf.d`).
+`NETDATA_PLUGINS_DIR`|The directory where all netdata plugins are stored.
+`NETDATA_WEB_DIR`|The directory where the web files of netdata are saved.
+`NETDATA_CACHE_DIR`|The directory where the cache files of netdata are stored. Use this directory if the plugin requires a place to store data. A new directory should be created for the plugin for this purpose, inside this directory.
+`NETDATA_LOG_DIR`|The directory where the log files are stored. By default the `stderr` output of the plugin will be saved in the `error.log` file of netdata.
+`NETDATA_HOST_PREFIX`|This is used in environments where system directories like `/sys` and `/proc` have to be accessed at a different path.
+`NETDATA_DEBUG_FLAGS`|This is a number (probably in hex starting with `0x`), that enables certain netdata debugging features. Check **[[Tracing Options]]** for more information.
+`NETDATA_UPDATE_EVERY`|The minimum number of seconds between chart refreshes. This is like the **internal clock** of netdata (it is user configurable, defaulting to `1`). There is no meaning for a plugin to update its values more frequently than this number of seconds.
+
+
+### the output of the plugin
+
+The plugin should output instructions for netdata to its output (`stdout`). Since this uses pipes, please make sure you flush stdout after every iteration.
+
+#### DISABLE
+
+`DISABLE` will disable this plugin. This will prevent netdata from restarting the plugin. You can also exit with the value `1` to have the same effect.
+
+#### CHART
+
+`CHART` defines a new chart.
+
+the template is:
+
+> CHART type.id name title units [family [context [charttype [priority [update_every [options [plugin [module]]]]]]]]
+
+ where:
+  - `type.id`
+
+    uniquely identifies the chart,
+    this is what will be needed to add values to the chart
+
+    the `type` part controls the menu the charts will appear in
+
+  - `name`
+
+    is the name that will be presented to the user instead of `id` in `type.id`. This means that only the `id` part of `type.id` is changed. When a name has been given, the chart is index (and can be referred) as both `type.id` and `type.name`. You can set name to `''`, or `null`, or `(null)` to disable it.
+
+  - `title`
+
+    the text above the chart
+
+  - `units`
+
+    the label of the vertical axis of the chart,
+    all dimensions added to a chart should have the same units
+    of measurement
+
+  - `family`
+
+    is used to group charts together
+    (for example all eth0 charts should say: eth0),
+    if empty or missing, the `id` part of `type.id` will be used
+    
+    this controls the sub-menu on the dashboard
+
+  - `context`
+
+    the context is giving the template of the chart. For example, if multiple charts present the same information for a different family, they should have the same `context`
+
+    this is used for looking up rendering information for the chart (colors, sizes, informational texts) and also apply alarms to it
+
+  - `charttype`
+
+    one of `line`, `area` or `stacked`,
+    if empty or missing, the `line` will be used
+
+  - `priority`
+
+    is the relative priority of the charts as rendered on the web page,
+    lower numbers make the charts appear before the ones with higher numbers,
+    if empty or missing, `1000` will be used
+
+  - `update_every`
+
+    overwrite the update frequency set by the server,
+    if empty or missing, the user configured value will be used
+
+  - `options`
+
+    a space separated list of options, enclosed in quotes. 4 options are currently supported: `obsolete` to mark a chart as obsolete (netdata will hide it and delete it after some time), `detail` to mark a chart as insignificant (this may be used by dashboards to make the charts smaller, or somehow visualize properly a less important chart), `store_first` to make netdata store the first collected value, assuming there was an invisible previous value set to zero (this is used by statsd charts - if the first data collected value of incremental dimensions is not zero based, unrealistic spikes will appear with this option set) and `hidden` to perform all operations on a chart, but do not offer it on dashboards (the chart will be send to backends). `CHART` options have been added in netdata v1.7 and the `hidden` option was added in 1.10.
+
+  - `plugin` and `module`
+
+    both are just names that are used to let the user the plugin and its module that generated the chart. If `plugin` is unset or empty, netdata will automatically set the filename of the plugin that generated the chart. `module` has not default.
+
+
+#### DIMENSION
+
+`DIMENSION` defines a new dimension for the chart
+
+the template is:
+
+> DIMENSION id [name [algorithm [multiplier [divisor [hidden]]]]]
+
+ where:
+
+  - `id`
+
+    the `id` of this dimension (it is a text value, not numeric),
+    this will be needed later to add values to the dimension
+
+    We suggest to avoid using `.` in dimension ids. Backends expect metrics to be `.` separated and people will get confused if a dimension id contains a dot.
+
+  - `name`
+
+    the name of the dimension as it will appear at the legend of the chart,
+    if empty or missing the `id` will be used
+
+  - `algorithm`
+
+    one of:
+
+    * `absolute`
+
+      the value is to drawn as-is (interpolated to second boundary),
+      if `algorithm` is empty, invalid or missing, `absolute` is used
+
+    * `incremental`
+
+      the value increases over time,
+      the difference from the last value is presented in the chart,
+      the server interpolates the value and calculates a per second figure
+
+    * `percentage-of-absolute-row`
+
+      the % of this value compared to the total of all dimensions
+
+    * `percentage-of-incremental-row`
+
+      the % of this value compared to the incremental total of
+      all dimensions
+
+  - `multiplier`
+
+    an integer value to multiply the collected value,
+    if empty or missing, `1` is used
+
+  - `divisor`
+
+    an integer value to divide the collected value,
+    if empty or missing, `1` is used
+
+  - `hidden`
+
+    giving the keyword `hidden` will make this dimension hidden,
+    it will take part in the calculations but will not be presented in the chart
+
+
+#### VARIABLE
+
+> VARIABLE [SCOPE] name = value
+
+`VARIABLE` defines a variable that can be used in alarms. This is to used for setting constants (like the max connections a server may accept).
+
+Variables support 2 scopes:
+
+- `GLOBAL` or `HOST` to define the variable at the host level.
+- `LOCAL` or `CHART` to define the variable at the chart level. Use chart-local variables when the same variable may exist for different charts (i.e. netdata monitors 2 mysql servers, and you need to set the `max_connections` each server accepts). Using chart-local variables is the ideal to build alarm templates.
+
+The position of the `VARIABLE` line, sets its default scope (in case you do not specify a scope). So, defining a `VARIABLE` before any `CHART`, or between `END` and `BEGIN` (outside any chart), sets `GLOBAL` scope, while defining a `VARIABLE` just after a `CHART` or a `DIMENSION`, or within the `BEGIN` - `END` block of a chart, sets `LOCAL` scope.
+
+These variables can be set and updated at any point.
+
+Variable names should use alphanumeric characters, the `.` and the `_`.
+
+The `value` is floating point (netdata used `long double`).
+
+Variables are transferred to upstream netdata servers (streaming and database replication).
+
+## data collection
+
+data collection is defined as a series of `BEGIN` -> `SET` -> `END` lines
+
+> BEGIN type.id [microseconds]
+
+  - `type.id`
+
+    is the unique identification of the chart (as given in `CHART`)
+
+  - `microseconds`
+
+    is the number of microseconds since the last update of the chart. It is optional.
+
+    Under heavy system load, the system may have some latency transferring
+    data from the plugins to netdata via the pipe. This number improves
+    accuracy significantly, since the plugin is able to calculate the
+    duration between its iterations better than netdata.
+
+    The first time the plugin is started, no microseconds should be given
+    to netdata.
+
+> SET id = value
+
+   - `id`
+
+     is the unique identification of the dimension (of the chart just began)
+
+   - `value`
+
+     is the collected value, only integer values are collected. If you want to push fractional values, multiply this value by 100 or 1000 and set the `DIMENSION` divider to 1000.
+
+> END
+
+  END does not take any parameters, it commits the collected values for all dimensions to the chart. If a dimensions was not `SET`, its value will be empty for this commit.
+
+More `SET` lines may appear to update all the dimensions of the chart.
+All of them in one `BEGIN` -> `END` block.
+
+All `SET` lines within a single `BEGIN` -> `END` block have to refer to the
+same chart.
+
+If more charts need to be updated, each chart should have its own
+`BEGIN` -> `SET` -> `END` block.
+
+If, for any reason, a plugin has issued a `BEGIN` but wants to cancel it,
+it can issue a `FLUSH`. The `FLUSH` command will instruct netdata to ignore
+all the values collected since the last `BEGIN` command.
+
+If a plugin does not behave properly (outputs invalid lines, or does not
+follow these guidelines), will be disabled by netdata.
+
+### collected values
+
+netdata will collect any **signed** value in the 64bit range:
+`-9.223.372.036.854.775.808` to `+9.223.372.036.854.775.807`
diff --git a/collectors/plugins.d/plugins_d.c b/collectors/plugins.d/plugins_d.c
new file mode 100644
index 0000000000..465ecd7963
--- /dev/null
+++ b/collectors/plugins.d/plugins_d.c
@@ -0,0 +1,696 @@
+// SPDX-License-Identifier: GPL-3.0-or-later
+
+#include "plugins_d.h"
+
+char *plugin_directories[PLUGINSD_MAX_DIRECTORIES] = { NULL };
+char *netdata_configured_plugins_dir_base;
+
+struct plugind *pluginsd_root = NULL;
+
+static inline int pluginsd_space(char c) {
+    switch(c) {
+    case ' ':
+    case '\t':
+    case '\r':
+    case '\n':
+    case '=':
+        return 1;
+
+    default:
+        return 0;
+    }
+}
+
+inline int config_isspace(char c) {
+    switch(c) {
+        case ' ':
+        case '\t':
+        case '\r':
+        case '\n':
+        case ',':
+            return 1;
+
+        default:
+            return 0;
+    }
+}
+
+// split a text into words, respecting quotes
+inline int quoted_strings_splitter(char *str, char **words, int max_words, int (*custom_isspace)(char)) {
+    char *s = str, quote = 0;
+    int i = 0, j;
+
+    // skip all white space
+    while(unlikely(custom_isspace(*s))) s++;
+
+    // check for quote
+    if(unlikely(*s == '\'' || *s == '"')) {
+        quote = *s; // remember the quote
+        s++;        // skip the quote
+    }
+
+    // store the first word
+    words[i++] = s;
+
+    // while we have something
+    while(likely(*s)) {
+        // if it is escape
+        if(unlikely(*s == '\\' && s[1])) {
+            s += 2;
+            continue;
+        }
+
+        // if it is quote
+        else if(unlikely(*s == quote)) {
+            quote = 0;
+            *s = ' ';
+            continue;
+        }
+
+        // if it is a space
+        else if(unlikely(quote == 0 && custom_isspace(*s))) {
+
+            // terminate the word
+            *s++ = '\0';
+
+            // skip all white space
+            while(likely(custom_isspace(*s))) s++;
+
+            // check for quote
+            if(unlikely(*s == '\'' || *s == '"')) {
+                quote = *s; // remember the quote
+                s++;        // skip the quote
+            }
+
+            // if we reached the end, stop
+            if(unlikely(!*s)) break;
+
+            // store the next word
+            if(likely(i < max_words)) words[i++] = s;
+            else break;
+        }
+
+        // anything else
+        else s++;
+    }
+
+    // terminate the words
+    j = i;
+    while(likely(j < max_words)) words[j++] = NULL;
+
+    return i;
+}
+
+inline int pluginsd_split_words(char *str, char **words, int max_words) {
+    return quoted_strings_splitter(str, words, max_words, pluginsd_space);
+}
+
+inline size_t pluginsd_process(RRDHOST *host, struct plugind *cd, FILE *fp, int trust_durations) {
+    int enabled = cd->enabled;
+
+    if(!fp || !enabled) {
+        cd->enabled = 0;
+        return 0;
+    }
+
+    size_t count = 0;
+
+    char line[PLUGINSD_LINE_MAX + 1];
+
+    char *words[PLUGINSD_MAX_WORDS] = { NULL };
+    uint32_t BEGIN_HASH = simple_hash(PLUGINSD_KEYWORD_BEGIN);
+    uint32_t END_HASH = simple_hash(PLUGINSD_KEYWORD_END);
+    uint32_t FLUSH_HASH = simple_hash(PLUGINSD_KEYWORD_FLUSH);
+    uint32_t CHART_HASH = simple_hash(PLUGINSD_KEYWORD_CHART);
+    uint32_t DIMENSION_HASH = simple_hash(PLUGINSD_KEYWORD_DIMENSION);
+    uint32_t DISABLE_HASH = simple_hash(PLUGINSD_KEYWORD_DISABLE);
+    uint32_t VARIABLE_HASH = simple_hash(PLUGINSD_KEYWORD_VARIABLE);
+
+    RRDSET *st = NULL;
+    uint32_t hash;
+
+    errno = 0;
+    clearerr(fp);
+
+    if(unlikely(fileno(fp) == -1)) {
+        error("file descriptor given is not a valid stream");
+        goto cleanup;
+    }
+
+    while(!ferror(fp)) {
+        if(unlikely(netdata_exit)) break;
+
+        char *r = fgets(line, PLUGINSD_LINE_MAX, fp);
+        if(unlikely(!r)) {
+            error("read failed");
+            break;
+        }
+
+        if(unlikely(netdata_exit)) break;
+
+        line[PLUGINSD_LINE_MAX] = '\0';
+
+        int w = pluginsd_split_words(line, words, PLUGINSD_MAX_WORDS);
+        char *s = words[0];
+        if(unlikely(!s || !*s || !w)) {
+            continue;
+        }
+
+        // debug(D_PLUGINSD, "PLUGINSD: words 0='%s' 1='%s' 2='%s' 3='%s' 4='%s' 5='%s' 6='%s' 7='%s' 8='%s' 9='%s'", words[0], words[1], words[2], words[3], words[4], words[5], words[6], words[7], words[8], words[9]);
+
+        if(likely(!simple_hash_strcmp(s, "SET", &hash))) {
+            char *dimension = words[1];
+            char *value = words[2];
+
+            if(unlikely(!dimension || !*dimension)) {
+                error("requested a SET on chart '%s' of host '%s', without a dimension. Disabling it.", st->id, host->hostname);
+                enabled = 0;
+                break;
+            }
+
+            if(unlikely(!value || !*value)) value = NULL;
+
+            if(unlikely(!st)) {
+                error("requested a SET on dimension %s with value %s on host '%s', without a BEGIN. Disabling it.", dimension, value?value:"<nothing>", host->hostname);
+                enabled = 0;
+                break;
+            }
+
+            if(unlikely(rrdset_flag_check(st, RRDSET_FLAG_DEBUG)))
+                debug(D_PLUGINSD, "is setting dimension %s/%s to %s", st->id, dimension, value?value:"<nothing>");
+
+            if(value) {
+                RRDDIM *rd = rrddim_find(st, dimension);
+                if(unlikely(!rd)) {
+                    error("requested a SET to dimension with id '%s' on stats '%s' (%s) on host '%s', which does not exist. Disabling it.", dimension, st->name, st->id, st->rrdhost->hostname);
+                    enabled = 0;
+                    break;
+                }
+                else
+                    rrddim_set_by_pointer(st, rd, strtoll(value, NULL, 0));
+            }
+        }
+        else if(likely(hash == BEGIN_HASH && !strcmp(s, PLUGINSD_KEYWORD_BEGIN))) {
+            char *id = words[1];
+            char *microseconds_txt = words[2];
+
+            if(unlikely(!id)) {
+                error("requested a BEGIN without a chart id for host '%s'. Disabling it.", host->hostname);
+                enabled = 0;
+                break;
+            }
+
+            st = rrdset_find(host, id);
+            if(unlikely(!st)) {
+                error("requested a BEGIN on chart '%s', which does not exist on host '%s'. Disabling it.", id, host->hostname);
+                enabled = 0;
+                break;
+            }
+
+            if(likely(st->counter_done)) {
+                usec_t microseconds = 0;
+                if(microseconds_txt && *microseconds_txt) microseconds = str2ull(microseconds_txt);
+
+                if(likely(microseconds)) {
+                    if(trust_durations)
+                        rrdset_next_usec_unfiltered(st, microseconds);
+                    else
+                        rrdset_next_usec(st, microseconds);
+                }
+                else rrdset_next(st);
+            }
+        }
+        else if(likely(hash == END_HASH && !strcmp(s, PLUGINSD_KEYWORD_END))) {
+            if(unlikely(!st)) {
+                error("requested an END, without a BEGIN on host '%s'. Disabling it.", host->hostname);
+                enabled = 0;
+                break;
+            }
+
+            if(unlikely(rrdset_flag_check(st, RRDSET_FLAG_DEBUG)))
+                debug(D_PLUGINSD, "requested an END on chart %s", st->id);
+
+            rrdset_done(st);
+            st = NULL;
+
+            count++;
+        }
+        else if(likely(hash == CHART_HASH && !strcmp(s, PLUGINSD_KEYWORD_CHART))) {
+            st = NULL;
+
+            char *type           = words[1];
+            char *name           = words[2];
+            char *title          = words[3];
+            char *units          = words[4];
+            char *family         = words[5];
+            char *context        = words[6];
+            char *chart          = words[7];
+            char *priority_s     = words[8];
+            char *update_every_s = words[9];
+            char *options        = words[10];
+            char *plugin         = words[11];
+            char *module         = words[12];
+
+            // parse the id from type
+            char *id = NULL;
+            if(likely(type && (id = strchr(type, '.')))) {
+                *id = '\0';
+                id++;
+            }
+
+            // make sure we have the required variables
+            if(unlikely(!type || !*type || !id || !*id)) {
+                error("requested a CHART, without a type.id, on host '%s'. Disabling it.", host->hostname);
+                enabled = 0;
+                break;
+            }
+
+            // parse the name, and make sure it does not include 'type.'
+            if(unlikely(name && *name)) {
+                // when data are coming from slaves
+                // name will be type.name
+                // so we have to remove 'type.' from name too
+                size_t len = strlen(type);
+                if(strncmp(type, name, len) == 0 && name[len] == '.')
+                    name = &name[len + 1];
+
+                // if the name is the same with the id,
+                // or is just 'NULL', clear it.
+                if(unlikely(strcmp(name, id) == 0 || strcasecmp(name, "NULL") == 0 || strcasecmp(name, "(NULL)") == 0))
+                    name = NULL;
+            }
+
+            int priority = 1000;
+            if(likely(priority_s && *priority_s)) priority = str2i(priority_s);
+
+            int update_every = cd->update_every;
+            if(likely(update_every_s && *update_every_s)) update_every = str2i(update_every_s);
+            if(unlikely(!update_every)) update_every = cd->update_every;
+
+            RRDSET_TYPE chart_type = RRDSET_TYPE_LINE;
+            if(unlikely(chart)) chart_type = rrdset_type_id(chart);
+
+            if(unlikely(name && !*name)) name = NULL;
+            if(unlikely(family && !*family)) family = NULL;
+            if(unlikely(context && !*context)) context = NULL;
+            if(unlikely(!title)) title = "";
+            if(unlikely(!units)) units = "unknown";
+
+            debug(D_PLUGINSD, "creating chart type='%s', id='%s', name='%s', family='%s', context='%s', chart='%s', priority=%d, update_every=%d"
+                  , type, id
+                  , name?name:""
+                  , family?family:""
+                  , context?context:""
+                  , rrdset_type_name(chart_type)
+                  , priority
+                  , update_every
+            );
+
+            st = rrdset_create(
+                    host
+                    , type
+                    , id
+                    , name
+                    , family
+                    , context
+                    , title
+                    , units
+                    , (plugin && *plugin)?plugin:cd->filename
+                    , module
+                    , priority
+                    , update_every
+                    , chart_type
+            );
+
+            if(options && *options) {
+                if(strstr(options, "obsolete"))
+                    rrdset_is_obsolete(st);
+                else
+                    rrdset_isnot_obsolete(st);
+
+                if(strstr(options, "detail"))
+                    rrdset_flag_set(st, RRDSET_FLAG_DETAIL);
+                else
+                    rrdset_flag_clear(st, RRDSET_FLAG_DETAIL);
+
+                if(strstr(options, "hidden"))
+                    rrdset_flag_set(st, RRDSET_FLAG_HIDDEN);
+                else
+                    rrdset_flag_clear(st, RRDSET_FLAG_HIDDEN);
+
+                if(strstr(options, "store_first"))
+                    rrdset_flag_set(st, RRDSET_FLAG_STORE_FIRST);
+                else
+                    rrdset_flag_clear(st, RRDSET_FLAG_STORE_FIRST);
+            }
+            else {
+                rrdset_isnot_obsolete(st);
+                rrdset_flag_clear(st, RRDSET_FLAG_DETAIL);
+                rrdset_flag_clear(st, RRDSET_FLAG_STORE_FIRST);
+            }
+        }
+        else if(likely(hash == DIMENSION_HASH && !strcmp(s, PLUGINSD_KEYWORD_DIMENSION))) {
+            char *id = words[1];
+            char *name = words[2];
+            char *algorithm = words[3];
+            char *multiplier_s = words[4];
+            char *divisor_s = words[5];
+            char *options = words[6];
+
+            if(unlikely(!id || !*id)) {
+                error("requested a DIMENSION, without an id, host '%s' and chart '%s'. Disabling it.", host->hostname, st?st->id:"UNSET");
+                enabled = 0;
+                break;
+            }
+
+            if(unlikely(!st)) {
+                error("requested a DIMENSION, without a CHART, on host '%s'. Disabling it.", host->hostname);
+                enabled = 0;
+                break;
+            }
+
+            long multiplier = 1;
+            if(multiplier_s && *multiplier_s) multiplier = strtol(multiplier_s, NULL, 0);
+            if(unlikely(!multiplier)) multiplier = 1;
+
+            long divisor = 1;
+            if(likely(divisor_s && *divisor_s)) divisor = strtol(divisor_s, NULL, 0);
+            if(unlikely(!divisor)) divisor = 1;
+
+            if(unlikely(!algorithm || !*algorithm)) algorithm = "absolute";
+
+            if(unlikely(rrdset_flag_check(st, RRDSET_FLAG_DEBUG)))
+                debug(D_PLUGINSD, "creating dimension in chart %s, id='%s', name='%s', algorithm='%s', multiplier=%ld, divisor=%ld, hidden='%s'"
+                      , st->id
+                      , id
+                      , name?name:""
+                      , rrd_algorithm_name(rrd_algorithm_id(algorithm))
+                      , multiplier
+                      , divisor
+                      , options?options:""
+                );
+
+            RRDDIM *rd = rrddim_add(st, id, name, multiplier, divisor, rrd_algorithm_id(algorithm));
+            rrddim_flag_clear(rd, RRDDIM_FLAG_HIDDEN);
+            rrddim_flag_clear(rd, RRDDIM_FLAG_DONT_DETECT_RESETS_OR_OVERFLOWS);
+            if(options && *options) {
+                if(strstr(options, "hidden") != NULL) rrddim_flag_set(rd, RRDDIM_FLAG_HIDDEN);
+                if(strstr(options, "noreset") != NULL) rrddim_flag_set(rd, RRDDIM_FLAG_DONT_DETECT_RESETS_OR_OVERFLOWS);
+                if(strstr(options, "nooverflow") != NULL) rrddim_flag_set(rd, RRDDIM_FLAG_DONT_DETECT_RESETS_OR_OVERFLOWS);
+            }
+        }
+        else if(likely(hash == VARIABLE_HASH && !strcmp(s, PLUGINSD_KEYWORD_VARIABLE))) {
+            char *name = words[1];
+            char *value = words[2];
+            int global = (st)?0:1;
+
+            if(name && *name) {
+                if((strcmp(name, "GLOBAL") == 0 || strcmp(name, "HOST") == 0)) {
+                    global = 1;
+                    name = words[2];
+                    value  = words[3];
+                }
+                else if((strcmp(name, "LOCAL") == 0 || strcmp(name, "CHART") == 0)) {
+                    global = 0;
+                    name = words[2];
+                    value  = words[3];
+                }
+            }
+
+            if(unlikely(!name || !*name)) {
+                error("requested a VARIABLE on host '%s', without a variable name. Disabling it.", host->hostname);
+                enabled = 0;
+                break;
+            }
+
+            if(unlikely(!value || !*value))
+                value = NULL;
+
+            if(value) {
+                char *endptr = NULL;
+                calculated_number v = (calculated_number)str2ld(value, &endptr);
+
+                if(unlikely(endptr && *endptr)) {
+                    if(endptr == value)
+                        error("the value '%s' of VARIABLE '%s' on host '%s' cannot be parsed as a number", value, name, host->hostname);
+                    else
+                        error("the value '%s' of VARIABLE '%s' on host '%s' has leftovers: '%s'", value, name, host->hostname, endptr);
+                }
+
+                if(global) {
+                    RRDVAR *rv = rrdvar_custom_host_variable_create(host, name);
+                    if (rv) rrdvar_custom_host_variable_set(host, rv, v);
+                    else error("cannot find/create HOST VARIABLE '%s' on host '%s'", name, host->hostname);
+                }
+                else if(st) {
+                    RRDSETVAR *rs = rrdsetvar_custom_chart_variable_create(st, name);
+                    if (rs) rrdsetvar_custom_chart_variable_set(rs, v);
+                    else error("cannot find/create CHART VARIABLE '%s' on host '%s', chart '%s'", name, host->hostname, st->id);
+                }
+                else
+                    error("cannot find/create CHART VARIABLE '%s' on host '%s' without a chart", name, host->hostname);
+            }
+            else
+                error("cannot set %s VARIABLE '%s' on host '%s' to an empty value", (global)?"HOST":"CHART", name, host->hostname);
+        }
+        else if(likely(hash == FLUSH_HASH && !strcmp(s, PLUGINSD_KEYWORD_FLUSH))) {
+            debug(D_PLUGINSD, "requested a FLUSH");
+            st = NULL;
+        }
+        else if(unlikely(hash == DISABLE_HASH && !strcmp(s, PLUGINSD_KEYWORD_DISABLE))) {
+            info("called DISABLE. Disabling it.");
+            enabled = 0;
+            break;
+        }
+        else {
+            error("sent command '%s' which is not known by netdata, for host '%s'. Disabling it.", s, host->hostname);
+            enabled = 0;
+            break;
+        }
+    }
+
+cleanup:
+    cd->enabled = enabled;
+
+    if(likely(count)) {
+        cd->successful_collections += count;
+        cd->serial_failures = 0;
+    }
+    else
+        cd->serial_failures++;
+
+    return count;
+}
+
+static void pluginsd_worker_thread_cleanup(void *arg) {
+    struct plugind *cd = (struct plugind *)arg;
+