Refactor ML code and add support for multiple KMeans models (#14198)

* Add profile.plugin

Creates the specified number of charts/dimensions, and supports
backfilling with pseudo-historical data.

* Bump

* Remove wrongly merged line.

* Use the number of models specified from the config section.

* Add option to consult all ML models.

* Remove profiling option consuming all models.

* Add underscore after chart name prefix.

* prediction -> dimensions chart

* reorder funcs

* Split charts across types with correct priority

* Ignore training request when chart is under replication.

* Track global number of models consulted.

* Cleanup config.

* initial readme updates

* fix readme

* readme

* Fix function definition when ML is disabled.

* Add dummy ml_chart_update_{begin,end}

* Remove profile_plugin

* Define chart priorities under collectors/all.h

* s/curr_t/current_time/

* Use libnetdata's lock/thread wrappers.

* Fix autotools & cmake builds.

* Delete ML dimensions & charts.

* Let users of buffer preprocessing to handle memory.

* Add separate API calls to start/stop ML threads.

Co-authored-by: Andrew Maguire <andrewm4894@gmail.com>

1 files changed, 43 insertions, 8 deletions

diff --git a/ml/ml-dummy.c b/ml/ml-dummy.c
index 492dfe2fce..178018898f 100644
--- a/ml/ml-dummy.c
+++ b/ml/ml-dummy.c
@@ -15,9 +15,37 @@ bool ml_enabled(RRDHOST *RH) {
 
 void ml_init(void) {}
 
-void ml_new_host(RRDHOST *RH) { (void) RH; }
+void ml_host_new(RRDHOST *RH) {
+    UNUSED(RH);
+}
+
+void ml_host_delete(RRDHOST *RH) {
+    UNUSED(RH);
+}
+
+void ml_chart_new(RRDSET *RS) {
+    UNUSED(RS);
+}
+
+void ml_chart_delete(RRDSET *RS) {
+    UNUSED(RS);
+}
+
+void ml_dimension_new(RRDDIM *RD) {
+    UNUSED(RD);
+}
+
+void ml_dimension_delete(RRDDIM *RD) {
+    UNUSED(RD);
+}
 
-void ml_delete_host(RRDHOST *RH) { (void) RH; }
+void ml_start_anomaly_detection_threads(RRDHOST *RH) {
+    UNUSED(RH);
+}
+
+void ml_stop_anomaly_detection_threads(RRDHOST *RH) {
+    UNUSED(RH);
+}
 
 char *ml_get_host_info(RRDHOST *RH) {
     (void) RH;
@@ -29,17 +57,24 @@ char *ml_get_host_runtime_info(RRDHOST *RH) {
     return NULL;
 }
 
+void ml_chart_update_begin(RRDSET *RS) {
+    (void) RS;
+}
+
+void ml_chart_update_end(RRDSET *RS) {
+    (void) RS;
+}
+
 char *ml_get_host_models(RRDHOST *RH) {
     (void) RH;
     return NULL;
 }
 
-void ml_new_dimension(RRDDIM *RD) { (void) RD; }
-
-void ml_delete_dimension(RRDDIM *RD) { (void) RD; }
-
-bool ml_is_anomalous(RRDDIM *RD, double Value, bool Exists) {
-    (void) RD; (void) Value; (void) Exists;
+bool ml_is_anomalous(RRDDIM *RD, time_t CurrT, double Value, bool Exists) {
+    (void) RD;
+    (void) CurrT;
+    (void) Value;
+    (void) Exists;
     return false;
 }