Refactor ML code and add support for multiple KMeans models (#14198)

* Add profile.plugin

Creates the specified number of charts/dimensions, and supports
backfilling with pseudo-historical data.

* Bump

* Remove wrongly merged line.

* Use the number of models specified from the config section.

* Add option to consult all ML models.

* Remove profiling option consuming all models.

* Add underscore after chart name prefix.

* prediction -> dimensions chart

* reorder funcs

* Split charts across types with correct priority

* Ignore training request when chart is under replication.

* Track global number of models consulted.

* Cleanup config.

* initial readme updates

* fix readme

* readme

* Fix function definition when ML is disabled.

* Add dummy ml_chart_update_{begin,end}

* Remove profile_plugin

* Define chart priorities under collectors/all.h

* s/curr_t/current_time/

* Use libnetdata's lock/thread wrappers.

* Fix autotools & cmake builds.

* Delete ML dimensions & charts.

* Let users of buffer preprocessing to handle memory.

* Add separate API calls to start/stop ML threads.

Co-authored-by: Andrew Maguire <andrewm4894@gmail.com>

1 files changed, 46 insertions, 0 deletions

diff --git a/ml/Stats.h b/ml/Stats.h
new file mode 100644
index 0000000000..b99bc39da5
--- /dev/null
+++ b/ml/Stats.h
@@ -0,0 +1,46 @@
+// SPDX-License-Identifier: GPL-3.0-or-later
+
+#ifndef ML_STATS_H
+#define ML_STATS_H
+
+#include "ml-private.h"
+
+namespace ml {
+
+struct MachineLearningStats {
+    size_t NumMachineLearningStatusEnabled;
+    size_t NumMachineLearningStatusDisabledUE;
+    size_t NumMachineLearningStatusDisabledSP;
+
+    size_t NumMetricTypeConstant;
+    size_t NumMetricTypeVariable;
+
+    size_t NumTrainingStatusUntrained;
+    size_t NumTrainingStatusPendingWithoutModel;
+    size_t NumTrainingStatusTrained;
+    size_t NumTrainingStatusPendingWithModel;
+
+    size_t NumAnomalousDimensions;
+    size_t NumNormalDimensions;
+};
+
+struct TrainingStats {
+    struct rusage TrainingRU;
+
+    size_t QueueSize;
+    size_t NumPoppedItems;
+
+    usec_t AllottedUT;
+    usec_t ConsumedUT;
+    usec_t RemainingUT;
+
+    size_t TrainingResultOk;
+    size_t TrainingResultInvalidQueryTimeRange;
+    size_t TrainingResultNotEnoughCollectedValues;
+    size_t TrainingResultNullAcquiredDimension;
+    size_t TrainingResultChartUnderReplication;
+};
+
+} // namespace ml
+
+#endif /* ML_STATS_H */