QUERY_TARGET: new query engine for Netdata Agent (#13697)

* initial implementation of QUERY_TARGET

* rrd2rrdr() interface

* rrddim_find_best_tier_for_timeframe() ported

* added dimension filtering

* added db object in query target

* rrd2rrdr() ported

* working on formatters

* working on jsonwrapper

* finally, it compiles...

* 1st run without crashes

* query planer working

* cleanup old code

* review changes

* fix also changing data collection frequency

* fix signess

* fix rrdlabels and dimension ordering

* fixes

* remove unused variable

* ml should accept NULL response from rrd2rrdr()

* number formatting fixes

* more number formatting fixes

* more number formatting fixes

* support mc parallel queries

* formatting and cleanup

* added rrd2rrdr_legacy() as a simplified interface to run a query

* make sure rrdset_find_natural_update_every_for_timeframe() returns a value

* make signed comparisons

* weights endpoint using rrdcontexts

* fix for legacy db modes and cleanup

* fix for chart_ids and remove AR chart from weights endpoint

* Ignore command if not initialized yet

* remove unused members

* properly initialize window

* code cleanup - rrddim linked list is gone; rrdset rwlock is gone too

* reviewed RRDR.internal members

* eliminate unnecessary members of QUERY_TARGET

* more complete query ids; more detailed information on aborted queries

* properly terminate option strings

* query id contains group_options which is controlled by users, so escaping is necessary

* tense in query id

* tense in query id - again

* added the remaining query options to the query id

* Expose hidden option to the dimension

* use the hidden flag when loading context dimensions

* Specify table alias for option

* dont update chart last access time, unless at least a dimension of the chart will be queried

Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>

1 files changed, 22 insertions, 21 deletions

diff --git a/ml/ADCharts.cc b/ml/ADCharts.cc
index 5a78b97918..5c207ed199 100644
--- a/ml/ADCharts.cc
+++ b/ml/ADCharts.cc
@@ -133,31 +133,32 @@ void ml::updateHostAndDetectionRateCharts(RRDHOST *RH, collected_number AnomalyR
     time_t After = Before - Cfg.AnomalyDetectionQueryDuration;
     RRDR_OPTIONS Options = static_cast<RRDR_OPTIONS>(0x00000000);
 
-    RRDR *R = rrd2rrdr(
-        OWA, HostRateRS,
-        1 /* points wanted */,
-        After,
-        Before,
-        Cfg.AnomalyDetectionGroupingMethod,
-        0 /* resampling time */,
-        Options, "anomaly_rate",
-        NULL /* context param list */,
-        NULL /* group options */,
-        0, /* timeout */
-        0 /* tier */
+    RRDR *R = rrd2rrdr_legacy(
+            OWA, HostRateRS,
+            1 /* points wanted */,
+            After,
+            Before,
+            Cfg.AnomalyDetectionGroupingMethod,
+            0 /* resampling time */,
+            Options, "anomaly_rate",
+            NULL /* group options */,
+            0, /* timeout */
+            0 /* tier */
     );
-    assert(R->d == 1 && R->n == 1 && R->rows == 1);
+    if(R) {
+        assert(R->d == 1 && R->n == 1 && R->rows == 1);
 
-    static thread_local bool PrevAboveThreshold = false;
-    bool AboveThreshold = R->v[0] >= Cfg.HostAnomalyRateThreshold;
-    bool NewAnomalyEvent = AboveThreshold && !PrevAboveThreshold;
-    PrevAboveThreshold = AboveThreshold;
+        static thread_local bool PrevAboveThreshold = false;
+        bool AboveThreshold = R->v[0] >= Cfg.HostAnomalyRateThreshold;
+        bool NewAnomalyEvent = AboveThreshold && !PrevAboveThreshold;
+        PrevAboveThreshold = AboveThreshold;
 
-    rrddim_set_by_pointer(AnomalyDetectionRS, AboveThresholdRD, AboveThreshold);
-    rrddim_set_by_pointer(AnomalyDetectionRS, NewAnomalyEventRD, NewAnomalyEvent);
-    rrdset_done(AnomalyDetectionRS);
+        rrddim_set_by_pointer(AnomalyDetectionRS, AboveThresholdRD, AboveThreshold);
+        rrddim_set_by_pointer(AnomalyDetectionRS, NewAnomalyEventRD, NewAnomalyEvent);
+        rrdset_done(AnomalyDetectionRS);
 
-    rrdr_free(OWA, R);
+        rrdr_free(OWA, R);
+    }
     onewayalloc_destroy(OWA);
 }