diff options
author | Markos Fountoulakis <44345837+mfundul@users.noreply.github.com> | 2019-05-15 08:28:06 +0300 |
---|---|---|
committer | Paul Emm. Katsoulakis <34388743+paulkatsoulakis@users.noreply.github.com> | 2019-05-15 08:28:06 +0300 |
commit | 6ca6d840dd19d5d7e9bacf93e011803ea5861496 (patch) | |
tree | f20393187806d642f94eab87f87180440089fb0a /backends | |
parent | fed63b6e99dd70beb2cf9ccadd7c396aa05b2ae0 (diff) |
Database engine (#5282)
* Database engine prototype version 0
* Database engine initial integration with netdata POC
* Scalable database engine with file and memory management.
* Database engine integration with netdata
* Added MIN MAX definitions to fix alpine build of travis CI
* Bugfix for backends and new DB engine, remove useless rrdset_time2slot() calls and erroneous checks
* DB engine disk protocol correction
* Moved DB engine storage file location to /var/cache/netdata/{host}/dbengine
* Fix configure to require openSSL for DB engine
* Fix netdata daemon health not holding read lock when iterating chart dimensions
* Optimized query API for new DB engine and old netdata DB fallback code-path
* netdata database internal query API improvements and cleanup
* Bugfix for DB engine queries returning empty values
* Added netdata internal check for data queries for old and new DB
* Added statistics to DB engine and fixed memory corruption bug
* Added preliminary charts for DB engine statistics
* Changed DB engine ratio statistics to incremental
* Added netdata statistics charts for DB engine internal statistics
* Fix for netdata not compiling successfully when missing dbengine dependencies
* Added DB engine functional test to netdata unittest command parameter
* Implemented DB engine dataset generator based on example.random chart
* Fix build error in CI
* Support older versions of libuv1
* Fixes segmentation fault when using multiple DB engine instances concurrently
* Fix memory corruption bug
* Fixed createdataset advanced option not exiting
* Fix for DB engine not working on FreeBSD
* Support FreeBSD library paths of new dependencies
* Workaround for unsupported O_DIRECT in OS X
* Fix unittest crashing during cleanup
* Disable DB engine FS caching in Apple OS X since O_DIRECT is not available
* Fix segfault when unittest and DB engine dataset generator don't have permissions to create temporary host
* Modified DB engine dataset generator to create multiple files
* Toned down overzealous page cache prefetcher
* Reduce internal memory fragmentation for page-cache data pages
* Added documentation describing the DB engine
* Documentation bugfixes
* Fixed unit tests compilation errors since last rebase
* Added note to back-up the DB engine files in documentation
* Added codacy fix.
* Support old gcc versions for atomic counters in DB engine
Diffstat (limited to 'backends')
-rw-r--r-- | backends/backends.c | 21 |
1 files changed, 19 insertions, 2 deletions
diff --git a/backends/backends.c b/backends/backends.c index deced30c46..5a8f3aa7b0 100644 --- a/backends/backends.c +++ b/backends/backends.c @@ -62,9 +62,11 @@ calculated_number backend_calculate_value_from_stored_data( (void)host; // find the edges of the rrd database for this chart - time_t first_t = rrdset_first_entry_t(st); - time_t last_t = rrdset_last_entry_t(st); + time_t first_t = rd->state->query_ops.oldest_time(rd); + time_t last_t = rd->state->query_ops.latest_time(rd); time_t update_every = st->update_every; + struct rrddim_query_handle handle; + storage_number n; // step back a little, to make sure we have complete data collection // for all metrics @@ -105,6 +107,7 @@ calculated_number backend_calculate_value_from_stored_data( size_t counter = 0; calculated_number sum = 0; +/* long start_at_slot = rrdset_time2slot(st, before), stop_at_slot = rrdset_time2slot(st, after), slot, stop_now = 0; @@ -126,7 +129,21 @@ calculated_number backend_calculate_value_from_stored_data( counter++; } +*/ + for(rd->state->query_ops.init(rd, &handle, before, after) ; !rd->state->query_ops.is_finished(&handle) ; ) { + n = rd->state->query_ops.next_metric(&handle); + if(unlikely(!does_storage_number_exist(n))) { + // not collected + continue; + } + + calculated_number value = unpack_storage_number(n); + sum += value; + + counter++; + } + rd->state->query_ops.finalize(&handle); if(unlikely(!counter)) { debug(D_BACKEND, "BACKEND: %s.%s.%s: no values stored in database for range %lu to %lu", host->hostname, st->id, rd->id, |