| Age | Commit message (Collapse) | Author |
|---|
|
* Ignore build/ dir.
This directory is the default dir for many LSPs and for IDES using
cmake. "Reserve" it by ignoring it in .gitignore.
* Fix format specifier.
* Use unsigned literals when shifting.
* Do not sanitize shifts in libjudy.
* Fix unaligned loads/stores of dbengine's CRCs
* Fix unaligned load when partitioning metrics.
* Use unsigned literals when shifting.
|
|
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Austin S. Hemmelgarn <austin@netdata.cloud>
Co-authored-by: Tasos Katsoulas <12612986+tkatsoulas@users.noreply.github.com>
Co-authored-by: Emmanuel Vasilakis <mrzammler@mm.st>
Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
Co-authored-by: netdatabot <bot@netdata.cloud>
Co-authored-by: Ilya Mashchenko <ilya@netdata.cloud>
|
|
|
|
|
|
/api/v2/weights nonzero output
|
|
optimization (#14493)
* first work on standardizing json formatting
* renamed old grouping to time_grouping and added group_by
* add dummy functions to enable compilation
* buffer json api work
* jsonwrap opening with buffer_json_X() functions
* cleanup
* storage for quotes
* optimize buffer printing for both numbers and strings
* removed ; from define
* contexts json generation using the new json functions
* fix buffer overflow at unit test
* weights endpoint using new json api
* fixes to weights endpoint
* check buffer overflow on all buffer functions
* do synchronous queries for weights
* buffer_flush() now resets json state too
* content type typedef
* print double values that are above the max 64-bit value
* str2ndd() can now parse values above UINT64_MAX
* faster number parsing by avoiding double calculations as much as possible
* faster number parsing
* faster hex parsing
* accurate printing and parsing of double values, even for very large numbers that cannot fit in 64bit integers
* full printing and parsing without using library functions - and related unit tests
* added IEEE754 streaming capability to enable streaming of double values in hex
* streaming and replication to transfer all values in hex
* use our own str2ndd for set2
* remove subnormal check from ieee
* base64 encoding for numbers, instead of hex
* when increasing double precision, also make sure the fractional number printed is aligned to the wanted precision
* str2ndd_encoded() parses all encoding formats, including integers
* prevent uninitialized use
* /api/v1/info using the new json API
* Fix error when compiling with --disable-ml
* Remove redundant 'buffer_unittest' declaration
* Fix formatting
* Fix formatting
* Fix formatting
* fix buffer unit test
* apps.plugin using the new JSON API
* make sure the metrics registry does not accept negative timestamps
* do not allow pages with negative timestamps to be loaded from db files; do not accept pages with negative timestamps in the cache
* Fix more formatting
---------
Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
|
|
* Move installation before concepts
* Reorder installation links
* Remove single node monitoring
* Reorg libnetdata content
|
|
* first commit - untested
* fix wrong begin command
* added set v2 too
* debug to log stream buffer
* debug to log stream buffer
* faster streaming printing
* mark charts and dimensions as collected
* use stream points even if sender is not enabled
* comment out stream debug log
* parse null as nan
* custom begin v2
* custom set v2; replication now copies the anomalous flag too
* custom end v2
* enabled stream log test
* renamed to BEGIN2, SET2, END2
* dont mix up replay and v2 members in user object
* fix typo
* cleanup
* support to v2 to v1 proxying
* mark updated dimensions as such
* do not log unknown flags
* comment out stream debug log
* send also the chart id on BEGIN2, v2 to v2
* update the data collections counter
* v2 values are transferred in hex
* faster hex parsing
* a little more generic hex and dec printing and parsing
* fix hex parsing
* minor optimization in dbengine api
* turn debugging into info message
* generalized the timings tracking, so that it can be used in more places
* commented out debug info
* renamed conflicting variable with macro
* remove wrong edits
* integrated ML and added cleanup in case parsing is interrupted
* disable data collection locking during v2
* cleanup stale ML locks; send updated chart variables during v2; add info to find stale locks
* inject an END2 between repeated BEGIN2 from rrdset_done()
* test: remove lockless single-threaded logic from dictionary and aral and apply the right acquire/release memory order to reference counters
* more fine grained dictionary atomics
* remove unecessary return values
* pointer validation under NETDATA_DICTIONARY_VALIDATE_POINTERS
* Revert "pointer validation under NETDATA_DICTIONARY_VALIDATE_POINTERS"
This reverts commit 846cdf2713e2a7ee2ff797f38db11714228800e9.
* Revert "remove unecessary return values"
This reverts commit 8c87d30f4d86f0f5d6b4562cf74fe7447138bbff.
* Revert "more fine grained dictionary atomics"
This reverts commit 984aec4234a340d197d45239ff9a10fd479fcf3c.
* Revert "test: remove lockless single-threaded logic from dictionary and aral and apply the right acquire/release memory order to reference counters"
This reverts commit c460b3d0ad497d2641bd0ea1d63cec7c052e74e4.
* Apply again "pointer validation under NETDATA_DICTIONARY_VALIDATE_POINTERS" while keeping the improved atomic operations.
This reverts commit f158d009
* fix last commit
* fix last commit again
* optimizations in dbengine
* do not send anomaly bit on non-supporting agents (send it when the INTERPOLATED capability is available)
* break long empty-points-loops in rrdset_done()
* decide page alignment on new page allocation, not on every point collected
* create max size pages but no smaller than 1/3
* Fix compilation when --disable-ml is specified
* Return false
* fixes for NETDATA_LOG_REPLICATION_REQUESTS
* added compile option NETDATA_WITHOUT_WORKERS_LATENCY
* put timings in BEGIN2, SET2, END2
* isolate begin2 ml
* revert repositioning data collection lock
* fixed multi-threading of statistics
* do not lookup dimensions all the time if they come in the same order
* update used on iteration, not on every points; also do better error handling
---------
Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
|
|
* Add remaining libnetdata readmes to learn
* Move all libnetdata readmes in learn under Developer/libnetdata libraries except for the main README
* Add the moved array allocator
|
|
Revert "Add libnetdata readmes to learn, delete empty (#14371)"
This reverts commit a2fa9180c567395752b57df1895d677fe129a790.
|
|
Co-authored-by: Christopher Akritidis <cakrit@DESKTOP-C7OKV71.localdomain>
|
|
* dbengine free from RRDSET and RRDDIM
* fix for excess parameters to query ops
* add comment about ML
* update_every from int to uint32_t
* rrddim_mem storage engine working
* fixes for update_every_s
* working dbengine
* a lot of changes in dbengine regarding timestamps
* better logging of not sequential points
* rrdset_done() now gives aligned timestamps for higher tiers
* dont change the end_time of descriptors, because they cant be loaded back
* fixes for cmake
* fixes for db mode ram
* Global counters for dbengine loading errors.
Ensure dbengine store metrics always has aligned metrics or breaks the page when storing new data.
* update lgtm config
* fixes for 32-bit systems
* update unittests
* Don't try to find and create a host on the fly if not already in memory
* Remove unused functions
* print backtrace in case of fatal
* always set ctx to page_index
* detect ctx and metric uuid discrepancies
* use legacy uuid if multihost is not available
* fix for last commit
* prevent repeating log
* Do not try to access archived charts when executing a data query
* Remove unused function
* log inconsistent collections once every 10 mins
Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
|
|
* set value to SN_EMPTY_SLOT if flags is SN_EMPTY_SLOT
* SN_EMPTY_SLOT should be SN_ANOMALOUS_ZERO
* added the const attribute to pack_storage_number()
* tier1 uses floats
* zero should not be empty slot
* add unlikely
* rename all SN flags to be more meaningful
* proper check for zero double value
* properly check if pages are full with empty points
Co-authored-by: Costa Tsaousis <costa@netdata.cloud>
|
|
* Tier part 1
* Tier part 2
* Tier part 3
* Tier part 4
* Tier part 5
* Fix some ML compilation errors
* fix more conflicts
* pass proper tier
* move metric_uuid from state to RRDDIM
* move aclk_live_status from state to RRDDIM
* move ml_dimension from state to RRDDIM
* abstracted the data collection interface
* support flushing for mem db too
* abstracted the query api
* abstracted latest/oldest time per metric
* cleanup
* store_metric for tier1
* fix for store_metric
* allow multiple tiers, more than 2
* state to tier
* Change storage type in db. Query param to request min, max, sum or average
* Store tier data correctly
* Fix skipping tier page type
* Add tier grouping in the tier
* Fix to handle archived charts (part 1)
* Temp fix for query granularity when requesting tier1 data
* Fix parameters in the correct order and calculate the anomaly based on the anomaly count
* Proper tiering grouping
* Anomaly calculation based on anomaly count
* force type checking on storage handles
* update cmocka tests
* fully dynamic number of storage tiers
* fix static allocation
* configure grouping for all tiers; disable tiers for unittest; disable statsd configuration for private charts mode
* use default page dt using the tiering info
* automatic selection of tier
* fix for automatic selection of tier
* working prototype of dynamic tier selection
* automatic selection of tier done right (I hope)
* ask for the proper tier value, based on the grouping function
* fixes for unittests and load_metric_next()
* fixes for lgtm findings
* minor renames
* add dbengine to page cache size setting
* add dbengine to page cache with malloc
* query engine optimized to loop as little are required based on the view_update_every
* query engine grouping methods now do not assume a constant number of points per group and they allocate memory with OWA
* report db points per tier in jsonwrap
* query planer that switches database tiers on the fly to satisfy the query for the entire timeframe
* dbegnine statistics and documentation (in progress)
* calculate average point duration in db
* handle single point pages the best we can
* handle single point pages even better
* Keep page type in the rrdeng_page_descr
* updated doc
* handle future backwards compatibility - improved statistics
* support &tier=X in queries
* enfore increasing iterations on tiers
* tier 1 is always 1 iteration
* backfilling higher tiers on first data collection
* reversed anomaly bit
* set up to 5 tiers
* natural points should only be offered on tier 0, except a specific tier is selected
* do not allow more than 65535 points of tier0 to be aggregated on any tier
* Work only on actually activated tiers
* fix query interpolation
* fix query interpolation again
* fix lgtm finding
* Activate one tier for now
* backfilling of higher tiers using raw metrics from lower tiers
* fix for crash on start when storage tiers is increased from the default
* more statistics on exit
* fix bug that prevented higher tiers to get any values; added backfilling options
* fixed the statistics log line
* removed limit of 255 iterations per tier; moved the code of freezing rd->tiers[x]->db_metric_handle
* fixed division by zero on zero points_wanted
* removed dead code
* Decide on the descr->type for the type of metric
* dont store metrics on unknown page types
* free db_metric_handle on sql based context queries
* Disable STORAGE_POINT value check in the exporting engine unit tests
* fix for db modes other than dbengine
* fix for aclk archived chart queries destroying db_metric_handles of valid rrddims
* fix left-over freez() instead of OWA freez on median queries
Co-authored-by: Costa Tsaousis <costa@netdata.cloud>
Co-authored-by: Vladimir Kobal <vlad@prokk.net>
|
|
* netdata doubles
* fix cmocka test
* fix cmocka test again
* fix left-overs of long double to NETDATA_DOUBLE
* RRDDIM detached from disk representation; db settings in [db] section of netdata.conf
* update the memory before saving
* rrdset is now detached from file structures too
* on memory mode map, update the memory mapped structures on every iteration
* allow RRD_ID_LENGTH_MAX to be changed
* granularity secs, back to update every
* fix formatting
* more formatting
|
|
* set grouping functions
* storage engine should check the validity of timestamps, not the query engine
* calculate and store in RRDR anomaly rates for every query
* anomaly rate used by volume metric correlations
* mc volume should use absolute data, to avoid cancelling effect
* return anomaly-rates in jasonwrap with jw-anomaly-rates option to data queries
* dont return null on anomaly rates
* allow passing group query options from the URL
* added countif to the query engine and used it in metric correlations
* fix configure
* fix countif and anomaly rate percentages
* added group_options to metric correlations; updated swagger
* added newline at the end of yaml file
* always check the time the highlighted window was above/below the highlighted window
* properly track time in memory queries
* error for internal checks only
* moved pack_storage_number() into the storage engines
* moved unpack_storage_number() inside the storage engines
* remove old comment
* pass unit tests
* properly detect zero or subnormal values in pack_storage_number()
* fill nulls before the value, not after
* make sure math.h is included
* workaround for isfinite()
* fix for isfinite()
* faster isfinite() alternative
* fix for faster isfinite() alternative
* next_metric() now returns end_time too
* variable step implemented in a generic way
* remove left-over variables
* ensure we always complete the wanted number of points
* fixes
* ensure no infinite loop
* mc-volume-improvements: Add information about invalid condition
* points should have a duration in the past
* removed unneeded info() line
* Fix unit tests for exporting engine
* new_point should only be checked when it is fetched from the db; better comment about the premature breaking of the main query loop
Co-authored-by: Thiago Marques <thiagoftsm@gmail.com>
Co-authored-by: Vladimir Kobal <vlad@prokk.net>
|
|
* faster correlations
* 4x times faster correlations
* a little bit more help
* 10x times faster metrics correlations
* 6 digits precision; better comments
* enabled metrics correlations by default
* abstracted DIFFS_NUMBER to allow easily changing it
* reworked the entire logic to have more accuracy and support a baseline that is power of two multiple of highlight
* properly calculate shifts
* even more improved version
* added support for timeout; fixed another memory leak; skipped hidden dimensions
* default timeout 1min
* reduce memory even further
* use dictionary for the list of charts and optimize locks
* return 403 forbidden, when mc is not enabled
* added query options
* dont process zero dimensions
* added volume method as an option to metric correlations ; now metric correlations can support multiple implementations
* make sure we will never crash
* spread results evenly for both kstwo and volume
* fixed bug in query engine that was missing misaligned queries when a single point was requested from the db; improved comments; improved query flags
* updated swagger and added sane defaults; query options are now supported, including anomaly-bit
* added "raw" option to allow cross node correlations; added "group" option to allow different time aggregations; allowed calling metric correlations without any parameters; allowed calling metric correlations with relative timestamps; added timeout to volume method; properly handled timeout on ks2 method; json output now sends all parameters back - same for json_wrap; modified query engine to use present time for relative timestamps; modified "allow_past" to mean both past backwards and forwards
* emulate the old behaviour about zero points
* 100% accuracy against python ks_2samp(); now the default is volume and the default points are 500
* added config option to change default metric correlations method
* removed work-arounds now that rrdlabels are merged
|
|
* faster rrdeng_load_metric_next()
* no need to check validity for number - already done at the query side
* solve discrepancy between query create and free
* inline unpack_storage_number
|
|
The LUT contains precomputed values of multiplier/divisors that are
used to unpack storage numbers into calculated numbers efficiently.
|
|
|
|
* Replace all usages of SN_EXISTS with SN_DEFAULT_FLAGS.
* Remove references to SN_NOT_EXISTS in comments.
* Replace raw zero constant with SN_EMPTY_SLOT.
* Use get_storage_number_flags only in storage_number.{c,h}
* Compare against SN_EMPTY_SLOT to check if a storage_number exists.
This is safe because:
1. rrdset_done_interpolate() is the only place where we call store_metric(),
2. All store_metric() calls, except for one, store an SN_EMPTY_SLOT value.
3. When we are not storing an SN_EMPTY_SLOT value, the flags that we pass to
pack_storage_number() can be either SN_EXISTS *or* SN_EXISTS_RESET.
* Compare only the SN_EXISTS_RESET bit to find reset values.
* Remove get_storage_number_flags from storage_number.h
* Do not set storage_number flags outside of rrdset_done_interpolate().
This is a NFC intended to limit the scope of storage_number flags
processing to just one function.
* Set reset bit without overwriting the rest of the flags.
* Rename SN_EXISTS to SN_ANOMALY_BIT.
* Use GOTOs in pack_storage_number to return from a single place.
* Teach pack_storage_number how to handle anomalous zero values.
Up until now, a storage_number had always either the SN_EXISTS or
SN_EXISTS_RESET bit set. This meant that it was not possible for any
packed storage_number to compare equal to the SN_EMPTY_SLOT.
However, the SN_ANOMALY_BIT can be set to zero. This is fine for every
value other than the anomalous 0 value, because it would compare equal to
SN_EMPTY_SLOT. We address this issue by mapping the anomalous zero value
to SN_EXISTS_100 (a number which was not possible to generate with the
previous versions of the agent, ie. it won't exist in older dbengine files).
This change was tested manually by intentionally flipping the anomaly
bit for odd/even iterations in rrdset_done_interpolate. Prior to this
change, charts whose dimensions had 0 values, where showing up in the
dashboard as gaps (SN_EMPTY_SLOT), whereas with this commit the values
are displayed correctly.
|
|
|
|
* First pass to get the script working right
* Finish adding analytics tags
|
|
Improved stored number accuracy
|
|
* Bulk add frontmatter
* A few extra edge cases
|
|
* Use 4 spaces for indentation of non-recipe lines in Makefile.am files
* Be more consistent in the use of space before = in Makefile.am files
|
|
* Removed support for 16-bit and 8-bit counter overflow
* Improve behaviour of counter overflow detection versus counter resets.
* Added support for signed 32-bit and 64-bit limits for counter overflows.
* Fixed signed incremental counter issues and added unit tests.
|
|
* Add str2ld test
* Build test with Autotools
* Add storage_number test
* Configure tests in CMake
|
|
* make remark access all directories
* detailed fix after autofix by remark lint
* cross check autofix for this set of files
* crosscheck more files
* crosschecking and small fixes
* crosscheck autofixed md files
|
|
* First pass of changing netdata to Netdata
* Second pass of netdata -> Netdata
* Starting work on netdata with no whitespace after
* Pass for netdata with no whitespace at the end
* Pass for netdata with no whitespace at the front
|
|
|
|
* Added GA tags to markdowns
* Add GA tags to mds
|
|
* incremental overflows do not show zeros; fixes #4533
* use the max per metric per session for detecting counter size
|
|
* moved related wiki pages into the repo
* updated web server docs
* fixed typos
|
|
* modularized all external plugins
* added README.md in plugins
* fixed title
* fixed typo
* relative link to external plugins
* external plugins configuration README
* added plugins link
* remove plugins link
* plugin names are links
* added links to external plugins
* removed unecessary spacing
* list to table
* added language
* fixed typo
* list to table on internal plugins
* added more documentation to internal plugins
* moved python, node, and bash code and configs into the external plugins
* added statsd README
* fix bug with corrupting config.h every 2nd compilation
* moved all config files together with their code
* more documentation
* diskspace info
* fixed broken links in apps.plugin
* added backends docs
* updated plugins readme
* move nc-backend.sh to backends
* created daemon directory
* moved all code outside src/
* fixed readme identation
* renamed plugins.d.plugin to plugins.d
* updated readme
* removed linux- from linux plugins
* updated readme
* updated readme
* updated readme
* updated readme
* updated readme
* updated readme
* fixed README.md links
* fixed netdata tree links
* updated codacy, codeclimate and lgtm excluded paths
* update CMakeLists.txt
* updated automake options at top directory
* libnetdata slit into directories
* updated READMEs
* updated READMEs
* updated ARL docs
* updated ARL docs
* moved /plugins to /collectors
* moved all external plugins outside plugins.d
* updated codacy, codeclimate, lgtm
* updated README
* updated url
* updated readme
* updated readme
* updated readme
* updated readme
* moved api and web into webserver
* web/api web/gui web/server
* modularized webserver
* removed web/gui/version.txt
|
