summaryrefslogtreecommitdiffstats
path: root/collectors
AgeCommit message (Collapse)Author
2023-03-03fix simple_pattern_create on freebsd (#14656)Ilya Mashchenko
2023-03-02Fix doc links (#14650)Chris Akritidis
* Update freebsd.md * Update REFERENCE.md * Update README.md * Update COLLECTORS.md
2023-03-02/api/v2/contexts (#14592)Costa Tsaousis
* preparation for /api/v2/contexts * working /api/v2/contexts * add anomaly rate information in all statistics; when sum-count is requested, return sums and counts instead of averages * minor fix * query targegt now accurately counts hosts, contexts, instances, dimensions, metrics * cleanup /api/v2/contexts * full text search with /api/v2/contexts * simple patterns now support the option to search ignoring case * full text search API with /api/v2/q * simple pattern execution optimization * do not show q when not given * full text search accounting * separated /api/v2/nodes from /api/v2/contexts * fix ssv queries for group_by * count query instances queried and failed per context and host * split rrdcontext.c to multiple files * add query totals * fix anomaly rate calculation; provide "ni" for indexing hosts * do not generate zero valued members * faster calculation of anomaly rate; by just summing integers for each db points and doing math once for every generated point * fix typo when printing dimensions totals * added option minify to remove spaces and newlines fron JSON output * send instance ids and names when they differ * do not add in query target dimensions, instances, contexts and hosts for which there is no retention in the current timeframe * fix for the previous + renames and code cleanup * when a dimension is filtered, include in the response all the other dimensions that are selectable * do not add nodes that do not have retention in the current window * move selection of dimensions to query_dimension_add(), instead of query_metric_add() * increase the pre-processing capacity of queries * generate instance fqdn ids and names only when they are needed * provide detailed statistics about tiers retention, queries, points, update_every * late allocation of query dimensions * cleanup * more cleanup * support for annotations per displayed point, RESET and PARTIAL * new type annotations * if a chart is not linked to contexts and it is collected, link it when it is collected * make ML run reentrant * make ML rrdr query synchronous * optimize replication memory allocation of replication_sort_entry * change units to percentage, when requesting a coefficinet of variation, or a percentage query * initialize replication before starting main threads * properly decrement no room requests counter * propagate the non-zero flag to group-by * the same by avoiding the extra loop * respect non-zero in all dimension arrays * remove dictionary garbage collection from dictionary_entries() and dictionary_version() * be more verbose when jv2 indexing is postponed * prevent infinite loop * use hidden dimensions even when dimensions pattern is unset * traverse hosts using dictionaries * fix dictionary unittests
2023-03-02Update README.md (#14647)Chris Akritidis
2023-02-28Make the title metadata H1 in all markdown files (#14625)Fotis Voutsas
* make the title metadta the H1 * Update collectors/python.d.plugin/zscores/README.md * Update libnetdata/ebpf/README.md * Update ml/README.md * Update libnetdata/string/README.md --------- Co-authored-by: Chris Akritidis <43294513+cakrit@users.noreply.github.com>
2023-02-28fix python version check to work for 3.10 and above (#14616)Andrew Maguire
2023-02-27Use vector allocation whenever is possible (eBPF) (#14591)thiagoftsm
2023-02-25Fix links to chart interactions (#14609)Chris Akritidis
2023-02-24Fix broken links (#14605)Fotis Voutsas
* fix broken links * fix a link to kernel-collector readme, make it to not point at .md
2023-02-22Change link text to collectors.md (#14590)Chris Akritidis
* Change link text to collectors.md * Change doc title
2023-02-21Memory management eBPF (#14462)thiagoftsm
2023-02-21Fix coverity issues (#14543)Stelios Fragkakis
* Fix coverity 383236: Resource leak * Fix coverity 382915 : Logically dead code * Fix coverity 379133 : Division or modulo by float zero * Fix coverity 382783 : Copy into fixed size buffer * Fix coverity 381151 : Missing unlock * Fix coverity 381903 : Dereference after null check
2023-02-20pandas collector add `read_sql()` support (#14563)Andrew Maguire
* import sqlalchemy and os to support read_sql
2023-02-20bump go.d to v0.51.0 (#14572)Ilya Mashchenko
2023-02-20Fix doc capitalization and remove obsolete section (#14569)Chris Akritidis
2023-02-20Include troubleshooting section in the docs of python.d modules (#13721)Tasos Katsoulas
Add troubleshooting sections to all python modules Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
2023-02-20Fix broken links in our documentation (#14565)Fotis Voutsas
* fix broken link in ml/README.md * fix broken link across all files * fix broken link across all files * fix broken links and remove what's next sections * fix broken links and remove what's next section * Remove related links sections with broken links that link to removed files * fix broken links
2023-02-17Reorg learn 021723 (#14556)Chris Akritidis
* Change titles of agent alert notifications * Reintroduce netdata for iot * Eliminate guides category, merge health config docs * Rename setup to configuration * Codacy fixes and move health config reference
2023-02-17More reorg learn 021623 (#14550)Chris Akritidis
* Moved contents of statsd guide inside the plugin documentation. * Remove remaining guides, content verified to exist elsewhere
2023-02-16Update learn path of python plugin readme (#14549)Chris Akritidis
2023-02-16Reorg markdown files for learn (#14547)Chris Akritidis
* Move export graphite metrics guide content to new integrations landing page and exporter readme * Merge info on how to write python collector and reorg file * Codacy warning fixes
2023-02-15JSON internal API, IEEE754 base64/hex streaming, weights endpoint ↵Costa Tsaousis
optimization (#14493) * first work on standardizing json formatting * renamed old grouping to time_grouping and added group_by * add dummy functions to enable compilation * buffer json api work * jsonwrap opening with buffer_json_X() functions * cleanup * storage for quotes * optimize buffer printing for both numbers and strings * removed ; from define * contexts json generation using the new json functions * fix buffer overflow at unit test * weights endpoint using new json api * fixes to weights endpoint * check buffer overflow on all buffer functions * do synchronous queries for weights * buffer_flush() now resets json state too * content type typedef * print double values that are above the max 64-bit value * str2ndd() can now parse values above UINT64_MAX * faster number parsing by avoiding double calculations as much as possible * faster number parsing * faster hex parsing * accurate printing and parsing of double values, even for very large numbers that cannot fit in 64bit integers * full printing and parsing without using library functions - and related unit tests * added IEEE754 streaming capability to enable streaming of double values in hex * streaming and replication to transfer all values in hex * use our own str2ndd for set2 * remove subnormal check from ieee * base64 encoding for numbers, instead of hex * when increasing double precision, also make sure the fractional number printed is aligned to the wanted precision * str2ndd_encoded() parses all encoding formats, including integers * prevent uninitialized use * /api/v1/info using the new json API * Fix error when compiling with --disable-ml * Remove redundant 'buffer_unittest' declaration * Fix formatting * Fix formatting * Fix formatting * fix buffer unit test * apps.plugin using the new JSON API * make sure the metrics registry does not accept negative timestamps * do not allow pages with negative timestamps to be loaded from db files; do not accept pages with negative timestamps in the cache * Fix more formatting --------- Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
2023-02-15Reorganize learn documents under Integrations part 2 (#14538)Chris Akritidis
* Reorge exporter integrations * Reorg alert notifications. change mdx to md for alert related files * Move all .mdx to .md, including links.
2023-02-14Fix broken links in markdown files (#14513)Fotis Voutsas
* fix broken links in claim/README.md * delete broken link in docs/guidelines.md * fix broken links * fix broken link * fix broken links * fix broken links * fix broken links * fix broken links * fix broken links * remove broken link * fix broken link * fix broken links * fix broken links * fix broken links * fix broken link * fix linking phrasing * fix broken links batch * fix broken links second batch * fix broken links * fix broken links * fix broken links * Update COLLECTORS.md * fix broken links * fix broken links
2023-02-12Make external plugins a category page in learn (#14511)Chris Akritidis
Make external plugins a category page
2023-02-12Learn integrations category changes (#14510)Chris Akritidis
* Better present netdata pandas and fix how to add new charts to netdata * Properly categorize statsd and show it as a generic collector. Move asterisk and k6 Elevate anomaly detection to "monitor anything"
2023-02-12Move collectors under Integrations/Monitoring (#14509)Chris Akritidis
* Move collectors under Integrations/Monitoring * Correct Monitoring to Monitor
2023-02-12Guides and collectors reorg and cleanup part 1 (#14507)Chris Akritidis
* rm github star important * Remove some guides and rework the data collection docs * link fixes on collectors/README.md * link fixes on collectors/REFERENCE.md * Fix codacy * More codacy list formatting
2023-02-10Fix broken links in collectors/COLLECTORS.md (#14502)Fotis Voutsas
* Publish collectors/python.d.plugin/anomalies/README.md to Learn * Update README.md * fix broken links in COLLECTORS.md
2023-02-09Virtual hosts for data collection (#14464)Costa Tsaousis
* support multiple hosts at pluginsd structures * cleanup obsolete code * use a lookup hashtable to quickly find the keyword to execute, without traversing the whole linked list of keywords * more cleanup * move new hash function to inlined.h * minimize comparisons, eliminate a pre-parsing of the first keyword for each line * cleanup parser from old code * move parser into libnetdata * unique entries in parser keywords hashtable * move all hashing functions to inlined.h, name their sources, simple_hash() now defaults to FNV1a, it was FNV1 * small_hash() for parser * plugins.d now can switch hosts, and also create/update them * update hash function and hashtable size * updated message * unittest all hashing functions * reset the chart when setting a new host * remove host tags * enable archived hosts when a collector pushes host info * do not need localhost to swtich to localhost * disable ARAL and OWA with -DFSANITIZE_ADDRESS=1
2023-02-08Fix random crash on agent shutdown (#14470)Stelios Fragkakis
Check MMAP failure. If fail to create semaphore, clear the shm_cgroup_ebpf.header to avoid crash during shutdown on thread cleanup
2023-02-08Add markdown files in Learn (#14466)Fotis Voutsas
* add metadata for learn * first batch of adding metadata to md files * second batch of adding metadata to md files * third batch of adding metadata to md files * test one sidebar_label * add missing sidebar_labels * add missing sidebar_labels to files left behind * test, ansible doc is stubborn * fix * fix * fix * don't use questionmarks in the sidebar label * don't use exclamation marks and symbols in the sidebar label * fix style guide * fixes * fixes
2023-02-08remove deprecated python.d collectors announced in v1.38.0 (#14454)Ilya Mashchenko
2023-02-07Streaming interpolated values (#14431)Costa Tsaousis
* first commit - untested * fix wrong begin command * added set v2 too * debug to log stream buffer * debug to log stream buffer * faster streaming printing * mark charts and dimensions as collected * use stream points even if sender is not enabled * comment out stream debug log * parse null as nan * custom begin v2 * custom set v2; replication now copies the anomalous flag too * custom end v2 * enabled stream log test * renamed to BEGIN2, SET2, END2 * dont mix up replay and v2 members in user object * fix typo * cleanup * support to v2 to v1 proxying * mark updated dimensions as such * do not log unknown flags * comment out stream debug log * send also the chart id on BEGIN2, v2 to v2 * update the data collections counter * v2 values are transferred in hex * faster hex parsing * a little more generic hex and dec printing and parsing * fix hex parsing * minor optimization in dbengine api * turn debugging into info message * generalized the timings tracking, so that it can be used in more places * commented out debug info * renamed conflicting variable with macro * remove wrong edits * integrated ML and added cleanup in case parsing is interrupted * disable data collection locking during v2 * cleanup stale ML locks; send updated chart variables during v2; add info to find stale locks * inject an END2 between repeated BEGIN2 from rrdset_done() * test: remove lockless single-threaded logic from dictionary and aral and apply the right acquire/release memory order to reference counters * more fine grained dictionary atomics * remove unecessary return values * pointer validation under NETDATA_DICTIONARY_VALIDATE_POINTERS * Revert "pointer validation under NETDATA_DICTIONARY_VALIDATE_POINTERS" This reverts commit 846cdf2713e2a7ee2ff797f38db11714228800e9. * Revert "remove unecessary return values" This reverts commit 8c87d30f4d86f0f5d6b4562cf74fe7447138bbff. * Revert "more fine grained dictionary atomics" This reverts commit 984aec4234a340d197d45239ff9a10fd479fcf3c. * Revert "test: remove lockless single-threaded logic from dictionary and aral and apply the right acquire/release memory order to reference counters" This reverts commit c460b3d0ad497d2641bd0ea1d63cec7c052e74e4. * Apply again "pointer validation under NETDATA_DICTIONARY_VALIDATE_POINTERS" while keeping the improved atomic operations. This reverts commit f158d009 * fix last commit * fix last commit again * optimizations in dbengine * do not send anomaly bit on non-supporting agents (send it when the INTERPOLATED capability is available) * break long empty-points-loops in rrdset_done() * decide page alignment on new page allocation, not on every point collected * create max size pages but no smaller than 1/3 * Fix compilation when --disable-ml is specified * Return false * fixes for NETDATA_LOG_REPLICATION_REQUESTS * added compile option NETDATA_WITHOUT_WORKERS_LATENCY * put timings in BEGIN2, SET2, END2 * isolate begin2 ml * revert repositioning data collection lock * fixed multi-threading of statistics * do not lookup dimensions all the time if they come in the same order * update used on iteration, not on every points; also do better error handling --------- Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
2023-02-07Fix agent build failure on FreeBSD 14.0 due to new tcpstat struct (#14446)Dim-P
* Fix agent build failure on FreeBSD 14.0 due to different tcpstat struct * Add comment with link to commit that introduced the bug
2023-02-06Updated w1sensor.chart.py (#14435)Martin Due
Changed regex to allow for negative temperatures. Previously negative temperatures did not show up in the chart.
2023-02-03Check on parents the microseconds delta sent by agents (#14422)Costa Tsaousis
check streaming BEGIN microseconds for negative values
2023-02-03Move under Developer in Learn (#14417)Chris Akritidis
Move to references in Learn
2023-02-03Reduce service exit (#14381)thiagoftsm
2023-02-02add help line to functions response (#14399)Costa Tsaousis
* add help line to functions response * re-arranged columns to make it more usefull and added the new fields for controlling values * renames and re-arrangements
2023-02-02Minor fixes for markdown links (#14415)Tasos Katsoulas
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
2023-02-02fix(cgroups.plugin): fix collecting full pressure stall time (#14410)Ilya Mashchenko
2023-02-02Covert our documentation links to GH absolute links (#14344)Tasos Katsoulas
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
2023-02-02feat(charts.d): add load usage (Watts) to nuts collector (#14407)Ilya Mashchenko
Fixes https://github.com/netdata/netdata/issues/14396
2023-02-02DBENGINE v2 - improvements part 12 (#14379)Costa Tsaousis
* parallel initialization of tiers * do not spawn multiple dbengine event loops * user configurable dbengine parallel initialization * size netdata based on the real cpu cores available on the system netdata runs, not on the system monitored * user configurable system cpus * move cpuset parsing to os.c/.h * fix replication of misaligned chart dimensions * give a different path to each tier thread * statically allocate the path into the initialization structure * use aral for reusing dbengine pages * dictionaries uses ARAL for fixed sized values * fix compilation without internal checks * journal v2 index uses aral * test to see judy allocations * judy allocations using aral * Add config option to select if dbengine will use direct I/O (default is yes) * V1 journafiles will use uv_fs_read instead of mmap (respect the direct I/O setting) * Remove sqlite3IsMemdb as it is unused * Fix compilation error when --disable-dbengine is used * use aral for dbengine work_cmds * changed aral API to support new features * pgc and mrg aral overheads * rrdeng opcodes using aral * better structuring and naming * dbegnine query handles using aral * page descriptors using aral * remove obsolete linking * extent io descriptors using aral * aral keeps one last page alive * add missing return value * added judy aral overhead * pdc now uses aral * page_details now use aral * epdl and deol using aral - make sure ARALs are initialized before spawning the event loop * remove unused linking * pgc now uses one aral per partition * aral measure maximum allocation queue * aral to allocate pages in parallel * aral parallel pages allocation when needed * aral cleanup * track page allocation and page population separately --------- Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
2023-02-01fix link to ebpf collector (#14405)Ilya Mashchenko
* fix link to ebpf collector * fix syntax
2023-02-01Add nvidia smi pci bandwidth percent collector (#14315)Van Phan Quang
Co-authored-by: Ilya Mashchenko <ilya@netdata.cloud>
2023-01-31Add missing entries in README.md (#14351)thiagoftsm
Fixes https://github.com/netdata/learn/issues/1298
2023-01-30DBENGINE v2 - improvements part 11 (#14337)Costa Tsaousis
* acquiring / releasing interface for metrics * metrics registry statistics * cleanup metrics registry by deleting metrics when they dont have retention anymore; do not double copy the data of pages to be flushed * print the tier in retention summary * Open files with buffered instead of direct I/O (test) * added more metrics stats and fixed unittest * rename writer functions to avoid confusion with refcounting * do not release a metric that is not acquired * Revert to use direct I/O on write -- use direct I/O on read as well * keep track of ARAL overhead and add it to the memory chart * aral full check via api * Cleanup * give names to ARALs and PGCs * aral improvements * restore query expansion to the future * prefer higher resolution tier when switching plans * added extent read statistics * smoother joining of tiers at query engine * fine tune aral max allocation size * aral restructuring to hide its internals from the rest of netdata * aral restructuring; addtion of defrag option to aral to keep the linked list sorted - enabled by default to test it * fully async aral * some statistics and cleanup * fix infinite loop while calculating retention * aral docs and defragmenting disabled by default * fix bug and add optimization when defragmenter is not enabled * aral stress test * aral speed report and documentation * added internal checks that all pages are full * improve internal log about metrics deletion * metrics registry uses one aral per partition * metrics registry aral max size to 512 elements per page * remove data_structures/README.md dependency --------- Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
2023-01-30Add collectors main readme to learn (#14370)Chris Akritidis
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-27 17:38:13 +0000