summaryrefslogtreecommitdiffstats
path: root/collectors
AgeCommit message (Collapse)Author
2023-03-17Add ethtool in third party collectors (#14753)Van Phan Quang
2023-03-15pandas collector replace `self.warn()` with `self.warning()` (#14736)Andrew Maguire
2023-03-14add go.d example collector to #etc section (#14722)Andrew Maguire
* add go.d example collector to #etc section * type
2023-03-13Change H1 of collector docs to separate from the website (#14715)Chris Akritidis
2023-03-10/api/v2/X improvements part 3 (#14665)Costa Tsaousis
* max web request size to 64KB * fix the request too big message * increase max request reading tries to 100 * support for bigger web requests * add "avg" as a shortcut for "average" to both group by aggregation and time aggregation; discard the last partial points of a query in play mode, up to max update every; group by hidden dimensions too * better implementation for partial data trimming * added group_by=selected to return only one dimension for all selected metrics * fix acceptance of group_by=selected * passing option "raw" disables partial data trimming * remove obsolete option "plan"; use "debug" * fix view.min and view.max calculation - there were 2 bugs: a) min and max were reset for every row and b) min and max were corrupted by GBC and AR printing * per row annotations * added time column to point annotations * disable caching for /api/v2/contexts responses * added api format json2 that returns an array for each points, having all the point values and annotations in them * work on swagger about /api/v2 * prevent infinite loop * cleanup and swagger work * allow negative simple pattern expressions to work as expected * do not lookup in the dictionary empty names * garbage collect dictionaries * make query_target allocate less aggressively; queries fill the remaining points with nulls * reusable query ops to save memory on huge queries * move parts of query plans into query ops to save query target memory * remove storage engine from query metric tiers, to save memory, and recalculate it when it is needed
2023-03-10Refactor ML code. (#14659)vkalintiris
* Refactor ML code. This commit introduces only non-functional changes. Originally, the C++ code exposed C functions to be called from the rest of the agent. When we migrated from C++ to C, we did not eliminate these wrapper functions to make the PR easier to understand and keep the total LOC low. This commit removes the wrapper functions and "reclaims" the `ml_` prefix that we used for the public API of the old implementation. Also, the nlohmann Json library has been removed and its functionality was replaced with the equivalent Json functionality that we added in libnetdata's BUFFERs. * Remove missing headers from build systems. * Fix CMake build. * rrddim_free is outside of rrd "internals" now.
2023-03-08Send an EOF from charts.d.plugin before exit (#14680)Emmanuel Vasilakis
* send an EOF before exit * use empty * make an EXIT keyword
2023-03-08Fix cloud node stale status when a virtual host is created (#14660)Stelios Fragkakis
* Schedule direct metadata update on host creation Virtual hosts do not have a receiver but they are not orphan Schedule node info update on host activation New function to store host info and host_system_info If the host is just created, create tables and sync thread If the host exists during startup it is not live but reschedule node update if it is reactivated * New opcode to send current node state * Remove debug messages * Fix system host info
2023-03-08eBPF new charts (user ring) (#14623)thiagoftsm
2023-03-03fix simple_pattern_create on freebsd (#14656)Ilya Mashchenko
2023-03-02Fix doc links (#14650)Chris Akritidis
* Update freebsd.md * Update REFERENCE.md * Update README.md * Update COLLECTORS.md
2023-03-02/api/v2/contexts (#14592)Costa Tsaousis
* preparation for /api/v2/contexts * working /api/v2/contexts * add anomaly rate information in all statistics; when sum-count is requested, return sums and counts instead of averages * minor fix * query targegt now accurately counts hosts, contexts, instances, dimensions, metrics * cleanup /api/v2/contexts * full text search with /api/v2/contexts * simple patterns now support the option to search ignoring case * full text search API with /api/v2/q * simple pattern execution optimization * do not show q when not given * full text search accounting * separated /api/v2/nodes from /api/v2/contexts * fix ssv queries for group_by * count query instances queried and failed per context and host * split rrdcontext.c to multiple files * add query totals * fix anomaly rate calculation; provide "ni" for indexing hosts * do not generate zero valued members * faster calculation of anomaly rate; by just summing integers for each db points and doing math once for every generated point * fix typo when printing dimensions totals * added option minify to remove spaces and newlines fron JSON output * send instance ids and names when they differ * do not add in query target dimensions, instances, contexts and hosts for which there is no retention in the current timeframe * fix for the previous + renames and code cleanup * when a dimension is filtered, include in the response all the other dimensions that are selectable * do not add nodes that do not have retention in the current window * move selection of dimensions to query_dimension_add(), instead of query_metric_add() * increase the pre-processing capacity of queries * generate instance fqdn ids and names only when they are needed * provide detailed statistics about tiers retention, queries, points, update_every * late allocation of query dimensions * cleanup * more cleanup * support for annotations per displayed point, RESET and PARTIAL * new type annotations * if a chart is not linked to contexts and it is collected, link it when it is collected * make ML run reentrant * make ML rrdr query synchronous * optimize replication memory allocation of replication_sort_entry * change units to percentage, when requesting a coefficinet of variation, or a percentage query * initialize replication before starting main threads * properly decrement no room requests counter * propagate the non-zero flag to group-by * the same by avoiding the extra loop * respect non-zero in all dimension arrays * remove dictionary garbage collection from dictionary_entries() and dictionary_version() * be more verbose when jv2 indexing is postponed * prevent infinite loop * use hidden dimensions even when dimensions pattern is unset * traverse hosts using dictionaries * fix dictionary unittests
2023-03-02Update README.md (#14647)Chris Akritidis
2023-02-28Make the title metadata H1 in all markdown files (#14625)Fotis Voutsas
* make the title metadta the H1 * Update collectors/python.d.plugin/zscores/README.md * Update libnetdata/ebpf/README.md * Update ml/README.md * Update libnetdata/string/README.md --------- Co-authored-by: Chris Akritidis <43294513+cakrit@users.noreply.github.com>
2023-02-28fix python version check to work for 3.10 and above (#14616)Andrew Maguire
2023-02-27Use vector allocation whenever is possible (eBPF) (#14591)thiagoftsm
2023-02-25Fix links to chart interactions (#14609)Chris Akritidis
2023-02-24Fix broken links (#14605)Fotis Voutsas
* fix broken links * fix a link to kernel-collector readme, make it to not point at .md
2023-02-22Change link text to collectors.md (#14590)Chris Akritidis
* Change link text to collectors.md * Change doc title
2023-02-21Memory management eBPF (#14462)thiagoftsm
2023-02-21Fix coverity issues (#14543)Stelios Fragkakis
* Fix coverity 383236: Resource leak * Fix coverity 382915 : Logically dead code * Fix coverity 379133 : Division or modulo by float zero * Fix coverity 382783 : Copy into fixed size buffer * Fix coverity 381151 : Missing unlock * Fix coverity 381903 : Dereference after null check
2023-02-20pandas collector add `read_sql()` support (#14563)Andrew Maguire
* import sqlalchemy and os to support read_sql
2023-02-20bump go.d to v0.51.0 (#14572)Ilya Mashchenko
2023-02-20Fix doc capitalization and remove obsolete section (#14569)Chris Akritidis
2023-02-20Include troubleshooting section in the docs of python.d modules (#13721)Tasos Katsoulas
Add troubleshooting sections to all python modules Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
2023-02-20Fix broken links in our documentation (#14565)Fotis Voutsas
* fix broken link in ml/README.md * fix broken link across all files * fix broken link across all files * fix broken links and remove what's next sections * fix broken links and remove what's next section * Remove related links sections with broken links that link to removed files * fix broken links
2023-02-17Reorg learn 021723 (#14556)Chris Akritidis
* Change titles of agent alert notifications * Reintroduce netdata for iot * Eliminate guides category, merge health config docs * Rename setup to configuration * Codacy fixes and move health config reference
2023-02-17More reorg learn 021623 (#14550)Chris Akritidis
* Moved contents of statsd guide inside the plugin documentation. * Remove remaining guides, content verified to exist elsewhere
2023-02-16Update learn path of python plugin readme (#14549)Chris Akritidis
2023-02-16Reorg markdown files for learn (#14547)Chris Akritidis
* Move export graphite metrics guide content to new integrations landing page and exporter readme * Merge info on how to write python collector and reorg file * Codacy warning fixes
2023-02-15JSON internal API, IEEE754 base64/hex streaming, weights endpoint ↵Costa Tsaousis
optimization (#14493) * first work on standardizing json formatting * renamed old grouping to time_grouping and added group_by * add dummy functions to enable compilation * buffer json api work * jsonwrap opening with buffer_json_X() functions * cleanup * storage for quotes * optimize buffer printing for both numbers and strings * removed ; from define * contexts json generation using the new json functions * fix buffer overflow at unit test * weights endpoint using new json api * fixes to weights endpoint * check buffer overflow on all buffer functions * do synchronous queries for weights * buffer_flush() now resets json state too * content type typedef * print double values that are above the max 64-bit value * str2ndd() can now parse values above UINT64_MAX * faster number parsing by avoiding double calculations as much as possible * faster number parsing * faster hex parsing * accurate printing and parsing of double values, even for very large numbers that cannot fit in 64bit integers * full printing and parsing without using library functions - and related unit tests * added IEEE754 streaming capability to enable streaming of double values in hex * streaming and replication to transfer all values in hex * use our own str2ndd for set2 * remove subnormal check from ieee * base64 encoding for numbers, instead of hex * when increasing double precision, also make sure the fractional number printed is aligned to the wanted precision * str2ndd_encoded() parses all encoding formats, including integers * prevent uninitialized use * /api/v1/info using the new json API * Fix error when compiling with --disable-ml * Remove redundant 'buffer_unittest' declaration * Fix formatting * Fix formatting * Fix formatting * fix buffer unit test * apps.plugin using the new JSON API * make sure the metrics registry does not accept negative timestamps * do not allow pages with negative timestamps to be loaded from db files; do not accept pages with negative timestamps in the cache * Fix more formatting --------- Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
2023-02-15Reorganize learn documents under Integrations part 2 (#14538)Chris Akritidis
* Reorge exporter integrations * Reorg alert notifications. change mdx to md for alert related files * Move all .mdx to .md, including links.
2023-02-14Fix broken links in markdown files (#14513)Fotis Voutsas
* fix broken links in claim/README.md * delete broken link in docs/guidelines.md * fix broken links * fix broken link * fix broken links * fix broken links * fix broken links * fix broken links * fix broken links * remove broken link * fix broken link * fix broken links * fix broken links * fix broken links * fix broken link * fix linking phrasing * fix broken links batch * fix broken links second batch * fix broken links * fix broken links * fix broken links * Update COLLECTORS.md * fix broken links * fix broken links
2023-02-12Make external plugins a category page in learn (#14511)Chris Akritidis
Make external plugins a category page
2023-02-12Learn integrations category changes (#14510)Chris Akritidis
* Better present netdata pandas and fix how to add new charts to netdata * Properly categorize statsd and show it as a generic collector. Move asterisk and k6 Elevate anomaly detection to "monitor anything"
2023-02-12Move collectors under Integrations/Monitoring (#14509)Chris Akritidis
* Move collectors under Integrations/Monitoring * Correct Monitoring to Monitor
2023-02-12Guides and collectors reorg and cleanup part 1 (#14507)Chris Akritidis
* rm github star important * Remove some guides and rework the data collection docs * link fixes on collectors/README.md * link fixes on collectors/REFERENCE.md * Fix codacy * More codacy list formatting
2023-02-10Fix broken links in collectors/COLLECTORS.md (#14502)Fotis Voutsas
* Publish collectors/python.d.plugin/anomalies/README.md to Learn * Update README.md * fix broken links in COLLECTORS.md
2023-02-09Virtual hosts for data collection (#14464)Costa Tsaousis
* support multiple hosts at pluginsd structures * cleanup obsolete code * use a lookup hashtable to quickly find the keyword to execute, without traversing the whole linked list of keywords * more cleanup * move new hash function to inlined.h * minimize comparisons, eliminate a pre-parsing of the first keyword for each line * cleanup parser from old code * move parser into libnetdata * unique entries in parser keywords hashtable * move all hashing functions to inlined.h, name their sources, simple_hash() now defaults to FNV1a, it was FNV1 * small_hash() for parser * plugins.d now can switch hosts, and also create/update them * update hash function and hashtable size * updated message * unittest all hashing functions * reset the chart when setting a new host * remove host tags * enable archived hosts when a collector pushes host info * do not need localhost to swtich to localhost * disable ARAL and OWA with -DFSANITIZE_ADDRESS=1
2023-02-08Fix random crash on agent shutdown (#14470)Stelios Fragkakis
Check MMAP failure. If fail to create semaphore, clear the shm_cgroup_ebpf.header to avoid crash during shutdown on thread cleanup
2023-02-08Add markdown files in Learn (#14466)Fotis Voutsas
* add metadata for learn * first batch of adding metadata to md files * second batch of adding metadata to md files * third batch of adding metadata to md files * test one sidebar_label * add missing sidebar_labels * add missing sidebar_labels to files left behind * test, ansible doc is stubborn * fix * fix * fix * don't use questionmarks in the sidebar label * don't use exclamation marks and symbols in the sidebar label * fix style guide * fixes * fixes
2023-02-08remove deprecated python.d collectors announced in v1.38.0 (#14454)Ilya Mashchenko
2023-02-07Streaming interpolated values (#14431)Costa Tsaousis
* first commit - untested * fix wrong begin command * added set v2 too * debug to log stream buffer * debug to log stream buffer * faster streaming printing * mark charts and dimensions as collected * use stream points even if sender is not enabled * comment out stream debug log * parse null as nan * custom begin v2 * custom set v2; replication now copies the anomalous flag too * custom end v2 * enabled stream log test * renamed to BEGIN2, SET2, END2 * dont mix up replay and v2 members in user object * fix typo * cleanup * support to v2 to v1 proxying * mark updated dimensions as such * do not log unknown flags * comment out stream debug log * send also the chart id on BEGIN2, v2 to v2 * update the data collections counter * v2 values are transferred in hex * faster hex parsing * a little more generic hex and dec printing and parsing * fix hex parsing * minor optimization in dbengine api * turn debugging into info message * generalized the timings tracking, so that it can be used in more places * commented out debug info * renamed conflicting variable with macro * remove wrong edits * integrated ML and added cleanup in case parsing is interrupted * disable data collection locking during v2 * cleanup stale ML locks; send updated chart variables during v2; add info to find stale locks * inject an END2 between repeated BEGIN2 from rrdset_done() * test: remove lockless single-threaded logic from dictionary and aral and apply the right acquire/release memory order to reference counters * more fine grained dictionary atomics * remove unecessary return values * pointer validation under NETDATA_DICTIONARY_VALIDATE_POINTERS * Revert "pointer validation under NETDATA_DICTIONARY_VALIDATE_POINTERS" This reverts commit 846cdf2713e2a7ee2ff797f38db11714228800e9. * Revert "remove unecessary return values" This reverts commit 8c87d30f4d86f0f5d6b4562cf74fe7447138bbff. * Revert "more fine grained dictionary atomics" This reverts commit 984aec4234a340d197d45239ff9a10fd479fcf3c. * Revert "test: remove lockless single-threaded logic from dictionary and aral and apply the right acquire/release memory order to reference counters" This reverts commit c460b3d0ad497d2641bd0ea1d63cec7c052e74e4. * Apply again "pointer validation under NETDATA_DICTIONARY_VALIDATE_POINTERS" while keeping the improved atomic operations. This reverts commit f158d009 * fix last commit * fix last commit again * optimizations in dbengine * do not send anomaly bit on non-supporting agents (send it when the INTERPOLATED capability is available) * break long empty-points-loops in rrdset_done() * decide page alignment on new page allocation, not on every point collected * create max size pages but no smaller than 1/3 * Fix compilation when --disable-ml is specified * Return false * fixes for NETDATA_LOG_REPLICATION_REQUESTS * added compile option NETDATA_WITHOUT_WORKERS_LATENCY * put timings in BEGIN2, SET2, END2 * isolate begin2 ml * revert repositioning data collection lock * fixed multi-threading of statistics * do not lookup dimensions all the time if they come in the same order * update used on iteration, not on every points; also do better error handling --------- Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
2023-02-07Fix agent build failure on FreeBSD 14.0 due to new tcpstat struct (#14446)Dim-P
* Fix agent build failure on FreeBSD 14.0 due to different tcpstat struct * Add comment with link to commit that introduced the bug
2023-02-06Updated w1sensor.chart.py (#14435)Martin Due
Changed regex to allow for negative temperatures. Previously negative temperatures did not show up in the chart.
2023-02-03Check on parents the microseconds delta sent by agents (#14422)Costa Tsaousis
check streaming BEGIN microseconds for negative values
2023-02-03Move under Developer in Learn (#14417)Chris Akritidis
Move to references in Learn
2023-02-03Reduce service exit (#14381)thiagoftsm
2023-02-02add help line to functions response (#14399)Costa Tsaousis
* add help line to functions response * re-arranged columns to make it more usefull and added the new fields for controlling values * renames and re-arrangements
2023-02-02Minor fixes for markdown links (#14415)Tasos Katsoulas
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>