summaryrefslogtreecommitdiffstats
path: root/exporting/graphite
AgeCommit message (Collapse)Author
2022-09-26Fix warnings during compilation time on ARM (32 bits) (#13681)thiagoftsm
2022-09-07Obsolete RRDSET state (#13635)Costa Tsaousis
* move chart_labels to rrdset * rename chart_labels to rrdlabels * renamed hash_id to uuid * turned is_ar_chart into an rrdset flag * removed rrdset state * removed unused senders_connected member of rrdhost * removed unused host flag RRDHOST_FLAG_MULTIHOST * renamed rrdhost host_labels to rrdlabels * Update exporting unit tests Co-authored-by: Vladimir Kobal <vlad@prokk.net>
2022-09-05Deduplicate all netdata strings (#13570)Costa Tsaousis
* rrdfamily * rrddim * rrdset plugin and module names * rrdset units * rrdset type * rrdset family * rrdset title * rrdset title more * rrdset context * rrdcalctemplate context and removal of context hash from rrdset * strings statistics * rrdset name * rearranged members of rrdset * eliminate rrdset name hash; rrdcalc chart converted to STRING * rrdset id, eliminated rrdset hash * rrdcalc, alarm_entry, alert_config and some of rrdcalctemplate * rrdcalctemplate * rrdvar * eval_variable * rrddimvar and rrdsetvar * rrdhost hostname, os and tags * fix master commits * added thread cache; implemented string_dup without locks * faster thread cache * rrdset and rrddim now use dictionaries for indexing * rrdhost now uses dictionary * rrdfamily now uses DICTIONARY * rrdvar using dictionary instead of AVL * allocate the right size to rrdvar flag members * rrdhost remaining char * members to STRING * * better error handling on indexing * strings now use a read/write lock to allow parallel searches to the index * removed AVL support from dictionaries; implemented STRING with native Judy calls * string releases should be negative * only 31 bits are allowed for enum flags * proper locking on strings * string threading unittest and fixes * fix lgtm finding * fixed naming * stream chart/dimension definitions at the beginning of a streaming session * thread stack variable is undefined on thread cancel * rrdcontext garbage collect per host on startup * worker control in garbage collection * relaxed deletion of rrdmetrics * type checking on dictfe * netdata chart to monitor rrdcontext triggers * Group chart label updates * rrdcontext better handling of collected rrdsets * rrdpush incremental transmition of definitions should use as much buffer as possible * require 1MB per chart * empty the sender buffer before enabling metrics streaming * fill up to 50% of buffer * reset signaling metrics sending * use the shared variable for status * use separate host flag for enabling streaming of metrics * make sure the flag is clear * add logging for streaming * add logging for streaming on buffer overflow * circular_buffer proper sizing * removed obsolete logs * do not execute worker jobs if not necessary * better messages about compression disabling * proper use of flags and updating rrdset last access time every time the obsoletion flag is flipped * monitor stream sender used buffer ratio * Update exporting unit tests * no need to compare label value with strcmp * streaming send workers now monitor bandwidth * workers now use strings * streaming receiver monitors incoming bandwidth * parser shift of worker ids * minor fixes * Group chart label updates * Populate context with dimensions that have data * Fix chart id * better shift of parser worker ids * fix for streaming compression * properly count received bytes * ensure LZ4 compression ring buffer does not wrap prematurely * do not stream empty charts; do not process empty instances in rrdcontext * need_to_send_chart_definition() does not need an rrdset lock any more * rrdcontext objects are collected, after data have been written to the db * better logging of RRDCONTEXT transitions * always set all variables needed by the worker utilization charts * implemented double linked list for most objects; eliminated alarm indexes from rrdhost; and many more fixes * lockless strings design - string_dup() and string_freez() are totally lockless when they dont need to touch Judy - only Judy is protected with a read/write lock * STRING code re-organization for clarity * thread_cache improvements; double numbers precision on worker threads * STRING_ENTRY now shadown STRING, so no duplicate definition is required; string_length() renamed to string_strlen() to follow the paradigm of all other functions, STRING internal statistics are now only compiled with NETDATA_INTERNAL_CHECKS * rrdhost index by hostname now cleans up; aclk queries of archieved hosts do not index hosts * Add index to speed up database context searches * Removed last_updated optimization (was also buggy after latest merge with master) Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com> Co-authored-by: Vladimir Kobal <vlad@prokk.net>
2022-07-11Exporting/send variables (#13221)boxjan
2022-06-29Get rid of extra semicolon in Graphite exporting (#13261)Vladimir Kobal
2022-06-28netdata doubles (#13217)Costa Tsaousis
* netdata doubles * fix cmocka test * fix cmocka test again * fix left-overs of long double to NETDATA_DOUBLE * RRDDIM detached from disk representation; db settings in [db] section of netdata.conf * update the memory before saving * rrdset is now detached from file structures too * on memory mode map, update the memory mapped structures on every iteration * allow RRD_ID_LENGTH_MAX to be changed * granularity secs, back to update every * fix formatting * more formatting
2022-06-13Labels with dictionary (#13070)Costa Tsaousis
* squashed and rebased to master * fix overflow and single character bug in sanitize; include rrd.h instead of node_info.h * added unittest for UTF-8 multibyte sanitization * Fix unit test compilation * Fix CMake build * remove double sanitizer for opentsdb; cleanup sanitize_json_string() * rename error_description to error_message to avoid conflict with json-c * revert last and undef error_description from json-c * more unittests; attempt to fix protobuf map issue * get rid of rrdlabels_get() and replace it with a safe version that writes the value to a buffer * added dictionary sorting unittest; rrdlabels_to_buffer() now is sorted * better sorted dictionary checking * proper unittesting for sorted dictionaries * call dictionary deletion callback when destroying the dictionary * remove obsolete variable * Fix exporting unit tests * Fix k8s label parsing test * workaround for cmocka and strdupz() * Bypass cmocka memory allocation check * Revert "Bypass cmocka memory allocation check" This reverts commit 4c49923839d9229bea23ca914dd8a0be1ebe2bf4. * Revert "workaround for cmocka and strdupz()" This reverts commit 7bebee04801db1865c748a7896d5fa54bb7104a5. * Bypass cmocka memory allocation checks * respect json formatting for chart labels * cloud sends colons * print the value only once * allow parenthesis in values and spaces; make stream sender send quotes for values Co-authored-by: Vladimir Kobal <vlad@prokk.net>
2022-02-17Docs: Removed Google Analytics tags (#12145)Tina Luedtke
2021-08-04Add HTTP basic authentication to some exporting connectors (#11394)Vladimir Kobal
2021-04-14Spelling exporting (#10915)Josh Soref
2020-12-14Kubernetes labels (#10107)Ilya Mashchenko
Co-authored-by: Markos Fountoulakis <markos.fountoulakis.senior@gmail.com> Co-authored-by: Vladimir Kobal <vlad@prokk.net>
2020-12-14Fix hostname configuration in the exporting engine (#10361)Vladimir Kobal
2020-11-08Fix repeated frontmatter (#10211)Joel Hans
2020-11-05Add HTTP and HTTPS support to the simple exporting connector (#9911)Vladimir Kobal
2020-09-29Add docsv2 project to master branch (#10000)Joel Hans
* Add overview docs to docsv2 project * Add quickstart docs to docsv2 project (#9865) * Init quickstart docs * Begin work on quickstart guides * Finish quickstart drafts * Tweaks to both quickstarts * Add titles * Copyedit pass to both docs * Fixes for Amy and Jen * Add Get doc to docsv2 project (#9854) * Init get file * Add some links * Change h2 to h1 * Rephrase * Add configure docs to docsv2 project (#9878) * Add overview docs to docsv2 project (#9849) * Init files * Add drafts of what and why * Finish initial drafts * Fix edit URL * Copyedit pass * Finish initial drafts of configure docs * Copyedit all docs * Fixes for Amy * Fixes for Jen * Add collect docs to the docsv2 project (#9932) * Init files * Finish first two collect docs * Finish drafts of collect docs * Copyedit pass * Fixes for Amy * Fix for Jen * Add visualize docs to the docsv2 project (#9943) * Add visualize docs * Copyedits and cleanup * New images and features * Copyedit pass and cleanup * Missing word * Fixes for Jen * Add monitor docs to docsv2 project (#9949) * Finish drafts of monitor docs * Copyedit pass * Remove whitespace * Fixes for Jen * Add store docs to docsv2 project (#9969) * Finalize store documents * Fix import path * Finishing edit section * Copyedit pass * Add export docs to docsv2 project (#9986) * Add install and claim videos to Get doc * Finish drafts of exporting docs plus other tweaks * Init new exporting READMEs * Copyedit pass and new links * Fixes for Amy, Vlad, Jen * Fix links in docsv2 project (#9993) * Fix links * Fix a bunch of links ahead of export merge * Fix additional links * Fix links, nuke what-is-netdata * Fixing a few last links * Improve product images in overview * Remove extra paren * Quick tweaks for Jen * Fixes for Jen * Access fix * Remove extra word
2020-07-02Read the prefix option from instance config sections (#9463)Vladimir Kobal
2020-05-26Missing checks exporting (#9034)thiagoftsm
Add tests to uv_mutex_init and uv_cond_init inside exporting.
2020-05-25Exporting cleanup (#9098)thiagoftsm
Cleanup allocated variables for the majority of the databases.
2020-05-14Fix wrong hostnames in the exporting engine (#8892)Vladimir Kobal
2020-04-13Revert "Revert changes since v1.21 in pereparation for hotfix release."Austin S. Hemmelgarn
This reverts commit e2874320fc027f7ab51ab3e115d5b1889b8fd747.
2020-04-13Revert changes since v1.21 in pereparation for hotfix release.Austin S. Hemmelgarn
2020-04-10Show internal stats for the exporting engine (#8635)Vladimir Kobal
* Add a print function for internal exporting statistics * Send statistics for simple connectors * Flush sending buffers on failures * Send statistics for the Kinesis connector * Send statistics for the MongoDB connector * Add unit tests
2020-03-12Add a Prometheus Remote Write connector to the exporting engine (#8292)Vladimir Kobal
* Copy files from the Prometheus remote write backend * Update the documentation * Rename backend -> exporting * Add the connector to the Netdata build * Separate files for the remote write connector * Add an initializer and formatters * Read a connector specific configuration option * Add a separate function for header sending * Use labels instead of tags * Separate write request for every instance * Add unit tests
2020-02-25Add an AWS Kinesis connector to the exporting engine (#8145)Vladimir Kobal
* Prepare files for the AWS Kinesis exporting connector * Update the documentation * Rename functions in backends * Include the connector to the Netdata buid * Add initializers and a worker * Add Kinesis specific configuration options * Add a compile time configuration check * Remove the connector data structure * Restore unit tests * Fix the compile-time configuration check * Initialize AWS SDK only once * Don't create an instance for an unknown exporting connector * Separate client and request outcome data for every instance * Fix memory cleanup, document functions * Add unit tests * Update the documentation
2020-02-01Parse host tags (#7702)Vladimir Kobal
* Fix memory leaks * Check for configuration options * Parse simple tags * Parse JSON tags * Remove an unnecessary check * Parse a JSON object * Parse a JSON array * Update the documentation * Fix host locks
2020-01-09Send host labels via exporting connectors (#7554)Vladimir Kobal
* Add labels to the JSON exporting connector * Add labels to the Graphite exporting connector * Add labels to the OpenTSDB telnet exporting connector * Add labels to the OpenTSDB HTTP exporting connector * Replace control characters in JSON strings * Add unit tests
2019-12-12Implement the main flow for the Exporting Engine (#7149)Vladimir Kobal
* Add top level tests * Add a skeleton for preparing buffers * Initialize graphite instance * Prepare buffers for all instances * Add Grafite collected value formatter * Add support for exporting.conf read and parsing * - Use new exporting_config instead of netdata_config * Implement Grafite worker * Disable exporting engine compilation if libuv is not available * Add mutex locks - Configure connectors as connector_<type> in sections of exporting.conf - Change exporting_select_type to check for connector_ fields * - Override exporting_config structure if there no exporting.conf so that look ups don't fail and we maintain backwards compatibility * Separate fixtures in unit tests * Test exporting_discard_responce * Test response receiving * Test buffer sending * Test simple connector worker - Instance section has the format connector:instance_name e.g graphite:my_graphite_instance - Connectors with : in their name e.g graphite:plaintext are reserved So graphite:plaintext is not accepted because it would activate an instance with name "plaintext" It should be graphite:plaintext:instance_name * - Enable the add_connector_instance to cleanup the internal structure by passing NULL,not NULL arguments * Implement configurable update interval - Add additional check to verify instance uniqueness across connectors * Add host and chart filters * Add the value calculation over a database series * Add the calculated over stored data graphite connector * Add tests for graphite connector * Add JSON connector * Add tests for JSON formatting functions * Add OpenTSDB connector * Add tests for the OpenTSDB connector * Add temporaty notes to the documentation