| Age | Commit message (Collapse) | Author |
|---|
|
* type checking on dictionary return values
* first STRING implementation, used by DICTIONARY and RRDLABEL
* enable AVL compilation of STRING
* Initial functions to store context info
* Call simple test functions
* Add host_id when getting charts
* Allow host to be null and in this case it will process the localhost
* Simplify init
Do not use strdupz - link directly to sqlite result set
* Init the database during startup
* make it compile - no functionality yet
* intermediate commit
* intermidiate
* first interface to sql
* loading instances
* check if we need to update cloud
* comparison of rrdcontext on conflict
* merge context titles
* rrdcontext public interface; statistics on STRING; scratchpad on DICTIONARY
* dictionaries maintain version numbers; rrdcontext api
* cascading changes
* first operational cleanup
* string unittest
* proper cleanup of referenced dictionaries
* added rrdmetrics
* rrdmetric starting retention
* Add fields to context
Adjuct context creation and delete
* Memory cleanup
* Fix get context list
Fix memory double free in tests
Store context with two hosts
* calculated retention
* rrdcontext retention with collection
* Persist database and shutdown
* loading all from sql
* Get chart list and dimension list changes
* fully working attempt 1
* fully working attempt 2
* missing archived flag from log
* fixed archived / collected
* operational
* proper cleanup
* cleanup - implemented all interface functions - dictionary react callback triggers after the dictionary is unlocked
* track all reasons for changes
* proper tracking of reasons of changes
* fully working thread
* better versioning of contexts
* fix string indexing with AVL
* running version per context vs hub version; ifdef dbengine
* added option to disable rrdmetrics
* release old context when a chart changes context
* cleanup properly
* renamed config
* cleanup contexts; general cleanup;
* deletion inline with dequeue; lots of cleanup; child connected/disconnected
* ml should start after rrdcontext
* added missing NULL to ri->rrdset; rrdcontext flags are now only changed under a mutex lock
* fix buggy STRING under AVL
* Rework database initialization
Add migration logic to the context database
* fix data race conditions during context deletion
* added version hash algorithm
* fix string over AVL
* update aclk-schemas
* compile new ctx related protos
* add ctx stream message utils
* add context messages
* add dummy rx message handlers
* add the new topics
* add ctx capability
* add helper functions to send the new messages
* update cmake build to not fail
* update topic names
* handle rrdcontext_enabled
* add more functions
* fatal on OOM cases instead of return NULL
* silence unknown query type error
* fully working attempt 1
* fully working attempt 2
* allow compiling without ACLK
* added family to the context
* removed excess character in UUID
* smarter merging of titles and families
* Database migration code to add family
Add family to SQL_CHART_DATA and VERSIONED_CONTEXT_DATA
* add family to context message
* enable ctx in communication
* hardcoded enabled contexts
* Add hard code for CTX
* add update node collectors to json
* add context message log
* fix log about last_time_t
* fix collected flags for queued items
* prevent crash on charts cleanup
* fix bug in AVL indexing of dictionaries; make sure react callback of dictionaries has a reference counter, which is acquired while the dictionary is locked
* fixed dictionary unittest
* strict policy to cleanup and garbage collector
* fix db rotation and garbage collection timings
* remove deadlock
* proper garbage collection - a lot faster retention recalculation
* Added not NULL in database columns
Remove migration code for context -- we will ship with version 1 of the table schema
Added define for query in tests to detect localhost
* Use UUID_STR_LEN instead of GUID_LEN + 1
Use realistic timestamps when adding test data in the database
* Add NULL checks for passed parameters
* Log deleted context when compiled with NETDATA_INTERNAL_CHECKS
* Error checking for null host id
* add missing ContextsCheckpoint log convertor
* Fix spelling in VACCUM
* Hold additional information for host -- prepare to load archived hosts on startup
* Make sure claim id is valid
* is_get_claimed is actually get the current claim id
* Simplify ctx get chart list query
* remove env negotiation
* fix string unittest when there are some strings already in the index
* propagate live-retention flag upstream; cleanup all update reasons; updated instances logging; automated attaching started/stopped collecting flags;
* first implementation of /api/v1/contexts
* full contexts API; updated swagger
* disabled debugging; rrdcontext enabled by default
* final cleanup and renaming of global variables
* return current time on currently collected contexts, charts and dimensions
* added option "deepscan" to the API to have the server refresh the retention and recalculate the contexts on the fly
* fixed identation of yaml
* Add constrains to the host table
* host->node_id may not be available
* new capabilities
* lock the context while rendering json
* update aclk-schemas
* added permanent labels to all charts about plugin, module and family; added labels to all proc plugin modules
* always add the labels
* allow merging of families down to [x]
* dont show uuids by default, added option to enable them; response is now accepting after,before to show only data for a specific timeframe; deleted items are only shown when "deleted" is requested; hub version is now shown when "queue" is requested
* Use the localhost claim id
* Fix to handle host constrains better
* cgroups: add "k8s." prefix to chart context in k8s
* Improve sqlite metadata version migration check
* empty values set to "[none]"; fix labels unit test to reflect that
* Check if we reached the version we want first (address CODACY report re: Array index 'i' is used before limits check)
* Rewrite condition to address CODACY report (Redundant condition: t->filter_callback. '!A || (A && B)' is equivalent to '!A || B')
* Properly unlock context
* fixed memory leak on rrdcontexts - it was not freeing all dictionaries in rrdhost; added wait of up to 100ms on dictionary_destroy() to give time to dictionaries to release their items before destroying them
* fixed memory leak on rrdlabels not freed on rrdinstances
* fixed leak when dimensions and charts are redefined
* Mark entries for charts and dimensions as submitted to the cloud 3600 seconds after their creation
Mark entries for charts and dimensions as updated (confirmed by the cloud) 1800 seconds after their submission
* renamed struct string
* update cgroups alarms
* fixed codacy suggestions
* update dashboard info
* fix k8s_cgroup_10s_received_packets_storm alarm
* added filtering options to /api/v1/contexts and /api/v1/context
* fix eslint
* fix eslint
* Fix pointer binding for host / chart uuids
* Fix cgroups unit tests
* fixed non-retention updates not propagated upstream
* removed non-fatal fatals
* Remove context from 2 way string merge.
* Move string_2way_merge to dictionary.c
* Add 2-way string merge tests.
* split long lines
* fix indentation in netdata-swagger.yaml
* update netdata-swagger.json
* yamllint please
* remove the deleted flag when a context is collected
* fix yaml warning in swagger
* removed non-fatal fatals
* charts should now be able to switch contexts
* allow deletion of unused metrics, instances and contexts
* keep the queued flag
* cleanup old rrdinstance labels
* dont hide objects when there is no filter; mark objects as deleted when there are no sub-objects
* delete old instances once they changed context
* delete all instances and contexts that do not have sub-objects
* more precise transitions
* Load archived hosts on startup (part 1)
* update the queued time every time
* disable by default; dedup deleted dimensions after snapshot
* Load archived hosts on startup (part 2)
* delayed processing of events until charts are being collected
* remove dont-trigger flag when object is collected
* polish all triggers given the new dont_process flag
* Remove always true condition
Enums for readbility / create_host_callback only if ACLK is enabled (for now)
* Skip retention message if context streaming is enabled
Add messages in the access log if context streaming is enabled
* Check for node id being a UUID that can be parsed
Improve error check / reporting when loading archived hosts and creating ACLK sync threads
* collected, archived, deleted are now mutually exclusive
* Enable the "orphan" handling for now
Remove dead code
Fix memory leak on free host
* Queue charts and dimensions will be no-op if host is set to stream contexts
* removed unused parameter and made sure flags are set on rrdcontext insert
* make the rrdcontext thread abort mid-work when exiting
* Skip chart hash computation and storage if contexts streaming is enabled
Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
Co-authored-by: Timo <timotej@netdata.cloud>
Co-authored-by: ilyam8 <ilya@netdata.cloud>
Co-authored-by: Vladimir Kobal <vlad@prokk.net>
Co-authored-by: Vasilis Kalintiris <vasilis@netdata.cloud>
|
|
Currently, we add the repository's top-level dir in the compiler's
header search path. This means that code in every top-level directory
within the repo can include headers sibling top-level directories.
This patch makes header inclusion consistent when it comes to files
that are included from sibling top-level directories within the repo.
|
|
This PR merges the feature-branch to make the cloud live. It contains the following work:
Co-authored-by: Andrew Moss <1043609+amoss@users.noreply.github.com(opens in new tab)>
Co-authored-by: Jacek Kolasa <jacek.kolasa@gmail.com(opens in new tab)>
Co-authored-by: Austin S. Hemmelgarn <austin@netdata.cloud(opens in new tab)>
Co-authored-by: James Mills <prologic@shortcircuit.net.au(opens in new tab)>
Co-authored-by: Markos Fountoulakis <44345837+mfundul@users.noreply.github.com(opens in new tab)>
Co-authored-by: Timotej S <6674623+underhood@users.noreply.github.com(opens in new tab)>
Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com(opens in new tab)>
* dashboard with new navbars, v1.0-alpha.9: PR #8478
* dashboard v1.0.11: netdata/dashboard#76
Co-authored-by: Jacek Kolasa <jacek.kolasa@gmail.com(opens in new tab)>
* Added installer code to bundle JSON-c if it's not present. PR #8836
Co-authored-by: James Mills <prologic@shortcircuit.net.au(opens in new tab)>
* Fix claiming config PR #8843
* Adds JSON-c as hard dep. for ACLK PR #8838
* Fix SSL renegotiation errors in old versions of openssl. PR #8840. Also - we have a transient problem with opensuse CI so this PR disables them with a commit from @prologic.
Co-authored-by: James Mills <prologic@shortcircuit.net.au(opens in new tab)>
* Fix claiming error handling PR #8850
* Added CI to verify JSON-C bundling code in installer PR #8853
* Make cloud-enabled flag in web/api/v1/info be independent of ACLK build success PR #8866
* Reduce ACLK_STABLE_TIMEOUT from 10 to 3 seconds PR #8871
* remove old-cloud related UI from old dashboard (accessible now via /old suffix) PR #8858
* dashboard v1.0.13 PR #8870
* dashboard v1.0.14 PR #8904
* Provide feedback on proxy setting changes PR #8895
* Change the name of the connect message to update during an ongoing session PR #8927
* Fetch active alarms from alarm_log PR #8944
|
|
* - Add initial mqtt support
* [WIP] Agent cloud link
- Setup main mqtt thread to connect to a broker using V5 of the MQTT protocol (TBD)
- Send alarms to "netdata/alarm"
- Add error checks to handle connection failures
- Add params for
Broker, port
Maximum concurrent sent / recev messages
- Dummy function to check claiming status
- Generic mqtt_send command to publish message to a base topic , sub topic
It will end up in the form base_topic/sub_topic
- Add host/port in the connection failure error message
* Test libmosquitto libs
* connect to broker locally (assume localhost:1883)
* subscribe to channel netdata/command
* Test try a reload command to trigger health reload
* publish alerts to netdata/alarm
* - Fix compile issues
* - Use sleep_usec instead of usleep
* - Delay reconnection on failure due to misconfiguration (high cpu usage)
* - Remove the TLS connection config
* - Fix NETDATA_MQTT_INITIALIZATION_SLEEP_WAIT to use seconds
* - Gather ACLK related code under aclk folder
- Add aclk_ functions for abstract layer
- Moved low level libs intergration in mqtt.c
* - Add README.md file with initial comment
* - Clean MQTT v5
* - Code cleanup
* - Remove alarm log for now
- Remove the heart beat
* - Remove message properties for V5
* - Remove message properties for V5 (header)
* Fixed the netdata target to use a local static version of libmosquitto.
The installer does not yet have steps to pull and build the local library.
cd project_root
git clone ssh://git@github.com/netdata/mosquitto mosquitto/
(cd mosquitto/lib && make) # Ignore the cpp error
This will leave mosquitto/lib/libmosquitto.a for the build process to use.
* - Fix compile issues with older < 1.6 libmosquitto lib
* - Enable alarm events to check it works
- Re arrange includes
- Rework topic to be agent/guid/. Actual id will be
returned by the is_agent_claimed
* - Add initial metadata info
- Added helper function in web_api
- Added a debug command (info)
* Update the claiming state to retrieve the claimed id.
* - Use define for constants like command and metadata topics
- Function to wait for initialization of the ACLK link
- New aclk_subscribe command with QOS parameter for the mqtt subscription
- Use the is_agent_claimed function to get the real claim id and use it to build the topics
that will be used for the cloud communication
- Change in netdata-claim.sh.in to write the claim id without a trailing \n
* - Use define for constants like command and metadata topics
- Function to wait for initialization of the ACLK link
- New aclk_subscribe command with QOS parameter for the mqtt subscription
- Use the is_agent_claimed function to get the real claim id and use it to build the topics
that will be used for the cloud communication
- Change in netdata-claim.sh.in to write the claim id without a trailing \n
* - Remove the alarm log for now
- Add code (but disabled) to send charts
* - Use dummy anon, anon as username and password for testing purposes
* - Use client id anon as well
* Testing without TLS
* Switching TLS back on to fix docker environment.
* - Added query processing
An incoming URL now calls web_client_api_request_v1_data to handle a request and push the results
back to the "data" topic
- Move the above processing from the message callback to the query handle loop
- Added helper "pause" , "resume" commands to stop and resume query processing to stress test loading the queue
with queries before executing them
- Changed the endpoint topics to "meta", and "cmd" (previously metadata and command)
* make info message follow protocol
* move metadata msg generation into new func
* move metadata msg generation into new func
* - Add metadata to the responses
- Add hook to queue chart changes on creation and dimensions
- Changed the queue mechanism to include delay for X seconds
- Add delayed submittion of charts to the cloud so that all DIMs are defined to avoid resubmission
* - Add additional data info for aclk_queue command
* - Use web_clinet_api_request_v1 to handle the incoming request
This will handle all requests coming from the cloud
* - Cleanup and aclk_query structure
- Add msg_id parameter
- Enable the incoming JSON request
- Enable the outgoing JSON response
* - Added new thread to handle query processing
- Add lock and cond wait to wakeup thread when queries are submitted
- Cleanup on the main init function
* - Add wait time on agent init, to allow for chart, alarms and other definitions to be completed.
- During the wait time, no queries will be queued
* - Send metadata on query thread init
- New generic create header function for the JSON response
- Pack info and charts into one message
- Modified chart to remove entries (test)
- Modified charts mod to remove entries e.g alarms and volatile info
- Change input to aclk_update_chart (RRDHOST / instead of hostname)
* - When a request fails, add to the payload
- We may need to handle in a different key
- Error check in json parsing
* - Add dummy aclk_update_alarm command
* - Move incoming request JSON parsing code away from mqtt.c
- Added #ifdef ACLK_ENABLE so that we can have code merged but disabled by default
- Added version in incoming and outgoing JSON dict
* - Disable code if ACLK_ENABLE is not defined
- Remove references to the mqtt (mosquitto) lib
- Add dummy stubs in mqtt.c for completeness if ACLK_ENABLE is not defined
* - Disable challenge sample code for now
* - Remove libmosquitto from makefile
* - Fix spaces in Makefile.am
- Remove ifdef to avoid warning from LGTM
* - Remove for now the code that builds an along log test message to send to the cloud
* - Add check for ACLK_ENABLE definition and avoid calling the chart update functions
* - Remove commented code
* - Move source files to the correct place (ACLK_PLUGIN_FILES)
* - Remove include file thats not needed
* - Remove include file thats not needed
- Add improved checks for load_claiming_state()
* - Fix error message. Used error() that also logs errno and message
* - Fix some codacy issues
* - Fix more codacy issues, code cleanup
* - Revert code to address codacy warnings
* - Revert spaces added in a previous commit by mistake
* clean up if/else nest
* print error if fopen fails
* minor - error already logs errno
* - Fix version formatting
* - Cleanup all ACLK related compiler warnings
- Re-arrange include files
- Removed unused defines
* - More compilation warnings fixed
- Bug with thread creation fixed
* - Add condition to skip compilation of the ACLK code entirely. Add env variable ACLK="yes" to enable
* - Add condition to skip the libmosquitto
* - Change feature flag from ACLK_ENABLE to ENABLE_ACLK in accordance with the rest of ENABLE_xx flags
- Typo in info message fix
Co-authored-by: Andrew Moss <1043609+amoss@users.noreply.github.com>
Co-authored-by: Timo <6674623+underhood@users.noreply.github.com>
|
|
Initial infrastructure support for agent claiming.
This feature is not currently enabled as we are still finalizing the details of the cloud infrastructure w.r.t. agent claiming. The feature will be enabled when we are ready
to release it.
|
