| Age | Commit message (Collapse) | Author |
|---|
|
* Hard code a node for non-legacy multidb test
Skip dbengine initialization for new incoming children
Add code to switch to multidb ctx when accessing the dbengine
* When a non-legacy streaming connection is detected, use the multidb metadata log context
* Clear the superblock memory to avoid random data written in the metadata log
* Activate the host detection during compaction
Activate the host detection during metadata log chart updates
Keep the host in the user object during replay of the HOST command
* Add defaults for health / rrdpush on HOST metadata replay
Check for legacy status on host creation by checking is_archived and if not conclusive, call is_legacy_child()
Use defaults from the stream.conf
* Count hosts only if not archived
When host switches from archived to active update rrd_hosts_available
Remove archived hosts from charts and info
* Change parameter from "multidb disk space" to "dbengine multihost disk space"
Remove unused variables
Fix compilation error when dbengine is disabled
Fix condition for machine_guid directory creation under cache_dir
* Enable multidb disk space file creation.
* Stop deleting dimensions when rotating archived metrics if the dimension is active in a different database engine.
* Fix old bug in the code that confused obsolete hosts with orphan hosts.
* Do not delete multi-host DB host files.
* Discard dbengine state when a legacy memory mode instantiates to avoid inconsistencies.
* Identify metadata that collide with non-dbengine memory mode hosts and ignore them.
* Handle non-dbengine localhost with dbengine archived charts in localhost and streaming.
* Ignore archived hosts in streaming.
* Add documentation before merging to master.
Co-authored-by: Markos Fountoulakis <markos.fountoulakis.senior@gmail.com>
|
|
* Add lock dir
* Clean directory on startup
* Update environment variable name
* Fix file removal
* Add error message
* collectors/python.d: change lock file name
* collectors/python.d: add `nolock` cmd option
Co-authored-by: ilyam8 <ilya@netdata.cloud>
|
|
Bring new options to ebpf.plugin.
|
|
Implemented default disk space size calculation for multihost db (#9504)
|
|
This causes failure to detect virtualization to be reported as no
virtualization instead of unknown virtulization.
|
|
Fix unit test execution( ` Cannot create unique machine id file`).
|
|
Move registry to do integration with spawn server.
|
|
* Replace all assert() calls with the new fatal_assert() for proper logging.
|
|
* Get netdata execution path early to avoid user permission issues
|
|
* Intial pass through docs
* Dash instead of slash
* To parent/child
* Child nodes
* Change diagrams
* Allowlist
* Fixes for Andrew
* Remove from build_external
* Change in proc
|
|
|
|
* Implemented collector metadata logging
* Added persistent GUIDs for charts and dimensions
* Added metadata log replay and automatic compaction
* Added detection of charts with no active collector (archived)
* Added new endpoint to report archived charts via `/api/v1/archivedcharts`
* Added support for collector metadata update
Co-authored-by: Markos Fountoulakis <44345837+mfundul@users.noreply.github.com>
|
|
Removed uses of the host lock that could deadlock senders and replaced with the new fine-grained mutex.
|
|
* Add a `ping` command to netdatacli to check if agent is alive.
This provides a way to trivially check if the agent itself appears to be
running (namely, the command parser for netdatacli in the agent itself
is working and responding), allowing users to check this without having
to rely on us continuing to have `help` be a command sent to the agent
instead of executing locally.
* Add a basic health check to our docke rimages.
This adds a relatively basic health checker script to our Docker images.
By default it verifies that the `/api/v1/info` endpoint returns a 200
status code.
It also supports checking different endpoints or using `netdatacli ping`
to check that Netdata is running, all controlled by a new Docker
environment variable: `NETDATA_HEALTH_CHECK`.
* Avoid unnessecary `chmod` in Dockerfile.
Suggested by @prologic.
* Fix typo in docs.
* Update environment variable name to be more clear.
Also add `-L` to `curl` command in health check to follow redirects.
|
|
the system's disk capacity. (#7902)
* Remove trailing whitespace in system-info.sh.
* Fix handling of APFS on macOS.
APFS can have multiple volumes in a single partition, which means that
the same functional 'volume' can appear multiple times in the output of
`df`. Duplicate lines for such volumes will show the same total size and
available space along with a common prefix for the device name.
This updates the parsing logic for `df` on macOS to account for this by
deduplicating lines in the `df` output that have the same total size,
available space, and same normalized device name.
This has the potential to incorrectly under-account space in some cases,
but the liklihood of that happeing is much less than the certainty of
overaccounting space on standard APFS configurations.
* Properly handle VirtIO block devices when using /sys.
The VirtIO Block device driver uses a dynamically allocated device major
number, meaning that we can't trivially match on it.
This updates the handling to properly look it up in `/proc/devices`
instead of just using the whole dynamic device number range.
* Add handling for NVMe block devices in sysfs code.
They use dynamic major numbers just like VirtIO Block devices do.
* Switch to device major discovery in /proc/devices for all device types.
This converts the code to use `/proc/devices` to look up correct device
major numbers for block devices taht we treat as disks just like we are
already doing for those that have dynamically assigned numbers. This
makes the code both more robust and easier to understand and modify.
This also excludes some particularly old hardware that we were
originally looking for. If needed, we can add in the required device
names, but for now it's better to keep the list concise.
* Correct handling of device major discovery.
We need to strip leading whitespace before calling cut, not after.
* Only use /sys/block if we can read /proc/devices.
We use `/proc/devices` to do device number lookups that we then use to
filter devices under `/sys/block`. As a result, if we can't read
`/proc/devices`, then we won't actually parse anything out of
`/sys/block` either, so we need to just fall back to parsing `df`
output.
* Deduplicate `df` output by device name on Linux.
This ensures that we properly handle BTRFS subvolumes, counting each
actual volume only once.
* Use POSIX math expansion instead of awk to sum disk sizes.
This avoids the rather annoying habit of AWK of printing integers in
scientific notation instead of as exact values.
* Correct `sed` options for POSIX complaince.
* Fix disk info fetching for macOS>
POSIX tools, as found on macOS, lack a number of rather useful filtering
and sorting features, so we need to get rather creative with the
handling on macOS to make the disk space computation work correctly.
This unfortunately makes the calculation a bit less reliable than it
would have been had the existing calculations worked correctly, but it's
the best I can come up with without making things exponentiall more
complicated.
* Properly handle sector size when using sysfs.
|
|
|
|
* Add options to daemon, clean up claiming
* Caught one more old conf
* Remove cloud docs
* I did a lot of things
* Rewrite tutorial step 3
* Remove my nodes menu, sync what-is-netdata
* Restore ACLK/claim/daemon docs to be handled by docs-go-live
* Fix up what-is-netdata
* More cleanup of README/what-is-netdata
* Restore daemon/config/README.md
* Fix frontmatter
* Change title, fix broken link
* Copyediting fixes
* Remove symbols
* Add a few more GIFs
* Fix hash
* Fix other hash
* Fix wording in web gui
* Address Andrew's and Jacek's comments
|
|
* Add support for spawning processes without pipes.
* Port health_alarm_execute() from mypopen() to netdata_spawn()
* Make alarm notifications asynchronous within a single health thread iteration
* Initial version of spawn server.
* preliminary integration of spawn client with health
|
|
|
|
|
|
documentation. (#8964)
|
|
|
|
* Restore docs from naughty PR
* Address Andrew's comments
* Ini to conf
* Changes based on meeting with Andrew
* Tweak text around claiming
* Some grammar/typo fixes
* Add /var/lib/netdata to Docker instructions on README
* Added a few more ACLK links per Chris
Co-authored-by: Joel Hans <joel@netdata.cloud>
|
|
This PR merges the feature-branch to make the cloud live. It contains the following work:
Co-authored-by: Andrew Moss <1043609+amoss@users.noreply.github.com(opens in new tab)>
Co-authored-by: Jacek Kolasa <jacek.kolasa@gmail.com(opens in new tab)>
Co-authored-by: Austin S. Hemmelgarn <austin@netdata.cloud(opens in new tab)>
Co-authored-by: James Mills <prologic@shortcircuit.net.au(opens in new tab)>
Co-authored-by: Markos Fountoulakis <44345837+mfundul@users.noreply.github.com(opens in new tab)>
Co-authored-by: Timotej S <6674623+underhood@users.noreply.github.com(opens in new tab)>
Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com(opens in new tab)>
* dashboard with new navbars, v1.0-alpha.9: PR #8478
* dashboard v1.0.11: netdata/dashboard#76
Co-authored-by: Jacek Kolasa <jacek.kolasa@gmail.com(opens in new tab)>
* Added installer code to bundle JSON-c if it's not present. PR #8836
Co-authored-by: James Mills <prologic@shortcircuit.net.au(opens in new tab)>
* Fix claiming config PR #8843
* Adds JSON-c as hard dep. for ACLK PR #8838
* Fix SSL renegotiation errors in old versions of openssl. PR #8840. Also - we have a transient problem with opensuse CI so this PR disables them with a commit from @prologic.
Co-authored-by: James Mills <prologic@shortcircuit.net.au(opens in new tab)>
* Fix claiming error handling PR #8850
* Added CI to verify JSON-C bundling code in installer PR #8853
* Make cloud-enabled flag in web/api/v1/info be independent of ACLK build success PR #8866
* Reduce ACLK_STABLE_TIMEOUT from 10 to 3 seconds PR #8871
* remove old-cloud related UI from old dashboard (accessible now via /old suffix) PR #8858
* dashboard v1.0.13 PR #8870
* dashboard v1.0.14 PR #8904
* Provide feedback on proxy setting changes PR #8895
* Change the name of the connect message to update during an ongoing session PR #8927
* Fetch active alarms from alarm_log PR #8944
|
|
|
|
* Fix parsing issue in system-info.sh.
Depending on the exact hardware it's run on, `lscpu` may or may not
report a maximum and/or minimum CPU frequency. We want to preferentially
match on the maximum if it's there because the regular CPU frequency
entry from `lscpu` shows the _current_ frequency most of the time, and
we want to report the 'intended' frequency for the CPU.
* Fix the check to see if we found a CPU frequency value.
* Actually fix parsing.
|
|
* Trying out some absolute-ish links
* Try one out on installer
* Testing logic
* Trying out some more links
* Fixing links
* Fix links in python collectors
* Changed a bunch more links
* Fix build errors
* Another push of links
* Fix build error and add more links
* Complete first pass
* Fix final broken links
* Fix links to files
* Fix for Netlify
* Two more fixes
|
|
This reverts commit e2874320fc027f7ab51ab3e115d5b1889b8fd747.
|
|
|
|
* Change MacOS to macOS
* Change Mac as noun to macOS system
|
|
* Fix the Prometheus web API code in the exporting engine
* Rename connector types
* Remove the conditional compilation of the exporting engine
* Use labels instead of tags
* Fix the exporter configuration
* Document functions
* Add unit tests
|
|
* tls13: This commit brings TLS 1.3 to Netdata
* tls13: Update variables on slave side
* tls13: Fix compilation error for old libraries
* tls13: Fix compilation error for old libraries 2
* tls13 remove ciphers
* tls13: TLS versions
This commit brings the missing tls versions accpeted for Netdata
and it also brings documentation update related to these versions
* tls13: Remove dupplication
This commit removes wrong dupplication of code
* tls13: Documentation
This commit brings fix for the documentation
* tls13: Remove magic number
This commit removes the magic number to allow the code to be readable
* tls13: TLS version
Small adjust with TLS version
* tls13: Security Init
This commit removes array from the function and overwrite the magic number
with a string
* tls13: Remove new variable name from stream
* tls13: OpenSSL versions and old key name
This commit removes the new key names and also update the names
used to define openssl version
|
|
Preparing for the cloud release. This changes how we handle the feature flag so that it no longer requires installer switches and can be set from the config file. This still requires internal access to use and is not ready for public access yet.
|
|
The default cloud url has been updated to app.netdata.cloud ready for the release. The claiming process now checks the current user executing claiming and refuses to perform the claim for the wrong user. If the current UID is 0 then claiming proceeds but the file ownership is adjusted to be the correct netdata user. The default expected user is `netdata` unless the script can identify the user from the current configuration. After the claiming script is executed the CLI is used to reload the claiming state.
|
|
Reports ACLK build failures to GA (if the user didn't opt-out)
|
|
Improved the stability of the ACLK
|
|
* Fix flushing error threshold to account for inactive producers of dbengine instance
* Disable page invalidation during shutdown of dbengine
* Fix crash during netdata shutdown if command server has failed to initialize
* Add fallback for uv_listen to retry with backlog = 1 on failure.
* Adhere to the API change of libuv v1.35
|
|
* Disallow multiple streaming connections to the same master agent
* Reject multiple streaming connections quickly without blocking
* Increase timeout for systemd service shutdown to give time to flush the db.
* Optimize page correlation ID to use atomic counter instead of locks
* Reduce contention in global configuration mutex
* Optimize complexity of inserting configuration sections from O(N) to O(1)
* Reduce overhead of clockgettime() by utilizing CLOCK_MONOTONIC_COARSE when applicable.
* Fix unit test compile errors
|
|
* Bulk add frontmatter
* A few extra edge cases
|
|
* Fixes to DOCS home and README
* Edit conf-guide and getting-started
* Add dbengine settings to map
* Fix tutorial and step-by-step
* Fix artifacts of old memory mode types
* A few tweaks
* Push a little harder on README
* Fix for Markos
|
|
* Fixes for database/readme.md
* Fixes for registry/readme.md
* Fixes for daemon/readme.md
* Fixes for database/engine/readme.md
* Fixes for registry/readme.md
* Fix for cli/readme.md
* Fixes on docs/a-github-star-is-important.md
* A few more documents
|
|
* update_info: New variables
This commit creates inside script and it reads them to Netdata
* update_info: API
This commit changes the web api response
* update_info: Disk space
This commit brings the disk space to info and renames the environment variables inside Netdata
* update_info: Rename variable
This commit renames the environment variable
* update_info: Rename response variable
This commit renames a response variable
* update_info: Labels
This commit creates the missing labels
* update_info: test before free
* update_info: Doc function
This commit brings docummentation to the functions to give instructions to developer
* update_info: Fix info message
This commit removes some info messages from the error.log
* update_info: Remove unecessary ifs, considering free manual
|
|
* Introduce dirty page pressure handling in the dbengine page cache that invalidates pages when the disk cannot keep up with the flushing speed.
|
|
* - Add initial mqtt support
* [WIP] Agent cloud link
- Setup main mqtt thread to connect to a broker using V5 of the MQTT protocol (TBD)
- Send alarms to "netdata/alarm"
- Add error checks to handle connection failures
- Add params for
Broker, port
Maximum concurrent sent / recev messages
- Dummy function to check claiming status
- Generic mqtt_send command to publish message to a base topic , sub topic
It will end up in the form base_topic/sub_topic
- Add host/port in the connection failure error message
* Test libmosquitto libs
* connect to broker locally (assume localhost:1883)
* subscribe to channel netdata/command
* Test try a reload command to trigger health reload
* publish alerts to netdata/alarm
* - Fix compile issues
* - Use sleep_usec instead of usleep
* - Delay reconnection on failure due to misconfiguration (high cpu usage)
* - Remove the TLS connection config
* - Fix NETDATA_MQTT_INITIALIZATION_SLEEP_WAIT to use seconds
* - Gather ACLK related code under aclk folder
- Add aclk_ functions for abstract layer
- Moved low level libs intergration in mqtt.c
* - Add README.md file with initial comment
* - Clean MQTT v5
* - Code cleanup
* - Remove alarm log for now
- Remove the heart beat
* - Remove message properties for V5
* - Remove message properties for V5 (header)
* Fixed the netdata target to use a local static version of libmosquitto.
The installer does not yet have steps to pull and build the local library.
cd project_root
git clone ssh://git@github.com/netdata/mosquitto mosquitto/
(cd mosquitto/lib && make) # Ignore the cpp error
This will leave mosquitto/lib/libmosquitto.a for the build process to use.
* - Fix compile issues with older < 1.6 libmosquitto lib
* - Enable alarm events to check it works
- Re arrange includes
- Rework topic to be agent/guid/. Actual id will be
returned by the is_agent_claimed
* - Add initial metadata info
- Added helper function in web_api
- Added a debug command (info)
* Update the claiming state to retrieve the claimed id.
* - Use define for constants like command and metadata topics
- Function to wait for initialization of the ACLK link
- New aclk_subscribe command with QOS parameter for the mqtt subscription
- Use the is_agent_claimed function to get the real claim id and use it to build the topics
that will be used for the cloud communication
- Change in netdata-claim.sh.in to write the claim id without a trailing \n
* - Use define for constants like command and metadata topics
- Function to wait for initialization of the ACLK link
- New aclk_subscribe command with QOS parameter for the mqtt subscription
- Use the is_agent_claimed function to get the real claim id and use it to build the topics
that will be used for the cloud communication
- Change in netdata-claim.sh.in to write the claim id without a trailing \n
* - Remove the alarm log for now
- Add code (but disabled) to send charts
* - Use dummy anon, anon as username and password for testing purposes
* - Use client id anon as well
* Testing without TLS
* Switching TLS back on to fix docker environment.
* - Added query processing
An incoming URL now calls web_client_api_request_v1_data to handle a request and push the results
back to the "data" topic
- Move the above processing from the message callback to the query handle loop
- Added helper "pause" , "resume" commands to stop and resume query processing to stress test loading the queue
with queries before executing them
- Changed the endpoint topics to "meta", and "cmd" (previously metadata and command)
* make info message follow protocol
* move metadata msg generation into new func
* move metadata msg generation into new func
* - Add metadata to the responses
- Add hook to queue chart changes on creation and dimensions
- Changed the queue mechanism to include delay for X seconds
- Add delayed submittion of charts to the cloud so that all DIMs are defined to avoid resubmission
* - Add additional data info for aclk_queue command
* - Use web_clinet_api_request_v1 to handle the incoming request
This will handle all requests coming from the cloud
* - Cleanup and aclk_query structure
- Add msg_id parameter
- Enable the incoming JSON request
- Enable the outgoing JSON response
* - Added new thread to handle query processing
- Add lock and cond wait to wakeup thread when queries are submitted
- Cleanup on the main init function
* - Add wait time on agent init, to allow for chart, alarms and other definitions to be completed.
- During the wait time, no queries will be queued
* - Send metadata on query thread init
- New generic create header function for the JSON response
- Pack info and charts into one message
- Modified chart to remove entries (test)
- Modified charts mod to remove entries e.g alarms and volatile info
- Change input to aclk_update_chart (RRDHOST / instead of hostname)
* - When a request fails, add to the payload
- We may need to handle in a different key
- Error check in json parsing
* - Add dummy aclk_update_alarm command
* - Move incoming request JSON parsing code away from mqtt.c
- Added #ifdef ACLK_ENABLE so that we can have code merged but disabled by default
- Added version in incoming and outgoing JSON dict
* - Disable code if ACLK_ENABLE is not defined
- Remove references to the mqtt (mosquitto) lib
- Add dummy stubs in mqtt.c for completeness if ACLK_ENABLE is not defined
* - Disable challenge sample code for now
* - Remove libmosquitto from makefile
* - Fix spaces in Makefile.am
- Remove ifdef to avoid warning from LGTM
* - Remove for now the code that builds an along log test message to send to the cloud
* - Add check for ACLK_ENABLE definition and avoid calling the chart update functions
* - Remove commented code
* - Move source files to the correct place (ACLK_PLUGIN_FILES)
* - Remove include file thats not needed
* - Remove include file thats not needed
- Add improved checks for load_claiming_state()
* - Fix error message. Used error() that also logs errno and message
* - Fix some codacy issues
* - Fix more codacy issues, code cleanup
* - Revert code to address codacy warnings
* - Revert spaces added in a previous commit by mistake
* clean up if/else nest
* print error if fopen fails
* minor - error already logs errno
* - Fix version formatting
* - Cleanup all ACLK related compiler warnings
- Re-arrange include files
- Removed unused defines
* - More compilation warnings fixed
- Bug with thread creation fixed
* - Add condition to skip compilation of the ACLK code entirely. Add env variable ACLK="yes" to enable
* - Add condition to skip the libmosquitto
* - Change feature flag from ACLK_ENABLE to ENABLE_ACLK in accordance with the rest of ENABLE_xx flags
- Typo in info message fix
Co-authored-by: Andrew Moss <1043609+amoss@users.noreply.github.com>
Co-authored-by: Timo <6674623+underhood@users.noreply.github.com>
|
|
|
|
* Fix memory leaks
* Check for configuration options
* Parse simple tags
* Parse JSON tags
* Remove an unnecessary check
* Parse a JSON object
* Parse a JSON array
* Update the documentation
* Fix host locks
|
|
* Fixed bug in DO_NOT_TRACK expression
* Fix kickstart-static64 checksum in docs.
Co-authored-by: Austin S. Hemmelgarn <ahferroin7@gmail.com>
|
|
variable (#7846)
* Added support for opting out of telemtry via the DO_NOT_TRACK environment variable
* Added support for DO_NOT_TRACK=1 in anonymous-statistics.sh and minor cleanup in Dockerfile and run.sh entrypoint
* Allow DO_NOT_TRACK to be either non-zero or non-empty
* Update md5sum of kickstart-static64.sh in docs
* Fixed a bug in netdata-installer.sh
* Revert changes to daemon/main.c (testing onyl)
* Update docs/anonymous-statistics.md
Co-Authored-By: Mansour Behabadi <57921115+ncmans@users.noreply.github.com>
Co-authored-by: Mansour Behabadi <57921115+ncmans@users.noreply.github.com>
|
|
* Add rudimentary disk size detection to system-info.sh.
This adds really basic detection of total disk size to the
system-info.sh script. This detection is _wildly_ inaccurate in many
cases, but it's not realistically possible to make it much better than
what's added here without making the script exponentially more
complicated (there are just way too many edge cases and special
conditions to worry about).
This adds the following keys to the output of the script:
* NETDATA_TOTAL_DISK_SIZE: This indicates the total detected disk size
in bytes.
* NETDATA_DISK_DETECTION: This indicates what detection method was used
for determining the total disk size. It may return either `df` (which
works almost everywhere but is wildly inaccurate) or `sysfs` (which is
linux specific, but is 100% accurate for a majority of cases).
On most platforms, this parses the output of the `df` command, limiting
only to filesystems that are used on fixed disks, which provides
reasonably accurate results in most trvial cases, but kind of falls
apart the moment people are doing anything remotely complicated in terms
of storage (like using a logical volume manager or under-comitting their
disk space).
On Linux, it will preferentially try to parse the info out of
`/sys/block`, filtering on device major numbers that are actually used
for fixed disks and excluding devices that are indicated to be
removable. This provides a very accurate result if the system does not
use removable media as primary storage, but requires that the user who
runs the script can read the contents of `/sys/block`.
* Add VirtIO block device major number to the list of scanned devices.
* Actually handle VirtIO block devices correctly.
* Fixes for macOS handling.
|
|
* missing_extern: Fix missing
Fix few externs that were missing in global variables
* missing_extern: Variables
This commit declares the variables inside .c files
|
