| Age | Commit message (Collapse) | Author |
|---|
|
* add metadata for learn
* first batch of adding metadata to md files
* second batch of adding metadata to md files
* third batch of adding metadata to md files
* test one sidebar_label
* add missing sidebar_labels
* add missing sidebar_labels to files left behind
* test, ansible doc is stubborn
* fix
* fix
* fix
* don't use questionmarks in the sidebar label
* don't use exclamation marks and symbols in the sidebar label
* fix style guide
* fixes
* fixes
|
|
Revert "Misc SSL improvements (#14317)"
This reverts commit f4d017ccf7385df3e152eeb5ea1259702ab01043.
|
|
* set web client to poll when ssl error want read or write
* turn to function
|
|
* enable retries for SSL_ERROR_WANT_READ
* only when bytes is <= 0
* treat ERROR_WANT_READ/WRITE as 0 bytes
* dont close connection on zero bytes
* reuse ssl connection
* treat zero bytes
* ifdef for old openssl
* revert check
|
|
* count open cache pages refering to datafile
* eliminate waste flush attempts
* remove eliminated variable
* journal v2 scanning split functions
* avoid locking open cache for a long time while migrating to journal v2
* dont acquire datafile for the loop; disable thread cancelability while a query is running
* work on datafile acquiring
* work on datafile deletion
* work on datafile deletion again
* logs of dbengine should start with DBENGINE
* thread specific key for queries to check if a query finishes without a finalize
* page_uuid is not used anymore
* Cleanup judy traversal when building new v2
Remove not needed calls to metric registry
* metric is 8 bytes smaller; timestamps are protected with a spinlock; timestamps in metric are now always coherent
* disable checks for invalid time-ranges
* Remove type from page details
* report scanning time
* remove infinite loop from datafile acquire for deletion
* remove infinite loop from datafile acquire for deletion again
* trace query handles
* properly allocate array of dimensions in replication
* metrics cleanup
* metrics registry uses arrayalloc
* arrayalloc free should be protected by lock
* use array alloc in page cache
* journal v2 scanning fix
* datafile reference leaking hunding
* do not load metrics of future timestamps
* initialize reasons
* fix datafile reference leak
* do not load pages that are entirely overlapped by others
* expand metric retention atomically
* split replication logic in initialization and execution
* replication prepare ahead queries
* replication prepare ahead queries fixed
* fix replication workers accounting
* add router active queries chart
* restore accounting of pages metadata sources; cleanup replication
* dont count skipped pages as unroutable
* notes on services shutdown
* do not migrate to journal v2 too early, while it has pending dirty pages in the main cache for the specific journal file
* do not add pages we dont need to pdc
* time in range re-work to provide info about past and future matches
* finner control on the pages selected for processing; accounting of page related issues
* fix invalid reference to handle->page
* eliminate data collection handle of pg_lookup_next
* accounting for queries with gaps
* query preprocessing the same way the processing is done; cache now supports all operations on Judy
* dynamic libuv workers based on number of processors; minimum libuv workers 8; replication query init ahead uses libuv workers - reserved ones (3)
* get into pdc all matching pages from main cache and open cache; do not do v2 scan if main cache and open cache can satisfy the query
* finner gaps calculation; accounting of overlapping pages in queries
* fix gaps accounting
* move datafile deletion to worker thread
* tune libuv workers and thread stack size
* stop netdata threads gradually
* run indexing together with cache flush/evict
* more work on clean shutdown
* limit the number of pages to evict per run
* do not lock the clean queue for accesses if it is not possible at that time - the page will be moved to the back of the list during eviction
* economies on flags for smaller page footprint; cleanup and renames
* eviction moves referenced pages to the end of the queue
* use murmur hash for indexing partition
* murmur should be static
* use more indexing partitions
* revert number of partitions to number of cpus
* cancel threads first, then stop services
* revert default thread stack size
* dont execute replication requests of disconnected senders
* wait more time for services that are exiting gradually
* fixed last commit
* finer control on page selection algorithm
* default stacksize of 1MB
* fix formatting
* fix worker utilization going crazy when the number is rotating
* avoid buffer full due to replication preprocessing of requests
* support query priorities
* add count of spins in spinlock when compiled with netdata internal checks
* remove prioritization from dbengine queries; cache now uses mutexes for the queues
* hot pages are now in sections judy arrays, like dirty
* align replication queries to optimal page size
* during flushing add to clean and evict in batches
* Revert "during flushing add to clean and evict in batches"
This reverts commit 8fb2b69d068499eacea6de8291c336e5e9f197c7.
* dont lock clean while evicting pages during flushing
* Revert "dont lock clean while evicting pages during flushing"
This reverts commit d6c82b5f40aeba86fc7aead062fab1b819ba58b3.
* Revert "Revert "during flushing add to clean and evict in batches""
This reverts commit ca7a187537fb8f743992700427e13042561211ec.
* dont cross locks during flushing, for the fastest flushes possible
* low-priority queries load pages synchronously
* Revert "low-priority queries load pages synchronously"
This reverts commit 1ef2662ddcd20fe5842b856c716df134c42d1dc7.
* cache uses spinlock again
* during flushing, dont lock the clean queue at all; each item is added atomically
* do smaller eviction runs
* evict one page at a time to minimize lock contention on the clean queue
* fix eviction statistics
* fix last commit
* plain should be main cache
* event loop cleanup; evictions and flushes can now happen concurrently
* run flush and evictions from tier0 only
* remove not needed variables
* flushing open cache is not needed; flushing protection is irrelevant since flushing is global for all tiers; added protection to datafiles so that only one flusher can run per datafile at any given time
* added worker jobs in timer to find the slow part of it
* support fast eviction of pages when all_of_them is set
* revert default thread stack size
* bypass event loop for dispatching read extent commands to workers - send them directly
* Revert "bypass event loop for dispatching read extent commands to workers - send them directly"
This reverts commit 2c08bc5bab12881ae33bc73ce5dea03dfc4e1fce.
* cache work requests
* minimize memory operations during flushing; caching of extent_io_descriptors and page_descriptors
* publish flushed pages to open cache in the thread pool
* prevent eventloop requests from getting stacked in the event loop
* single threaded dbengine controller; support priorities for all queries; major cleanup and restructuring of rrdengine.c
* more rrdengine.c cleanup
* enable db rotation
* do not log when there is a filter
* do not run multiple migration to journal v2
* load all extents async
* fix wrong paste
* report opcodes waiting, works dispatched, works executing
* cleanup event loop memory every 10 minutes
* dont dispatch more work requests than the number of threads available
* use the dispatched counter instead of the executing counter to check if the worker thread pool is full
* remove UV_RUN_NOWAIT
* replication to fill the queues
* caching of extent buffers; code cleanup
* caching of pdc and pd; rework on journal v2 indexing, datafile creation, database rotation
* single transaction wal
* synchronous flushing
* first cancel the threads, then signal them to exit
* caching of rrdeng query handles; added priority to query target; health is now low prio
* add priority to the missing points; do not allow critical priority in queries
* offload query preparation and routing to libuv thread pool
* updated timing charts for the offloaded query preparation
* caching of WALs
* accounting for struct caches (buffers); do not load extents with invalid sizes
* protection against memory booming during replication due to the optimal alignment of pages; sender thread buffer is now also reset when the circular buffer is reset
* also check if the expanded before is not the chart later updated time
* also check if the expanded before is not after the wall clock time of when the query started
* Remove unused variable
* replication to queue less queries; cleanup of internal fatals
* Mark dimension to be updated async
* caching of extent_page_details_list (epdl) and datafile_extent_offset_list (deol)
* disable pgc stress test, under an ifdef
* disable mrg stress test under an ifdef
* Mark chart and host labels, host info for async check and store in the database
* dictionary items use arrayalloc
* cache section pages structure is allocated with arrayalloc
* Add function to wakeup the aclk query threads and check for exit
Register function to be called during shutdown after signaling the service to exit
* parallel preparation of all dimensions of queries
* be more sensitive to enable streaming after replication
* atomically finish chart replication
* fix last commit
* fix last commit again
* fix last commit again again
* fix last commit again again again
* unify the normalization of retention calculation for collected charts; do not enable streaming if more than 60 points are to be transferred; eliminate an allocation during replication
* do not cancel start streaming; use high priority queries when we have locked chart data collection
* prevent starvation on opcodes execution, by allowing 2% of the requests to be re-ordered
* opcode now uses 2 spinlocks one for the caching of allocations and one for the waiting queue
* Remove check locks and NETDATA_VERIFY_LOCKS as it is not needed anymore
* Fix bad memory allocation / cleanup
* Cleanup ACLK sync initialization (part 1)
* Don't update metric registry during shutdown (part 1)
* Prevent crash when dashboard is refreshed and host goes away
* Mark ctx that is shutting down.
Test not adding flushed pages to open cache as hot if we are shutting down
* make ML work
* Fix compile without NETDATA_INTERNAL_CHECKS
* shutdown each ctx independently
* fix completion of quiesce
* do not update shared ML charts
* Create ML charts on child hosts.
When a parent runs a ML for a child, the relevant-ML charts
should be created on the child host. These charts should use
the parent's hostname to differentiate multiple parents that might
run ML for a child.
The only exception to this rule is the training/prediction resource
usage charts. These are created on the localhost of the parent host,
because they provide information specific to said host.
* check new ml code
* first save the database, then free all memory
* dbengine prep exit before freeing all memory; fixed deadlock in cache hot to dirty; added missing check to query engine about metrics without any data in the db
* Cleanup metadata thread (part 2)
* increase refcount before dispatching prep command
* Do not try to stop anomaly detection threads twice.
A separate function call has been added to stop anomaly detection threads.
This commit removes the left over function calls that were made
internally when a host was being created/destroyed.
* Remove allocations when smoothing samples buffer
The number of dims per sample is always 1, ie. we are training and
predicting only individual dimensions.
* set the orphan flag when loading archived hosts
* track worker dispatch callbacks and threadpool worker init
* make ML threads joinable; mark ctx having flushing in progress as early as possible
* fix allocation counter
* Cleanup metadata thread (part 3)
* Cleanup metadata thread (part 4)
* Skip metadata host scan when running unittest
* unittest support during init
* dont use all the libuv threads for queries
* break an infinite loop when sleep_usec() is interrupted
* ml prediction is a collector for several charts
* sleep_usec() now makes sure it will never loop if it passes the time expected; sleep_usec() now uses nanosleep() because clock_nanosleep() misses signals on netdata exit
* worker_unregister() in netdata threads cleanup
* moved pdc/epdl/deol/extent_buffer related code to pdc.c and pdc.h
* fixed ML issues
* removed engine2 directory
* added dbengine2 files in CMakeLists.txt
* move query plan data to query target, so that they can be exposed by in jsonwrap
* uniform definition of query plan according to the other query target members
* event_loop should be in daemon, not libnetdata
* metric_retention_by_uuid() is now part of the storage engine abstraction
* unify time_t variables to have the suffix _s (meaning: seconds)
* old dbengine statistics become "dbengine io"
* do not enable ML resource usage charts by default
* unify ml chart families, plugins and modules
* cleanup query plans from query target
* cleanup all extent buffers
* added debug info for rrddim slot to time
* rrddim now does proper gap management
* full rewrite of the mem modes
* use library functions for madvise
* use CHECKSUM_SZ for the checksum size
* fix coverity warning about the impossible case of returning a page that is entirely in the past of the query
* fix dbengine shutdown
* keep the old datafile lock until a new datafile has been created, to avoid creating multiple datafiles concurrently
* fine tune cache evictions
* dont initialize health if the health service is not running - prevent crash on shutdown while children get connected
* rename AS threads to ACLK[hostname]
* prevent re-use of uninitialized memory in queries
* use JulyL instead of JudyL for PDC operations - to test it first
* add also JulyL files
* fix July memory accounting
* disable July for PDC (use Judy)
* use the function to remove datafiles from linked list
* fix july and event_loop
* add july to libnetdata subdirs
* rename time_t variables that end in _t to end in _s
* replicate when there is a gap at the beginning of the replication period
* reset postponing of sender connections when a receiver is connected
* Adjust update every properly
* fix replication infinite loop due to last change
* packed enums in rrd.h and cleanup of obsolete rrd structure members
* prevent deadlock in replication: replication_recalculate_buffer_used_ratio_unsafe() deadlocking with replication_sender_delete_pending_requests()
* void unused variable
* void unused variables
* fix indentation
* entries_by_time calculation in VD was wrong; restored internal checks for checking future timestamps
* macros to caclulate page entries by time and size
* prevent statsd cleanup crash on exit
* cleanup health thread related variables
Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
Co-authored-by: vkalintiris <vasilis@netdata.cloud>
|
|
prevent concurrent initialization of SSL from multiple threads; prevent re-initialization of SSL
|
|
remove retries
|
|
* streaming compression, query planner and replication fixes
* remove journal v2 stats from global statistics
* disable sql for checking past sql UUIDs
* single threaded replication
* final replication thread using dictionaries and JudyL for sorting the pending requests
* do not timeout the sending socket when there are pending replication requests
* streaming receiver using read() instead of fread()
* remove FILE * from streaming - now using posix read() and write()
* increase timeouts to 10 minutes
* apply sender timeout only when there are metrics that are supposed to be streamed
* error handling in replication
* remove retries on socket read timeout; better error messages
* take into account inbound traffic too to detect that a connection is stale
* remove race conditions from replication thread
* make sure deleted entries are marked as executed, so that even if deletion fails, they will not be executed
* 2 minutes timeout to retry streaming to a parent that already has this node
* remove unecessary condition check
* fix compilation warnings
* include judy in replication
* wrappers to handle retries for SSL_read and SSL_write
* compressed bytes read monitoring
* recursive locks on replication to make it faster during flush or cleanup
* replication completion chart at the receiver side
* simplified recursive mutex
* simplified recursive mutex again
|
|
* setup default certificates path
* change logic
* change default in stream.conf
|
|
* error_limit() function to limit number of error lines per instance
* count should be more than 1
* protect settings ERROR_LIMIT members
|
|
* Revert "Use llvm's ar and ranlib when compiling with clang (#13854)"
This reverts commit a9135f47bbb36e9cb437b18a7109607569580db7.
* Profile plugin
* Fix macos static thread
* Add support for replication
- Add a new capability for replication, when not supported the agent
should behave as previously.
- When replication is supported, the text protocol supports the
following new commands:
- CHART_DEFINITION_END: send the first/last entry of the child
- REPLAY_RRDSET_BEGIN: sends the name of the chart we are
replicating
- REPLAY_RRDSET_HEADER: sends a line describing the columns of the
following command (ie. start-time, end-time, dim1-name, ...)
- REPLAY_RRDSET_DONE: sends values to push for a specific start/end
time
- REPLAY_RRDSET_END: send the (a) update every of the chart, (b)
first/last entries in DB, (c) whether the child's been told to
start streaming, (d) original after/before period to replicate.
- REPLAY_CHART: Sent from a parent to a child, specifying (a)
the chart name we want data for, (b) whether the child should
start streaming once it has fullfilled the request with the
aforementioned commands, (c) after/before of the data the parent
wants
- As a consequence of the new protocol, streaming is disabled for all
charts on a new connection. It's enabled once replication is finished.
- The configuration parameters are specified from within stream.conf:
- "enable replication = yes|no"
- "seconds to replicate = 3600"
- "replication step = 600" (ie. how many seconds to fill per
roundtrip request.
* Minor fixes
- quote set and dim ids
- start streaming after writing replicated data to the buffer
- write replicated data only when buffer is less than 50% full.
- use reentrant iteration for charts
* Do not send chart definitions on connection.
* Track replication status through rrdset flags.
* Add debug flag for noisy log messages.
* Add license notice.
* Iterate charts with reentrant loop
* Set replication finished flag when streaming is disabled.
* Revert "Profile plugin"
This reverts commit 468fc9386e5283e0865fae56e9989b8ec83de14d.
Used only for testing purposes.
* Revert "Revert "Use llvm's ar and ranlib when compiling with clang (#13854)""
This reverts commit 27c955c58d95aed6c44d42e8b675f0cf3ca45c6d.
Reapply commit that I had to revert in order to be able to build the
agent on MacOS.
* Build replication source files with CMake.
* Pass number of words in plugind functions.
* Use get_word instead of indexing words.
* Use size_t instead of int.
* Pay only what we use when splitting words.
* no need to redefine PLUGINSD_MAX_WORDS
* fix formatting warning
* all usages of pluginsd_split_words() should use the return value to ensure non-cached results reuse; no need to lock the host to find a chart
* keep a sender dictionary with all the replication commands received and remove replication commands from charts
* do not replicate future data
* use last_updated to find the end of the db
* uniformity of replication logs
* rewrite of the query logic
* replication.c in C; debug info in human readable dates
* update the chart on every replication row
* update all chart members so that rrdset_done() can continue
* update the protocol to push one dimension per line and transfer data collection state to parent
* fix formatting
* remove replication object from pluginsd
* shorter communication
* fix typo
* support for replication proxies
* proper use of flags
* set receiver replication finished flag on charts created after the sender has been connected
* clear RRDSET_FLAG_SYNC_CLOCK on replicated charts
* log storing of nulls
* log first store
* log update every switches
* test ignoring timestamps but sending a point just after replication end
* replication should work on end_time
* use replicated timestamps
* at the final replication step, replicate all the remaining points
* cleanup code from tests
* print timestamps as unsigned long long
* more formating changes; fix conflicting type of replicate_chart_response()
* updated stream.conf
* always respond to replication requests
* in non-dbengine db modes, do not replicate more than the database size
* advance the db pointer of legacy db modes
* should be multiplied by update_every
* fix buggy label parsing - identified by codacy
* dont log error on history mismatches for db mode dbengine
* allow SSL requests to streaming children
* dont use ssl variable
Co-authored-by: Costa Tsaousis <costa@netdata.cloud>
|
|
By default functions are declared as extern in C/C++ headers. The goal
of this PR is to reduce the wall of text that many headers have and,
more importantly, to make the declaration of extern'd variables - of
which we have many dispersed in various places - easily and quickly
identifiable.
Automatically generated with:
$ git grep -l '^extern.*(' '**.h' | \
grep -v libjudy | \
grep -v 'sqlite3.h' | \
xargs sed -i -e 's/extern \(.*(.*$\)/\1/'
This is a NFC.
|
|
|
|
* fix bad merge
* fix for ssl_security_cert
|
|
about specific charts (#13720)
* function renames and code cleanup in popen.c; no actual code changes
* netdata popen() now opens both child process stdin and stdout and returns FILE * for both
* pass both input and output to parser structures
* updated rrdset to call custom functions
* RRDSET FUNCTION leading calls for both sync and async operation
* put RRDSET functions to a separate file
* added format and timeout at function definition
* support for synchronous (internal plugins) and asynchronous (external plugins and children) functions
* /api/v1/function endpoint
* functions are now attached to the host and there is a dictionary view per chart
* functions implemented at plugins.d
* remove the defer until keyword hook from plugins.d when it is done
* stream sender implementation of functions
* sanitization of all functions so that certain characters are only allowed
* strictier sanitization
* common max size
* 1st working plugins.d example
* always init inflight dictionary
* properly destroy dictionaries to avoid parallel insertion of items
* add more debugging on disconnection reasons
* add more debugging on disconnection reasons again
* streaming receiver respects newlines
* dont use the same fp for both streaming receive and send
* dont free dbengine memory with internal checks
* make sender proceed in the buffer
* added timing info and garbage collection at plugins.d
* added info about routing nodes
* added info about routing nodes with delay
* added more info about delays
* added more info about delays again
* signal sending thread to wake up
* streaming version labeling and commented code to support capabilities
* added functions to /api/v1/data, /api/v1/charts, /api/v1/chart, /api/v1/info
* redirect top output to stdout
* address coverity findings
* fix resource leaks of popen
* log attempts to connect to individual destinations
* better messages
* properly parse destinations
* try to find a function from the most matching to the least matching
* log added streaming destinations
* rotate destinations bypassing a node in the middle that does not accept our connection
* break the loops properly
* use typedef to define callbacks
* capabilities negotiation during streaming
* functions exposed upstream based on capabilities; compression disabled per node persisting reconnects; always try to connect with all capabilities
* restore functionality to lookup functions
* better logging of capabilities
* remove old versions from capabilities when a newer version is there
* fix formatting
* optimization for plugins.d rrdlabels to avoid creating and destructing dictionaries all the time
* delayed health initialization for rrddim and rrdset
* cleanup health initialization
* fix for popen() not returning the right value
* add health worker jobs for initializing rrdset and rrddim
* added content type support for functions; apps.plugin permanent function to display all the processes
* fixes for functions parameters parsing in apps.plugin
* fix for process matching in apps.plugiin
* first working function for apps.plugin
* Dashboard ACL is disabled for functions; Function errors are all in JSON format
* apps.plugin function processes returns json table
* use json_escape_string() to escape message
* fix formatting
* apps.plugin exposes all its metrics to function processes
* fix json formatting when filtering out some rows
* reopen the internal pipe of rrdpush in case of errors
* misplaced statement
* do not use buffer->len
* support for GLOBAL functions (functions that are not linked to a chart
* added /api/v1/functions endpoint; removed format from the FUNCTIONS api;
* swagger documentation about the new api end points
* added plugins.d documentation about functions
* never re-close a file
* remove uncessesary ifdef
* fixed issues identified by codacy
* fix for null label value
* make edit-config copy-and-paste friendly
* Revert "make edit-config copy-and-paste friendly"
This reverts commit 54500c0e0a97f65a0c66c4d34e966f6a9056698e.
* reworked sender handshake to fix coverity findings
* timeout is zero, for both send_timeout() and recv_timeout()
* properly detect that parent closed the socket
* support caching of function responses; limit function response to 10MB; added protection from malformed function responses
* disabled excessive logging
* added units to apps.plugin function processes and normalized all values to be human readable
* shorter field names
* fixed issues reported
* fixed apps.plugin error response; tested that pluginsd can properly handle faulty responses
* use double linked list macros for double linked list management
* faster apps.plugin function printing by minimizing file operations
* added memory percentage
* fix compatibility issues with older compilers and FreeBSD
* rrdpush sender code cleanup; rrhost structure cleanup from sender flags and variables;
* fix letftover variable in ifdef
* apps.plugin: do not call detach from the thread; exit immediately when input is broken
* exclude AR charts from health
* flush cleaner; prefer sender output
* clarity
* do not fill the cbuffer if not connected
* fix
* dont enabled host->sender if streaming is not enabled; send host label updates to parent;
* functions are only available through ACLK
* Prepared statement reports only in dev mode
* fix AR chart detection
* fix for streaming not being enabling itself
* more cleanup of sender and receiver structures
* moved read-only flags and configuration options to rrdhost->options
* fixed merge with master
* fix for incomplete rename
* prevent service thread from working on charts that are being collected
Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
|
|
* remove_warnings_openssl_v3: Add new macro to define latest OpenSSL version
* remove_warnings_openssl_v3: Add headers necessary for new API
* remove_warnings_openssl_v3: Add compatible variables and adjst code inside load_private_key
* remove_warnings_openssl_v3: Adjust function aclk_get_mqtt_otp according to openssl version
* remove_warnings_openssl_v3: Adjust function private_decrypt
* remove_warnings_openssl_v3: Fix function private_decrypt
* remove_warnings_openssl_v3: Update error message
* remove_warnings_openssl_v3: Update missing error message
|
|
* optimize poll_events() to spread the work over the threads more evenly
* fixed typos, code cleanup
* better error handling
* prevent crash in case callbacks manipulate the sockets arrays - added warnings
|
|
|
|
|
|
|
|
* Fix broken links
* Fix dimension template guide
* Update collectors/COLLECTORS.md
Co-authored-by: Ilya Mashchenko <ilya@netdata.cloud>
Co-authored-by: Ilya Mashchenko <ilya@netdata.cloud>
|
|
|
|
Fix proxy redirect considering variables available on proxy side.
|
|
|
|
Brings TLS to OpenTSDB connector and InfluxDB
|
|
* tls13: This commit brings TLS 1.3 to Netdata
* tls13: Update variables on slave side
* tls13: Fix compilation error for old libraries
* tls13: Fix compilation error for old libraries 2
* tls13 remove ciphers
* tls13: TLS versions
This commit brings the missing tls versions accpeted for Netdata
and it also brings documentation update related to these versions
* tls13: Remove dupplication
This commit removes wrong dupplication of code
* tls13: Documentation
This commit brings fix for the documentation
* tls13: Remove magic number
This commit removes the magic number to allow the code to be readable
* tls13: TLS version
Small adjust with TLS version
* tls13: Security Init
This commit removes array from the function and overwrite the magic number
with a string
* tls13: Remove new variable name from stream
* tls13: OpenSSL versions and old key name
This commit removes the new key names and also update the names
used to define openssl version
|
|
* Cleaning up the ACLK part 2 (#8187)
* Initial proxy support implementation (#8146)
* Implement the ACLK Challenge-Response Authentication (#8317)
Co-authored-by: Timo <6674623+underhood@users.noreply.github.com>
|
|
* Use 4 spaces for indentation of non-recipe lines in Makefile.am files
* Be more consistent in the use of space before = in Makefile.am files
|
|
Our default configuration includes:
allow connections from = localhost *
allow management from = localhost
The problem occurs when a connection is received that passes the `allow connections` pattern
match, but fails the ACL check for `allow management`. During the failure processing path the
DNS lookup is triggered to allow the FQDN to be checked against the pattern. On a FreeBSD
system this lookup fails more slowly than linux and causes a visible performance problem
during stress-testing.
The fix adds a heuristic to analyse the patterns and determine if it is possible to match a DNS name,
or only match a numeric IP address (either IPv4 or IPv6), or only match a constant value. This
heuristic is used to disable the DNS checks when they cannot produce anything that may match
the pattern. Each heuristic is evaluated once, when the configuration is loaded, not per-connection to the agent.
Because the heuristic is not exact it can be overridden using the new config options for each of the ACL connection filters to set it to "yes", "no" or "heuristic". The default for everything *except* the netdata.conf ACL is "heuristic". Because of the numeric-patterns in the netdata.conf ACL the default is set to "no".
|
|
|
|
##### Summary
The Access Control List (ACL) configuration parameters can now use hostnames with simple patterns. Incoming connections are resolved using reverse DNS to obtain the hostname. Where a hostname is resolved, forward DNS resolution is performed to check the IP address is really associated with the hostname. If the checks pass then the patterns supplied are checked. Any patterns supplied for numeric ip addresses are also checked.
##### Component Name
daemon
##### Additional Information
Fixes #6438
* Reverse lookup on ip to get hostname
* Forward lookup on hostname to get IP addresses.
* Validation that the incomming ip is associated with the host.
If these checks fail the hostname is discarded so it cannot match against the access-list
patterns. If these checks validate the ip successfully then the resolved hostname is
pattern-matched as described in the previous commit.
|
|
* make remark access all directories
* detailed fix after autofix by remark lint
* cross check autofix for this set of files
* crosscheck more files
* crosschecking and small fixes
* crosscheck autofixed md files
|
|
* sslcertificate: Trust certificate
The netdata could not allow invalid certificate or certificate with invalid chain
this commit fixes this!
* sslcertificate: Changing name
We are binging the same names used by the OpenSSL library to simplify the understand of the parameters
* sslcertificate: Name changes and explicity directory
This commit fix the problem with Streams and rename correctly the files in the option, it also uses stat to define the existence of a file
* sslcertificate: Documentation
Fix grammar for the newest section in the documentation
* sslcertificate: Rename variables
The old variables did not represent well what they are doing, so it was renamed
|
|
* sslstream: ACL parser
It was noticed in the issue 6457 that the some ACLs were not parsing
correctly when they were along SSL acl, this commit fixes this'
* sslstream: remove comments
This commit removes the comments that were present while I was testing the code
* sslstream: Tests
This commit adds ACL tests to check the Netdata response to them
* sslstream: Tests
Fix the extension to upload the files
* sslstream: more tests
In this commit I am bringing more tests, including the ssl tests'
* sslstream: leading space
Remove leading space from variable that was creating problem with shellcheck
* sslstream: glob
Remove special character from script
* sslstream: Makefile
The Makefile diretives were pointed to wrong files
* sslstream: Missing stream encrypt
This commit solves the problem of the stream not be encrypted, but
it is not the final solution, because the parser made is incomplete.
* sslstream: Finish encrypt channel
This commit brings the step that I was missing, the complete encryptation
in the communication between Master and Slave
* sslstream: Fix argument in script
After the latest tests, it was verified that two arguments given to a function
inside the script were not correct, with this PR I am fixing this!
* sslstream: Fix argument in info
Instead to call a function to deliver an integer I was passing a size_t value.
Only cmake showed this, but not in my clion! :/
* sslstream: Fix redirect
When we were having different SSL configuration, the system were not applying
the option for all
* sslstream: Update documentation
Our documentation was not clear about the rules according our code
so I am updating the text to explain for the users
* sslstream: Adjust script
With this last commit, I am adjusting the tests to avoid false positive
* sslstream: Missing elif
The previous commit had a missing elif in the shell script
* sslstream: Split ports
Before this commit Netdata was having SSL as a global option, now it has as a real ACL.
* sslstream: reduce context
The stream variable will not be affected in the master side, it is only necessary
on the slave side, so I am reducing the context of it
* sslstream: Force SSL
When the user has certificate and he does not set any SSL flag, it is necessary
to append the SSL=force flag
* sslstream: Default flag
It is necessary to have a default flag when the SSL flags are not SET
* sslstream: remove comments
Remove comments from the scrip
* sslstream: moving flag
It is better the flag to be set inside socket instead everytime there is a new connection
* sslstream: documentation
Fix a sentence in the web/server/README.md
|
|
* configure.ac: Add support for GCC's __attribute__((unused))
When compiling under GCC with -Wextra (along with -Wall) there are a lot
of compiler warnings such as
collectors/cgroups.plugin/cgroup-network.c:89:45: warning: unused parameter ‘scope’ [-Wunused-parameter]
89 | struct iface *read_proc_net_dev(const char *scope, const char *prefix) {
| ~~~~~~~~~~~~^~~~~
Some arguments may be able to be got rid off, others won't.
GCC (and at least clang[0]) provide an __attribute__((unused))
annotation that can be used on function parameters (also on variables,
functions, labels, enums, structs etc) to inform the compiler of such
and will squash warnings of the above nature.
A check is added to configure.ac for the use of GCC (I believe $GCC will
be set to 'yes' for clang also) and if found it creates __always_unused
& __maybe_unused #define's set to __attribute__((unused)) otherwise it
just sets them empty.
If other compilers have a similar feature this check can be adjusted to
accommodate them.
The reason for the two defines is that some variables may always be
unused in a function, others may or may not depending on #ifdef's for
example. So we are able to document both cases.
Subsequent commits will start making use of these to squash such
compiler warnings.
[0]: https://clang.llvm.org/docs/AttributeReference.html#maybe-unused-unused
Signed-off-by: Andrew Clayton <andrew@zeta.digital-domain.net>
* collectors/statsd.plugin: Mark a function argument as __maybe_unused
In collectors/statsd.plugin/statsd.c the app function argument to
STATSD_APP_CHART_DIM() might be unused if NETDATA_INTERNAL_CHECKS is not
defined, then the debug() macro that it's used in from
libnetdata/log/log.h will be defined to a dummy function where none of
the arguments are used.
This fixes a compiler warning [-Wunused-parameter] when compiling with
-Wextra *and* -Wall.
Signed-off-by: Andrew Clayton <andrew@zeta.digital-domain.net>
* collectors/apps.plugin: Mark a function argument as __maybe_unused
In collectors/apps.plugin/apps_plugin.c the function
debug_print_process_tree() takes an argument 'msg' that might be unused
if NETDATA_INTERNAL_CHECKS is not defined, then debug_log() will be set
to a dummy function that takes no arguments.
This fixes a compiler warning [-Wunused-parameter] when compiling with
-Wextra *and* -Wall.
Signed-off-by: Andrew Clayton <andrew@zeta.digital-domain.net>
* libnetdata/locks/locks: Mark function arguments as __maybe_unused
In libnetdata/locks/locks.c there a number of functions that take
arguments 'file', 'function' & 'line' that might be unused if
NETDATA_INTERNAL_CHECKS is not defined, then the debug() macro that it's
used in from libnetdata/log/log.h will be defined to a dummy function
where none of the arguments are used.
This fixes compiler warnings [-Wunused-parameter] when compiling with
-Wextra *and* -Wall.
Signed-off-by: Andrew Clayton <andrew@zeta.digital-domain.net>
* libnetdata/socket/security: Mark a function argument as __maybe_unused
In libnetdata/socket/security.c the function security_info_callback()
takes an argument 'ret' that might be unused if NETDATA_INTERNAL_CHECKS
is not defined, then the debug() macro that it's used in from
libnetdata/log/log.h will be defined to a dummy function where none of
the arguments are used.
This fixes a compiler warning [-Wunused-parameter] when compiling with
-Wextra *and* -Wall.
Signed-off-by: Andrew Clayton <andrew@zeta.digital-domain.net>
* collectors/cgroups.plugin: Mark a function argument as __maybe_unused
In collectors/cgroups.plugin/cgroup-network.c the function
read_proc_net_dev() takes an argument 'scope' that might be unused if
the NETDATA_INTERNAL_CHECKS is not defined.
This fixes a compiler warning [-Wunused-parameter] when compiling with
-Wextra *and* -Wall".
Signed-off-by: Andrew Clayton <andrew@zeta.digital-domain.net>
* database/rrddim: Mark function arguments as __maybe_unused
In database/rrddim.c there a couple of functions that take a 'st'
argument that might be unused if NETDATA_INTERNAL_CHECKS is not defined,
then the debug() macro that it's used in from libnetdata/log/log.h will
be defined to a dummy function where none of the arguments are used.
This fixes compiler warnings [-Wunused-parameter] when compiling with
-Wextra *and* -Wall.
Signed-off-by: Andrew Clayton <andrew@zeta.digital-domain.net>
* database/rrdvar: Mark a function argument as __maybe_unused
In database/rrdvar.c the function rrdvar_create_and_index() take an
argument 'scope' that might be unused if NETDATA_INTERNAL_CHECKS is not
defined, then the debug() macro that it's used in from
libnetdata/log/log.h will be defined to a dummy function where none of
the arguments are used.
This fixes a compiler warning [-Wunused-parameter] when compiling with
-Wextra *and* -Wall.
Signed-off-by: Andrew Clayton <andrew@zeta.digital-domain.net>
|
|
* SSL_backend Begin of the encryptation of backend!
* SSL_backend changing opentsdb!
* SSL_backend fix HTTP message with JSON!
* SSL_backend HTTP API done!
* SSL_fix_format preparing to connect with proxy!
* SSL_backend wip SSL send/receive !
* SSL_backend working with proxy
* SSL_backend removing comments!
* SSL_backend docummentation!
* SSL_backend review]!
* SSL_backend organizing!
* Alarm_backend remove comments!
* SSL_backend!
* SSL_backend typedef!
* SSL_backend bring switch!
* SSL_backend commiting format changes!
* SSL_backend fix github parser!
* SSL_Backend fix format!
* SSL_backend switch everything!
* SSL_backend reviewing!
* SSL_backend comments!
* SSL_backend indentation!
* SSL_backend indentation 3!
* SSL_backend documentation!
* SSL_backend hidden pointer!
* SSL_backend missing space
* SSL_backend change documentation!
* SSL_backend change documentation 2!
|
|
|
|
* Add Prometheus remote write backend prototype
* Fix autotools issues
* Send HTTP POST request
* Add parameters to HTTP header
* Discard HTTP responce 200
* Update CMake build configuration
* Fix Codacy issue
* Check for C++ binary
* Fix compilation without remote write backend
* Add options to the installer script
* Fix configure script warning
* Fix make dist
* Downgrade to ByteSize for better compatibility
* Integrate remote write more tightly into the existing backends code
* Cleanup
* Fix build error
* Parse host tags
* Fix Codacy issue
* Fix counters for buffered data
* Rename preprocessor symbol
* Better error handling
* Cleanup
* Update the documentation
|
|
* SSL_fix_format Fix wrong format used with SSL!
* SSL_fix_format Remove unnecessary space!
* SSL_fix_format fixing last requests!
* SSL_fix_format fixing spaces!
* SSL_fix_format killing spaces!
|
