summaryrefslogtreecommitdiffstats
path: root/web/server
AgeCommit message (Collapse)Author
2021-03-22Add data query support for archived charts (#10771)Stelios Fragkakis
2021-03-04Log ACLK cloud commands to access.log (#10697)Stelios Fragkakis
2021-01-18Spelling md (#10508)Josh Soref
* spelling: activity Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: adding Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: addresses Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: administrators Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: alarm Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: alignment Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: analyzing Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: apcupsd Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: apply Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: around Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: associated Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: automatically Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: availability Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: background Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: bandwidth Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: berkeley Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: between Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: celsius Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: centos Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: certificate Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: cockroach Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: collectors Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: concatenation Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: configuration Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: configured Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: continuous Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: correctly Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: corresponding Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: cyberpower Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: daemon Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: dashboard Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: database Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: deactivating Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: dependencies Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: deployment Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: determine Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: downloading Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: either Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: electric Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: entity Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: entrant Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: enumerating Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: environment Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: equivalent Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: etsy Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: everything Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: examining Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: expectations Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: explicit Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: explicitly Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: finally Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: flexible Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: further Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: hddtemp Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: humidity Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: identify Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: importance Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: incoming Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: individual Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: initiate Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: installation Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: integration Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: integrity Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: involuntary Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: issues Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: kernel Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: language Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: libwebsockets Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: lighttpd Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: maintained Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: meaningful Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: memory Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: metrics Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: miscellaneous Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: monitoring Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: monitors Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: monolithic Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: multi Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: multiplier Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: navigation Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: noisy Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: number Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: observing Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: omitted Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: orchestrator Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: overall Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: overridden Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: package Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: packages Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: packet Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: pages Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: parameter Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: parsable Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: percentage Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: perfect Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: phpfpm Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: platform Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: preferred Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: prioritize Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: probabilities Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: process Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: processes Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: program Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: qos Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: quick Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: raspberry Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: received Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: recvfile Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: red hat Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: relatively Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: reliability Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: repository Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: requested Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: requests Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: retrieved Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: scenarios Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: see all Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: supported Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: supports Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: temporary Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: tsdb Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: tutorial Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: updates Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: utilization Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: value Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: variables Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: visualize Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: voluntary Signed-off-by: Josh Soref <jsoref@users.noreply.github.com> * spelling: your Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2021-01-07Docs housekeeping for SEO and syntax, part 1 (#10388)Joel Hans
* First pass to get the script working right * Finish adding analytics tags
2020-12-15Fix a buffer overflow when extracting information from a STREAM connection ↵Stelios Fragkakis
(#10391)
2020-11-24Migrate metadata log to SQLite (#10139)Stelios Fragkakis
2020-09-29Add missing tests (#10008)thiagoftsm
2020-09-10Implements ACLK v2 http message with compression (#9895)Timotej S
Allows cloud to use v2 queries which support compression.
2020-08-28Fix redirect with parametersthiagoftsm
Netdata was missing query string when there was redirect, this PR fixes this.
2020-08-19Fix proxy redirect thiagoftsm
Fix proxy redirect considering variables available on proxy side.
2020-08-19Add missing value to list simple patternthiagoftsm
Fix access using Unix sockets when Netdata is installed with kickstart-static64
2020-06-12Change streaming terminology to parent/child in docs (#9312)Joel Hans
* Intial pass through docs * Dash instead of slash * To parent/child * Child nodes * Change diagrams * Allowlist * Fixes for Andrew * Remove from build_external * Change in proc
2020-06-12Change streaming terminology to parent-child in the code (#9323)Andrew Moss
2020-06-03Fix bugs in streaming and enable support for gap filling (#9214)Andrew Moss
This PR adds (inactive) support that we will use to fill the gaps on chart when a receiving agent goes offline and the sender reconnects. The streaming component has been reworked to make the connection bi-directional and fix several outstanding bugs in the area. * Fixed an incorrect case of version negotiation. Removed fatal() on exhaustion of fds. * Fixed cases that fell through to polling the socket after closing. * Fixed locking of data related to sender and receiver in the host structure. * Added fine-grained locks to reduce contention. * Added circular buffer to sender to prevent starvation in high-latency conditions. * Fixed case where agent is a proxy and negotiated different streaming versions with sender and receiver. * Changed interface to new parser to put the buffering code in streaming. * Fixed the bug that stopped senders from reconnecting after their socket times out - this was part of the scaling fixes that provide an early shortcut path for rejecting connections without lock contention. * Uses fine-grained locking and a different approach to thread shutdown instead. * Added liveness detection to connections to allow selection of the best connection.
2020-05-11Docs: Fix internal links and remove obsolete admonitions (#8946)Joel Hans
* Fixed a few more links * Remove old syntax * Abs-relative links to files in docs folder * Trying to fix nother doc learn link * Fix a few more links * Add testing doc * Tracking down mysteries * Cleanup * Update broken external links * Remove index.html that appeared from testing * Fix remainder of links
2020-04-22Add http headers to responses (#8760)Andrew Moss
The MQTT payloads for responses to API requests from the cloud now include a headers field with the raw http headers encoded into unicode. This exposes the `Date` and `Expired` fields to the cloud backend.
2020-04-14Docs: Standardize links between documentation (#8638)Joel Hans
* Trying out some absolute-ish links * Try one out on installer * Testing logic * Trying out some more links * Fixing links * Fix links in python collectors * Changed a bunch more links * Fix build errors * Another push of links * Fix build error and add more links * Complete first pass * Fix final broken links * Fix links to files * Fix for Netlify * Two more fixes
2020-03-31Extend TLS Support (#8505)thiagoftsm
* tls13: This commit brings TLS 1.3 to Netdata * tls13: Update variables on slave side * tls13: Fix compilation error for old libraries * tls13: Fix compilation error for old libraries 2 * tls13 remove ciphers * tls13: TLS versions This commit brings the missing tls versions accpeted for Netdata and it also brings documentation update related to these versions * tls13: Remove dupplication This commit removes wrong dupplication of code * tls13: Documentation This commit brings fix for the documentation * tls13: Remove magic number This commit removes the magic number to allow the code to be readable * tls13: TLS version Small adjust with TLS version * tls13: Security Init This commit removes array from the function and overwrite the magic number with a string * tls13: Remove new variable name from stream * tls13: OpenSSL versions and old key name This commit removes the new key names and also update the names used to define openssl version
2020-03-10Bulk add frontmatter to all documentation (#8354)Joel Hans
* Bulk add frontmatter * A few extra edge cases
2020-01-30Adds support for opting out of telemetry via the DO_NOT_TRACK envirnment ↵James Mills
variable (#7846) * Added support for opting out of telemtry via the DO_NOT_TRACK environment variable * Added support for DO_NOT_TRACK=1 in anonymous-statistics.sh and minor cleanup in Dockerfile and run.sh entrypoint * Allow DO_NOT_TRACK to be either non-zero or non-empty * Update md5sum of kickstart-static64.sh in docs * Fixed a bug in netdata-installer.sh * Revert changes to daemon/main.c (testing onyl) * Update docs/anonymous-statistics.md Co-Authored-By: Mansour Behabadi <57921115+ncmans@users.noreply.github.com> Co-authored-by: Mansour Behabadi <57921115+ncmans@users.noreply.github.com>
2020-01-04Redirect when url =~ \/host\/hostname$ (#7539) (#7643)Timo
* Redirect when url =~ \/host\/hostname$ (#7539)
2019-11-21CMocka tests for Issue 7274 (#7308)Andrew Moss
* Start of testing partial requests. Need to stash this to checkout a PR to test. * Disambiguated error messages during header validation. The mocking has blown up in the linker, need to wipe out repo local changes and restart from a known good state. * Test failures. CMocka is really not designed for parametric tests which is making it difficult to test the http validation properly. We have some problems in the web_client.c code that are causing early failures in the testing sequence, and it is causing CMocka to abort the sequence. Need to try a different approach to building the tests... * Pedantic style pass. * Test generation. There must be another value hidden in the system that CMocka uses. This sets up 3278 tests but the results from cmocka_run_group_tests_name show 0 tests were run. * The problem was the "helper"-macro. Calling CMocka directly, moved the setup/teardown into explicit fixtures. Successfully runs the family of tests over the same (empty) state. * Parameterised family of tests runs. The api_next() acts as a counter, the least significant digit is the prefix_len using the web_buffer in the test_family struct as a template to walk throufh. The most significant digit is the number of headers to use in the request. Checked that this walk executes correctly and all the tests run before putting the test payloads back in. We trigger a failure about 3-4 tests in that takes down the process. Currently investigating which parts are not mocked correctly. * Pedantic style pass * Adding a mocking for fatal. That weird thing with the linker has happened again, need to clean repo and rebuild fresh. * Full test sequence executes. The test parameter counter jammed after a failure - we cannot rely on anything in the main test body being executed after we call the functionailty under test. A failure will skip the rest of the execution. Moved the counter stepping to the top of the function (i.e. it is now a ++i instead of a i++). Adjusted the initial state to compensate. This now steps through all of the test-sequence, but it raises an ugly issue - the post-test cleanup will not be executed on a failure. TODO: * Move the test-state into the test_family. * Do the clean-up of the previous test (if necesarry) in the step function. * Fix the assertion on the web_client state. * Pedantic style pass * Test state is now in the test_family. This addresses the issue with leaking on failure and not performing clean-up - we don't really care about memory leaks during unit-testing, but we do care about reseting the system-under-test back to a known state to guarantee independence across the tests. The clean-up is now triggered in api_next(). * Flip the wait flag assertions. Partial requests should leave the web_client waiting to receive more data. * Fixing ACL flags in test-driver. This makes some tests pass - but far too many. Probably need a proper debugging function to show the request / response in a readable format. * Result from the api mocking. Setting a successful return code in the api mocking makes the non-partial tests pass. Zero'ing out the web_client before use has not fixed the initialization errors, there is still some history on the parse_tries that needs to be tracked down. Some of the other errors are spurious - they result from stream multiplexing in the testdriver - be careful with less. * Fix warnings. Switched the build configuration to CFLAGS="-O1 -ggdb -Wall -Wextra -Wformat-signedness -fstack-protector-all -DNETDATA_INTERNAL_CHECKS=1 -D_FORTIFY_SOURCE=2 -DNETDATA_VERIFY_LOCKS=1". The memset introduced last night to zero out the initial web_client state had transposed parameters. Now that the state is initially zero before hitting the http request processing most issues have disappeared. There are 3000+ passing tests and 48 boundary cases to track down. * Pushing log entries from each test into a buffer. This will allow suppression of logs from tests that pass. * Switched to a unique test definition structure per test. This cleans up the code as it means that a list of tests can be constructed during the first walk through the parameter space. There is no need to walk the space twice and keep both walks aligned. Removed the cmocka_unit_test macro and build the CMUnitTest structures directly -> this allows a real name per test instead of the procedure name. The walking/step function api_info_next has been folded back into the test procedure as it is simpler to walk the list in the shared test state. Current TODO: * There is a bug, the check on the wait state in the buffer is not being handled properly, investigate why everything fails. * The results don't match the old code, are we handing the correct web_buffer to each tested piece of code? * Capture the test success state -> dump the log buffer on failures. * State is properly passed through the tests. Spent a long time chasing a horrible bug that seems to be inside CMocka? The state parameter being passed to each unit test is different on each call, i.e. it looks like a unique void** where the void * (*state) has been overwritten with the original value on each iteration through the testing loop. This behaviour does not match the CMocka source code, which does thread the given valud through the unit test calls. It could be a side-effect of the memory check-pointing, but the net-effect is that we cannot change the shared state between tests. It can be set in the setup-fixture and used in each test, but not altered for the subsequent test. This took a long time to diagnose - the fix is simple, we just share the state in a global pointer. This shared state is used to walk through the list of test_def structures so that each unit-test knows where it is in the parameter- space. * With the correct state the bug in triggering the correct assertions is gone. * Dump out the buffered logs on test failure. * The only failing case (relative to these assertions) are the ulta-short partial-requests. * Check the web_client->mode is set properly. * Style pass * Checking values passed to the API despatch point. * Disabled the parametric tests to do some low-level testings. Later on both sets of tests will be active. While the low-level url encoding tests are being developed the dynamically generated set is disabled to make the output easier to read. Working through the W3C URL spec, against RFC3986 and comparing the cases in available url-parsing test-suites to build our test-suite. * Start of the URL test-suite. The percent-decoding in the current implementation is in the wrong place - it happens too early and causes non-delimitor characters in the URL to be treated as delimitors. Current unit-tests seem to cover the range of checks that we need CMocka to make. The handling of output is a little awkward - need something like the dynamic cases that can output the log on a failure or skip it on a pass. * Raw material for low-level testing. * Adding more families in here is getting too messy. About to switch over to multiple testdrivers. * Need to clean repo to work around wrapping failures in CMocka. * CMocka is not compatible with LTO. The weird wrapping issues that come and go are as a result of LTO. My typical netdata-installer command-line that I use to reboot the project state disables LTO, while my normal autoreconf / configure command-line does not causing this problem to reappear seemingly-randomly. To build a single test-driver target this works: autoreconf -ivf && CFLAGS='-O1 -ggdb -Wall -Wextra -Wformat-signedness -fstack-protector-all -DNETDATA_INTERNAL_CHECKS=1 -D_FORTIFY_SOURCE=2 -DNETDATA_VERIFY_LOCKS=1' ./configure --disable-lto && make web/api/tests/web_api_valid_urls The actual change in this commit is just a bug-fix. * Ripping out the parameterized test generator. Each of the URL cases is slightly and subtly different. This can't be done using the parameterization and will need a healthy dose of cut and paste. CMocka does not recognise the mocking for mysendfile, which is necessary to capture the exit route from the URL parsing. * Weird bug in CMocka? For some reason CMocka will not mock out the mysendfile() procedure. We need to mock this to capture the behaviour of the URL parsing as it is one of the exit paths. The wrapping is setup the same way as for the procedures so I cannot see any reason that the library would not overwrite the calls. The only difference that I can find is that mysendfile is in the unit being tested and the other mocked procedures are in different translation units. This should not make a difference, but we have to disable LTO to get CMocka to work and the symbol patchs is some kind of linker hack so there could be an issue if LTO is not running and the patch target is inside the same translation unit. Hiding it for now with a #ifndef UNIT_TESTING, which then compiles find and control flow hits the mock... * Converting the ascii comments into unit_tests. * More nasty cases for unit testing. The commented out case will trigger a buffer overflow in the netdata agent and crash it. * Last of the individual unit tests planned before the demo. * Removing warnings. * Switching on the rest of the parametric set - the other case with CRs. * Fix Travis build failure under docker. * Change the name of a define so it does not collide with existing testing in Travis. * Add CMocka unit tests to CMake * Linting pass * Adding RFC comment to test. * Buffer overflow checks on the captured logs. This fixes the seg-fault seen by @vlvkobal and @thiagoftsm during testing. * Chasing down other valgrind reports. This gets rid of all of the uninitialised variable warnings. We stil have a memory leak, the headers that are set during the unit testing switch on compression. This causes the web_client code to call deflatInit2 and allocate structures for the compressor. We do not have a matching call to deflateEnd anywhere in the code so the memory leaks. * Cleaning up a comment. * Fixing review comments from @vlvkobal. Also noticed that the buffer overflow fix this morning was killing the logfile output, fixed this as well. * Addressing @thiagoftsm's concerns about the changing number of failures. Switched the log dump for failing cases to repr(). Found a bug in the test case generator (not storing the flag for `\r`. Verified that the 58 failing cases are the correct set of failures for the tested code.
2019-11-11Makefile.am files indentation (#7252)Konstantinos Natsakis
* Use 4 spaces for indentation of non-recipe lines in Makefile.am files * Be more consistent in the use of space before = in Makefile.am files
2019-10-24Fixing DNS-lookup performance issue on FreeBSD. (#7132)Andrew Moss
Our default configuration includes: allow connections from = localhost * allow management from = localhost The problem occurs when a connection is received that passes the `allow connections` pattern match, but fails the ACL check for `allow management`. During the failure processing path the DNS lookup is triggered to allow the FQDN to be checked against the pattern. On a FreeBSD system this lookup fails more slowly than linux and causes a visible performance problem during stress-testing. The fix adds a heuristic to analyse the patterns and determine if it is possible to match a DNS name, or only match a numeric IP address (either IPv4 or IPv6), or only match a constant value. This heuristic is used to disable the DNS checks when they cannot produce anything that may match the pattern. Each heuristic is evaluated once, when the configuration is loaded, not per-connection to the agent. Because the heuristic is not exact it can be overridden using the new config options for each of the ACL connection filters to set it to "yes", "no" or "heuristic". The default for everything *except* the netdata.conf ACL is "heuristic". Because of the numeric-patterns in the netdata.conf ACL the default is set to "no".
2019-09-23Suggest using /run or /var/run for the unix socket (#6916)Chris Akritidis
2019-09-19Permit x-auth-token in Access-Control-Allow-Headers (#6894)Chris Akritidis
Fixes #6893
2019-09-18Remove Dollar sign from Bash code in documentation and fix remark-lint ↵Promise Akpan
warnings (#6880) * remove dollar sign from bash code * remove dollar sign from sh * clean up uninstall doc * remove dollar sign from shell code * fix remark after removing dollar sign * fix HAProxy dollar sign and clean up remark
2019-09-18Allow hostnames in Access Control Lists (#6796)Andrew Moss
##### Summary The Access Control List (ACL) configuration parameters can now use hostnames with simple patterns. Incoming connections are resolved using reverse DNS to obtain the hostname. Where a hostname is resolved, forward DNS resolution is performed to check the IP address is really associated with the hostname. If the checks pass then the patterns supplied are checked. Any patterns supplied for numeric ip addresses are also checked. ##### Component Name daemon ##### Additional Information Fixes #6438 * Reverse lookup on ip to get hostname * Forward lookup on hostname to get IP addresses. * Validation that the incomming ip is associated with the host. If these checks fail the hostname is discarded so it cannot match against the access-list patterns. If these checks validate the ip successfully then the resolved hostname is pattern-matched as described in the previous commit.
2019-09-13Buffer overflow (#6817)Andrew Moss
* Buffer overflow The host field in the web_client is to store the value of the Host HTTP header, but it is an arbitrary size and there are no length checks. I could not see an easy way to exploit it but this checks it will not overflow the buffer. * Fix warnings on @thiagoftsm build system.
2019-09-03HTTP response message (#6595)thiagoftsm
* health_connection: http_error_pattern This commit brings an unique pattern for the Netdata webserver errors, now Netdata uses define for all web error * http_error_pattern: API v1 This PR also brings the pattern for the web_api_v1.c
2019-08-28url_registry: Remove of unecessary NULL (#6714)thiagoftsm
Summary With the recent changes on URL parsing, we caused an issue in the registry. Requests were not being logged, so the counters remained the same. The reason was that we modified the request buffer and the daemon couldn't access the cookie required to store the info. After @cosmix review, I notice that there was a code that is not necessary on Netdata. I executed tests with the stress.sh and urls/request.sh scripts, this last I used during 4 hours with cron calling it each hour.
2019-08-15Fix Markdown Lint warnings (#6664)Promise Akpan
* make remark access all directories * detailed fix after autofix by remark lint * cross check autofix for this set of files * crosscheck more files * crosschecking and small fixes * crosscheck autofixed md files
2019-08-13 Change "netdata" to "Netdata" in all docs (#6621)Joel Hans
* First pass of changing netdata to Netdata * Second pass of netdata -> Netdata * Starting work on netdata with no whitespace after * Pass for netdata with no whitespace at the end * Pass for netdata with no whitespace at the front
2019-08-01Fix crash in malloc (#6583)thiagoftsm
* Server Crashing: URL search path The system was setting NULL in an address without to have the values * Server Crashing: URL script After to fix the SSL, the script were not 100% compatible, so I am bringing the solution here * Server Crashing: Fixes reported in the issue related a possible NULL value to be kept and wrong variable * Server Crashing: Readable code and missing if It was a missing if yet, so I changed it, no less important I inverted the check order inside if to be more readable
2019-07-25minor - code readibility HTTP CODES as defines (#6539)Timo
2019-07-25Fix parsing SSL ACL along with others (#6468)thiagoftsm
* sslstream: ACL parser It was noticed in the issue 6457 that the some ACLs were not parsing correctly when they were along SSL acl, this commit fixes this' * sslstream: remove comments This commit removes the comments that were present while I was testing the code * sslstream: Tests This commit adds ACL tests to check the Netdata response to them * sslstream: Tests Fix the extension to upload the files * sslstream: more tests In this commit I am bringing more tests, including the ssl tests' * sslstream: leading space Remove leading space from variable that was creating problem with shellcheck * sslstream: glob Remove special character from script * sslstream: Makefile The Makefile diretives were pointed to wrong files * sslstream: Missing stream encrypt This commit solves the problem of the stream not be encrypted, but it is not the final solution, because the parser made is incomplete. * sslstream: Finish encrypt channel This commit brings the step that I was missing, the complete encryptation in the communication between Master and Slave * sslstream: Fix argument in script After the latest tests, it was verified that two arguments given to a function inside the script were not correct, with this PR I am fixing this! * sslstream: Fix argument in info Instead to call a function to deliver an integer I was passing a size_t value. Only cmake showed this, but not in my clion! :/ * sslstream: Fix redirect When we were having different SSL configuration, the system were not applying the option for all * sslstream: Update documentation Our documentation was not clear about the rules according our code so I am updating the text to explain for the users * sslstream: Adjust script With this last commit, I am adjusting the tests to avoid false positive * sslstream: Missing elif The previous commit had a missing elif in the shell script * sslstream: Split ports Before this commit Netdata was having SSL as a global option, now it has as a real ACL. * sslstream: reduce context The stream variable will not be affected in the master side, it is only necessary on the slave side, so I am reducing the context of it * sslstream: Force SSL When the user has certificate and he does not set any SSL flag, it is necessary to append the SSL=force flag * sslstream: Default flag It is necessary to have a default flag when the SSL flags are not SET * sslstream: remove comments Remove comments from the scrip * sslstream: moving flag It is better the flag to be set inside socket instead everytime there is a new connection * sslstream: documentation Fix a sentence in the web/server/README.md
2019-07-25Url parser refactoring (#6247)thiagoftsm
* URL_parser_review comments 1 * URL_parser_review restoring web_client.c * URL_parser_review restoring url.h * URL_parser_review restoring web_client.h * URL_parser_review restoring inlined.h * URL_parser_review restoring various * URL_parser_review commenting! * URL_parser_review last checks! * URL_parser_review registry! * URL_parser_review codacy errors! * URL_parser_review codacy errors 2! * URL_parser_review end of request! * URL_parser_review * URL_parser_review format fix * URL_parser_review restoring * URL_parser_review stopped at 5! * URL_parser_review formatting! * URL_parser_review: Started the map of the query string when it is necessary * URL_parser_review: With these adjusts in the URL library we are now able to parser all the escape characters! * URL_parser_review: code review Fixes problems and format asked by coworkers! * URL_parser_review: adjust script The script was not 100% according the shellcheck specifications, no less important it was a direct script instead a .in file * sslstream: Rebase 2 It was necessary to change a function due the UTF-8 * sslstream: Fixing 6426 We had a cast error introduced by other PR, so I am fixing here * URL_parser_review Change .gitignore to avoid considering a script file.
2019-07-24Utf8 Badge Fix And URL Parser International Support (initial) (#6426)Timo
#### Summary Fixes #3117 Additionally it adds support for UTF-8 in URL parser (as it should). Label sizes now are updated by browser with JavaScript (although guess is still calculated by verdana11_widths with minor improvements) #### Component Name API/Badges, LibNetData/URL #### Additional Information It was found that not only verdana11_widths need to be updated but the url parser replaces international characters with spaces (one space per each byte of multibyte character). Therefore I update both to support international chars.
2019-07-10SSL vs. TLS consistency and clarification in documentation (#6414)Joel Hans
* Changed references of SSL to TLS where appropriate * A few more tweaks on SSL->TLS * Fixed whitespace issue from @vlvkobal * Fixing sentence on OpenTSDB for @thiagoftsm
2019-06-27Backend and SSL! (#6220)thiagoftsm
* SSL_backend Begin of the encryptation of backend! * SSL_backend changing opentsdb! * SSL_backend fix HTTP message with JSON! * SSL_backend HTTP API done! * SSL_fix_format preparing to connect with proxy! * SSL_backend wip SSL send/receive ! * SSL_backend working with proxy * SSL_backend removing comments! * SSL_backend docummentation! * SSL_backend review]! * SSL_backend organizing! * Alarm_backend remove comments! * SSL_backend! * SSL_backend typedef! * SSL_backend bring switch! * SSL_backend commiting format changes! * SSL_backend fix github parser! * SSL_Backend fix format! * SSL_backend switch everything! * SSL_backend reviewing! * SSL_backend comments! * SSL_backend indentation! * SSL_backend indentation 3! * SSL_backend documentation! * SSL_backend hidden pointer! * SSL_backend missing space * SSL_backend change documentation! * SSL_backend change documentation 2!
2019-06-21Handle file descriptors running out (#6303)Markos Fountoulakis
* Handle file descriptors running out * Added alarm for dbengine FS and I/O errors * more verbose alarm message * * Added File-Descriptor budget to Database Engine instances. * Changed FD budget of the web server from 50% to 25%. * Allocated 25% of FDs to dbengine. * Created a new dbengine global FD utilization chart.
2019-06-07Revert "New URL parser (#6070)"Pavlos Emm. Katsoulakis
This reverts commit 58b7d95a7ec9c576f8a06bbab07f755846b5349a. --- As agreed with @thiago and @cakrit we revert URL parser changes, to buy the time on a more detailed investigation ---
2019-06-06New URL parser (#6070)thiagoftsm
* URL_parser 3 * URL_parser rebase 2! * URL_parameter parsing 3 * URL_parameter parsing 4 * URL_parameter parsing 5 * URL_parser alarms * URL_parser finish the basic structure * URL_parser codacity fixes! * URL_parser scripts! * URL_parser codacy! * URL_parser rebase 3! * URL_parser host fixes! * URL_parser host fixes 2! * URL_parser fix spaces! * URL_parser error message! * URL_parser Christopher requests! * URL_parser alarms fixed! * URL_parser health fixed! * URL_parser rebase 4! * URL_parser C fix write format! * URL_parser fix bugs due cache!
2019-06-05netdata/daemon: Eliminate a couple of warnings, plus tabs removal (#6222)Paul Emm. Katsoulakis
* netdata/daemon: Eliminate a couple of warnings, plus tabs removal on that file * netdata/daemon: fix indent
2019-05-31SSL implementation for Netdata (#5956)thiagoftsm
* SSL implementation for Netdata * Upload of fixes asked by @paulkatsoulakis and @cakrit * Fix local computer * Adding openssl to webserver * fixing.. * HTTPS almost there * Codacity * HTTPS day 3 * HTTPS without Bio step 1 * HTTPS without Bio step 2 * HTTPS without Bio step 3 * HTTPS without Bio step 4 * HTTPS without Bio step 5 * HTTPS without Bio step 6 * HTTPS without Bio step 7 * HTTPS without Bio step 8 * HTTPS without Bio step 9 * HTTPS without Bio step 10 * SSL on streaming 1 * Daily pull * HTTPS without Bio step 11 * HTTPS without Bio step 12 * HTTPS without Bio step 13 * HTTPS without Bio step 14 * SSL_Interception change documentation * HTTPS without Bio step 15 * HTTPS without Bio step 16 * SSL_Interception fix codacity * SSL_Interception fix doc * SSL_Interception comments * SSL_Interception fixing problems! * SSL_Interception killing bugs * SSL_Interception changing parameter * SSL_Implementation documentation and script * SSL_Implementation multiple fixes * SSL_Implementation installer and cipher * SSL_Implementation Redirect 301 * SSL_Implementation webserver doc and install-or-update.sh * SSL_Implementation error 00000001:lib(0):func(0):reason(1) * SSL_Implementation web server doc * SSL_Implementation SEGFAULT on Fedora * SSL_Implementation fix ^SSL=force|optional * SSL_Implementation Redirect and Ciphers * SSL_Implementation race condition 1 * SSL_Implementation Fix Location * SSL_Implementation Fix Location 2 * SSL_Implementation Fix stream * SSL_Implementation Fix stream 2 * SSL_Implementation Fix stream 3 * SSL_Implementation last problems! * SSL_Implementation adjusts to commit! * SSL_Implementation documentation permission! * SSL_Implementation documentation permission 2! * SSL_Implementation documentation permission 3!
2019-05-13Do not cache alarms and info api calls. Extend no-cache headers (#5999)Chris Akritidis
2019-04-09Return responses even in memory mode none (#5819)Chris Akritidis
2019-02-21Fix timer crash during exit (#5464)Markos Fountoulakis
2019-02-20Remove v from Server header (#5440)Chris Akritidis
2019-01-18Bug fix for netdata behind authenticated proxies (#5216)Chris Akritidis
* Was incorrectly updating the headers when the Authorization header was being sent * Use X-Auth-Token instead of Authorization header, to allow the management API to work authenticated behind proxies as well
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-16 04:07:49 +0000