New logging layer (#16357)

* cleanup of logging - wip

* first working iteration

* add errno annotator

* replace old logging functions with netdata_logger()

* cleanup

* update error_limit

* fix remanining error_limit references

* work on fatal()

* started working on structured logs

* full cleanup

* default logging to files; fix all plugins initialization

* fix formatting of numbers

* cleanup and reorg

* fix coverity issues

* cleanup obsolete code

* fix formatting of numbers

* fix log rotation

* fix for older systems

* add detection of systemd journal via stderr

* finished on access.log

* remove left-over transport

* do not add empty fields to the logs

* journal get compact uuids; X-Transaction-ID header is added in web responses

* allow compiling on systems without memfd sealing

* added libnetdata/uuid directory

* move datetime formatters to libnetdata

* add missing files

* link the makefiles in libnetdata

* added uuid_parse_flexi() to parse UUIDs with and without hyphens; the web server now read X-Transaction-ID and uses it for functions and web responses

* added stream receiver, sender, proc plugin and pluginsd log stack

* iso8601 advanced usage; line_splitter module in libnetdata; code cleanup

* add message ids to streaming inbound and outbound connections

* cleanup line_splitter between lines to avoid logging garbage; when killing children, kill them with SIGABRT if internal checks is enabled

* send SIGABRT to external plugins only if we are not shutting down

* fix cross cleanup in pluginsd parser

* fatal when there is a stack error in logs

* compile netdata with -fexceptions

* do not kill external plugins with SIGABRT

* metasync info logs to debug level

* added severity to logs

* added json output; added options per log output; added documentation; fixed issues mentioned

* allow memfd only on linux

* moved journal low level functions to journal.c/h

* move health logs to daemon.log with proper priorities

* fixed a couple of bugs; health log in journal

* updated docs

* systemd-cat-native command to push structured logs to journal from the command line

* fix makefiles

* restored NETDATA_LOG_SEVERITY_LEVEL

* fix makefiles

* systemd-cat-native can also work as the logger of Netdata scripts

* do not require a socket to systemd-journal to log-as-netdata

* alarm notify logs in native format

* properly compare log ids

* fatals log alerts; alarm-notify.sh working

* fix overflow warning

* alarm-notify.sh now logs the request (command line)

* anotate external plugins logs with the function cmd they run

* added context, component and type to alarm-notify.sh; shell sanitization removes control character and characters that may be expanded by bash

* reformatted alarm-notify logs

* unify cgroup-network-helper.sh

* added quotes around params

* charts.d.plugin switched logging to journal native

* quotes for logfmt

* unify the status codes of streaming receivers and senders

* alarm-notify: dont log anything, if there is nothing to do

* all external plugins log to stderr when running outside netdata; alarm-notify now shows an error when notifications menthod are needed but are not available

* migrate cgroup-name.sh to new logging

* systemd-cat-native now supports messages with newlines

* socket.c logs use priority

* cleanup log field types

* inherit the systemd set INVOCATION_ID if found

* allow systemd-cat-native to send messages to a systemd-journal-remote URL

* log2journal command that can convert structured logs to journal export format

* various fixes and documentation of log2journal

* updated log2journal docs

* updated log2journal docs

* updated documentation of fields

* allow compiling without libcurl

* do not use socket as format string

* added version information to newly added tools

* updated documentation and help messages

* fix the namespace socket path

* print errno with error

* do not timeout

* updated docs

* updated docs

* updated docs

* log2journal updated docs and params

* when talking to a remote journal, systemd-cat-native batches the messages

* enable lz4 compression for systemd-cat-native when sending messages to a systemd-journal-remote

* Revert "enable lz4 compression for systemd-cat-native when sending messages to a systemd-journal-remote"

This reverts commit b079d53c11f6687cd64d804fdd7b24c0492bf245.

* note about uncompressed traffic

* log2journal: code reorg and cleanup to make modular

* finished rewriting log2journal

* more comments

* rewriting rules support

* increased limits

* updated docs

* updated docs

* fix old log call

* use journal only when stderr is connected to journal

* update netdata.spec for libcurl, libpcre2 and log2journal

* pcre2-devel

* do not require pcre2 in centos < 8, amazonlinux < 2023, open suse

* log2journal only on systems pcre2 is available

* ignore log2journal in .gitignore

* avoid log2journal on centos 7, amazonlinux 2 and opensuse

* add pcre2-8 to static build

* undo last commit

* Bundle to static

Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>

* Add build deps for deb packages

Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>

* Add dependencies; build from source

Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>

* Test build for amazon linux and centos expect to fail for suse

Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>

* fix minor oversight

Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>

* Reorg code

* Add the install from source (deps) as a TODO
* Not enable the build on suse ecosystem

Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>

---------

Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
Co-authored-by: Tasos Katsoulas <tasos@netdata.cloud>

1 files changed, 81 insertions, 0 deletions

diff --git a/libnetdata/datetime/iso8601.c b/libnetdata/datetime/iso8601.c
new file mode 100644
index 0000000000..8e3f4e0276
--- /dev/null
+++ b/libnetdata/datetime/iso8601.c
@@ -0,0 +1,81 @@
+// SPDX-License-Identifier: GPL-3.0-or-later
+
+#include "../libnetdata.h"
+
+size_t iso8601_datetime_ut(char *buffer, size_t len, usec_t now_ut, ISO8601_OPTIONS options) {
+    if(unlikely(!buffer || len == 0))
+        return 0;
+
+    time_t t = (time_t)(now_ut / USEC_PER_SEC);
+    struct tm *tmp, tmbuf;
+
+    if(options & ISO8601_UTC)
+        // Use gmtime_r for UTC time conversion.
+        tmp = gmtime_r(&t, &tmbuf);
+    else
+        // Use localtime_r for local time conversion.
+        tmp = localtime_r(&t, &tmbuf);
+
+    if (unlikely(!tmp)) {
+        buffer[0] = '\0';
+        return 0;
+    }
+
+    // Format the date and time according to the ISO 8601 format.
+    size_t used_length = strftime(buffer, len, "%Y-%m-%dT%H:%M:%S", tmp);
+    if (unlikely(used_length == 0)) {
+        buffer[0] = '\0';
+        return 0;
+    }
+
+    if(options & ISO8601_MILLISECONDS) {
+        // Calculate the remaining microseconds
+        int milliseconds = (int) ((now_ut % USEC_PER_SEC) / USEC_PER_MS);
+        if(milliseconds && len - used_length > 4)
+            used_length += snprintfz(buffer + used_length, len - used_length, ".%03d", milliseconds);
+    }
+    else if(options & ISO8601_MICROSECONDS) {
+        // Calculate the remaining microseconds
+        int microseconds = (int) (now_ut % USEC_PER_SEC);
+        if(microseconds && len - used_length > 7)
+            used_length += snprintfz(buffer + used_length, len - used_length, ".%06d", microseconds);
+    }
+
+    if(options & ISO8601_UTC) {
+        if(used_length + 1 < len) {
+            buffer[used_length++] = 'Z';
+            buffer[used_length] = '\0'; // null-terminate the string.
+        }
+    }
+    else {
+        // Calculate the timezone offset in hours and minutes from UTC.
+        long offset = tmbuf.tm_gmtoff;
+        int hours = (int) (offset / 3600); // Convert offset seconds to hours.
+        int minutes = (int) ((offset % 3600) / 60); // Convert remainder to minutes (keep the sign for minutes).
+
+        // Check if timezone is UTC.
+        if(hours == 0 && minutes == 0) {
+            // For UTC, append 'Z' to the timestamp.
+            if(used_length + 1 < len) {
+                buffer[used_length++] = 'Z';
+                buffer[used_length] = '\0'; // null-terminate the string.
+            }
+        }
+        else {
+            // For non-UTC, format the timezone offset. Omit minutes if they are zero.
+            if(minutes == 0) {
+                // Check enough space is available for the timezone offset string.
+                if(used_length + 3 < len) // "+hh\0"
+                    used_length += snprintfz(buffer + used_length, len - used_length, "%+03d", hours);
+            }
+            else {
+                // Check enough space is available for the timezone offset string.
+                if(used_length + 6 < len) // "+hh:mm\0"
+                    used_length += snprintfz(buffer + used_length, len - used_length,
+                                             "%+03d:%02d", hours, abs(minutes));
+            }
+        }
+    }
+
+    return used_length;
+}