diff options
author | Costa Tsaousis <costa@netdata.cloud> | 2023-11-22 08:27:25 +0000 |
---|---|---|
committer | GitHub <noreply@github.com> | 2023-11-22 10:27:25 +0200 |
commit | 3e508c8f95ab0bdf8b6d74501437210d7b8d2919 (patch) | |
tree | 965caf50e73854f638bc9fbc4aebfbd4690619e5 /database | |
parent | 8f31356a0c0cb5956b9a31ffd5abb45d85de1656 (diff) |
New logging layer (#16357)
* cleanup of logging - wip
* first working iteration
* add errno annotator
* replace old logging functions with netdata_logger()
* cleanup
* update error_limit
* fix remanining error_limit references
* work on fatal()
* started working on structured logs
* full cleanup
* default logging to files; fix all plugins initialization
* fix formatting of numbers
* cleanup and reorg
* fix coverity issues
* cleanup obsolete code
* fix formatting of numbers
* fix log rotation
* fix for older systems
* add detection of systemd journal via stderr
* finished on access.log
* remove left-over transport
* do not add empty fields to the logs
* journal get compact uuids; X-Transaction-ID header is added in web responses
* allow compiling on systems without memfd sealing
* added libnetdata/uuid directory
* move datetime formatters to libnetdata
* add missing files
* link the makefiles in libnetdata
* added uuid_parse_flexi() to parse UUIDs with and without hyphens; the web server now read X-Transaction-ID and uses it for functions and web responses
* added stream receiver, sender, proc plugin and pluginsd log stack
* iso8601 advanced usage; line_splitter module in libnetdata; code cleanup
* add message ids to streaming inbound and outbound connections
* cleanup line_splitter between lines to avoid logging garbage; when killing children, kill them with SIGABRT if internal checks is enabled
* send SIGABRT to external plugins only if we are not shutting down
* fix cross cleanup in pluginsd parser
* fatal when there is a stack error in logs
* compile netdata with -fexceptions
* do not kill external plugins with SIGABRT
* metasync info logs to debug level
* added severity to logs
* added json output; added options per log output; added documentation; fixed issues mentioned
* allow memfd only on linux
* moved journal low level functions to journal.c/h
* move health logs to daemon.log with proper priorities
* fixed a couple of bugs; health log in journal
* updated docs
* systemd-cat-native command to push structured logs to journal from the command line
* fix makefiles
* restored NETDATA_LOG_SEVERITY_LEVEL
* fix makefiles
* systemd-cat-native can also work as the logger of Netdata scripts
* do not require a socket to systemd-journal to log-as-netdata
* alarm notify logs in native format
* properly compare log ids
* fatals log alerts; alarm-notify.sh working
* fix overflow warning
* alarm-notify.sh now logs the request (command line)
* anotate external plugins logs with the function cmd they run
* added context, component and type to alarm-notify.sh; shell sanitization removes control character and characters that may be expanded by bash
* reformatted alarm-notify logs
* unify cgroup-network-helper.sh
* added quotes around params
* charts.d.plugin switched logging to journal native
* quotes for logfmt
* unify the status codes of streaming receivers and senders
* alarm-notify: dont log anything, if there is nothing to do
* all external plugins log to stderr when running outside netdata; alarm-notify now shows an error when notifications menthod are needed but are not available
* migrate cgroup-name.sh to new logging
* systemd-cat-native now supports messages with newlines
* socket.c logs use priority
* cleanup log field types
* inherit the systemd set INVOCATION_ID if found
* allow systemd-cat-native to send messages to a systemd-journal-remote URL
* log2journal command that can convert structured logs to journal export format
* various fixes and documentation of log2journal
* updated log2journal docs
* updated log2journal docs
* updated documentation of fields
* allow compiling without libcurl
* do not use socket as format string
* added version information to newly added tools
* updated documentation and help messages
* fix the namespace socket path
* print errno with error
* do not timeout
* updated docs
* updated docs
* updated docs
* log2journal updated docs and params
* when talking to a remote journal, systemd-cat-native batches the messages
* enable lz4 compression for systemd-cat-native when sending messages to a systemd-journal-remote
* Revert "enable lz4 compression for systemd-cat-native when sending messages to a systemd-journal-remote"
This reverts commit b079d53c11f6687cd64d804fdd7b24c0492bf245.
* note about uncompressed traffic
* log2journal: code reorg and cleanup to make modular
* finished rewriting log2journal
* more comments
* rewriting rules support
* increased limits
* updated docs
* updated docs
* fix old log call
* use journal only when stderr is connected to journal
* update netdata.spec for libcurl, libpcre2 and log2journal
* pcre2-devel
* do not require pcre2 in centos < 8, amazonlinux < 2023, open suse
* log2journal only on systems pcre2 is available
* ignore log2journal in .gitignore
* avoid log2journal on centos 7, amazonlinux 2 and opensuse
* add pcre2-8 to static build
* undo last commit
* Bundle to static
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
* Add build deps for deb packages
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
* Add dependencies; build from source
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
* Test build for amazon linux and centos expect to fail for suse
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
* fix minor oversight
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
* Reorg code
* Add the install from source (deps) as a TODO
* Not enable the build on suse ecosystem
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
---------
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
Co-authored-by: Tasos Katsoulas <tasos@netdata.cloud>
Diffstat (limited to 'database')
-rw-r--r-- | database/contexts/rrdcontext.c | 65 | ||||
-rw-r--r-- | database/engine/cache.c | 7 | ||||
-rw-r--r-- | database/engine/datafile.c | 11 | ||||
-rw-r--r-- | database/engine/metric.c | 27 | ||||
-rw-r--r-- | database/engine/pdc.c | 10 | ||||
-rw-r--r-- | database/engine/rrdengine.c | 16 | ||||
-rwxr-xr-x | database/engine/rrdengineapi.c | 18 | ||||
-rw-r--r-- | database/rrd.h | 2 | ||||
-rw-r--r-- | database/rrdcalc.c | 8 | ||||
-rw-r--r-- | database/rrdfunctions.c | 4 | ||||
-rw-r--r-- | database/rrdhost.c | 269 | ||||
-rw-r--r-- | database/rrdset.c | 9 | ||||
-rw-r--r-- | database/sqlite/sqlite_aclk_alert.c | 58 | ||||
-rw-r--r-- | database/sqlite/sqlite_aclk_node.c | 9 | ||||
-rw-r--r-- | database/sqlite/sqlite_health.c | 4 | ||||
-rw-r--r-- | database/sqlite/sqlite_metadata.c | 28 |
16 files changed, 330 insertions, 215 deletions
diff --git a/database/contexts/rrdcontext.c b/database/contexts/rrdcontext.c index 8538d17f28..9dee39be2a 100644 --- a/database/contexts/rrdcontext.c +++ b/database/contexts/rrdcontext.c @@ -224,26 +224,31 @@ void rrdcontext_hub_checkpoint_command(void *ptr) { struct ctxs_checkpoint *cmd = ptr; if(!rrdhost_check_our_claim_id(cmd->claim_id)) { - netdata_log_error("RRDCONTEXT: received checkpoint command for claim_id '%s', node id '%s', but this is not our claim id. Ours '%s', received '%s'. Ignoring command.", - cmd->claim_id, cmd->node_id, - localhost->aclk_state.claimed_id?localhost->aclk_state.claimed_id:"NOT SET", - cmd->claim_id); + nd_log(NDLS_DAEMON, NDLP_WARNING, + "RRDCONTEXT: received checkpoint command for claim_id '%s', node id '%s', " + "but this is not our claim id. Ours '%s', received '%s'. Ignoring command.", + cmd->claim_id, cmd->node_id, + localhost->aclk_state.claimed_id?localhost->aclk_state.claimed_id:"NOT SET", + cmd->claim_id); return; } RRDHOST *host = rrdhost_find_by_node_id(cmd->node_id); if(!host) { - netdata_log_error("RRDCONTEXT: received checkpoint command for claim id '%s', node id '%s', but there is no node with such node id here. Ignoring command.", - cmd->claim_id, - cmd->node_id); + nd_log(NDLS_DAEMON, NDLP_WARNING, + "RRDCONTEXT: received checkpoint command for claim id '%s', node id '%s', " + "but there is no node with such node id here. Ignoring command.", + cmd->claim_id, cmd->node_id); return; } if(rrdhost_flag_check(host, RRDHOST_FLAG_ACLK_STREAM_CONTEXTS)) { - netdata_log_info("RRDCONTEXT: received checkpoint command for claim id '%s', node id '%s', while node '%s' has an active context streaming.", - cmd->claim_id, cmd->node_id, rrdhost_hostname(host)); + nd_log(NDLS_DAEMON, NDLP_NOTICE, + "RRDCONTEXT: received checkpoint command for claim id '%s', node id '%s', " + "while node '%s' has an active context streaming.", + cmd->claim_id, cmd->node_id, rrdhost_hostname(host)); // disable it temporarily, so that our worker will not attempt to send messages in parallel rrdhost_flag_clear(host, RRDHOST_FLAG_ACLK_STREAM_CONTEXTS); @@ -252,8 +257,10 @@ void rrdcontext_hub_checkpoint_command(void *ptr) { uint64_t our_version_hash = rrdcontext_version_hash(host); if(cmd->version_hash != our_version_hash) { - netdata_log_error("RRDCONTEXT: received version hash %"PRIu64" for host '%s', does not match our version hash %"PRIu64". Sending snapshot of all contexts.", - cmd->version_hash, rrdhost_hostname(host), our_version_hash); + nd_log(NDLS_DAEMON, NDLP_NOTICE, + "RRDCONTEXT: received version hash %"PRIu64" for host '%s', does not match our version hash %"PRIu64". " + "Sending snapshot of all contexts.", + cmd->version_hash, rrdhost_hostname(host), our_version_hash); #ifdef ENABLE_ACLK // prepare the snapshot @@ -275,41 +282,55 @@ void rrdcontext_hub_checkpoint_command(void *ptr) { #endif } - internal_error(true, "RRDCONTEXT: host '%s' enabling streaming of contexts", rrdhost_hostname(host)); + nd_log(NDLS_DAEMON, NDLP_DEBUG, + "RRDCONTEXT: host '%s' enabling streaming of contexts", + rrdhost_hostname(host)); + rrdhost_flag_set(host, RRDHOST_FLAG_ACLK_STREAM_CONTEXTS); char node_str[UUID_STR_LEN]; uuid_unparse_lower(*host->node_id, node_str); - netdata_log_access("ACLK REQ [%s (%s)]: STREAM CONTEXTS ENABLED", node_str, rrdhost_hostname(host)); + nd_log(NDLS_ACCESS, NDLP_DEBUG, + "ACLK REQ [%s (%s)]: STREAM CONTEXTS ENABLED", + node_str, rrdhost_hostname(host)); } void rrdcontext_hub_stop_streaming_command(void *ptr) { struct stop_streaming_ctxs *cmd = ptr; if(!rrdhost_check_our_claim_id(cmd->claim_id)) { - netdata_log_error("RRDCONTEXT: received stop streaming command for claim_id '%s', node id '%s', but this is not our claim id. Ours '%s', received '%s'. Ignoring command.", - cmd->claim_id, cmd->node_id, - localhost->aclk_state.claimed_id?localhost->aclk_state.claimed_id:"NOT SET", - cmd->claim_id); + nd_log(NDLS_DAEMON, NDLP_WARNING, + "RRDCONTEXT: received stop streaming command for claim_id '%s', node id '%s', " + "but this is not our claim id. Ours '%s', received '%s'. Ignoring command.", + cmd->claim_id, cmd->node_id, + localhost->aclk_state.claimed_id?localhost->aclk_state.claimed_id:"NOT SET", + cmd->claim_id); return; } RRDHOST *host = rrdhost_find_by_node_id(cmd->node_id); if(!host) { - netdata_log_error("RRDCONTEXT: received stop streaming command for claim id '%s', node id '%s', but there is no node with such node id here. Ignoring command.", - cmd->claim_id, cmd->node_id); + nd_log(NDLS_DAEMON, NDLP_WARNING, + "RRDCONTEXT: received stop streaming command for claim id '%s', node id '%s', " + "but there is no node with such node id here. Ignoring command.", + cmd->claim_id, cmd->node_id); return; } if(!rrdhost_flag_check(host, RRDHOST_FLAG_ACLK_STREAM_CONTEXTS)) { - netdata_log_error("RRDCONTEXT: received stop streaming command for claim id '%s', node id '%s', but node '%s' does not have active context streaming. Ignoring command.", - cmd->claim_id, cmd->node_id, rrdhost_hostname(host)); + nd_log(NDLS_DAEMON, NDLP_NOTICE, + "RRDCONTEXT: received stop streaming command for claim id '%s', node id '%s', " + "but node '%s' does not have active context streaming. Ignoring command.", + cmd->claim_id, cmd->node_id, rrdhost_hostname(host)); return; } - internal_error(true, "RRDCONTEXT: host '%s' disabling streaming of contexts", rrdhost_hostname(host)); + nd_log(NDLS_DAEMON, NDLP_DEBUG, + "RRDCONTEXT: host '%s' disabling streaming of contexts", + rrdhost_hostname(host)); + rrdhost_flag_clear(host, RRDHOST_FLAG_ACLK_STREAM_CONTEXTS); } diff --git a/database/engine/cache.c b/database/engine/cache.c index 1bc2b383a5..af7600298b 100644 --- a/database/engine/cache.c +++ b/database/engine/cache.c @@ -1171,9 +1171,10 @@ static bool evict_pages_with_filter(PGC *cache, size_t max_skip, size_t max_evic if(all_of_them && !filter) { pgc_ll_lock(cache, &cache->clean); if(cache->clean.stats->entries) { - error_limit_static_global_var(erl, 1, 0); - error_limit(&erl, "DBENGINE CACHE: cannot free all clean pages, %zu are still in the clean queue", - cache->clean.stats->entries); + nd_log_limit_static_global_var(erl, 1, 0); + nd_log_limit(&erl, NDLS_DAEMON, NDLP_NOTICE, + "DBENGINE CACHE: cannot free all clean pages, %zu are still in the clean queue", + cache->clean.stats->entries); } pgc_ll_unlock(cache, &cache->clean); } diff --git a/database/engine/datafile.c b/database/engine/datafile.c index fcda84bd60..7bb30e2345 100644 --- a/database/engine/datafile.c +++ b/database/engine/datafile.c @@ -479,14 +479,18 @@ int create_new_datafile_pair(struct rrdengine_instance *ctx, bool having_lock) int ret; char path[RRDENG_PATH_MAX]; - netdata_log_info("DBENGINE: creating new data and journal files in path %s", ctx->config.dbfiles_path); + nd_log(NDLS_DAEMON, NDLP_DEBUG, + "DBENGINE: creating new data and journal files in path %s", + ctx->config.dbfiles_path); + datafile = datafile_alloc_and_init(ctx, 1, fileno); ret = create_data_file(datafile); if(ret) goto error_after_datafile; generate_datafilepath(datafile, path, sizeof(path)); - netdata_log_info("DBENGINE: created data file \"%s\".", path); + nd_log(NDLS_DAEMON, NDLP_INFO, + "DBENGINE: created data file \"%s\".", path); journalfile = journalfile_alloc_and_init(datafile); ret = journalfile_create(journalfile, datafile); @@ -494,7 +498,8 @@ int create_new_datafile_pair(struct rrdengine_instance *ctx, bool having_lock) goto error_after_journalfile; journalfile_v1_generate_path(datafile, path, sizeof(path)); - netdata_log_info("DBENGINE: created journal file \"%s\".", path); + nd_log(NDLS_DAEMON, NDLP_INFO, + "DBENGINE: created journal file \"%s\".", path); ctx_current_disk_space_increase(ctx, datafile->pos + journalfile->unsafe.pos); datafile_list_insert(ctx, datafile, having_lock); diff --git a/database/engine/metric.c b/database/engine/metric.c index 07ab251a97..571887797b 100644 --- a/database/engine/metric.c +++ b/database/engine/metric.c @@ -592,27 +592,30 @@ inline void mrg_update_metric_retention_and_granularity_by_uuid( time_t update_every_s, time_t now_s) { if(unlikely(last_time_s > now_s)) { - error_limit_static_global_var(erl, 1, 0); - error_limit(&erl, "DBENGINE JV2: wrong last time on-disk (%ld - %ld, now %ld), " - "fixing last time to now", - first_time_s, last_time_s, now_s); + nd_log_limit_static_global_var(erl, 1, 0); + nd_log_limit(&erl, NDLS_DAEMON, NDLP_WARNING, + "DBENGINE JV2: wrong last time on-disk (%ld - %ld, now %ld), " + "fixing last time to now", + first_time_s, last_time_s, now_s); last_time_s = now_s; } if (unlikely(first_time_s > last_time_s)) { - error_limit_static_global_var(erl, 1, 0); - error_limit(&erl, "DBENGINE JV2: wrong first time on-disk (%ld - %ld, now %ld), " - "fixing first time to last time", - first_time_s, last_time_s, now_s); + nd_log_limit_static_global_var(erl, 1, 0); + nd_log_limit(&erl, NDLS_DAEMON, NDLP_WARNING, + "DBENGINE JV2: wrong first time on-disk (%ld - %ld, now %ld), " + "fixing first time to last time", + first_time_s, last_time_s, now_s); first_time_s = last_time_s; } if (unlikely(first_time_s == 0 || last_time_s == 0)) { - error_limit_static_global_var(erl, 1, 0); - error_limit(&erl, "DBENGINE JV2: zero on-disk timestamps (%ld - %ld, now %ld), " - "using them as-is", - first_time_s, last_time_s, now_s); + nd_log_limit_static_global_var(erl, 1, 0); + nd_log_limit(&erl, NDLS_DAEMON, NDLP_WARNING, + "DBENGINE JV2: zero on-disk timestamps (%ld - %ld, now %ld), " + "using them as-is", + first_time_s, last_time_s, now_s); } bool added = false; diff --git a/database/engine/pdc.c b/database/engine/pdc.c index 028265b0e3..d27db5be78 100644 --- a/database/engine/pdc.c +++ b/database/engine/pdc.c @@ -772,7 +772,7 @@ VALIDATED_PAGE_DESCRIPTOR validate_page( if(unlikely(!vd.is_valid || updated)) { #ifndef NETDATA_INTERNAL_CHECKS - error_limit_static_global_var(erl, 1, 0); + nd_log_limit_static_global_var(erl, 1, 0); #endif char uuid_str[UUID_STR_LEN + 1]; uuid_unparse(*uuid, uuid_str); @@ -788,7 +788,7 @@ VALIDATED_PAGE_DESCRIPTOR validate_page( #ifdef NETDATA_INTERNAL_CHECKS internal_error(true, #else - error_limit(&erl, + nd_log_limit(&erl, NDLS_DAEMON, NDLP_ERR, #endif "DBENGINE: metric '%s' %s invalid page of type %u " "from %ld to %ld (now %ld), update every %ld, page length %zu, entries %zu (flags: %s)", @@ -808,7 +808,7 @@ VALIDATED_PAGE_DESCRIPTOR validate_page( #ifdef NETDATA_INTERNAL_CHECKS internal_error(true, #else - error_limit(&erl, + nd_log_limit(&erl, NDLS_DAEMON, NDLP_ERR, #endif "DBENGINE: metric '%s' %s page of type %u " "from %ld to %ld (now %ld), update every %ld, page length %zu, entries %zu (flags: %s), " @@ -915,8 +915,8 @@ static void epdl_extent_loading_error_log(struct rrdengine_instance *ctx, EPDL * if(end_time_s) log_date(end_time_str, LOG_DATE_LENGTH, end_time_s); - error_limit_static_global_var(erl, 1, 0); - error_limit(&erl, + nd_log_limit_static_global_var(erl, 1, 0); + nd_log_limit(&erl, NDLS_DAEMON, NDLP_ERR, "DBENGINE: error while reading extent from datafile %u of tier %d, at offset %" PRIu64 " (%u bytes) " "%s from %ld (%s) to %ld (%s) %s%s: " "%s", diff --git a/database/engine/rrdengine.c b/database/engine/rrdengine.c index 5eadd5ca7f..5cbbb4a870 100644 --- a/database/engine/rrdengine.c +++ b/database/engine/rrdengine.c @@ -1478,12 +1478,19 @@ static void *journal_v2_indexing_tp_worker(struct rrdengine_instance *ctx __mayb spinlock_unlock(&datafile->writers.spinlock); if(!available) { - netdata_log_info("DBENGINE: journal file %u needs to be indexed, but it has writers working on it - skipping it for now", datafile->fileno); + nd_log(NDLS_DAEMON, NDLP_NOTICE, + "DBENGINE: journal file %u needs to be indexed, but it has writers working on it - " + "skipping it for now", + datafile->fileno); + datafile = datafile->next; continue; } - netdata_log_info("DBENGINE: journal file %u is ready to be indexed", datafile->fileno); + nd_log(NDLS_DAEMON, NDLP_DEBUG, + "DBENGINE: journal file %u is ready to be indexed", + datafile->fileno); + pgc_open_cache_to_journal_v2(open_cache, (Word_t) ctx, (int) datafile->fileno, ctx->config.page_type, journalfile_migrate_to_v2_callback, (void *) datafile->journalfile); @@ -1496,7 +1503,10 @@ static void *journal_v2_indexing_tp_worker(struct rrdengine_instance *ctx __mayb } errno = 0; - internal_error(count, "DBENGINE: journal indexing done; %u files processed", count); + if(count) + nd_log(NDLS_DAEMON, NDLP_DEBUG, + "DBENGINE: journal indexing done; %u files processed", + count); worker_is_idle(); diff --git a/database/engine/rrdengineapi.c b/database/engine/rrdengineapi.c index 304e20661f..1ddce52438 100755 --- a/database/engine/rrdengineapi.c +++ b/database/engine/rrdengineapi.c @@ -361,12 +361,12 @@ static void rrdeng_store_metric_create_new_page(struct rrdeng_collect_handle *ha #ifdef NETDATA_INTERNAL_CHECKS internal_error(true, #else - error_limit_static_global_var(erl, 1, 0); - error_limit(&erl, - #endif - "DBENGINE: metric '%s' new page from %ld to %ld, update every %ld, has a conflict in main cache " - "with existing %s%s page from %ld to %ld, update every %ld - " - "is it collected more than once?", + nd_log_limit_static_global_var(erl, 1, 0); + nd_log_limit(&erl, NDLS_DAEMON, NDLP_WARNING, +#endif + "DBENGINE: metric '%s' new page from %ld to %ld, update every %ld, has a conflict in main cache " + "with existing %s%s page from %ld to %ld, update every %ld - " + "is it collected more than once?", uuid, page_entry.start_time_s, page_entry.end_time_s, (time_t)page_entry.update_every_s, pgc_is_page_hot(pgc_page) ? "hot" : "not-hot", @@ -521,8 +521,8 @@ static void store_metric_next_error_log(struct rrdeng_collect_handle *handle __m collect_page_flags_to_buffer(wb, handle->page_flags); } - error_limit_static_global_var(erl, 1, 0); - error_limit(&erl, + nd_log_limit_static_global_var(erl, 1, 0); + nd_log_limit(&erl, NDLS_DAEMON, NDLP_NOTICE, "DBENGINE: metric '%s' collected point at %ld, %s last collection at %ld, " "update every %ld, %s page from %ld to %ld, position %u (of %u), flags: %s", uuid, @@ -535,7 +535,7 @@ static void store_metric_next_error_log(struct rrdeng_collect_handle *handle __m (time_t)(handle->page_end_time_ut / USEC_PER_SEC), handle->page_position, handle->page_entries_max, wb ? buffer_tostring(wb) : "" - ); + ); buffer_free(wb); #else diff --git a/database/rrd.h b/database/rrd.h index 341a29e501..f3e7e17d8c 100644 --- a/database/rrd.h +++ b/database/rrd.h @@ -1023,7 +1023,7 @@ typedef enum __attribute__ ((__packed__)) rrdhost_flags { #ifdef NETDATA_INTERNAL_CHECKS #define rrdset_debug(st, fmt, args...) do { if(unlikely(debug_flags & D_RRD_STATS && rrdset_flag_check(st, RRDSET_FLAG_DEBUG))) \ - debug_int(__FILE__, __FUNCTION__, __LINE__, "%s: " fmt, rrdset_name(st), ##args); } while(0) + netdata_logger(NDLS_DEBUG, NDLP_DEBUG, __FILE__, __FUNCTION__, __LINE__, "%s: " fmt, rrdset_name(st), ##args); } while(0) #else #define rrdset_debug(st, fmt, args...) debug_dummy() #endif diff --git a/database/rrdcalc.c b/database/rrdcalc.c index 620883ec2f..1c0c3b584f 100644 --- a/database/rrdcalc.c +++ b/database/rrdcalc.c @@ -809,10 +809,10 @@ void rrdcalc_delete_alerts_not_matching_host_labels_from_this_host(RRDHOST *host continue; if(!rrdlabels_match_simple_pattern_parsed(host->rrdlabels, rc->host_labels_pattern, '=', NULL)) { - netdata_log_health("Health configuration for alarm '%s' cannot be applied, because the host %s does not have the label(s) '%s'", - rrdcalc_name(rc), - rrdhost_hostname(host), - rrdcalc_host_labels(rc)); + nd_log(NDLS_DAEMON, NDLP_DEBUG, + "Health configuration for alarm '%s' cannot be applied, " + "because the host %s does not have the label(s) '%s'", + rrdcalc_name(rc), rrdhost_hostname(host), rrdcalc_host_labels(rc)); rrdcalc_unlink_and_delete(host, rc, false); } diff --git a/database/rrdfunctions.c b/database/rrdfunctions.c index 6d68f3d33c..2659130f03 100644 --- a/database/rrdfunctions.c +++ b/database/rrdfunctions.c @@ -1007,11 +1007,11 @@ int rrd_function_run(RRDHOST *host, BUFFER *result_wb, int timeout, const char * // the function can only be executed in async mode // put the function into the inflight requests - char uuid_str[UUID_STR_LEN]; + char uuid_str[UUID_COMPACT_STR_LEN]; if(!transaction) { uuid_t uuid; uuid_generate_random(uuid); - uuid_unparse_lower(uuid, uuid_str); + uuid_unparse_lower_compact(uuid, uuid_str); transaction = uuid_str; } diff --git a/database/rrdhost.c b/database/rrdhost.c index dac3472cbf..686518a4bb 100644 --- a/database/rrdhost.c +++ b/database/rrdhost.c @@ -80,8 +80,6 @@ static inline void rrdhost_init() { } RRDHOST_ACQUIRED *rrdhost_find_and_acquire(const char *machine_guid) { - netdata_log_debug(D_RRD_CALLS, "rrdhost_find_and_acquire() host %s", machine_guid); - return (RRDHOST_ACQUIRED *)dictionary_get_and_acquire_item(rrdhost_root_index, machine_guid); } @@ -116,8 +114,9 @@ static inline RRDHOST *rrdhost_index_add_by_guid(RRDHOST *host) { rrdhost_option_set(host, RRDHOST_OPTION_INDEXED_MACHINE_GUID); else { rrdhost_option_clear(host, RRDHOST_OPTION_INDEXED_MACHINE_GUID); - netdata_log_error("RRDHOST: %s() host with machine guid '%s' is already indexed", - __FUNCTION__, host->machine_guid); + nd_log(NDLS_DAEMON, NDLP_NOTICE, + "RRDHOST: host with machine guid '%s' is already indexed. Not adding it again.", + host->machine_guid); } return host; @@ -126,8 +125,9 @@ static inline RRDHOST *rrdhost_index_add_by_guid(RRDHOST *host) { static void rrdhost_index_del_by_guid(RRDHOST *host) { if(rrdhost_option_check(host, RRDHOST_OPTION_INDEXED_MACHINE_GUID)) { if(!dictionary_del(rrdhost_root_index, host->machine_guid)) - netdata_log_error("RRDHOST: %s() failed to delete machine guid '%s' from index", - __FUNCTION__, host->machine_guid); + nd_log(NDLS_DAEMON, NDLP_NOTICE, + "RRDHOST: failed to delete machine guid '%s' from index", + host->machine_guid); rrdhost_option_clear(host, RRDHOST_OPTION_INDEXED_MACHINE_GUID); } @@ -148,8 +148,9 @@ static inline void rrdhost_index_del_hostname(RRDHOST *host) { if(rrdhost_option_check(host, RRDHOST_OPTION_INDEXED_HOSTNAME)) { if(!dictionary_del(rrdhost_root_index_hostname, rrdhost_hostname(host))) - netdata_log_error("RRDHOST: %s() failed to delete hostname '%s' from index", - __FUNCTION__, rrdhost_hostname(host)); + nd_log(NDLS_DAEMON, NDLP_NOTICE, + "RRDHOST: failed to delete hostname '%s' from index", + rrdhost_hostname(host)); rrdhost_option_clear(host, RRDHOST_OPTION_INDEXED_HOSTNAME); } @@ -303,11 +304,11 @@ static RRDHOST *rrdhost_create( int is_localhost, bool archived ) { - netdata_log_debug(D_RRDHOST, "Host '%s': adding with guid '%s'", hostname, guid); - if(memory_mode == RRD_MEMORY_MODE_DBENGINE && !dbengine_enabled) { - netdata_log_error("memory mode 'dbengine' is not enabled, but host '%s' is configured for it. Falling back to 'alloc'", - hostname); + nd_log(NDLS_DAEMON, NDLP_ERR, + "memory mode 'dbengine' is not enabled, but host '%s' is configured for it. Falling back to 'alloc'", + hostname); + memory_mode = RRD_MEMORY_MODE_ALLOC; } @@ -392,7 +393,9 @@ int is_legacy = 1; (host->rrd_memory_mode == RRD_MEMORY_MODE_DBENGINE && is_legacy))) { int r = mkdir(host->cache_dir, 0775); if(r != 0 && errno != EEXIST) - netdata_log_error("Host '%s': cannot create directory '%s'", rrdhost_hostname(host), host->cache_dir); + nd_log(NDLS_DAEMON, NDLP_CRIT, + "Host '%s': cannot create directory '%s'", + rrdhost_hostname(host), host->cache_dir); } } @@ -418,7 +421,9 @@ int is_legacy = 1; ret = mkdir(dbenginepath, 0775); if (ret != 0 && errno != EEXIST) - netdata_log_error("Host '%s': cannot create directory '%s'", rrdhost_hostname(host), dbenginepath); + nd_log(NDLS_DAEMON, NDLP_CRIT, + "Host '%s': cannot create directory '%s'", + rrdhost_hostname(host), dbenginepath); else ret = 0; // succeed @@ -459,8 +464,9 @@ int is_legacy = 1; } if (ret) { // check legacy or multihost initialization success - netdata_log_error("Host '%s': cannot initialize host with machine guid '%s'. Failed to initialize DB engine at '%s'.", - rrdhost_hostname(host), host->machine_guid, host->cache_dir); + nd_log(NDLS_DAEMON, NDLP_CRIT, + "Host '%s': cannot initialize host with machine guid '%s'. Failed to initialize DB engine at '%s'.", + rrdhost_hostname(host), host->machine_guid, host->cache_dir); rrd_wrlock(); rrdhost_free___while_having_rrd_wrlock(host, true); @@ -508,10 +514,13 @@ int is_legacy = 1; RRDHOST *t = rrdhost_index_add_by_guid(host); if(t != host) { - netdata_log_error("Host '%s': cannot add host with machine guid '%s' to index. It already exists as host '%s' with machine guid '%s'.", - rrdhost_hostname(host), host->machine_guid, rrdhost_hostname(t), t->machine_guid); + nd_log(NDLS_DAEMON, NDLP_NOTICE, + "Host '%s': cannot add host with machine guid '%s' to index. It already exists as host '%s' with machine guid '%s'.", + rrdhost_hostname(host), host->machine_guid, rrdhost_hostname(t), t->machine_guid); + if (!is_localhost) rrdhost_free___while_having_rrd_wrlock(host, true); + rrd_unlock(); return NULL; } @@ -527,21 +536,22 @@ int is_legacy = 1; // ------------------------------------------------------------------------ - netdata_log_info("Host '%s' (at registry as '%s') with guid '%s' initialized" - ", os '%s'" - ", timezone '%s'" - ", tags '%s'" - ", program_name '%s'" - ", program_version '%s'" - ", update every %d" - ", memory mode %s" - ", history entries %d" - ", streaming %s" - " (to '%s' with api key '%s')" - ", health %s" - ", cache_dir '%s'" - ", alarms default handler '%s'" - ", alarms default recipient '%s'" + nd_log(NDLS_DAEMON, NDLP_INFO, + "Host '%s' (at registry as '%s') with guid '%s' initialized" + ", os '%s'" + ", timezone '%s'" + ", tags '%s'" + ", program_name '%s'" + ", program_version '%s'" + ", update every %d" + ", memory mode %s" + ", history entries %d" + ", streaming %s" + " (to '%s' with api key '%s')" + ", health %s" + ", cache_dir '%s'" + ", alarms default handler '%s'" + ", alarms default recipient '%s'" , rrdhost_hostname(host) , rrdhost_registry_hostname(host) , host->machine_guid @@ -621,44 +631,56 @@ static void rrdhost_update(RRDHOST *host host->registry_hostname = string_strdupz((registry_hostname && *registry_hostname)?registry_hostname:hostname); if(strcmp(rrdhost_hostname(host), hostname) != 0) { - netdata_log_info("Host '%s' has been renamed to '%s'. If this is not intentional it may mean multiple hosts are using the same machine_guid.", rrdhost_hostname(host), hostname); + nd_log(NDLS_DAEMON, NDLP_WARNING, + "Host '%s' has been renamed to '%s'. If this is not intentional it may mean multiple hosts are using the same machine_guid.", + rrdhost_hostname(host), hostname); + rrdhost_init_hostname(host, hostname, true); } else { rrdhost_index_add_hostname(host); } if(strcmp(rrdhost_program_name(host), program_name) != 0) { - netdata_log_info("Host '%s' switched program name from '%s' to '%s'", rrdhost_hostname(host), rrdhost_program_name(host), program_name); + nd_log(NDLS_DAEMON, NDLP_NOTICE, + "Host '%s' switched program name from '%s' to '%s'", + rrdhost_hostname(host), rrdhost_program_name(host), program_name); + STRING *t = host->program_name; host->program_name = string_strdupz(program_name); string_freez(t); } if(strcmp(rrdhost_program_version(host), program_version) != 0) { - netdata_log_info("Host '%s' switched program version from '%s' to '%s'", rrdhost_hostname(host), rrdhost_program_version(host), program_version); + nd_log(NDLS_DAEMON, NDLP_NOTICE, + "Host '%s' switched program version from '%s' to '%s'", + rrdhost_hostname(host), rrdhost_program_version(host), program_version); + STRING *t = host->program_version; host->program_version = string_strdupz(program_version); string_freez(t); } if(host->rrd_update_every != update_every) - netdata_log_error("Host '%s' has an update frequency of %d seconds, but the wanted one is %d seconds. " - "Restart netdata here to apply the new settings.", - rrdhost_hostname(host), host->rrd_update_every, update_every); + nd_log(NDLS_DAEMON, NDLP_WARNING, + "Host '%s' has an update frequency of %d seconds, but the wanted one is %d seconds. " + "Restart netdata here to apply the new settings.", + rrdhost_hostname(host), host->rrd_update_every, update_every); |