diff options
author | Costa Tsaousis <costa@netdata.cloud> | 2024-11-17 13:19:50 +0200 |
---|---|---|
committer | GitHub <noreply@github.com> | 2024-11-17 13:19:50 +0200 |
commit | fbea64a02da5dc32fe17a9be741a1f9c6fa5949a (patch) | |
tree | d6ef470a7e9bfa17b840ee0f6faff6a4aecd1397 | |
parent | fa1abcf0cafd3e0c2f9b3a694e586c143040c666 (diff) |
-rw-r--r-- | docs/netdata-agent/sizing-netdata-agents/ram-requirements.md | 37 |
1 files changed, 37 insertions, 0 deletions
diff --git a/docs/netdata-agent/sizing-netdata-agents/ram-requirements.md b/docs/netdata-agent/sizing-netdata-agents/ram-requirements.md index a4ccf5507a..f45e4516e5 100644 --- a/docs/netdata-agent/sizing-netdata-agents/ram-requirements.md +++ b/docs/netdata-agent/sizing-netdata-agents/ram-requirements.md @@ -1,3 +1,4 @@ + # RAM Requirements With default configuration about database tiers, Netdata should need about 16KiB per unique metric collected, independently of the data collection frequency. @@ -12,6 +13,41 @@ This number can be lowered by limiting the number of database tier or switching ## On Metrics Centralization Points, Netdata Parents +|Description|Scope|RAM Required|Notes| +|:---|:---:|:---:|:---| +|metrics with retention|time-series in the db|1 KiB|Metadata and indexes. +|metrics currently collected|time-series collected|20 KiB|16 KiB for db + 4 KiB for collection structures. +|metrics with machine learning models|time-series collected|5 KiB|The trained models per dimension. +|nodes with retention|nodes in the db|10 KiB|Metadata and indexes. +|nodes currently received|nodes collected|512 KiB|Structures and reception buffers. +|nodes currently sent|nodes collected|512 KiB|Structures and dispatch buffers. + +These numbers vary depending on name length, the number of dimensions per instance and per context, the number and length of the labels added, the number of machine learning models maintained and similar parameters. For most use cases, they represent the worst case scenario, so you may find out Netdata actually needs less than that. + +Each metric currently being collected needs (1 index + 20 collection + 5 ml) = 26 KiB. When it stops being collected it needs 1 KiB (index). + +Each node currently being collected needs (10 index + 512 reception + 512 dispatch) = 1034 KiB. When it stops being collected it needs 10 KiB (index). + +### Example + +A Netdata Parents cluster (2 nodes) has 1 million currently collected metrics from 500 nodes, and 10 million archived metrics from 5000 nodes: + +|Description|Entries|RAM per Entry|Total RAM| +|:---|:---:|:---:|---:| +|metrics with retention|11 million|1 KiB|10742 MiB| +|metrics currently collected|1 million|20 KiB|19531 MiB| +|metrics with machine learning models|1 million|5 KiB|4883 MiB| +|nodes with retention|5500|10 KiB|52 MiB| +|nodes currently received|500|512 KiB|256 MiB| +|nodes currently sent|500|512 KiB|256 MiB| +|**Memory required per node**|||**35.7 GiB**| + +On highly volatile environments (like Kubernetes clusters), the database retention can significantly affect memory usage. Usually reducing retention on higher database tiers helps reducing memory usage. + +## Database Size + +Netdata supports memory ballooning to automatically adjust its database memory size based on the number of time-series concurrently being collected. + The general formula, with the default configuration of database tiers, is: ```text @@ -39,6 +75,7 @@ There are two cache sizes that can be configured in `netdata.conf`: Both of them are dynamically adjusted to use some of the total memory computed above. The configuration in `netdata.conf` allows providing additional memory to them, increasing their caching efficiency. + ## I have a Netdata Parent that is also a systemd-journal logs centralization point, what should I know? Logs usually require significantly more disk space and I/O bandwidth than metrics. For optimal performance, we recommend to store metrics and logs on separate, independent disks. |