summaryrefslogtreecommitdiffstats
path: root/docs
diff options
context:
space:
mode:
authorJoel Hans <joel@netdata.cloud>2020-11-30 15:16:34 -0700
committerGitHub <noreply@github.com>2020-11-30 15:16:34 -0700
commit795e8a9d615f362f8dedd947ad6ad11d27b6c1db (patch)
treecf184d58b87ad161d4294e0998361295eee5967f /docs
parentafd07313d0a10aff360529050b8a84a5d32d0f1d (diff)
Add guide: How to optimize Netdata's performance (#10271)
* Init guide * Working on guide * Finish first draft of performance guide * Retitle and fix links in other docs * Acutally add guide * Copyedit pass * Fixes for Amy
Diffstat (limited to 'docs')
-rw-r--r--docs/Performance.md226
-rw-r--r--docs/configuration-guide.md4
-rw-r--r--docs/getting-started.md2
-rw-r--r--docs/guides/configure/performance.md233
-rw-r--r--docs/guides/longer-metrics-storage.md4
-rw-r--r--docs/guides/monitor/pi-hole-raspberry-pi.md2
-rw-r--r--docs/guides/step-by-step/step-10.md2
-rw-r--r--docs/high-performance-netdata.md167
-rw-r--r--docs/netdata-for-IoT.md4
9 files changed, 242 insertions, 402 deletions
diff --git a/docs/Performance.md b/docs/Performance.md
deleted file mode 100644
index 219bb6ba42..0000000000
--- a/docs/Performance.md
+++ /dev/null
@@ -1,226 +0,0 @@
-<!--
----
-title: "Performance"
-custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/Performance.md
----
--->
-
-# Performance
-
-Netdata performance is affected by:
-
-**Data collection**
-
-- the number of charts for which data are collected
-- the number of plugins running
-- the technology of the plugins (i.e. BASH plugins are slower than binary plugins)
-- the frequency of data collection
-
-You can control all the above.
-
-**Web clients accessing the data**
-
-- the duration of the charts in the dashboard
-- the number of charts refreshes requested
-- the compression level of the web responses
-
-- - -
-
-## Netdata Daemon
-
-For most server systems, with a few hundred charts and a few thousand dimensions, the Netdata daemon, without any web clients accessing it, should not use more than 1% of a single core.
-
-To prove Netdata scalability, check issue [#1323](https://github.com/netdata/netdata/issues/1323#issuecomment-265501668) where Netdata collects 95.000 metrics per second, with 12% CPU utilization of a single core!
-
-In embedded systems, if the Netdata daemon is using a lot of CPU without any web clients accessing it, you should lower the data collection frequency. To set the data collection frequency, edit `/etc/netdata/netdata.conf` and set `update_every` to a higher number (this is the frequency in seconds data are collected for all charts: higher number of seconds = lower frequency, the default is 1 for per second data collection). You can also set this frequency per module or chart. Check the [daemon configuration](/daemon/config/README.md) for plugins and charts. For specific modules, the configuration needs to be changed in:
-
-- `python.d.conf` for [python](/collectors/python.d.plugin/README.md)
-- `node.d.conf` for [nodejs](/collectors/node.d.plugin/README.md)
-- `charts.d.conf` for [bash](/collectors/charts.d.plugin/README.md)
-
-## Plugins
-
-If a plugin is using a lot of CPU, you should lower its update frequency, or if you wrote it, re-factor it to be more CPU efficient. Check [External Plugins](/collectors/plugins.d/README.md) for more details on writing plugins.
-
-## CPU consumption when web clients are accessing dashboards
-
-Netdata is very efficient when servicing web clients. On most server platforms, Netdata should be able to serve **1800 web client requests per second per core** for auto-refreshing charts.
-
-Normally, each user connected will request less than 10 chart refreshes per second (the page may have hundreds of charts, but only the visible are refreshed). So you can expect 180 users per CPU core accessing dashboards before having any delays.
-
-Netdata runs with the lowest possible process priority, so even if 1000 users are accessing dashboards, it should not influence your applications. CPU utilization will reach 100%, but your applications should get all the CPU they need.
-
-To lower the CPU utilization of Netdata when clients are accessing the dashboard, set `web compression level = 1`, or disable web compression completely by setting `enable web responses gzip compression = no`. Both settings are in the `[web]` section.
-
-## Monitoring a heavily-loaded system
-
-While running, Netdata does not depend much on disk I/O aside from writing to log files and the [database
-engine](/database/engine/README.md) "spilling" historical metrics to disk when it uses all its available RAM.
-
-Under a heavy system load, plugins that need disk may stop and show gaps during heavy system load, but the Netdata
-daemon itself should be able to work and collect values from `/proc` and `/sys` and serve web clients accessing it.
-
-Keep in mind that Netdata saves its database when it exits, and loads it up again when started.
-
-## Netdata process priority
-
-By default, Netdata runs with the `idle` process scheduler, which assigns CPU resources to Netdata, only when the system has such resources to spare.
-
-The following `netdata.conf` settings control this:
-
-```
-[global]
- process scheduling policy = idle
- process scheduling priority = 0
- process nice level = 19
-```
-
-The policies supported by Netdata are `idle` (the Netdata default), `other` (also as `nice`), `batch`, `rr`, `fifo`. Netdata also recognizes `keep` and `none` to keep the current settings without changing them.
-
-For `other`, `nice` and `batch`, the setting `process nice level = 19` is activated to configure the nice level of Netdata. Nice gets values -20 (highest) to 19 (lowest).
-
-For `rr` and `fifo`, the setting `process scheduling priority = 0` is activated to configure the priority of the relative scheduling policy. Priority gets values 1 (lowest) to 99 (highest).
-
-For the details of each scheduler, see `man sched_setscheduler` and `man sched`.
-
-When Netdata is running under systemd, it can only lower its priority (the default is `other` with `nice level = 0`). If you want to make Netdata to get more CPU than that, you will need to set in `netdata.conf`:
-
-```
-[global]
- process scheduling policy = keep
-```
-
-and edit `/etc/systemd/system/netdata.service` and add:
-
-```
-CPUSchedulingPolicy=other | batch | idle | fifo | rr
-CPUSchedulingPriority=99
-Nice=-10
-```
-
-## Running Netdata in embedded devices
-
-Embedded devices usually have very limited CPU resources available, and in most cases, just a single core.
-
-> keep in mind that Netdata on RPi 2 and 3 does not require any tuning. The default settings will be good. The following tunables apply only when running Netdata on RPi 1 or other very weak IoT devices.
-
-We suggest to do the following:
-
-### 1. Disable External plugins
-
-External plugins can consume more system resources than the Netdata server. Disable the ones you don't need. If you need them, increase their `update every` value (again in `/etc/netdata/netdata.conf`), so that they do not run that frequently.
-
-Edit `/etc/netdata/netdata.conf`, find the `[plugins]` section:
-
-```
-[plugins]
- proc = yes
-
- tc = no
- idlejitter = no
- cgroups = no
- checks = no
- apps = no
- charts.d = no
- node.d = no
- python.d = no
-
- plugins directory = /usr/libexec/netdata/plugins.d
- enable running new plugins = no
- check for new plugins every = 60
-```
-
-In detail:
-
-| plugin|description|
-|:----:|:----------|
-| `proc`|the internal plugin used to monitor the system. Normally, you don't want to disable this. You can disable individual functions of it at the next section.|
-| `tc`|monitoring network interfaces QoS (tc classes)|
-| `idlejitter`|internal plugin (written in C) that attempts show if the systems starved for CPU. Disabling it will eliminate a thread.|
-| `cgroups`|monitoring linux containers. Most probably you are not going to need it. This will also eliminate another thread.|
-| `checks`|a debugging plugin, which is disabled by default.|
-| `apps`|a plugin that monitors system processes. It is very complex and heavy (consumes twice the CPU resources of the Netdata daemon), so if you don't need to monitor the process tree, you can disable it.|
-| `charts.d`|BASH plugins (squid, nginx, mysql, etc). This is a heavy plugin, that consumes twice the CPU resources of the Netdata daemon.|
-| `node.d`|node.js plugin, currently used for SNMP data collection and monitoring named (the name server).|
-| `python.d`|has many modules and can use over 20MB of memory.|
-
-For most IoT devices, you can disable all plugins except `proc`. For `proc` there is another section that controls which functions of it you need. Check the next section.
-
----
-
-### 2. Disable internal plugins
-
-In this section you can select which modules of the `proc` plugin you need. All these are run in a single thread, one after another. Still, each one needs some RAM and consumes some CPU cycles. With all the modules enabled, the `proc` plugin adds ~9 MiB on top of the 5 MiB required by the Netdata daemon.
-
-```
-[plugin:proc]
- # /proc/net/dev = yes # network interfaces
- # /proc/diskstats = yes # disks
-...
-```
-
-Refer to the [proc.plugins documentation](/collectors/proc.plugin/README.md) for the list and description of all the proc plugin modules.
-
-### 3. Lower internal plugin update frequency
-
-If Netdata is still using a lot of CPU, lower its update frequency. Going from per second updates, to once every 2 seconds updates, will cut the CPU resources of all Netdata programs **in half**, and you will still have very frequent updates.
-
-If the CPU of the embedded device is too weak, try setting even lower update frequency. Experiment with `update every = 5` or `update every = 10` (higher number = lower frequency) in `netdata.conf`, until you get acceptable results.
-
-Keep in mind this will also force dashboard chart refreshes to happen at the same rate. So increasing this number actually lowers data collection frequency but also lowers dashboard chart refreshes frequency.
-
-This is a dashboard on a device with `[global].update every = 5` (this device is a media player and is now playing a movie):
-
-![pi1](https://cloud.githubusercontent.com/assets/2662304/15338489/ca84baaa-1c88-11e6-9ab2-118208e11ce1.gif)
-
-### 4. Disable logs
-
-Normally, you will not need them. To disable them, set:
-
-```
-[global]
- debug log = none
- error log = none
- access log = none
-```
-
-### 5. Lower Netdata's memory usage
-
-You can change the amount of RAM and disk the database engine uses for all charts and their dimensions with the
-following settings in the `[global]` section of `netdata.conf`:
-
-```conf
-[global]
- # memory mode = dbengine
- # page cache size = 32
- # dbengine disk space = 256
-```
-
-See the [database engine documentation](/database/engine/README.md) or our [guide on metrics
-retention](/docs/guides/longer-metrics-storage.md) for more details on lowering the database engine's memory requirements.
-
-### 6. Disable gzip compression of responses
-
-Gzip compression of the web responses is using more CPU that the rest of Netdata. You can lower the compression level or disable gzip compression completely. You can disable it, like this:
-
-```
-[web]
- enable gzip compression = no
-```
-
-To lower the compression level, do this:
-
-```
-[web]
- enable gzip compression = yes
- gzip compression level = 1
-```
-
-Finally, if no web server is installed on your device, you can use port tcp/80 for Netdata:
-
-```
-[web]
- port = 80
-```
-
-[![analytics](https://www.google-analytics.com/collect?v=1&aip=1&t=pageview&_s=1&ds=github&dr=https%3A%2F%2Fgithub.com%2Fnetdata%2Fnetdata&dl=https%3A%2F%2Fmy-netdata.io%2Fgithub%2Fdocs%2FPerformance&_u=MAC~&cid=5792dfd7-8dc4-476b-af31-da2fdb9f93d2&tid=UA-64295674-3)](<>)
diff --git a/docs/configuration-guide.md b/docs/configuration-guide.md
index e9a6fdbe7e..6af45dfd1e 100644
--- a/docs/configuration-guide.md
+++ b/docs/configuration-guide.md
@@ -151,8 +151,8 @@ documentation](/web/server/README.md#binding-netdata-to-multiple-ports) for more
#### Reduce the resources Netdata uses
-The page on [Netdata performance](/docs/Performance.md) has an excellent guide on how to reduce the Netdata cpu/disk/RAM
-utilization to levels suitable even for the weakest [IoT devices](/docs/netdata-for-IoT.md).
+The [Netdata performance guide](/docs/guides/configure/performance.md) outlines many ways to reduce the Netdata
+CPU/disk/RAM utilization to levels suitable even for the weakest [IoT devices](/docs/netdata-for-IoT.md).
#### Change when Netdata saves metrics to disk
diff --git a/docs/getting-started.md b/docs/getting-started.md
index 6b6d306d3d..5ad1a48c00 100644
--- a/docs/getting-started.md
+++ b/docs/getting-started.md
@@ -157,7 +157,7 @@ changes based on your particular Nginx setup.
- Look at the [full list of data collection modules](/collectors/COLLECTORS.md)
to configure your sources for auto-detection and monitoring.
-- Improve the [performance](/docs/Performance.md) of Netdata on low-memory systems.
+- Improve the [performance](/docs/guides/configure/performance.md) of Netdata on low-memory systems.
- Configure `systemd` to expose [systemd services
utilization](/collectors/cgroups.plugin/README.md#monitoring-systemd-services) metrics automatically.
- [Reconfigure individual charts](/daemon/config/README.md#per-chart-configuration) in `netdata.conf`.
diff --git a/docs/guides/configure/performance.md b/docs/guides/configure/performance.md
new file mode 100644
index 0000000000..50087877b6
--- /dev/null
+++ b/docs/guides/configure/performance.md
@@ -0,0 +1,233 @@
+<!--
+title: How to optimize the Netdata Agent's performance
+description: "While the Netdata Agent is designed to monitor a system with only 1% CPU, you can optimize its performance for low-resource systems."
+image: /img/seo/guides/configure/performance.png
+custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/guides/configure/performance.md
+-->
+
+# How to optimize the Netdata Agent's performance
+
+We designed the Netdata Agent to be incredibly lightweight, even when it's collecting a few thousand dimensions every
+second and visualizing that data into hundreds of charts. The Agent itself should never use more than 1% of a single CPU
+core, roughly 100 MiB of RAM, and minimal disk I/O to collect, store, and visualize all this data.
+
+We take this scalability seriously. We have one user [running
+Netdata](https://github.com/netdata/netdata/issues/1323#issuecomment-266427841) on a system with 144 cores and 288
+threads. Despite collecting 100,000 metrics every second, the Agent still only uses 9% CPU utilization on a
+single core.
+
+But not everyone has such powerful systems at their disposal. For example, you might run the Agent on a cloud VM with
+only 512 MiB of RAM, or an IoT device like a [Raspberry Pi](/docs/guides/monitor/pi-hole-raspberry-pi.md). In these
+cases, reducing Netdata's footprint beyond its already diminuitive size can pay big dividends, giving your services more
+horsepower while still monitoring the health and the performance of the node, OS, hardware, and applications.
+
+## Prerequisites
+
+- A node running the Netdata Agent.
+- Familiarity with configuring the Netdata Agent with `edit-config`.
+
+If you're not familiar with how to configure the Netdata Agent, read our [node configuration
+doc](/docs/configure/nodes.md) before continuing with this guide. This guide assumes familiarity with the Netdata config
+directory, using `edit-config`, and the process of uncommenting/editing various settings in `netdata.conf` and other
+configuration files.
+
+## What affects Netdata's performance?
+
+Netdata's performance is primarily affected by **data collection/retention** and **clients accessing data**.
+
+You can configure almost all aspects of data collection/retention, and certain aspects of clients accessing data. For
+example, you can't control how many users might be viewing a local Agent dashboard, [viewing an
+infrastructure](/docs/visualize/overview-infrastructure.md) in real-time with Netdata Cloud, or running [Metric
+Correlations](https://learn.netdata.cloud/docs/cloud/insights/metric-correlations).
+
+The Netdata Agent runs with the lowest possible [process scheduling
+policy](/daemon/README.md#netdata-process-scheduling-policy), which is `nice 19`, and uses the `idle` process scheduler.
+Together, these settings ensure that the Agent only gets CPU resources when the node has CPU resources to space. If the
+node reaches 100% CPU utilization, the Agent is stopped first to ensure your applications get any available resources.
+In addition, under heavy load, collectors that require disk I/O may stop and show gaps in charts.
+
+Let's walk through the best ways to improve the Netdata Agent's performance.
+
+## Reduce collection frequency
+
+The fastest way to improve the Agent's resource utilization is to reduce how often it collects metrics.
+
+## Global
+
+If you don't need per-second metrics, or if the Agent uses a lot of CPU even when no one is viewing that node's
+dashboard, configure the Agent to collect metrics less often.
+
+Open `netdata.conf` and edit the `update every` setting. The default is `1`, meaning that the Agent updates every
+second.
+
+If you change this to `2`, Netdata collects metrics every other second, which will effectively halve the CPU utilization
+dedicated for metrics collection. Set this to `5` or `10` to collect metrics every 5 or 10 seconds, respectively.
+
+```conf
+[global]
+ update every: 5
+```
+
+## Specific plugin or collector
+
+If you did not [reduce the global collection frequency](#global) but find that a specific plugin/collector uses too many
+resources, you can reduce its frequency. You configure [internal
+collectors](/docs/collect/how-collectors-work.md#collector-architecture-and-terminolog) in `netdata.conf` and external
+collectors in their individual `.conf` files.
+
+To reduce the frequency of an [internal
+plugin/collector](/docs/collect/how-collectors-work.md#collector-architecture-and-terminology), open `netdata.conf` and
+find the appropriate section. For example, to reduce the frequency of the `apps` plugin, which collects and visualizes
+metrics on application resource utilization:
+
+```conf
+[plugin:apps]
+ update every = 5
+```
+
+To [configure an individual collector](/docs/collect/enable-configure.md), open its specific configuration file with
+`edit-config` and look for the `update_every` setting. For example, to reduce the frequency of the `nginx` collector,
+run `sudo ./edit-config go.d/nginx.conf`:
+
+```conf
+# [ GLOBAL ]
+update_every: 10
+```
+
+## Disable unneeded plugins or collectors
+
+If you know that you don't need an [entire plugin or a specific
+collector](/docs/collect/how-collectors-work.md#collector-architecture-and-terminology), you can disable any of them.
+Keep in mind that if a plugin/collector has nothing to do, it simply shuts down and does not consume system resources.
+You will only improve the Agent's performance by disabling plugins/collectors that are actively collecting metrics.
+
+Open `netdata.conf` and scroll down to the `[plugins]` section. To disable any plugin, uncomment it and set the value to
+`no`. For example, to explicitly keep the `proc` and `go.d` plugins enabled while disabling `python.d`, `charts.d`, and
+`node.d`.
+
+```conf
+[plugins]
+ proc = yes
+ python.d = no
+ charts.d = no
+ node.d = no
+ go.d = yes
+```
+
+Disable specific collectors by opening their respective plugin configuration files, uncommenting the line for the
+collector, and setting its value to `no`.
+
+```bash
+sudo ./edit-config go.d.conf
+sudo ./edit-config python.d.conf
+sudo ./edit-config node.d.conf
+sudo ./edit-config charts.d.conf
+```
+
+For example, to disable a few Python collectors:
+
+```conf
+ apache: no
+ dockerd: no
+ fail2ban: no
+```
+
+## Lower memory usage for metrics retention
+
+Reduce the disk space that the [database engine](/database/engine/README.md) uses to retain metrics by editing
+the `dbengine multihost disk space` option in `netdata.conf`. The default value is `256`, but can be set to a minimum of
+`64`. By reducing the disk space allocation, Netdata also needs to store less metadata in the node's memory.
+
+The `page cache size` option also directly impacts Netdata's memory usage, but has a minimum value of `32`.
+
+Reducing the value of `dbengine multihost disk space` does slim down Netdata's resource usage, but it also reduces how
+long Netdata retains metrics. Find the right balance of performance and metrics retention by using the [dbengine
+calculator](/docs/store/change-metrics-storage.md#calculate-the-system-resources-ram-disk-space-needed-to-store-metrics).
+
+All the settings are found in the `[global]` section of `netdata.conf`:
+
+```conf
+[global]
+ memory mode = dbengine
+ page cache size = 32
+ dbengine multihost disk space = 256
+```
+
+## Run Netdata behind Nginx
+
+A dedicated web server like Nginx provides far more robustness than the Agent's internal [web server](/web/README.md).
+Nginx can handle more concurrent connections, reuse idle connections, and use fast gzip compression to reduce payloads.
+
+For details on installing Nginx as a proxy for the local Agent dashboard, see our [Nginx
+doc](/docs/Running-behind-nginx.md).
+
+After you complete Nginx setup according to the doc linked above, we recommend setting `keepalive` to `1024`, and using
+gzip compression with the following options in the `location /` block:
+
+```conf
+ location / {
+ ...
+ gzip on;
+ gzip_proxied any;
+ gzip_types *;
+ }
+```
+
+Finally, edit `netdata.conf` with the following settings:
+
+```conf
+[global]
+ bind socket to IP = 127.0.0.1
+ access log = none
+ disconnect idle web clients after seconds = 3600
+ enable web responses gzip compression = no
+```
+
+## Disable/lower gzip compression for the dashboard
+
+If you choose not to run the Agent behind Nginx, you can disable or lower the Agent's web server's gzip compression.
+While gzip compression does reduce the size of the HTML/CSS/JS payload, it does use additional CPU while a user is
+looking at the local Agent dashboard.
+
+To disable gzip compression, open `netdata.conf` and find the `[web]` section:
+
+```conf
+[web]
+ enable gzip compression = no
+```
+
+Or to lower the default compression level:
+
+```conf
+[web]
+ enable gzip compression = yes
+ gzip compression level = 1
+```
+
+## Disable logs
+
+If you installation is working correctly, and you're not actively auditing Netdata's logs, disable them in
+`netdata.conf`.
+
+```conf
+[global]
+ debug log = none
+ error log = none
+ access log = none
+```
+
+## What's next?
+
+We hope this guide helped you better understand how to optimize the performance of the Netdata Agent.
+
+Now that your Agent is running smoothly, we recommend you [secure your nodes](/docs/configure/nodes.md) if you haven't
+already.
+
+Next, dive into some of Netdata's more complex features, such as configuring its health watchdog or exporting metrics to
+an external time-series database.
+
+- [Interact with dashboards and charts](/docs/visualize/interact-dashboards-charts.md)
+- [Configure health alarms](/docs/monitor/configure-alarms.md)
+- [Export metrics to external time-series databases](/docs/export/external-databases.md)
+
+[![analytics](https://www.google-analytics.com/collect?v=1&aip=1&t=pageview&_s=1&ds=github&dr=https%3A%2F%2Fgithub.com%2Fnetdata%2Fnetdata&dl=https%3A%2F%2Fmy-netdata.io%2Fgithub%2Fdocs%2Fguides%2Fconfigure%2Fperformance.md&_u=MAC~&cid=5792dfd7-8dc4-476b-af31-da2fdb9f93d2&tid=UA-64295674-3)](<>)
diff --git a/docs/guides/longer-metrics-storage.md b/docs/guides/longer-metrics-storage.md
index c4c9a3a05e..e58265c55f 100644
--- a/docs/guides/longer-metrics-storage.md
+++ b/docs/guides/longer-metrics-storage.md
@@ -154,7 +154,7 @@ probably want to see it in action!
For more information about how to pan charts to view historical metrics, see our documentation on [using
charts](/web/README.md#using-charts).
-And if you'd now like to reduce Netdata's resource usage, view our [performance guide](/docs/Performance.md) for
-our best practices on optimization.
+And if you'd now like to reduce Netdata's resource usage, view our [performance
+guide](/docs/guides/configure/performance.md) for our best practices on optimization.
[![analytics](https://www.google-analytics.com/collect?v=1&aip=1&t=pageview&_s=1&ds=github&dr=https%3A%2F%2Fgithub.com%2Fnetdata%2Fnetdata&dl=https%3A%2F%2Fmy-netdata.io%2Fgithub%2Fdocs%2Fguides%2Flonger-metrics-storage&_u=MAC~&cid=5792dfd7-8dc4-476b-af31-da2fdb9f93d2&tid=UA-64295674-3)](<>)
diff --git a/docs/guides/monitor/pi-hole-raspberry-pi.md b/docs/guides/monitor/pi-hole-raspberry-pi.md
index daed36910d..97db0a1bfa 100644
--- a/docs/guides/monitor/pi-hole-raspberry-pi.md
+++ b/docs/guides/monitor/pi-hole-raspberry-pi.md
@@ -149,7 +149,7 @@ configure Netdata to more specific goals.
Most importantly, you can always install additional services and instantly collect metrics from many of them with our
[300+ integrations](/collectors/COLLECTORS.md).
-- [Optimize performance](/docs/Performance.md) using tweaks developed for IoT devices.
+- [Optimize performance](/docs/guides/configure/performance.md) using tweaks developed for IoT devices.
- [Stream Raspberry Pi metrics](/streaming/README.md) to a parent host for easy access or longer-term storage.
- [Tweak alarms](/health/QUICKSTART.md) for either Pi-hole or the health of your Raspberry Pi.
- [Export metrics to external databases](/exporting/README.md) with the exporting engine.
diff --git a/docs/guides/step-by-step/step-10.md b/docs/guides/step-by-step/step-10.md
index 2a4d1732db..2fec3895a1 100644
--- a/docs/guides/step-by-step/step-10.md
+++ b/docs/guides/step-by-step/step-10.md
@@ -218,7 +218,7 @@ You're a real sysadmin now!
If you want to configure your Nginx proxy further, check out the following:
- [Running Netdata behind Nginx](/docs/Running-behind-nginx.md)
-- [High-performance Netdata](/docs/high-performance-netdata.md)
+- [How to optimize Netdata's performance](/docs/guides/configure/performance.md)
- [Enabling TLS on Netdata's dashboard](/web/server/README.md#enabling-tls-support)
And... you're _almost_ done with the Netdata guide.
diff --git a/docs/high-performance-netdata.md b/docs/high-performance-netdata.md
deleted file mode 100644
index 89272fe2b5..0000000000
--- a/docs/high-performance-netdata.md
+++ /dev/null
@@ -1,167 +0,0 @@
-<!--
----
-title: "High performance Netdata"
-custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/high-performance-netdata.md
----
--->
-
-# High performance Netdata
-
-If you plan to run a Netdata public on the internet, you will get the most performance out of it by following these
-rules:
-
-## 1. run behind nginx
-
-The internal web server is optimized to provide the best experience with few clients connected to it. Normally a web
-browser will make 4-6 concurrent connections to a web server, so that it can send requests in parallel. To best serve a
-single client, Netdata spawns a thread for each connection it receives (so 4-6 threads per connected web browser).
-
-If you plan to have your Netdata public on the internet, this strategy wastes resources. It provides a lock-free
-environment so each thread is autonomous to serve the browser, but it does not scale well. Running Netdata behind nginx,
-idle connections to Netdata can be reused, thus improving significantly the performance of Netdata.
-
-In the following nginx configuration we do the following:
-
-- allow nginx to maintain up to 1024 idle connections to Netdata (so Netdata will have up to 1024 threads waiting for
- requests)
-- allow nginx to compress the responses of Netdata (later we will disable gzip compression at Netdata)
-- we disable wordpress pingback attacks and allow only GET, HEAD and OPTIONS requests.
-
-```conf
-upstream backend {
- server 127.0.0.1:19999;
- keepalive 1024;
-}
-
-server {
- listen *:80;
- server_name my.web.server.name;
-
- location / {
- proxy_set_header X-Forwarded-Host $host;
- proxy_set_header X-Forwarded-Server $host;
- proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
- proxy_pass http://backend;
- proxy_http_version 1.1;
- proxy_pass_request_headers on;
- proxy_set_header Connection "keep-alive";
- proxy_store off;
- gzip on;
- gzip_proxied any;
- gzip_types *;
-
- # Block any HTTP requests other than GET, HEAD, and OPTIONS
- limit_except GET HEAD OPTIONS {
- deny all;
- }
- }
-
- # WordPress Pingback Request Denial
- if ($http_user_agent ~* "WordPress") {
- return 403;
- }
-
-}
-```
-
-Then edit `/etc/netdata/netdata.conf` and set these config options:
-
-```conf
-[global]
- bind socket to IP = 127.0.0.1
- access log = none
- disconnect idle web clients after seconds = 3600
- enable web responses gzip compression = no
-```
-
-These options:
-
-- `[global].bind socket to IP = 127.0.0.1` makes Netdata listen only for requests from localhost (nginx).
-- `[global].access log = none` disables the access.log of Netdata. It is not needed since Netdata only listens for
- requests on 127.0.0.1 and thus only nginx can access it. nginx has its own access.log for your record.
-- `[global].disconnect idle web clients after seconds = 3600` will kill inactive web threads after an hour of
- inactivity.
-- `[global].enable web responses gzip compression = no` disables gzip compression at Netdata (nginx will compress the
- responses).
-
-## 2. increase open files limit (non-systemd)
-
-By default Linux limits open file descriptors per process to 1024. This means that less than half of this number of
-client connections can be accepted by both nginx and Netdata. To increase them, create 2 new files:
-
-1. `/etc/security/limits.d/nginx.conf`, with these contents:
-
-```conf
-nginx soft nofile 10000
-nginx hard nofile 30000
-```
-
-2. `/etc/security/limits.d/netdata.conf`, with these contents:
-
-```conf
-netdata soft nofile 10000
-netdata hard nofile 30000
-```
-
-and to activate them, run:
-
-```sh
-sysctl -p
-```
-
-## 2b. increase open files limit (systemd)
-
-Thanks to [@leleobhz](https://github.com/netdata/netdata/issues/655#issue-163932584), this is what you need to raise the
-limits using systemd:
-
-This is based on <https://ma.ttias.be/increase-open-files-limit-in-mariadb-on-centos-7-with-systemd/> and here worked as
-following:
-
-1. Create the folders in /etc:
-
-```bash
-mkdir -p /etc/systemd/system/netdata.service.d
-mkdir -p /etc/systemd/system/nginx.service.d
-```
-
-2. Create limits.conf in each folder as following:
-
-```conf
-[Service]
-LimitNOFILE=30000
-```
-
-3. Reload systemd daemon list and restart services:
-
-```sh
-systemctl daemon-reload
-systemctl restart netdata.service
-systemctl restart nginx.service
-```
-
-You can check limits with following commands:
-
-```sh
-cat /proc/$(ps aux | grep "nginx: master process" | grep -v grep | awk '{print $2}')/limits | grep "Max open files"
-cat /proc/$(ps aux | grep "\/[n]etdata " | head -n1 | grep -v grep | awk '{print $2}')/limits | grep "Max open files"
-```
-
-View of the files:
-
-```sh
-# tree /etc/systemd/system/*service.d/etc/systemd/system/netdata.service.d
-/etc/systemd/system/netdata.service.d
-└── limits.conf
-/etc/systemd/system/nginx.service.d
-└── limits.conf
-
-0 directories, 2 files
-
-# cat /proc/$(ps aux | grep "nginx: master process" | grep -v grep | awk '{print $2}')/limits | grep "Max open files"
-Max open files 30000 30000 files
-
-# cat /proc/$(ps aux | grep "netdata" | head -n1 | grep -v grep | awk '{print $2}')/limits | grep "Max open files"
-Max open files 30000 30000 files
-```
-
-[![analytics](https://www.google-analytics.com/collect?v=1&aip=1&t=pageview&_s=1&ds=github&dr=https%3A%2F%2Fgithub.com%2Fnetdata%2Fnetdata&dl=https%3A%2F%2Fmy-netdata.io%2Fgithub%2Fdocs%2Fhigh-performance-netdata&_u=MAC~&cid=5792dfd7-8dc4-476b-af31-da2fdb9f93d2&tid=UA-64295674-3)](<>)
diff --git a/docs/netdata-for-IoT.md b/docs/netdata-for-IoT.md
index 8b6730ae53..abd2d5ddda 100644
--- a/docs/netdata-for-IoT.md
+++ b/docs/netdata-for-IoT.md
@@ -45,8 +45,8 @@ provider so it can directly be used by google sheets, google charts, google widg
![sensors](https://cloud.githubusercontent.com/assets/2662304/15339745/8be84540-1c8e-11e6-9e9a-106dea7539b6.gif)
Although Netdata has been significantly optimized to lower the CPU and RAM resources it consumes, the plethora of data
-collection plugins may be inappropriate for weak IoT devices. Please follow the guide on [running Netdata in embedded
-devices](Performance.md)
+collection plugins may be inappropriate for weak IoT devices. Please follow the [Netdata Agent performance
+guide](/docs/guides/configure/performance.md)
## Monitoring RPi temperature