Documentation changes, new files and restructuring the hierarchy (#17014)

* docs additions

* docs from writerside, not to be published in this state, links need work and Learn map needs to include them

* rename some files to reduce repetition on filenames

* use new packaging documentation and replace links to old

* change learn-rel-path to new category names

* replace configuration file with new one, add conf directory section to it, and replace links to point to that

* linkfix

* run integrations pipeline to get new links

* catoverpage

* fix writerside style links

* addition in on-prem mention

* comment out mermaid problematic line

* change path of alerting integrations docs for Learn

* fix

* fixes

* fix diagrams

40 files changed, 2756 insertions, 6 deletions

diff --git a/docs/category-overview-pages/developer-and-contributor-corner.md b/docs/category-overview-pages/developer-and-contributor-corner.md
new file mode 100644
index 0000000000..d4d86382ac
--- /dev/null
+++ b/docs/category-overview-pages/developer-and-contributor-corner.md
@@ -0,0 +1,3 @@
+# Developer and Contributor Corner
+
+In this section of our Documentation you will find more advanced information, suited for developers and contributors alike.
\ No newline at end of file
diff --git a/docs/category-overview-pages/installation-overview.md b/docs/category-overview-pages/installation-overview.md
index e60dd442c0..703ca26b9b 100644
--- a/docs/category-overview-pages/installation-overview.md
+++ b/docs/category-overview-pages/installation-overview.md
@@ -1,7 +1,7 @@
 # Installation
 
 In this category you can find instructions on all the possible ways you can install Netdata on the
-[supported platforms](https://github.com/netdata/netdata/blob/master/packaging/PLATFORM_SUPPORT.md).
+[supported platforms](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/versions-and-platforms.md).
 
 If this is your first time using Netdata, we recommend that you first start with the 
 [quick installation guide](https://github.com/netdata/netdata/edit/master/packaging/installer/README.md) and then 
diff --git a/docs/contributing/style-guide.md b/docs/contributing/style-guide.md
index b77927a9c1..2487f2eb1c 100644
--- a/docs/contributing/style-guide.md
+++ b/docs/contributing/style-guide.md
@@ -305,7 +305,7 @@ Don't include full paths, beginning from the system's root (`/`), as these might
 |                 |                                                                                                                                                                                                                                                                                                    |
 |-----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | Not recommended | Use `edit-config` to edit Netdata's configuration: `sudo /etc/netdata/edit-config netdata.conf`.                                                                                                                                                                                                   |
-| **Recommended** | Use `edit-config` to edit Netdata's configuration by first navigating to your [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory), which is typically at `/etc/netdata`, then running `sudo edit-config netdata.conf`. |
+| **Recommended** | Use `edit-config` to edit Netdata's configuration by first navigating to your [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration.md#the-netdata-config-directory), which is typically at `/etc/netdata`, then running `sudo edit-config netdata.conf`. |
 
 ### `sudo`
 
diff --git a/docs/deployment-guides/README.md b/docs/deployment-guides/README.md
new file mode 100644
index 0000000000..18f5788573
--- /dev/null
+++ b/docs/deployment-guides/README.md
@@ -0,0 +1,25 @@
+# Deployment Guides
+
+Netdata can be used to monitor all kinds of infrastructure, from stand-alone tiny IoT devices to complex hybrid setups combining on-premise and cloud infrastructure, mixing bare-metal servers, virtual machines and containers.
+
+There are 3 components to structure your Netdata ecosystem:
+
+1. **Netdata Agents**
+
+   To monitor the physical or virtual nodes of your infrastructure, including all applications and containers running on them.
+
+   Netdata Agents are Open-Source, licensed under GPL v3+.
+
+2. **Netdata Parents**
+
+   To create [observability centralization points](https://github.com/netdata/netdata/blob/master/docs/observability-centralization-points/README.md) within your infrastructure, to offload Netdata Agents functions from your production systems, to provide high-availability of your data, increased data retention and isolation of your nodes.
+
+   Netdata Parents are implemented using the Netdata Agent software. Any Netdata Agent can be an Agent for a node and a Parent  for other Agents, at the same time.
+
+   It is recommended to set up multiple Netdata Parents. They will all seamlessly be integrated by Netdata Cloud into one monitoring solution.
+
+3. **Netdata Cloud**
+
+   Our SaaS, combining all your infrastructure, all your Netdata Agents and Parents, into one uniform, distributed, scalable, monitoring database, offering advanced data slicing and dicing capabilities, custom dashboards, advanced troubleshooting tools, user management, centralized management of alerts, and more.
+
+The Netdata Agent is a highly modular software piece, providing data collection via numerous plugins, an in-house crafted time-series database, a query engine, health monitoring and alerts, machine learning and anomaly detection, metrics exporting to third party systems.
diff --git a/docs/deployment-guides/deployment-with-centralization-points.md b/docs/deployment-guides/deployment-with-centralization-points.md
new file mode 100644
index 0000000000..b3e2b40dc6
--- /dev/null
+++ b/docs/deployment-guides/deployment-with-centralization-points.md
@@ -0,0 +1,121 @@
+# Deployment with Centralization Points
+
+An observability centralization point can centralize both metrics and logs. The sending systems are called Children, while the receiving systems are called a Parents.
+
+When metrics and logs are centralized, the Children are never queried for metrics and logs. The Netdata Parents have all the data needed to satisfy queries.
+
+- **Metrics** are centralized by Netdata, with a feature we call **Streaming**. The Parents listen for incoming connections and permit access only to Children that connect to it with the right API key. Children are configured to push their metrics to the Parents and they initiate the connections to do so.
+
+- **Logs** are centralized with methodologies provided by `systemd-journald`. This involves installing `systemd-journal-remote` on both the Parent and the Children, and configuring the keys required for this communication.
+
+|                    Feature                    |                                                 How it works                                                  |
+|:---------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|
+| Unified infrastructure dashboards for metrics |                                             Yes, at Netdata Cloud                                             |
+|  Unified infrastructure dashboards for logs   | All logs are accessible via the same dashboard at Netdata Cloud, although they are unified per Netdata Parent |
+|          Centrally configured alerts          |                                            Yes, at Netdata Parents                                            |
+|   Centrally dispatched alert notifications    |                                             Yes, at Netdata Cloud                                             |
+|         Data are exclusively on-prem          |                                                 Yes, Netdata Cloud queries Netdata Agents to satisfy dashboard queries.                                                 |
+
+A configuration with 2 observability centralization points, looks like this:
+
+```mermaid
+flowchart LR
+    WEB[["One unified
+        dashboard
+        for all nodes"]]
+    NC(["<b>Netdata Cloud</b>
+        decides which agents
+        need to be queried"])
+    SA1["Netdata at AWS
+         A1"]
+    SA2["Netdata at AWS
+         A2"]
+    SAN["Netdata at AWS
+         AN"]
+    PA["<b>Netdata Parent A</b>
+        at AWS
+        having all metrics & logs
+        for all Ax nodes"]
+    SB1["Netdata On-Prem
+         B1"]
+    SB2["Netdata On-Prem
+         B2"]
+    SBN["Netdata On-Prem
+         BN"]
+    PB["<b>Netdata Parent B</b>
+        On-Prem
+        having all metrics & logs
+        for all Bx nodes"]
+    WEB -->|query| NC -->|query| PA & PB
+    PA ---|stream| SA1 & SA2 & SAN
+    PB ---|stream| SB1 & SB2 & SBN 
+```
+
+Netdata Cloud queries the Netdata Parents to provide aggregated dashboard views.
+
+For alerts, the dispatch of notifications looks like in the following chart:
+
+```mermaid
+flowchart LR
+    NC(["<b>Netdata Cloud</b>
+        applies silencing
+        & user settings"])
+    SA1["Netdata at AWS
+         A1"]
+    SA2["Netdata at AWS
+         A2"]
+    SAN["Netdata at AWS
+         AN"]
+    PA["<b>Netdata Parent A</b>
+        at AWS
+        having all metrics & logs
+        for all Ax nodes"]
+    SB1["Netdata On-Prem
+         B1"]
+    SB2["Netdata On-Prem
+         B2"]
+    SBN["Netdata On-Prem
+         BN"]
+    PB["<b>Netdata Parent B</b>
+        On-Prem
+        having all metrics & logs
+        for all Bx nodes"]
+    EMAIL{{"<b>e-mail</b>
+        notifications"}}
+    MOBILEAPP{{"<b>Netdata Mobile App</b>
+        notifications"}}
+    SLACK{{"<b>Slack</b>
+        notifications"}}
+    OTHER{{"Other
+        notifications"}}
+    PA & PB -->|alert transitions| NC -->|notification| EMAIL & MOBILEAPP & SLACK & OTHER 
+    SA1 & SA2 & SAN ---|stream| PA
+    SB1 & SB2 & SBN ---|stream| PB 
+```
+
+### Configuration steps for deploying Netdata with Observability Centralization Points
+
+For Metrics:
+
+- Install Netdata agents on all systems and the Netdata Parents.
+
+- Configure `stream.conf` at the Netdata Parents to enable streaming access with an API key.
+
+- Configure `stream.conf` at the Netdata Children to enable streaming to the configured Netdata Parents.
+
+For Logs:
+
+- Install `systemd-journal-remote` on all systems and the Netdata Parents.
+
+- Configure `systemd-journal-remote` at the Netdata Parents to enable logs reception.
+
+- Configure `systemd-journal-upload` at the Netdata Children to enable transmission of their logs to the Netdata Parents.
+
+Optionally:
+
+- Disable ML, health checks and dashboard access at Netdata Children to save resources and avoid duplicate notifications.
+
+When using Netdata Cloud:
+
+- Optionally: disable dashboard access on all Netdata agents (including Netdata Parents).
+- Optionally: disable alert notifications on all Netdata agents (including Netdata Parents).
diff --git a/docs/deployment-guides/standalone-deployment.md b/docs/deployment-guides/standalone-deployment.md
new file mode 100644
index 0000000000..5baef805a9
--- /dev/null
+++ b/docs/deployment-guides/standalone-deployment.md
@@ -0,0 +1,139 @@
+# Standalone Deployment
+
+To help our users have a complete experience of Netdata when they install it for the first time, a Netdata Agent with default configuration is a complete monitoring solution out of the box, having all its features enabled and available.
+
+So, each Netdata agent acts as a standalone monitoring system by default.
+
+## Standalone agents, without Netdata Cloud
+
+|                    Feature                    |                     How it works                     |
+|:---------------------------------------------:|:----------------------------------------------------:|
+| Unified infrastructure dashboards for metrics |  No, each Netdata agent provides its own dashboard   |
+|  Unified infrastructure dashboards for logs   |     No, each Netdata agent exposes its own logs      |
+|          Centrally configured alerts          |  No, each Netdata has its own alerts configuration   |
+|   Centrally dispatched alert notifications    | No, each Netdata agent sends notifications by itself |
+|         Data are exclusively on-prem          |                         Yes                          |
+
+When using Standalone Netdata agents, each of them offers an API and a dashboard, at its own unique URL, that looks like `http://agent-ip:19999`.
+
+So, each of the Netdata agents has to be accessed individually and independently of the others:
+
+```mermaid
+flowchart LR
+    WEB[["Multiple
+        Independent
+        Dashboards"]]
+    S1["Standalone
+        Netdata
+         1"]
+    S2["Standalone
+        Netdata
+         2"]
+    SN["Standalone
+        Netdata
+         N"]
+    WEB -->|URL 1| S1
+    WEB -->|URL 2| S2
+    WEB -->|URL N| SN
+```
+
+The same is true for alert notifications. Each of the Netdata agents runs its own alerts and sends notifications by itself, according to its configuration:
+
+```mermaid
+flowchart LR
+    S1["Standalone
+        Netdata
+         1"]
+    S2["Standalone
+        Netdata
+         2"]
+    SN["Standalone
+        Netdata
+         N"]
+    EMAIL{{"<b>e-mail</b>
+        notifications"}}
+    SLACK{{"<b>Slack</b>
+        notifications"}}
+    OTHER{{"Other
+        notifications"}}
+    S1 & S2 & SN .-> SLACK
+    S1 & S2 & SN ---> EMAIL
+    S1 & S2 & SN ==> OTHER
+```
+
+### Configuration steps for standalone Netdata agents without Netdata Cloud
+
+No special configuration needed.
+
+- Install Netdata agents on all your systems, then access each of them via its own unique URL, that looks like `http://agent-ip:19999/`.
+
+## Standalone agents, with Netdata Cloud
+
+|                    Feature                    |                                                                              How it works                                                                               |
+|:---------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
+| Unified infrastructure dashboards for metrics |                                                 Yes, via Netdata Cloud, all charts aggregate metrics from all servers.                                                  |
+|  Unified infrastructure dashboards for logs   | All logs are accessible via the same dashboard at Netdata Cloud, although they are not unified (ie. logs from different servers are not multiplexed into a single view) |
+|             Centrally configured alerts       |                                                            No, each Netdata has its own alerts configuration                                                            |
+|   Centrally dispatched alert notifications    |                                                                         Yes, via Netdata Cloud                                                                          |
+|         Data are exclusively on-prem          |                                                 Yes, Netdata Cloud queries Netdata Agents to satisfy dashboard queries.                                                 |
+
+By [connecting all Netdata agents to Netdata Cloud](https://github.com/netdata/netdata/blob/master/src/claim/README.md), you can have a unified infrastructure view of all your nodes, with aggregated charts, without configuring [observability centralization points](https://github.com/netdata/netdata/blob/master/docs/observability-centralization-points/README.md).
+
+```mermaid
+flowchart LR
+    WEB[["One unified
+        dashboard
+        for all nodes"]]
+    NC(["<b>Netdata Cloud</b>
+        decides which agents
+        need to be queried"])
+    S1["Standalone
+        Netdata
+         1"]
+    S2["Standalone
+        Netdata
+         2"]
+    SN["Standalone
+        Netdata
+         N"]
+    WEB -->|queries| NC
+    NC -->|queries| S1 & S2 & SN
+```
+
+Similarly for alerts, Netdata Cloud receives all alert transitions from all agents, decides which notifications should be sent and how, applies silencing rules, maintenance windows and based on each Netdata Cloud space and user settings, dispatches notifications:
+
+```mermaid
+flowchart LR
+    EMAIL{{"<b>e-mail</b>
+        notifications"}}
+    MOBILEAPP{{"<b>Netdata Mobile App</b>
+        notifications"}}
+    SLACK{{"<b>Slack</b>
+        notifications"}}
+    OTHER{{"Other
+        notifications"}}
+    NC(["<b>Netdata Cloud</b>
+        applies silencing
+        & user settings"])
+    S1["Standalone
+        Netdata
+         1"]
+    S2["Standalone
+        Netdata
+         2"]
+    SN["Standalone
+        Netdata
+         N"]
+    NC -->|notification| EMAIL & MOBILEAPP & SLACK & OTHER
+    S1 & S2 & SN -->|alert transition| NC
+```
+
+> Note that alerts are still triggered by Netdata agents. Netdata Cloud takes care of the notifications only.
+
+### Configuration steps for standalone Netdata agents with Netdata Cloud
+
+- Install Netdata agents using the commands given by Netdata Cloud, so that they will be automatically added to your Netdata Cloud space. Otherwise, install Netdata agents and then claim them via the command line or their dashboard.
+
+- Optionally: disable their direct dashboard access to secure them.
+
+- Optionally: disable their alert notifications to avoid receiving email notifications directly from them (email notifications are automatically enabled when a working MTA is found on the systems Netdata agents are installed).
diff --git a/docs/export/enable-connector.md b/docs/export/enable-connector.md
index dd3a55a4bf..f81074c88b 100644
--- a/docs/export/enable-connector.md
+++ b/docs/export/enable-connector.md
@@ -24,7 +24,7 @@ Once you understand the process of enabling a connector, you can translate that
 ## Enable the exporting engine
 
 Use `edit-config` from your
-[Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory)
+[Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration.md#the-netdata-config-directory)
 to open `exporting.conf`:
 
 ```bash
diff --git a/docs/guides/monitor/raspberry-pi-anomaly-detection.md b/docs/guides/monitor/raspberry-pi-anomaly-detection.md
index c046439586..e2d28c2d11 100644
--- a/docs/guides/monitor/raspberry-pi-anomaly-detection.md
+++ b/docs/guides/monitor/raspberry-pi-anomaly-detection.md
@@ -24,7 +24,7 @@ Read on to learn all the steps and enable unsupervised anomaly detection on your
 First make sure Netdata is using Python 3 when it runs Python-based data collectors. 
 
 Next, open `netdata.conf` using [`edit-config`](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#use-edit-config-to-edit-configuration-files)
-from within the [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). Scroll down to the
+from within the [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration.md#the-netdata-config-directory). Scroll down to the
 `[plugin:python.d]` section to pass in the `-ppython3` command option. 
 
 ```conf
diff --git a/docs/metrics-storage-management/enable-streaming.md b/docs/metrics-storage-management/enable-streaming.md
index 49c798804a..82443bedf2 100644
--- a/docs/metrics-storage-management/enable-streaming.md
+++ b/docs/metrics-storage-management/enable-streaming.md
@@ -114,7 +114,7 @@ itself while initiating a streaming connection. Copy that into a separate text f
 > Find out how to [install `uuidgen`](https://command-not-found.com/uuidgen) on your node if you don't already have it.
 
 Next, open `stream.conf` using [`edit-config`](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#use-edit-config-to-edit-configuration-files)
-from within the [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory).
+from within the [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration.md#the-netdata-config-directory).
 
 ```bash
 cd /etc/netdata
diff --git a/docs/netdata-agent/README.md b/docs/netdata-agent/README.md
new file mode 100644
index 0000000000..faf262fd44
--- /dev/null
+++ b/docs/netdata-agent/README.md
@@ -0,0 +1,84 @@
+# Netdata Agent
+
+The Netdata Agent is the main building block in a Netdata ecosystem. It is installed on all monitored systems to monitor system components, containers and applications.
+
+The Netdata Agent is an **observability pipeline in a box** that can either operate standalone, or blend into a bigger pipeline made by more Netdata Agents (Children and Parents).
+
+## Distributed Observability Pipeline
+
+The Netdata observability pipeline looks like in the following graph.
+
+The pipeline is extended by creating Metrics Observability Centralization Points that are linked all together (`from a remote Netdata`, `to a remote Netdata`), so that all Netdata installed become a vast integrated observability pipeline.
+
+```mermaid
+stateDiagram-v2
+    classDef userFeature fill:#f00,color:white,font-weight:bold,stroke-width:2px,stroke:yellow
+    classDef usedByNC fill:#090,color:white,font-weight:bold,stroke-width:2px,stroke:yellow
+    Local --> Discover
+    Local: Local Netdata
+    [*] --> Detect: from a remote Netdata
+    Others: 3rd party time-series DBs
+    Detect: Detect Anomalies
+    Dashboard:::userFeature
+    Dashboard: Netdata Dashboards
+    3rdDashboard:::userFeature
+    3rdDashboard: 3rd party Dashboards
+    Notifications:::userFeature
+    Notifications: Alert Notifications
+    Alerts: Alert Transitions
+    Discover --> Collect
+    Collect --> Detect
+    Store: Store
+    Store: Time-Series Database
+    Detect --> Store
+    Store --> Learn
+    Store --> Check
+    Store --> Query
+    Store --> Score
+    Store --> Stream
+    Store --> Export
+    Query --> Visualize
+    Score --> Visualize
+    Check --> Alerts
+    Learn --> Detect: trained ML models
+    Alerts --> Notifications
+    Stream --> [*]: to a remote Netdata
+    Export --> Others
+    Others --> 3rdDashboard
+    Visualize --> Dashboard
+    Score:::usedByNC
+    Query:::usedByNC
+    Alerts:::usedByNC
+```
+
+1. **Discover**: auto-detect metric sources on localhost, auto-discover metric sources on Kubernetes.
+2. **Collect**: query data sources to collect metric samples, using the optimal protocol for each data source. 800+ integrations supported, including dozens of native application protocols, OpenMetrics and StatsD.
+3. **Detect Anomalies**: use the trained machine learning models for each metric, to detect in real-time if each sample collected is an outlier (an anomaly), or not.
+4. **Store**: keep collected samples and their anomaly status, in the time-series database (database mode `dbengine`) or a ring buffer (database modes `ram` and `alloc`).
+5. **Learn**: train multiple machine learning models for each metric collected, learning behaviors and patterns for detecting anomalies.
+6. **Check**: a health engine, triggering alerts and sending notifications. Netdata comes with hundreds of alert configurations that are automatically attached to metrics when they get collected, detecting errors, common configuration errors and performance issues.
+7. **Query**: a query engine for querying time-series data.
+8. **Score**: a scoring engine for comparing and correlating metrics.
+9. **Stream**: a mechanism to connect Netdata agents and build Metrics Centralization Points (Netdata Parents).
+10. **Visualize**: Netdata's fully automated dashboards for all metrics.
+11. **Export**: export metric samples to 3rd party time-series databases, enabling the use of 3rd party tools for visualization, like Grafana.
+
+## Comparison to other observability solutions
+
+1. **One moving part**: Other monitoring solution require maintaining metrics exporters, time-series databases, visualization engines. Netdata has everything integrated into one package, even when [Metrics Centralization Points](https://github.com/netdata/netdata/blob/master/docs/observability-centralization-points/metrics-centralization-points/README.md) are required, making deployment and maintenance a lot simpler.
+
+2. **Automation**: Netdata is designed to automate most of the process of setting up and running an observability solution. It is designed to instantly provide comprehensive dashboards and fully automated alerts, with zero configuration.
+
+3. **High Fidelity Monitoring**: Netdata was born from our need to kill the console for observability. So, it provides metrics and logs in the same granularity and fidelity console tools do, but also comes with tools that go beyond metrics and logs, to provide a holistic view of the monitored infrastructure (e.g. check [Top Monitoring](https://github.com/netdata/netdata/blob/master/docs/cloud/netdata-functions.md)).  
+
+4. **Minimal impact on monitored systems and applications**: Netdata has been designed to have a minimal impact on the monitored systems and their applications. There are [independent studies](https://www.ivanomalavolta.com/files/papers/ICSOC_2023.pdf) reporting that Netdata excels in CPU usage, RAM utilization, Execution Time and the impact Netdata has on monitored applications and containers.
+
+5. **Energy efficiency**: [University of Amsterdam did a research to find the energy efficiency of monitoring tools](https://twitter.com/IMalavolta/status/1734208439096676680)