diff options
| author | Joel Hans <joel@netdata.cloud> | 2020-09-29 10:57:52 -0700 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2020-09-29 10:57:52 -0700 |
| commit | 61d7e23eed0503bf591274df70713970213b5c7f (patch) | |
| tree | 125879ff5f780c2937ee206fc510d89e9a91a868 /docs/collect/how-collectors-work.md | |
| parent | e3b04fb39a06991d9a2deed0488044dd7d340e3f (diff) | |
Add docsv2 project to master branch (#10000)
* Add overview docs to docsv2 project
* Add quickstart docs to docsv2 project (#9865)
* Init quickstart docs
* Begin work on quickstart guides
* Finish quickstart drafts
* Tweaks to both quickstarts
* Add titles
* Copyedit pass to both docs
* Fixes for Amy and Jen
* Add Get doc to docsv2 project (#9854)
* Init get file
* Add some links
* Change h2 to h1
* Rephrase
* Add configure docs to docsv2 project (#9878)
* Add overview docs to docsv2 project (#9849)
* Init files
* Add drafts of what and why
* Finish initial drafts
* Fix edit URL
* Copyedit pass
* Finish initial drafts of configure docs
* Copyedit all docs
* Fixes for Amy
* Fixes for Jen
* Add collect docs to the docsv2 project (#9932)
* Init files
* Finish first two collect docs
* Finish drafts of collect docs
* Copyedit pass
* Fixes for Amy
* Fix for Jen
* Add visualize docs to the docsv2 project (#9943)
* Add visualize docs
* Copyedits and cleanup
* New images and features
* Copyedit pass and cleanup
* Missing word
* Fixes for Jen
* Add monitor docs to docsv2 project (#9949)
* Finish drafts of monitor docs
* Copyedit pass
* Remove whitespace
* Fixes for Jen
* Add store docs to docsv2 project (#9969)
* Finalize store documents
* Fix import path
* Finishing edit section
* Copyedit pass
* Add export docs to docsv2 project (#9986)
* Add install and claim videos to Get doc
* Finish drafts of exporting docs plus other tweaks
* Init new exporting READMEs
* Copyedit pass and new links
* Fixes for Amy, Vlad, Jen
* Fix links in docsv2 project (#9993)
* Fix links
* Fix a bunch of links ahead of export merge
* Fix additional links
* Fix links, nuke what-is-netdata
* Fixing a few last links
* Improve product images in overview
* Remove extra paren
* Quick tweaks for Jen
* Fixes for Jen
* Access fix
* Remove extra word
Diffstat (limited to 'docs/collect/how-collectors-work.md')
| -rw-r--r-- | docs/collect/how-collectors-work.md | 80 |
1 files changed, 80 insertions, 0 deletions
diff --git a/docs/collect/how-collectors-work.md b/docs/collect/how-collectors-work.md new file mode 100644 index 0000000000..6818e79427 --- /dev/null +++ b/docs/collect/how-collectors-work.md @@ -0,0 +1,80 @@ +<!-- +title: "How Netdata's metrics collectors work" +description: "When Netdata starts, and with zero configuration, it auto-detects thousands of data sources and immediately collects per-second metrics." +custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/collect/how-collectors-work.md +--> + +# How Netdata's metrics collectors work + +When Netdata starts, and with zero configuration, it auto-detects thousands of data sources and immediately collects +per-second metrics. + +Netdata can immediately collect metrics from these endpoints thanks to 300+ **collectors**, which all come pre-installed +when you [install the Netdata Agent](/docs/get/README.md#install-the-netdata-agent). + +Every collector has two primary jobs: + +- Look for exposed metrics at a pre- or user-defined endpoint. +- Gather exposed metrics and use additional logic to build meaningful, interactive visualizations. + +If the collector finds compatible metrics exposed on the configured endpoint, it begins a per-second collection job. The +Netdata Agent gathers these metrics, sends them to the [database engine for +storage](/docs/store/change-metrics-storage.md), and immediately [visualizes them +meaningfully](/docs/visualize/interact-dashboards-charts.md) on dashboards. + +Each collector comes with a pre-defined configuration that matches the default setup for that application. This endpoint +can be a URL and port, a socket, a file, a web page, and more. + +For example, the [Nginx collector](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/nginx) searches +at `http://127.0.0.1/stub_status`, which is the default endpoint for exposing Nginx metrics. The [web log collector for +Nginx or Apache](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/weblog) searches at +`/var/log/nginx/access.log` and `/var/log/apache2/access.log`, respectively, both of which are standard locations for +access log files on Linux systems. + +The endpoint is user-configurable, as are many other specifics of what a given collector does. + +## What can Netdata collect? + +To quickly find your answer, see our [list of supported collectors](/collectors/COLLECTORS.md). + +Generally, Netdata's collectors can be grouped into three types: + +- [Systems](/docs/collect/system-metrics.md): Monitor CPU, memory, disk, networking, systemd, eBPF, and much more. + Every metric exposed by `/proc`, `/sys`, and other Linux kernel sources. +- [Containers](/docs/collect/container-metrics.md): Gather metrics from container agents, like `dockerd` or `kubectl`, + along with the resource usage of containers and the applications they run. +- [Applications](/docs/collect/application-metrics.md): Collect per-second metrics from web servers, databases, logs, + message brokers, APM tools, email servers, and much more. + +## Collector architecture and terminology + +**Collector** is a catch-all term for any Netdata process that gathers metrics from an endpoint. + +While we use _collector_ most often in documentation, release notes, and educational content, you may encounter other +terms related to collecting metrics. + +- **Modules** are a type of collector. +- **Orchestrators** are external plugins that run and manage one or more modules. They run as independent processes. + The Go orchestator is in active development. + - [go.d.plugin](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/): An orchestrator for data + collection modules written in `go`. + - [python.d.plugin](/collectors/python.d.plugin/README.md): An orchestrator for data collection modules written in + `python` v2/v3. + - [charts.d.plugin](/collectors/charts.d.plugin/README.md): An orchestrator for data collection modules written in + `bash` v4+. + - [node.d.plugin](/collectors/node.d.plugin/README.md): An orchestrator for data collection modules written in + `node.js`. +- **External plugins** gather metrics from external processes, such as a webserver or database, and run as independent + processes that communicate with the Netdata daemon via pipes. +- **Internal plugins** gather metrics from `/proc`, `/sys`, and other Linux kernel sources. They are written in `C`, + and run as threads within the Netdata daemon. + +## What's next? + +[Enable or configure a collector](/docs/collect/enable-configure.md) if the default settings are not compatible with +your infrastructure. + +See our [collectors reference](/collectors/REFERENCE.md) for detailed information on Netdata's collector architecture, +troubleshooting a collector, developing a custom collector, and more. + +[](<>) |