From 6430ad1211fb5b3274240276d6fb25840c2db979 Mon Sep 17 00:00:00 2001 From: Joel Hans Date: Wed, 12 Feb 2020 09:05:57 -0700 Subject: Refactor collectors documentation (#7996) * Initializing new structure * Continuing with cleanup * Modify old plugins conf * Get rid of Add-more-charts and move links over * More cleanup, add add-more-charts back in, build OK * Cleanup and move content to reference * Finish quickstart, add troubleshooting * Sync what-is-netdata and fix typos * Tweaks * Finish quickstart, revamp collectors * One more fix --- README.md | 269 +++----------------------------------------------------------- 1 file changed, 10 insertions(+), 259 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index a0c7c6b286..2bf9ce3c48 100644 --- a/README.md +++ b/README.md @@ -52,7 +52,7 @@ Netdata!](https://img.shields.io/twitter/url/http/shields.io.svg?style=social&la 7. [Infographic](#infographic) - Everything about Netdata in a single graphic 8. [Features](#features) - How you'll use Netdata on your systems 9. [Visualization](#visualization) - Learn about visual anomaly detection -10. [What does it monitor?](#what-does-it-monitor) - See which apps/services Netdata auto-detects +10. [What Netdata monitors](#what-netdata-monitors) - See which apps/services Netdata auto-detects 11. [Documentation](#documentation) - Read the documentation 12. [Community](#community) - Discuss Netdata with others and get support 13. [License](#license) - Check Netdata's licencing @@ -381,268 +381,19 @@ same range on all charts._ > Highlighted ranges are propagated from Netdata server to Netdata server, when navigating via the [My nodes > menu](registry/README.md). -## What does it monitor - -Netdata data collection is **extensible**. You can monitor anything you can get a metric for. Our [plugin -API](collectors/plugins.d/) supports a variety of programming languages to make nearly anything a Netdata plugin: Go, -Python, Node.js, Ruby, Java, Bash, Perl, and more! - -- For better performance, most system-related plugins (CPU, memory, disks, filesystems, networking, etc) have been - written in C. -- For faster development and easier contributions, most application related plugins (databases, web servers, etc) have - been written in Go and Python. - -### APM (Application Performance Monitoring) - -- **[statsd](collectors/statsd.plugin/)** - Netdata is a fully featured statsd server. -- **[Go expvar](collectors/python.d.plugin/go_expvar/)** - collects metrics exposed by applications written in the Go - programming language using the expvar package. -- **[Spring Boot](collectors/python.d.plugin/springboot/)** - monitors running Java Spring Boot applications that - expose their metrics with the use of the Spring Boot Actuator included in Spring Boot library. -- **[uWSGI](collectors/python.d.plugin/uwsgi/)** - collects performance metrics from uWSGI applications. - -### System Resources - -- **[CPU Utilization](collectors/proc.plugin/)** - total and per core CPU usage. -- **[Interrupts](collectors/proc.plugin/)** - total and per core CPU interrupts. -- **[SoftIRQs](collectors/proc.plugin/)** - total and per core SoftIRQs. -- **[SoftNet](collectors/proc.plugin/)** - total and per core SoftIRQs related to network activity. -- **[CPU Throttling](collectors/proc.plugin/)** - collects per core CPU throttling. -- **[CPU Frequency](collectors/proc.plugin/)** - collects the current CPU frequency. -- **[CPU Idle](collectors/proc.plugin/)** - collects the time spent per processor state. -- **[IdleJitter](collectors/idlejitter.plugin/)** - measures CPU latency. -- **[Entropy](collectors/proc.plugin/)** - random numbers pool, using in cryptography. -- **[Interprocess Communication - IPC](collectors/proc.plugin/)** - such as semaphores and semaphores arrays. - -### Memory - -- **[ram](collectors/proc.plugin/)** - collects info about RAM usage. -- **[swap](collectors/proc.plugin/)** - collects info about swap memory usage. -- **[available memory](collectors/proc.plugin/)** - collects the amount of RAM available for userspace processes. -- **[committed memory](collectors/proc.plugin/)** - collects the amount of RAM committed to userspace processes. -- **[Page Faults](collectors/proc.plugin/)** - collects the system page faults (major and minor). -- **[writeback memory](collectors/proc.plugin/)** - collects the system dirty memory and writeback activity. -- **[huge pages](collectors/proc.plugin/)** - collects the amount of RAM used for huge pages. -- **[KSM](collectors/proc.plugin/)** - collects info about Kernel Same Merging (memory dedupper). -- **[Numa](collectors/proc.plugin/)** - collects Numa info on systems that support it. -- **[slab](collectors/proc.plugin/)** - collects info about the Linux kernel memory usage. - -### Disks - -- **[block devices](collectors/proc.plugin/)** - per disk: I/O, operations, backlog, utilization, space, etc. -- **[BCACHE](collectors/proc.plugin/)** - detailed performance of SSD caching devices. -- **[DiskSpace](collectors/proc.plugin/)** - monitors disk space usage. -- **[mdstat](collectors/proc.plugin/)** - software RAID. -- **[hddtemp](collectors/python.d.plugin/hddtemp/)** - disk temperatures. -- **[smartd](collectors/python.d.plugin/smartd_log/)** - disk S.M.A.R.T. values. -- **[device mapper](collectors/proc.plugin/)** - naming disks. -- **[Veritas Volume Manager](collectors/proc.plugin/)** - naming disks. -- **[megacli](collectors/python.d.plugin/megacli/)** - adapter, physical drives and battery stats. -- **[adaptec_raid](collectors/python.d.plugin/adaptec_raid/)** - logical and physical devices health metrics. -- **[ioping](collectors/ioping.plugin/)** - to measure disk read/write latency. - -### Filesystems - -- **[BTRFS](collectors/proc.plugin/)** - detailed disk space allocation and usage. -- **[Ceph](collectors/python.d.plugin/ceph/)** - OSD usage, Pool usage, number of objects, etc. -- **[NFS file servers and clients](collectors/proc.plugin/)** - NFS v2, v3, v4: I/O, cache, read ahead, RPC calls -- **[Samba](collectors/python.d.plugin/samba/)** - performance metrics of Samba SMB2 file sharing. -- **[ZFS](collectors/proc.plugin/)** - detailed performance and resource usage. - -### Networking - -- **[Network Stack](collectors/proc.plugin/)** - everything about the networking stack (both IPv4 and IPv6 for all - protocols: TCP, UDP, SCTP, UDPLite, ICMP, Multicast, Broadcast, etc), and all network interfaces (per interface: - bandwidth, packets, errors, drops). -- **[Netfilter](collectors/proc.plugin/)** - everything about the netfilter connection tracker. -- **[SynProxy](collectors/proc.plugin/)** - collects performance data about the linux SYNPROXY (DDoS). -- **[NFacct](collectors/nfacct.plugin/)** - collects accounting data from iptables. -- **[Network QoS](collectors/tc.plugin/)** - the only tool that visualizes network `tc` classes in real-time. -- **[FPing](collectors/fping.plugin/)** - to measure latency and packet loss between any number of hosts. -- **[ISC dhcpd](collectors/python.d.plugin/isc_dhcpd/)** - pools utilization, leases, etc. -- **[AP](collectors/charts.d.plugin/ap/)** - collects Linux access point performance data (`hostapd`). -- **[SNMP](collectors/node.d.plugin/snmp/)** - SNMP devices can be monitored too (although you will need to configure - these). -- **[port_check](collectors/python.d.plugin/portcheck/)** - checks TCP ports for availability and response time. - -### Virtual Private Networks - -- **[OpenVPN](collectors/python.d.plugin/ovpn_status_log/)** - collects status per tunnel. -- **[LibreSwan](collectors/charts.d.plugin/libreswan/)** - collects metrics per IPSEC tunnel. -- **[Tor](collectors/python.d.plugin/tor/)** - collects Tor traffic statistics. - -### Processes - -- **[System Processes](collectors/proc.plugin/)** - running, blocked, forks, active. -- **[Applications](collectors/apps.plugin/)** - by grouping the process tree and reporting CPU, memory, disk reads, - disk writes, swap, threads, pipes, sockets - per process group. -- **[systemd](collectors/cgroups.plugin/)** - monitors systemd services using CGROUPS. - -### Users - -- **[Users and User Groups resource usage](collectors/apps.plugin/)** - by summarizing the process tree per user and - group, reporting: CPU, memory, disk reads, disk writes, swap, threads, pipes, sockets. -- **[logind](collectors/python.d.plugin/logind/)** - collects sessions, users and seats connected. - -### Containers and VMs - -- **[Containers](collectors/cgroups.plugin/)** - collects resource usage for all kinds of containers, using CGROUPS - (systemd-nspawn, lxc, lxd, docker, kubernetes, etc). -- **[libvirt VMs](collectors/cgroups.plugin/)** - collects resource usage for all kinds of VMs, using CGROUPS. -- **[dockerd](collectors/python.d.plugin/dockerd/)** - collects docker health metrics. - -### Web Servers - -- **[Apache and lighttpd](collectors/python.d.plugin/apache/)** - `mod-status` (v2.2, v2.4) and cache log statistics, - for multiple servers. -- **[IPFS](collectors/python.d.plugin/ipfs/)** - bandwidth, peers. -- **[LiteSpeed](collectors/python.d.plugin/litespeed/)** - reads the litespeed rtreport files to collect metrics. -- **[Nginx](collectors/python.d.plugin/nginx/)** - `stub-status`, for multiple servers. -- **[Nginx+](collectors/python.d.plugin/nginx_plus/)** - connects to multiple nginx_plus servers (local or remote) to - collect real-time performance metrics. -- **[PHP-FPM](collectors/python.d.plugin/phpfpm/)** - multiple instances, each reporting connections, requests, - performance, etc. -- **[Tomcat](collectors/python.d.plugin/tomcat/)** - accesses, threads, free memory, volume, etc. -- **[web server `access.log` files](collectors/python.d.plugin/web_log/)** - extracting in real-time, web server and - proxy performance metrics and applying several health checks, etc. -- **[HTTP check](collectors/python.d.plugin/httpcheck/)** - checks one or more web servers for HTTP status code and - returned content. - -### Proxies, Balancers, Accelerators - -- **[HAproxy](collectors/python.d.plugin/haproxy/)** - bandwidth, sessions, backends, etc. -- **[Squid](collectors/python.d.plugin/squid/)** - multiple servers, each showing: clients bandwidth and requests, - servers bandwidth and requests. -- **[Traefik](collectors/python.d.plugin/traefik/)** - connects to multiple traefik instances (local or remote) to - collect API metrics (response status code, response time, average response time and server uptime). -- **[Varnish](collectors/python.d.plugin/varnish/)** - threads, sessions, hits, objects, backends, etc. -- **[IPVS](collectors/proc.plugin/)** - collects metrics from the Linux IPVS load balancer. - -### Database Servers - -- **[CouchDB](collectors/python.d.plugin/couchdb/)** - reads/writes, request methods, status codes, tasks, - replication, per-db, etc. -- **[MemCached](collectors/python.d.plugin/memcached/)** - multiple servers, each showing: bandwidth, connections, - items, etc. -- **[MongoDB](collectors/python.d.plugin/mongodb/)** - operations, clients, transactions, cursors, connections, - asserts, locks, etc. -- **[MySQL and mariadb](collectors/python.d.plugin/mysql/)** - multiple servers, each showing: bandwidth, queries/s, - handlers, locks, issues, tmp operations, connections, binlog metrics, threads, innodb metrics, and more. -- **[PostgreSQL](collectors/python.d.plugin/postgres/)** - multiple servers, each showing: per database statistics - (connections, tuples read - written - returned, transactions, locks), backend processes, indexes, tables, write - ahead, background writer and more. -- **[Proxy SQL](collectors/python.d.plugin/proxysql/)** - collects Proxy SQL backend and frontend performance metrics. -- **[Redis](collectors/python.d.plugin/redis/)** - multiple servers, each showing: operations, hit rate, memory, keys, - clients, slaves. -- **[RethinkDB](collectors/python.d.plugin/rethinkdbs/)** - connects to multiple rethinkdb servers (local or remote) - to collect real-time metrics. +## What Netdata monitors -### Message Brokers +Netdata can collect metrics from 200+ popular services and applications, on top of dozens of system-related metrics +jocs, such as CPU, memory, disks, filesystems, networking, and more. We call these **collectors**, and they're managed +by [**plugins**](collectors/plugins.d/), which support a variety of programming languages, including Go and Python. -- **[beanstalkd](collectors/python.d.plugin/beanstalk/)** - global and per tube monitoring. -- **[RabbitMQ](collectors/python.d.plugin/rabbitmq/)** - performance and health metrics. +Popular collectors include **Nginx**, **Apache**, **MySQL**, **statsd**, **cgroups** (containers, Docker, Kubernetes, +LXC, and more), **Traefik**, **web server `access.log` files**, and much more. -### Search and Indexing +See the **full list of [supported collectors](collectors/COLLECTORS.md)**. -- **[ElasticSearch](collectors/python.d.plugin/elasticsearch/)** - search and index performance, latency, timings, - cluster statistics, threads statistics, etc. - -### DNS Servers - -- **[bind_rndc](collectors/python.d.plugin/bind_rndc/)** - parses `named.stats` dump file to collect real-time - performance metrics. All versions of bind after 9.6 are supported. -- **[dnsdist](collectors/python.d.plugin/dnsdist/)** - performance and health metrics. -- **[ISC Bind (named)](collectors/node.d.plugin/named/)** - multiple servers, each showing: clients, requests, - queries, updates, failures and several per view metrics. All versions of bind after 9.9.10 are supported. -- **[NSD](collectors/python.d.plugin/nsd/)** - queries, zones, protocols, query types, transfers, etc. -- **[PowerDNS](collectors/python.d.plugin/powerdns/)** - queries, answers, cache, latency, etc. -- **[dns_query_time](collectors/python.d.plugin/dns_query_time/)** - DNS query time statistics. - -### Time Servers - -- **[chrony](collectors/python.d.plugin/chrony/)** - uses the `chronyc` command to collect chrony statistics - (Frequency, Last offset, RMS offset, Residual freq, Root delay, Root dispersion, Skew, System time). -- **[ntpd](collectors/python.d.plugin/ntpd/)** - connects to multiple ntpd servers (local or remote) to provide - statistics of system variables and optional also peer variables. - -### Mail Servers - -- **[Dovecot](collectors/python.d.plugin/dovecot/)** - POP3/IMAP servers. -- **[Exim](collectors/python.d.plugin/exim/)** - message queue (emails queued). -- **[Postfix](collectors/python.d.plugin/postfix/)** - message queue (entries, size). - -### Hardware Sensors - -- **[IPMI](collectors/freeipmi.plugin/)** - enterprise hardware sensors and events. -- **[lm-sensors](collectors/python.d.plugin/sensors/)** - temperature, voltage, fans, power, humidity, etc. -- **[Nvidia](collectors/python.d.plugin/nvidia_smi/)** - collects information for Nvidia GPUs. -- **[RPi](collectors/charts.d.plugin/sensors/)** - Raspberry Pi temperature sensors. -- **[w1sensor](collectors/python.d.plugin/w1sensor/)** - collects data from connected 1-Wire sensors. - -### UPSes - -- **[apcupsd](collectors/charts.d.plugin/apcupsd/)** - load, charge, battery voltage, temperature, utility metrics, - output metrics. -- **[NUT](collectors/charts.d.plugin/nut/)** - load, charge, battery voltage, temperature, utility metrics, output - metrics. -- **[Linux Power Supply](collectors/proc.plugin/)** - collects metrics reported by power supply drivers on Linux. - -### Social Sharing Servers - -- **[RetroShare](collectors/python.d.plugin/retroshare/)** - connects to multiple retroshare servers (local or remote) - to collect real-time performance metrics. - -### Security - -- **[Fail2Ban](collectors/python.d.plugin/fail2ban/)** - monitors the fail2ban log file to check all bans for all - active jails. - -### Authentication, Authorization, Accounting (AAA, RADIUS, LDAP) Servers - -- **[FreeRadius](collectors/python.d.plugin/freeradius/)** - uses the `radclient` command to provide freeradius - statistics (authentication, accounting, proxy-authentication, proxy-accounting). - -### Telephony Servers - -- **[opensips](collectors/charts.d.plugin/opensips/)** - connects to an opensips server (localhost only) to collect - real-time performance metrics. - -### Household Appliances - -- **[SMA webbox](collectors/node.d.plugin/sma_webbox/)** - connects to multiple remote SMA webboxes to collect - real-time performance metrics of the photovoltaic (solar) power generation. -- **[Fronius](collectors/node.d.plugin/fronius/)** - connects to multiple remote Fronius Symo servers to collect - real-time performance metrics of the photovoltaic (solar) power generation. -- **[StiebelEltron](collectors/node.d.plugin/stiebeleltron/)** - collects the temperatures and other metrics from your - Stiebel Eltron heating system using their Internet Service Gateway (ISG web). - -### Game Servers - -- **[SpigotMC](collectors/python.d.plugin/spigotmc/)** - monitors Spigot Minecraft server ticks per second and number - of online players using the Minecraft remote console. - -### Distributed Computing - -- **[BOINC](collectors/python.d.plugin/boinc/)** - monitors task states for local and remote BOINC client software - using the remote GUI RPC interface. Also provides alarms for a handful of error conditions. - -### Media Streaming Servers - -- **[IceCast](collectors/python.d.plugin/icecast/)** - collects the number of listeners for active sources. - -### Monitoring Systems - -- **[Monit](collectors/python.d.plugin/monit/)** - collects metrics about monit targets (filesystems, applications, - networks). - -### Provisioning Systems - -- **[Puppet](collectors/python.d.plugin/puppet/)** - connects to multiple Puppet Server and Puppet DB instances (local - or remote) to collect real-time status metrics. - -You can easily extend Netdata, by writing plugins that collect data from any source, using any computer language. - ---- +Netdata's data collection is **extensible**, which means you can monitor anything you can get a metric for. You can even +write a collector for your custom application using our [plugin API](collectors/plugins.d/README.md). ## Documentation -- cgit v1.2.3