summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorCosta Tsaousis <costa@tsaousis.gr>2018-10-17 13:34:53 +0300
committerGitHub <noreply@github.com>2018-10-17 13:34:53 +0300
commit8cff4f255e62e0414527b2b01278adc4243baee1 (patch)
tree4d6a5c435ab7629948d71196ea6b6d44031dd099
parentb8f6925a8cad0621a6ae7293932e6c1cca58bc33 (diff)
fixed documentation links (#4418)
* fixed documentation links * updated apps.plugin info * updated apps.plugin info * updated apps.plugin info
-rw-r--r--collectors/apps.plugin/README.md217
-rw-r--r--collectors/charts.d.plugin/apache/README.md125
-rw-r--r--collectors/fping.plugin/README.md41
-rw-r--r--collectors/python.d.plugin/go_expvar/README.md88
4 files changed, 371 insertions, 100 deletions
diff --git a/collectors/apps.plugin/README.md b/collectors/apps.plugin/README.md
index c427e04223..a2cb9a70d6 100644
--- a/collectors/apps.plugin/README.md
+++ b/collectors/apps.plugin/README.md
@@ -1,48 +1,94 @@
# apps.plugin
-This plugin provides charts for 3 sections of the default dashboard:
+`apps.plugin` breaks down system resource usage to **processes**, **users** and **user groups**.
-1. Per application charts
-2. Per user charts
-3. Per user group charts
+To achieve this task, it iterates through the whole process tree, collecting resource usage information
+for every process found running.
-## Per application charts
+Since netdata needs to present this information in charts and track them through time,
+instead of presenting a `top` like list, `apps.plugin` uses a pre-defined list of **process groups**
+to which it assigns all running processes. This list is [customizable](apps_groups.conf) and netdata
+ships with a good default for most cases (to edit it on your system run `/etc/netdata/edit-config apps_groups.conf`).
-This plugin walks through the entire `/proc` filesystem and aggregates statistics for applications of interest, defined in `/etc/netdata/apps_groups.conf` (the default is [here](apps_groups.conf)) (to edit it on your system run `/etc/netdata/edit-config apps_groups.conf`).
+So, `apps.plugin` builds a process tree (much like `ps fax` does in Linux), and groups
+processes together (evaluating both child and parent processes) so that the result is always a list with
+a predefined set of members (of course, only process groups found running are reported).
-The plugin internally builds a process tree (much like `ps fax` does), and groups processes together (evaluating both child and parent processes) so that the result is always a chart with a predefined set of dimensions (of course, only application groups found running are reported).
+> If you find that `apps.plugin` categorizes standard applications as `other`, we would be
+> glad to accept pull requests improving the [defaults](apps_groups.conf) shipped with netdata.
-Using this information it provides the following charts (per application group defined in `/etc/netdata/apps_groups.conf` - to edit it on your system run `/etc/netdata/edit-config apps_groups.conf`):
+Unlike traditional process monitoring tools (like `top`), `apps.plugin` is able to account the resource
+utilization of exit processes. Their utilization is accounted at their currently running parents.
+So, `apps.plugin` is perfectly able to measure the resources used by shell scripts and other processes
+that fork/spawn other short lived processes hundreds of times per second.
-1. Total CPU usage
-2. Total User CPU usage
-3. Total System CPU usage
-4. Total Disk Physical Reads
-5. Total Disk Physical Writes
-6. Total Disk Logical Reads
-7. Total Disk Logical Writes
-8. Total Open Files (unique files - if a file is found open multiple times, it is counted just once)
-9. Total Dedicated Memory (non shared)
-10. Total Minor Page Faults
-11. Total Number of Processes
-12. Total Number of Threads
-13. Total Number of Pipes
-14. Total Swap Activity (Major Page Faults)
-15. Total Open Sockets
+For example, ssh to a server running netdata and execute this:
-## Per User Charts
+```sh
+while true; do ls -l /var/run >/dev/null; done
+```
+
+All the console tools will report that a a CPU core is 100% used, but they will fail to identify which
+process is using all that CPU (because there is no single process using it - thousands of `ls` per second
+are using it). Netdata however, will be able to identify that `ssh` is using it
+(`ssh` is the parent process group defined in its [default config](apps_groups.conf)):
+
+![](https://cloud.githubusercontent.com/assets/2662304/21076220/c9687848-bf2e-11e6-8d81-348592c5aca2.png)
+
+This feature makes `apps.plugin` unique in narrowing down the list of offending processes that may be
+responsible for slow downs, or abusing system resources.
+
+## Charts
+
+`apps.plugin` provides charts for 3 sections:
+
+1. Per application charts as **Applications** at netdata dashboards
+2. Per user charts as **Users** at netdata dashboards
+3. Per user group charts as **User Groups** at netdata dashboards
+
+Each of these sections provides the same number of charts:
+
+- CPU Utilization
+ - Total CPU usage
+ - User / System CPU usage
+- Disk I/O
+ - Physical Reads / Writes
+ - Logical Reads / Writes
+ - Open Unique Files (if a file is found open multiple times, it is counted just once)
+- Memory
+ - Real Memory Used (non shared)
+ - Virtual Memory Allocated
+ - Minor Page Faults (i.e. memory activity)
+- Processes
+ - Threads Running
+ - Processes Running
+ - Pipes Open
+- Swap Memory
+ - Swap Memory Used
+ - Major Page Faults (i.e. swap activity)
+- Network
+ - Sockets Open
-All the above charts, are also grouped by username, using the effective uid of each process.
+The above are reported:
-## Per Group Charts
+- For **Applications** per [target configured](apps_groups.conf).
+- For **Users** per username or UID (when the username is not available).
+- For **User Groups** per groupname or GID (when groupname is not available).
-All the above charts, are also grouped by group name, using the effective gid of each process.
+## Performance
-## CPU Usage
+`apps.plugin` is a complex piece of software and has a lot of work to do
+We are proud that `apps.plugin` is a lot faster compared to any other similar tool,
+while collecting a lot more information for the processes, however the fact is that
+this plugin requires more CPU resources than the netdata daemon itself.
-`apps.plugin` is a complex software piece and has a lot of work to do (actually this plugin requires more CPU resources that the netdata daemon). For each process running, `apps.plugin` reads several `/proc` files to get CPU usage, memory allocated, I/O usage, open file descriptors, etc. Doing this work per-second, especially on hosts with several thousands of processes, may increase the CPU resources consumed by the plugin.
+Under Linux, for each process running, `apps.plugin` reads several `/proc` files
+per process. Doing this work per-second, especially on hosts with several thousands
+of processes, may increase the CPU resources consumed by the plugin.
-In such cases, you many need to lower its data collection frequency. To do this, edit `/etc/netdata/netdata.conf` and find this section:
+In such cases, you many need to lower its data collection frequency.
+
+To do this, edit `/etc/netdata/netdata.conf` and find this section:
```
[plugin:apps]
@@ -50,8 +96,8 @@ In such cases, you many need to lower its data collection frequency. To do this,
# command options =
```
-Uncomment the line `update every` and set it to a higher number. If you just set it to ` 2 `, its CPU resources will be cut in half, and data collection will be once every 2 seconds.
-
+Uncomment the line `update every` and set it to a higher number. If you just set it to ` 2 `,
+its CPU resources will be cut in half, and data collection will be once every 2 seconds.
## Configuration
@@ -64,27 +110,63 @@ The configuration file works accepts multiple lines, each having this format:
group: process1 process2 ...
```
-Process names should be given as they appear when running `ps -e`. The program will actually match the process names in the `/proc/PID/status` file. So, to be sure the name is right for a process running with PID ` X `, do this:
+Each group can be given multiple times, to add more processes to it.
-```sh
-cat /proc/X/status
-```
+For the **Applications** section, only groups configured in this file are reported.
+All other processes will be reported as `other`.
+
+For each process given, its whole process tree will be grouped, not just the process matched.
+The plugin will include both parents and children.
+
+The process names are the ones returned by:
+
+ - `ps -e` or `cat /proc/PID/stat`
+ - in case of substring mode (see below): `/proc/PID/cmdline`
+
+To add process names with spaces, enclose them in quotes (single or double)
+example: ` 'Plex Media Serv' ` or ` "my other process" `.
+
+You can add an asterisk ` * ` at the beginning and/or the end of a process:
-The first line on the output is `Name: xxxxx`. This is the process name `apps.plugin` sees.
+ - `*name` *suffix* mode: will search for processes ending with `name` (at `/proc/PID/stat`)
+ - `name*` *prefix* mode: will search for processes beginning with `name` (at `/proc/PID/stat`)
+ - `*name*` *substring* mode: will search for `name` in the whole command line (at `/proc/PID/cmdline`)
-The order of the lines in the file is important only if you include the same process name to multiple groups.
+If you enter even just one *name* (substring), `apps.plugin` will process
+`/proc/PID/cmdline` for all processes (of course only once per process: when they are first seen).
-## Apps plugin is missing information
+To add processes with single quotes, enclose them in double quotes: ` "process with this ' single quote" `
-`apps.plugin` requires additional privileges to collect all the information it needs. The problem is described in issue #157.
+To add processes with double quotes, enclose them in single quotes: ` 'process with this " double quote' `
-When netdata is installed, `apps.plugin` is given the capabilities `cap_dac_read_search,cap_sys_ptrace+ep`. If that is not possible (i.e. `setcap` fails), `apps.plugin` is setuid to `root`.
+If a group or process name starts with a ` - `, the dimension will be hidden from the chart (cpu chart only).
-## linux capabilities in containers
+If a process starts with a ` + `, debugging will be enabled for it (debugging produces a lot of output - do not enable it in production systems).
-There are a few cases, like `docker` and `virtuozzo` containers, where `setcap` succeeds, but the capabilities are silently ignored (in `lxc` containers `setcap` fails).
+You can add any number of groups. Only the ones found running will affect the charts generated.
+However, producing charts with hundreds of dimensions may slow down your web browser.
-In these cases that `setcap` succeeds by capabilities do not work, you will have to setuid to root `apps.plugin` by running these commands:
+The order of the entries in this list is important: the first that matches a process is used, so put important
+ones at the top. Processes not matched by any row, will inherit it from their parents or children.
+
+The order also controls the order of the dimensions on the generated charts (although applications started
+after apps.plugin is started, will be appended to the existing list of dimensions the netdata daemon maintains).
+
+## Permissions
+
+`apps.plugin` requires additional privileges to collect all the information it needs.
+The problem is described in issue #157.
+
+When netdata is installed, `apps.plugin` is given the capabilities `cap_dac_read_search,cap_sys_ptrace+ep`.
+If this fails (i.e. `setcap` fails), `apps.plugin` is setuid to `root`.
+
+#### linux capabilities in containers
+
+There are a few cases, like `docker` and `virtuozzo` containers, where `setcap` succeeds, but the capabilities
+are silently ignored (in `lxc` containers `setcap` fails).
+
+In these cases ()`setcap` succeeds but capabilities do not work), you will have to setuid
+to root `apps.plugin` by running these commands:
```sh
chown root:netdata /usr/libexec/netdata/plugins.d/apps.plugin
@@ -93,11 +175,50 @@ chmod 4750 /usr/libexec/netdata/plugins.d/apps.plugin
You will have to run these, every time you update netdata.
+## Security
+
+`apps.plugin` performs a hard-coded function of building the process tree in memory,
+iterating forever, collecting metrics for each running process and sending them to netdata.
+This is a one-way communication, from `apps.plugin` to netdata.
+
+So, since `apps.plugin` cannot be instructed by netdata for the actions it performs,
+we think it is pretty safe to allow it have these increased privileges.
+
+Keep in mind that `apps.plugin` will still run without escalated permissions,
+but it will not be able to collect all the information.
+
+## Application Badges
+
+You can create badges that you can embed anywhere you like, with URLs like this:
+
+```
+https://your.netdata.ip:19999/api/v1/badge.svg?chart=apps.processes&dimensions=myapp&value_color=green%3E0%7Cred
+```
+
+The color expression unescaped is this: `value_color=green>0|red`.
+
+Here is an example for the process group `sql` at `https://registry.my-netdata.io`:
+
+![image](https://registry.my-netdata.io/api/v1/badge.svg?chart=apps.processes&dimensions=sql&value_color=green%3E0%7Cred)
-### Is is safe to give `apps.plugin` these privileges?
+Netdata is able give you a lot more badges for your app.
+Examples below for process group `sql`:
-`apps.plugin` performs a hard-coded function of building the process tree in memory, iterating forever, collecting metrics for each running process and sending them to netdata. This is a one-way communication, from `apps.plugin` to netdata.
+- CPU usage: ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.cpu&dimensions=sql&value_color=green=0%7Corange%3C50%7Cred)
+- Disk Physical Reads ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.preads&dimensions=sql&value_color=green%3C100%7Corange%3C1000%7Cred)
+- Disk Physical Writes ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.pwrites&dimensions=sql&value_color=green%3C100%7Corange%3C1000%7Cred)
+- Disk Logical Reads ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.lreads&dimensions=sql&value_color=green%3C100%7Corange%3C1000%7Cred)
+- Disk Logical Writes ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.lwrites&dimensions=sql&value_color=green%3C100%7Corange%3C1000%7Cred)
+- Open Files ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.files&dimensions=sql&value_color=green%3E30%7Cred)
+- Real Memory ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.mem&dimensions=sql&value_color=green%3C100%7Corange%3C200%7Cred)
+- Virtual Memory ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.vmem&dimensions=sql&value_color=green%3C100%7Corange%3C1000%7Cred)
+- Swap Memory ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.swap&dimensions=sql&value_color=green=0%7Cred)
+- Minor Page Faults ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.minor_faults&dimensions=sql&value_color=green%3C100%7Corange%3C1000%7Cred)
+- Processes ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.processes&dimensions=sql&value_color=green%3E0%7Cred)
+- Threads ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.threads&dimensions=sql&value_color=green%3E=28%7Cred)
+- Major Faults (swap activity) ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.major_faults&dimensions=sql&value_color=green=0%7Cred)
+- Open Pipes ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.pipes&dimensions=sql&value_color=green=0%7Cred)
+- Open Sockets ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.sockets&dimensions=sql&value_color=green%3E=3%7Cred)
-So, since `apps.plugin` cannot be instructed by netdata for the actions it performs, we think it is pretty safe to allow it have these increased privileges.
-Keep in mind that `apps.plugin` will still run without these permissions, but it will not be able to collect all the data for every process.
+For more information about badges check [Generating Badges](https://github.com/netdata/netdata/wiki/Generating-Badges) \ No newline at end of file
diff --git a/collectors/charts.d.plugin/apache/README.md b/collectors/charts.d.plugin/apache/README.md
index d82951aacc..890cee9844 100644
--- a/collectors/charts.d.plugin/apache/README.md
+++ b/collectors/charts.d.plugin/apache/README.md
@@ -1,2 +1,127 @@
> THIS MODULE IS OBSOLETE.
> USE THE PYTHON ONE - IT SUPPORTS MULTIPLE JOBS AND IT IS MORE EFFICIENT
+
+---
+
+# Apache Plugin (apache)
+
+The `apache` collector visualizes key performance data for an apache web server.
+
+## Example netdata charts
+
+For apache 2.2:
+
+![image](https://cloud.githubusercontent.com/assets/2662304/12530273/421c4d14-c1e2-11e5-9fb6-ca6d6dd3b1dd.png)
+
+For apache 2.4:
+
+![image](https://cloud.githubusercontent.com/assets/2662304/12530376/29ec26de-c1e6-11e5-9af1-e48aaf781795.png)
+
+## How it works
+
+It runs `curl "http://apache.host/server-status?auto` to fetch the current status of apache.
+
+It has been tested with apache 2.2 and apache 2.4. The latter also provides connections information (total and break down by status).
+
+Apache 2.2 response:
+
+```sh
+$ curl "http://127.0.0.1/server-status?auto"
+Total Accesses: 80057
+Total kBytes: 223017
+CPULoad: .018287
+Uptime: 64472
+ReqPerSec: 1.24173
+BytesPerSec: 3542.15
+BytesPerReq: 2852.59
+BusyWorkers: 1
+IdleWorkers: 49
+Scoreboard: _________________________......................................._W_______________________.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
+```
+
+Apache 2.4 response:
+
+```sh
+$ curl "http://127.0.0.1/server-status?auto"
+127.0.0.1
+ServerVersion: Apache/2.4.18 (Unix)
+ServerMPM: event
+Server Built: Dec 14 2015 08:05:54
+CurrentTime: Saturday, 23-Jan-2016 14:42:06 EET
+RestartTime: Saturday, 23-Jan-2016 04:57:13 EET
+ParentServerConfigGeneration: 2
+ParentServerMPMGeneration: 1
+ServerUptimeSeconds: 35092
+ServerUptime: 9 hours 44 minutes 52 seconds
+Load1: 0.32
+Load5: 0.32
+Load15: 0.27
+Total Accesses: 32403
+Total kBytes: 34464
+CPUUser: 30.37
+CPUSystem: 29.55
+CPUChildrenUser: 0
+CPUChildrenSystem: 0
+CPULoad: .170751
+Uptime: 35092
+ReqPerSec: .923373
+BytesPerSec: 1005.67
+BytesPerReq: 1089.13
+BusyWorkers: 1
+IdleWorkers: 99
+ConnsTotal: 0
+ConnsAsyncWriting: 0
+ConnsAsyncKeepAlive: 0
+ConnsAsyncClosing: 0
+Scoreboard: __________________________________________________________________________________________W_________............................................................................................................................................................................................................................................................................................................
+```
+
+From the apache status output it collects:
+
+ - total accesses (incremental value, rendered as requests/s)
+ - total bandwidth (incremental value, rendered as bandwidth/s)
+ - requests per second (this appears to be calculated by apache as an average for its lifetime, while the one calculated by netdata using the total accesses counter is real-time)
+ - bytes per second (average for the lifetime of the apache server)
+ - bytes per request (average for the lifetime of the apache server)
+ - workers by status (`busy` and `idle`)
+ - total connections (currently active connections - offered by apache 2.4+)
+ - async connections per status (`keepalive`, `writing`, `closing` - offered by apache 2.4+)
+
+## Configuration
+
+The configuration is stored in `/etc/netdata/charts.d/apache.conf`.
+To edit this file on your system run `/etc/netdata/edit-config charts.d/apache.conf`.
+
+The internal default is:
+
+```sh
+# the URL your apache server is responding with mod_status information.
+apache_url="http://127.0.0.1:80/server-status?auto"
+
+# use this to set custom curl options you may need
+apache_curl_opts=
+
+# set this to a NUMBER to overwrite the update frequency
+# it is in seconds
+apache_update_every=
+```
+
+The default `apache_update_every` is configured in netdata.
+
+## Auto-detection
+
+If you have configured your apache server to offer server-status information on localhost clients, the defaults should work fine.
+
+## Apache Configuration
+
+Apache configuration differs between distributions. Please check your distribution's documentation for information on enabling apache's `mod_status` module.
+
+If you are able to run successfully, by hand this command:
+
+```sh
+curl "http://127.0.0.1:80/server-status?auto"
+```
+
+netdata will be able to do it too.
+
+Notice: You may need to have the default `000-default.conf ` website enabled in order for the status mod to work.
diff --git a/collectors/fping.plugin/README.md b/collectors/fping.plugin/README.md
index 597d381717..0554a7edcf 100644
--- a/collectors/fping.plugin/README.md
+++ b/collectors/fping.plugin/README.md
@@ -1,8 +1,10 @@
# fping.plugin
-The fping plugin supports monitoring latency, packet loss and uptime of any number of hosts, by pinging them with fping.
+The fping plugin supports monitoring latency, packet loss and uptime of any number of network end points,
+by pinging them with `fping`.
-A recent version of `fping` is required (one that supports option ` -N `). The supplied plugin can install it. Run:
+A recent version of `fping` is required (one that supports option ` -N `).
+The supplied plugin can install it, by running:
```sh
/usr/libexec/netdata/plugins.d/fping.plugin install
@@ -10,7 +12,8 @@ A recent version of `fping` is required (one that supports option ` -N `). The s
The above will download, build and install the right version as `/usr/local/bin/fping`.
-Then you need to edit `/etc/netdata/fping.conf` (to edit it on your system run `/etc/netdata/edit-config fping.conf`) like this:
+Then you need to edit `/etc/netdata/fping.conf` (to edit it on your system run
+`/etc/netdata/edit-config fping.conf`) like this:
```sh
# uncomment the following line - it should already be there
@@ -31,12 +34,10 @@ ping_every=200
fping_opts="-R -b 56 -i 1 -r 0 -t 5000"
```
-The latest version of the config: https://github.com/netdata/netdata/blob/master/conf.d/fping.conf
-
## alarms
netdata will automatically attach a few alarms for each host.
-Check the latest versions of the fping alarms here: https://github.com/netdata/netdata/blob/master/conf.d/health.d/fping.conf
+Check the [latest versions of the fping alarms](https://github.com/netdata/netdata/blob/master/health/health.d/fping.conf)
## Additional Tips
@@ -56,7 +57,8 @@ ping_every=5000
### Multiple fping Plugins With Different Settings
-You may need to run multiple fping plugins with different settings for different hosts. For example, you may need to ping a few hosts 10 times per second, and others once per second.
+You may need to run multiple fping plugins with different settings for different end points.
+For example, you may need to ping a few hosts 10 times per second, and others once per second.
netdata allows you to add as many `fping` plugins as you like.
@@ -64,40 +66,31 @@ Follow this procedure:
**1. Create New fping Configuration File**
-Step Into Configuration Directory
```sh
+# Step Into Configuration Directory
cd /etc/netdata
-```
-
-Copy Original fping Configuration File To New Configuration File
-```sh
+# Copy Original fping Configuration File To New Configuration File
cp fping.conf fping2.conf
```
-Edit `fping2.conf` and set the settings and the hosts you need
+Edit `fping2.conf` and set the settings and the hosts you need for the seconds instance.
**2. Soft Link Original fping Plugin to New Plugin File**
-Become root (If The Step Step Is Performed As Non-Root User)
-
```sh
+# Become root (If The Step Step Is Performed As Non-Root User)
sudo su
-```
-Step Into The Plugins Directory
-
-```sh
+# Step Into The Plugins Directory
cd /usr/libexec/netdata/plugins.d
-```
-Link fping.plugin to fping2.plugin
-
-```sh
+# Link fping.plugin to fping2.plugin
ln -s fping.plugin fping2.plugin
```
That's it. netdata will detect the new plugin and start it.
-You can name the new plugin any name you like. Just make sure the plugin and the configuration file have the same name.
+You can name the new plugin any name you like.
+Just make sure the plugin and the configuration file have the same name.
diff --git a/collectors/python.d.plugin/go_expvar/README.md b/collectors/python.d.plugin/go_expvar/README.md
index ec5dbdc400..6309c195f3 100644
--- a/collectors/python.d.plugin/go_expvar/README.md
+++ b/collectors/python.d.plugin/go_expvar/README.md
@@ -1,9 +1,9 @@
# go_expvar
-The `go_expvar` module can monitor any Go application that exposes its metrics with the use of `expvar` package from the Go standard library.
+The `go_expvar` module can monitor any Go application that exposes its metrics with the use of
+`expvar` package from the Go standard library.
`go_expvar` produces charts for Go runtime memory statistics and optionally any number of custom charts.
-Please see the [wiki page](https://github.com/netdata/netdata/wiki/Monitoring-Go-Applications) for more info.
For the memory statistics, it produces the following charts:
@@ -32,11 +32,13 @@ For the memory statistics, it produces the following charts:
## Monitoring Go Applications
-Netdata can be used to monitor running Go applications that expose their metrics with the use of the [expvar package](https://golang.org/pkg/expvar/) included in Go standard library.
+Netdata can be used to monitor running Go applications that expose their metrics with
+the use of the [expvar package](https://golang.org/pkg/expvar/) included in Go standard library.
-The `expvar` package exposes these metrics over HTTP and is very easy to use. Consider this minimal sample below:
+The `expvar` package exposes these metrics over HTTP and is very easy to use.
+Consider this minimal sample below:
-```
+```go
package main
import (
@@ -49,18 +51,23 @@ func main() {
}
```
-When imported this way, the `expvar` package registers a HTTP handler at `/debug/vars` that exposes Go runtime's memory statistics in JSON format. You can inspect the output by opening the URL in your browser (or by using `wget` or `curl`). Sample output:
+When imported this way, the `expvar` package registers a HTTP handler at `/debug/vars` that
+exposes Go runtime's memory statistics in JSON format. You can inspect the output by opening
+the URL in your browser (or by using `wget` or `curl`).
-```
+Sample output:
+
+```json
{
"cmdline": ["./expvar-demo-binary"],
"memstats": {"Alloc":630856,"TotalAlloc":630856,"Sys":3346432,"Lookups":27, <ommited for brevity>}
}
```
-You can of course expose and monitor your own variables as well. Here is a sample Go application that exposes a few custom variables:
+You can of course expose and monitor your own variables as well.
+Here is a sample Go application that exposes a few custom variables:
-```
+```go
package main
import (
@@ -91,13 +98,17 @@ func main() {
}
```
-Apart from the runtime memory stats, this application publishes two counters and the number of currently running Goroutines and updates these stats every second.
+Apart from the runtime memory stats, this application publishes two counters and the
+number of currently running Goroutines and updates these stats every second.
-In the next section, we will cover how to monitor and chart these exposed stats with the use of `netdata`s ```go_expvar``` module.
+In the next section, we will cover how to monitor and chart these exposed stats with
+the use of `netdata`s ```go_expvar``` module.
### Using netdata go_expvar module
-The `go_expvar` module is disabled by default. To enable it, edit [`python.d.conf`](https://github.com/netdata/netdata/blob/master/conf.d/python.d.conf) (to edit it on your system run `/etc/netdata/edit-config python.d.conf`), and change the `go_expvar` variable to `yes`:
+The `go_expvar` module is disabled by default. To enable it, edit [`python.d.conf`](../python.d.conf)
+(to edit it on your system run `/etc/netdata/edit-config python.d.conf`), and change the `go_expvar`
+variable to `yes`:
```
# Enable / Disable python.d.plugin modules
@@ -113,7 +124,10 @@ go_expvar: yes
...
```
-Next, we need to edit the module configuration file (found at [`/etc/netdata/python.d/go_expvar.conf`](https://github.com/netdata/netdata/blob/master/conf.d/python.d/go_expvar.conf) by default) (to edit it on your system run `/etc/netdata/edit-config python.d/go_expvar.conf`). The module configuration consists of jobs, where each job can be used to monitor a separate Go application. Let's see a sample job configuration:
+Next, we need to edit the module configuration file (found at [`/etc/netdata/python.d/go_expvar.conf`](go_expvar.conf) by default)
+(to edit it on your system run `/etc/netdata/edit-config python.d/go_expvar.conf`).
+The module configuration consists of jobs, where each job can be used to monitor a separate Go application.
+Let's see a sample job configuration:
```
# /etc/netdata/python.d/go_expvar.conf
@@ -129,23 +143,29 @@ Let's go over each of the defined options:
name: 'app1'
-This is the job name that will appear at the netdata dashboard. If not defined, the job_name (top level key) will be used.
+This is the job name that will appear at the netdata dashboard.
+If not defined, the job_name (top level key) will be used.
url: 'http://127.0.0.1:8080/debug/vars'
-This is the URL of the expvar endpoint. As the expvar handler can be installed in a custom path, the whole URL has to be specified. This value is mandatory.
+This is the URL of the expvar endpoint. As the expvar handler can be installed
+in a custom path, the whole URL has to be specified. This value is mandatory.
collect_memstats: true
-Whether to enable collecting stats about Go runtime's memory. You can find more information about the exposed values at the [runtime package docs](https://golang.org/pkg/runtime/#MemStats).
+Whether to enable collecting stats about Go runtime's memory. You can find more
+information about the exposed values at the [runtime package docs](https://golang.org/pkg/runtime/#MemStats).
extra_charts: {}
-Enables the user to specify custom expvars to monitor and chart. Will be explained in more detail below.
+Enables the user to specify custom expvars to monitor and chart.
+Will be explained in more detail below.
-**Note: if `collect_memstats` is disabled and no `extra_charts` are defined, the plugin will disable itself, as there will be no data to collect!**
+**Note: if `collect_memstats` is disabled and no `extra_charts` are defined, the plugin will
+disable itself, as there will be no data to collect!**
-Apart from these options, each job supports options inherited from netdata's `python.d.plugin` and its base `UrlService` class. These are:
+Apart from these options, each job supports options inherited from netdata's `python.d.plugin`
+and its base `UrlService` class. These are:
update_every: 1 # the job's data collection frequency
priority: 60000 # the job's order on the dashboard
@@ -155,24 +175,30 @@ Apart from these options, each job supports options inherited from netdata's `py
### Monitoring custom vars with go_expvar
-Now, memory stats might be useful, but what if you want netdata to monitor some custom values that your Go application exposes? The `go_expvar` module can do that as well with the use of the `extra_charts` configuration variable.
+Now, memory stats might be useful, but what if you want netdata to monitor some custom values
+that your Go application exposes? The `go_expvar` module can do that as well with the use of
+the `extra_charts` configuration variable.
-The `extra_charts` variable is a YaML list of netdata chart definitions. Each chart definition has the following keys:
+The `extra_charts` variable is a YaML list of netdata chart definitions.
+Each chart definition has the following keys:
id: netdata chart ID
options: a key-value mapping of chart options
lines: a list of line definitions
-**Note: please do not use dots in the chart or line ID field. See [this issue](https://github.com/netdata/netdata/pull/1902#issuecomment-284494195) for explanation.**
+**Note: please do not use dots in the chart or line ID field.
+See [this issue](https://github.com/netdata/netdata/pull/1902#issuecomment-284494195) for explanation.**
Please see these two links to the official netdata documentation for more information about the values:
-- [External plugins - charts](https://github.com/netdata/netdata/wiki/External-Plugins#chart)
+- [External plugins - charts](../../plugins.d/#chart)
- [Chart variables](https://github.com/netdata/netdata/wiki/How-to-write-new-module#global-variables-order-and-chart)
**Line definitions**
-Each chart can define multiple lines (dimensions). A line definition is a key-value mapping of line options. Each line can have the following options:
+Each chart can define multiple lines (dimensions).
+A line definition is a key-value mapping of line options.
+Each line can have the following options:
# mandatory
expvar_key: the name of the expvar as present in the JSON output of /debug/vars endpoint
@@ -187,9 +213,11 @@ Each chart can define multiple lines (dimensions). A line definition is a key-va
hidden: False
Please see the following link for more information about the options and their default values:
-[External plugins - dimensions](https://github.com/netdata/netdata/wiki/External-Plugins#dimension)
+[External plugins - dimensions](../../plugins.d/#dimension)
-Apart from top-level expvars, this plugin can also parse expvars stored in a multi-level map; All dicts in the resulting JSON document are then flattened to one level. Expvar names are joined together with '.' when flattening.
+Apart from top-level expvars, this plugin can also parse expvars stored in a multi-level map;
+All dicts in the resulting JSON document are then flattened to one level.
+Expvar names are joined together with '.' when flattening.
Example:
```
@@ -199,11 +227,15 @@ Example:
}
```
-In the above case, the exported variables will be available under `runtime.goroutines`, `counters.cnt1` and `counters.cnt2` expvar_keys. If the flattening results in a key collision, the first defined key wins and all subsequent keys with the same name are ignored.
+In the above case, the exported variables will be available under `runtime.goroutines`,
+`counters.cnt1` and `counters.cnt2` expvar_keys. If the flattening results in a key collision,
+the first defined key wins and all subsequent keys with the same name are ignored.
**Configuration example**
-The configuration below matches the second Go application described above. Netdata will monitor and chart memory stats for the application, as well as a custom chart of running goroutines and two dummy counters.
+The configuration below matches the second Go application described above.
+Netdata will monitor and chart memory stats for the application, as well as a custom chart of
+running goroutines and two dummy counters.
```
app1: