diff options
author | Costa Tsaousis <costa@tsaousis.gr> | 2018-10-17 13:34:53 +0300 |
---|---|---|
committer | GitHub <noreply@github.com> | 2018-10-17 13:34:53 +0300 |
commit | 8cff4f255e62e0414527b2b01278adc4243baee1 (patch) | |
tree | 4d6a5c435ab7629948d71196ea6b6d44031dd099 | |
parent | b8f6925a8cad0621a6ae7293932e6c1cca58bc33 (diff) |
fixed documentation links (#4418)
* fixed documentation links
* updated apps.plugin info
* updated apps.plugin info
* updated apps.plugin info
-rw-r--r-- | collectors/apps.plugin/README.md | 217 | ||||
-rw-r--r-- | collectors/charts.d.plugin/apache/README.md | 125 | ||||
-rw-r--r-- | collectors/fping.plugin/README.md | 41 | ||||
-rw-r--r-- | collectors/python.d.plugin/go_expvar/README.md | 88 |
4 files changed, 371 insertions, 100 deletions
diff --git a/collectors/apps.plugin/README.md b/collectors/apps.plugin/README.md index c427e04223..a2cb9a70d6 100644 --- a/collectors/apps.plugin/README.md +++ b/collectors/apps.plugin/README.md @@ -1,48 +1,94 @@ # apps.plugin -This plugin provides charts for 3 sections of the default dashboard: +`apps.plugin` breaks down system resource usage to **processes**, **users** and **user groups**. -1. Per application charts -2. Per user charts -3. Per user group charts +To achieve this task, it iterates through the whole process tree, collecting resource usage information +for every process found running. -## Per application charts +Since netdata needs to present this information in charts and track them through time, +instead of presenting a `top` like list, `apps.plugin` uses a pre-defined list of **process groups** +to which it assigns all running processes. This list is [customizable](apps_groups.conf) and netdata +ships with a good default for most cases (to edit it on your system run `/etc/netdata/edit-config apps_groups.conf`). -This plugin walks through the entire `/proc` filesystem and aggregates statistics for applications of interest, defined in `/etc/netdata/apps_groups.conf` (the default is [here](apps_groups.conf)) (to edit it on your system run `/etc/netdata/edit-config apps_groups.conf`). +So, `apps.plugin` builds a process tree (much like `ps fax` does in Linux), and groups +processes together (evaluating both child and parent processes) so that the result is always a list with +a predefined set of members (of course, only process groups found running are reported). -The plugin internally builds a process tree (much like `ps fax` does), and groups processes together (evaluating both child and parent processes) so that the result is always a chart with a predefined set of dimensions (of course, only application groups found running are reported). +> If you find that `apps.plugin` categorizes standard applications as `other`, we would be +> glad to accept pull requests improving the [defaults](apps_groups.conf) shipped with netdata. -Using this information it provides the following charts (per application group defined in `/etc/netdata/apps_groups.conf` - to edit it on your system run `/etc/netdata/edit-config apps_groups.conf`): +Unlike traditional process monitoring tools (like `top`), `apps.plugin` is able to account the resource +utilization of exit processes. Their utilization is accounted at their currently running parents. +So, `apps.plugin` is perfectly able to measure the resources used by shell scripts and other processes +that fork/spawn other short lived processes hundreds of times per second. -1. Total CPU usage -2. Total User CPU usage -3. Total System CPU usage -4. Total Disk Physical Reads -5. Total Disk Physical Writes -6. Total Disk Logical Reads -7. Total Disk Logical Writes -8. Total Open Files (unique files - if a file is found open multiple times, it is counted just once) -9. Total Dedicated Memory (non shared) -10. Total Minor Page Faults -11. Total Number of Processes -12. Total Number of Threads -13. Total Number of Pipes -14. Total Swap Activity (Major Page Faults) -15. Total Open Sockets +For example, ssh to a server running netdata and execute this: -## Per User Charts +```sh +while true; do ls -l /var/run >/dev/null; done +``` + +All the console tools will report that a a CPU core is 100% used, but they will fail to identify which +process is using all that CPU (because there is no single process using it - thousands of `ls` per second +are using it). Netdata however, will be able to identify that `ssh` is using it +(`ssh` is the parent process group defined in its [default config](apps_groups.conf)): + +![](https://cloud.githubusercontent.com/assets/2662304/21076220/c9687848-bf2e-11e6-8d81-348592c5aca2.png) + +This feature makes `apps.plugin` unique in narrowing down the list of offending processes that may be +responsible for slow downs, or abusing system resources. + +## Charts + +`apps.plugin` provides charts for 3 sections: + +1. Per application charts as **Applications** at netdata dashboards +2. Per user charts as **Users** at netdata dashboards +3. Per user group charts as **User Groups** at netdata dashboards + +Each of these sections provides the same number of charts: + +- CPU Utilization + - Total CPU usage + - User / System CPU usage +- Disk I/O + - Physical Reads / Writes + - Logical Reads / Writes + - Open Unique Files (if a file is found open multiple times, it is counted just once) +- Memory + - Real Memory Used (non shared) + - Virtual Memory Allocated + - Minor Page Faults (i.e. memory activity) +- Processes + - Threads Running + - Processes Running + - Pipes Open +- Swap Memory + - Swap Memory Used + - Major Page Faults (i.e. swap activity) +- Network + - Sockets Open -All the above charts, are also grouped by username, using the effective uid of each process. +The above are reported: -## Per Group Charts +- For **Applications** per [target configured](apps_groups.conf). +- For **Users** per username or UID (when the username is not available). +- For **User Groups** per groupname or GID (when groupname is not available). -All the above charts, are also grouped by group name, using the effective gid of each process. +## Performance -## CPU Usage +`apps.plugin` is a complex piece of software and has a lot of work to do +We are proud that `apps.plugin` is a lot faster compared to any other similar tool, +while collecting a lot more information for the processes, however the fact is that +this plugin requires more CPU resources than the netdata daemon itself. -`apps.plugin` is a complex software piece and has a lot of work to do (actually this plugin requires more CPU resources that the netdata daemon). For each process running, `apps.plugin` reads several `/proc` files to get CPU usage, memory allocated, I/O usage, open file descriptors, etc. Doing this work per-second, especially on hosts with several thousands of processes, may increase the CPU resources consumed by the plugin. +Under Linux, for each process running, `apps.plugin` reads several `/proc` files +per process. Doing this work per-second, especially on hosts with several thousands +of processes, may increase the CPU resources consumed by the plugin. -In such cases, you many need to lower its data collection frequency. To do this, edit `/etc/netdata/netdata.conf` and find this section: +In such cases, you many need to lower its data collection frequency. + +To do this, edit `/etc/netdata/netdata.conf` and find this section: ``` [plugin:apps] @@ -50,8 +96,8 @@ In such cases, you many need to lower its data collection frequency. To do this, # command options = ``` -Uncomment the line `update every` and set it to a higher number. If you just set it to ` 2 `, its CPU resources will be cut in half, and data collection will be once every 2 seconds. - +Uncomment the line `update every` and set it to a higher number. If you just set it to ` 2 `, +its CPU resources will be cut in half, and data collection will be once every 2 seconds. ## Configuration @@ -64,27 +110,63 @@ The configuration file works accepts multiple lines, each having this format: group: process1 process2 ... ``` -Process names should be given as they appear when running `ps -e`. The program will actually match the process names in the `/proc/PID/status` file. So, to be sure the name is right for a process running with PID ` X `, do this: +Each group can be given multiple times, to add more processes to it. -```sh -cat /proc/X/status -``` +For the **Applications** section, only groups configured in this file are reported. +All other processes will be reported as `other`. + +For each process given, its whole process tree will be grouped, not just the process matched. +The plugin will include both parents and children. + +The process names are the ones returned by: + + - `ps -e` or `cat /proc/PID/stat` + - in case of substring mode (see below): `/proc/PID/cmdline` + +To add process names with spaces, enclose them in quotes (single or double) +example: ` 'Plex Media Serv' ` or ` "my other process" `. + +You can add an asterisk ` * ` at the beginning and/or the end of a process: -The first line on the output is `Name: xxxxx`. This is the process name `apps.plugin` sees. + - `*name` *suffix* mode: will search for processes ending with `name` (at `/proc/PID/stat`) + - `name*` *prefix* mode: will search for processes beginning with `name` (at `/proc/PID/stat`) + - `*name*` *substring* mode: will search for `name` in the whole command line (at `/proc/PID/cmdline`) -The order of the lines in the file is important only if you include the same process name to multiple groups. +If you enter even just one *name* (substring), `apps.plugin` will process +`/proc/PID/cmdline` for all processes (of course only once per process: when they are first seen). -## Apps plugin is missing information +To add processes with single quotes, enclose them in double quotes: ` "process with this ' single quote" ` -`apps.plugin` requires additional privileges to collect all the information it needs. The problem is described in issue #157. +To add processes with double quotes, enclose them in single quotes: ` 'process with this " double quote' ` -When netdata is installed, `apps.plugin` is given the capabilities `cap_dac_read_search,cap_sys_ptrace+ep`. If that is not possible (i.e. `setcap` fails), `apps.plugin` is setuid to `root`. +If a group or process name starts with a ` - `, the dimension will be hidden from the chart (cpu chart only). -## linux capabilities in containers +If a process starts with a ` + `, debugging will be enabled for it (debugging produces a lot of output - do not enable it in production systems). -There are a few cases, like `docker` and `virtuozzo` containers, where `setcap` succeeds, but the capabilities are silently ignored (in `lxc` containers `setcap` fails). +You can add any number of groups. Only the ones found running will affect the charts generated. +However, producing charts with hundreds of dimensions may slow down your web browser. -In these cases that `setcap` succeeds by capabilities do not work, you will have to setuid to root `apps.plugin` by running these commands: +The order of the entries in this list is important: the first that matches a process is used, so put important +ones at the top. Processes not matched by any row, will inherit it from their parents or children. + +The order also controls the order of the dimensions on the generated charts (although applications started +after apps.plugin is started, will be appended to the existing list of dimensions the netdata daemon maintains). + +## Permissions + +`apps.plugin` requires additional privileges to collect all the information it needs. +The problem is described in issue #157. + +When netdata is installed, `apps.plugin` is given the capabilities `cap_dac_read_search,cap_sys_ptrace+ep`. +If this fails (i.e. `setcap` fails), `apps.plugin` is setuid to `root`. + +#### linux capabilities in containers + +There are a few cases, like `docker` and `virtuozzo` containers, where `setcap` succeeds, but the capabilities +are silently ignored (in `lxc` containers `setcap` fails). + +In these cases ()`setcap` succeeds but capabilities do not work), you will have to setuid +to root `apps.plugin` by running these commands: ```sh chown root:netdata /usr/libexec/netdata/plugins.d/apps.plugin @@ -93,11 +175,50 @@ chmod 4750 /usr/libexec/netdata/plugins.d/apps.plugin You will have to run these, every time you update netdata. +## Security + +`apps.plugin` performs a hard-coded function of building the process tree in memory, +iterating forever, collecting metrics for each running process and sending them to netdata. +This is a one-way communication, from `apps.plugin` to netdata. + +So, since `apps.plugin` cannot be instructed by netdata for the actions it performs, +we think it is pretty safe to allow it have these increased privileges. + +Keep in mind that `apps.plugin` will still run without escalated permissions, +but it will not be able to collect all the information. + +## Application Badges + +You can create badges that you can embed anywhere you like, with URLs like this: + +``` +https://your.netdata.ip:19999/api/v1/badge.svg?chart=apps.processes&dimensions=myapp&value_color=green%3E0%7Cred +``` + +The color expression unescaped is this: `value_color=green>0|red`. + +Here is an example for the process group `sql` at `https://registry.my-netdata.io`: + +![image](https://registry.my-netdata.io/api/v1/badge.svg?chart=apps.processes&dimensions=sql&value_color=green%3E0%7Cred) -### Is is safe to give `apps.plugin` these privileges? +Netdata is able give you a lot more badges for your app. +Examples below for process group `sql`: -`apps.plugin` performs a hard-coded function of building the process tree in memory, iterating forever, collecting metrics for each running process and sending them to netdata. This is a one-way communication, from `apps.plugin` to netdata. +- CPU usage: ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.cpu&dimensions=sql&value_color=green=0%7Corange%3C50%7Cred) +- Disk Physical Reads ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.preads&dimensions=sql&value_color=green%3C100%7Corange%3C1000%7Cred) +- Disk Physical Writes ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.pwrites&dimensions=sql&value_color=green%3C100%7Corange%3C1000%7Cred) +- Disk Logical Reads ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.lreads&dimensions=sql&value_color=green%3C100%7Corange%3C1000%7Cred) +- Disk Logical Writes ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.lwrites&dimensions=sql&value_color=green%3C100%7Corange%3C1000%7Cred) +- Open Files ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.files&dimensions=sql&value_color=green%3E30%7Cred) +- Real Memory ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.mem&dimensions=sql&value_color=green%3C100%7Corange%3C200%7Cred) +- Virtual Memory ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.vmem&dimensions=sql&value_color=green%3C100%7Corange%3C1000%7Cred) +- Swap Memory ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.swap&dimensions=sql&value_color=green=0%7Cred) +- Minor Page Faults ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.minor_faults&dimensions=sql&value_color=green%3C100%7Corange%3C1000%7Cred) +- Processes ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.processes&dimensions=sql&value_color=green%3E0%7Cred) +- Threads ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.threads&dimensions=sql&value_color=green%3E=28%7Cred) +- Major Faults (swap activity) ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.major_faults&dimensions=sql&value_color=green=0%7Cred) +- Open Pipes ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.pipes&dimensions=sql&value_color=green=0%7Cred) +- Open Sockets ![image](http://registry.my-netdata.io/api/v1/badge.svg?chart=apps.sockets&dimensions=sql&value_color=green%3E=3%7Cred) -So, since `apps.plugin` cannot be instructed by netdata for the actions it performs, we think it is pretty safe to allow it have these increased privileges. -Keep in mind that `apps.plugin` will still run without these permissions, but it will not be able to collect all the data for every process. +For more information about badges check [Generating Badges](https://github.com/netdata/netdata/wiki/Generating-Badges)
\ No newline at end of file diff --git a/collectors/charts.d.plugin/apache/README.md b/collectors/charts.d.plugin/apache/README.md index d82951aacc..890cee9844 100644 --- a/collectors/charts.d.plugin/apache/README.md +++ b/collectors/charts.d.plugin/apache/README.md @@ -1,2 +1,127 @@ > THIS MODULE IS OBSOLETE. > USE THE PYTHON ONE - IT SUPPORTS MULTIPLE JOBS AND IT IS MORE EFFICIENT + +--- + +# Apache Plugin (apache) + +The `apache` collector visualizes key performance data for an apache web server. + +## Example netdata charts + +For apache 2.2: + +![image](https://cloud.githubusercontent.com/assets/2662304/12530273/421c4d14-c1e2-11e5-9fb6-ca6d6dd3b1dd.png) + +For apache 2.4: + +![image](https://cloud.githubusercontent.com/assets/2662304/12530376/29ec26de-c1e6-11e5-9af1-e48aaf781795.png) + +## How it works + +It runs `curl "http://apache.host/server-status?auto` to fetch the current status of apache. + +It has been tested with apache 2.2 and apache 2.4. The latter also provides connections information (total and break down by status). + +Apache 2.2 response: + +```sh +$ curl "http://127.0.0.1/server-status?auto" +Total Accesses: 80057 +Total kBytes: 223017 +CPULoad: .018287 +Uptime: 64472 +ReqPerSec: 1.24173 +BytesPerSec: 3542.15 +BytesPerReq: 2852.59 +BusyWorkers: 1 +IdleWorkers: 49 +Scoreboard: _________________________......................................._W_______________________....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... +``` + +Apache 2.4 response: + +```sh +$ curl "http://127.0.0.1/server-status?auto" +127.0.0.1 +ServerVersion: Apache/2.4.18 (Unix) +ServerMPM: event +Server Built: Dec 14 2015 08:05:54 +CurrentTime: Saturday, 23-Jan-2016 14:42:06 EET +RestartTime: Saturday, 23-Jan-2016 04:57:13 EET +ParentServerConfigGeneration: 2 +ParentServerMPMGeneration: 1 +ServerUptimeSeconds: 35092 +ServerUptime: 9 hours 44 minutes 52 seconds +Load1: 0.32 +Load5: 0.32 +Load15: 0.27 +Total Accesses: 32403 +Total kBytes: 34464 +CPUUser: 30.37 +CPUSystem: 29.55 +CPUChildrenUser: 0 +CPUChildrenSystem: 0 +CPULoad: .170751 +Uptime: 35092 +ReqPerSec: .923373 +BytesPerSec: 1005.67 +BytesPerReq: 1089.13 +BusyWorkers: 1 +IdleWorkers: 99 +ConnsTotal: 0 +ConnsAsyncWriting: 0 +ConnsAsyncKeepAlive: 0 +ConnsAsyncClosing: 0 +Scoreboard: __________________________________________________________________________________________W_________............................................................................................................................................................................................................................................................................................................ +``` + +From the apache status output it collects: + + - total accesses (incremental value, rendered as requests/s) + - total bandwidth (incremental value, rendered as bandwidth/s) + - requests per second (this appears to be calculated by apache as an average for its lifetime, while the one calculated by netdata using the total accesses counter is real-time) + - bytes per second (average for the lifetime of the apache server) + - bytes per request (average for the lifetime of the apache server) + - workers by status (`busy` and `idle`) + - total connections (currently active connections - offered by apache 2.4+) + - async connections per status (`keepalive`, `writing`, `closing` - offered by apache 2.4+) + +## Configuration + +The configuration is stored in `/etc/netdata/charts.d/apache.conf`. +To edit this file on your system run `/etc/netdata/edit-config charts.d/apache.conf`. + +The internal default is: + +```sh +# the URL your apache server is responding with mod_status information. +apache_url="http://127.0.0.1:80/server-status?auto" + +# use this to set custom curl options you may need +apache_curl_opts= + +# set this to a NUMBER to overwrite the update frequency +# it is in seconds +apache_update_every= +``` + +The default `apache_update_every` is configured in netdata. + +## Auto-detection + +If you have configured your apache server to offer server-status information on localhost clients, the defaults should work fine. + +## Apache Configuration + +Apache configuration differs between distributions. Please check your distribution's documentation for information on enabling apache's `mod_status` module. + +If you are able to run successfully, by hand this command: + +```sh +curl "http://127.0.0.1:80/server-status?auto" +``` + +netdata will be able to do it too. + +Notice: You may need to have the default `000-default.conf ` website enabled in order for the status mod to work. diff --git a/collectors/fping.plugin/README.md b/collectors/fping.plugin/README.md index 597d381717..0554a7edcf 100644 --- a/collectors/fping.plugin/README.md +++ b/collectors/fping.plugin/README.md @@ -1,8 +1,10 @@ # fping.plugin -The fping plugin supports monitoring latency, packet loss and uptime of any number of hosts, by pinging them with fping. +The fping plugin supports monitoring latency, packet loss and uptime of any number of network end points, +by pinging them with `fping`. -A recent version of `fping` is required (one that supports option ` -N `). The supplied plugin can install it. Run: +A recent version of `fping` is required (one that supports option ` -N `). +The supplied plugin can install it, by running: ```sh /usr/libexec/netdata/plugins.d/fping.plugin install @@ -10,7 +12,8 @@ A recent version of `fping` is required (one that supports option ` -N `). The s The above will download, build and install the right version as `/usr/local/bin/fping`. -Then you need to edit `/etc/netdata/fping.conf` (to edit it on your system run `/etc/netdata/edit-config fping.conf`) like this: +Then you need to edit `/etc/netdata/fping.conf` (to edit it on your system run +`/etc/netdata/edit-config fping.conf`) like this: ```sh # uncomment the following line - it should already be there @@ -31,12 +34,10 @@ ping_every=200 fping_opts="-R -b 56 -i 1 -r 0 -t 5000" ``` -The latest version of the config: https://github.com/netdata/netdata/blob/master/conf.d/fping.conf - ## alarms netdata will automatically attach a few alarms for each host. -Check the latest versions of the fping alarms here: https://github.com/netdata/netdata/blob/master/conf.d/health.d/fping.conf +Check the [latest versions of the fping alarms](https://github.com/netdata/netdata/blob/master/health/health.d/fping.conf) ## Additional Tips @@ -56,7 +57,8 @@ ping_every=5000 ### Multiple fping Plugins With Different Settings -You may need to run multiple fping plugins with different settings for different hosts. For example, you may need to ping a few hosts 10 times per second, and others once per second. +You may need to run multiple fping plugins with different settings for different end points. +For example, you may need to ping a few hosts 10 times per second, and others once per second. netdata allows you to add as many `fping` plugins as you like. @@ -64,40 +66,31 @@ Follow this procedure: **1. Create New fping Configuration File** -Step Into Configuration Directory ```sh +# Step Into Configuration Directory cd /etc/netdata -``` - -Copy Original fping Configuration File To New Configuration File -```sh +# Copy Original fping Configuration File To New Configuration File cp fping.conf fping2.conf ``` -Edit `fping2.conf` and set the settings and the hosts you need +Edit `fping2.conf` and set the settings and the hosts you need for the seconds instance. **2. Soft Link Original fping Plugin to New Plugin File** -Become root (If The Step Step Is Performed As Non-Root User) - ```sh +# Become root (If The Step Step Is Performed As Non-Root User) sudo su -``` -Step Into The Plugins Directory - -```sh +# Step Into The Plugins Directory cd /usr/libexec/netdata/plugins.d -``` -Link fping.plugin to fping2.plugin - -```sh +# Link fping.plugin to fping2.plugin ln -s fping.plugin fping2.plugin ``` That's it. netdata will detect the new plugin and start it. -You can name the new plugin any name you like. Just make sure the plugin and the configuration file have the same name. +You can name the new plugin any name you like. +Just make sure the plugin and the configuration file have the same name. diff --git a/collectors/python.d.plugin/go_expvar/README.md b/collectors/python.d.plugin/go_expvar/README.md index ec5dbdc400..6309c195f3 100644 --- a/collectors/python.d.plugin/go_expvar/README.md +++ b/collectors/python.d.plugin/go_expvar/README.md @@ -1,9 +1,9 @@ # go_expvar -The `go_expvar` module can monitor any Go application that exposes its metrics with the use of `expvar` package from the Go standard library. +The `go_expvar` module can monitor any Go application that exposes its metrics with the use of +`expvar` package from the Go standard library. `go_expvar` produces charts for Go runtime memory statistics and optionally any number of custom charts. -Please see the [wiki page](https://github.com/netdata/netdata/wiki/Monitoring-Go-Applications) for more info. For the memory statistics, it produces the following charts: @@ -32,11 +32,13 @@ For the memory statistics, it produces the following charts: ## Monitoring Go Applications -Netdata can be used to monitor running Go applications that expose their metrics with the use of the [expvar package](https://golang.org/pkg/expvar/) included in Go standard library. +Netdata can be used to monitor running Go applications that expose their metrics with +the use of the [expvar package](https://golang.org/pkg/expvar/) included in Go standard library. -The `expvar` package exposes these metrics over HTTP and is very easy to use. Consider this minimal sample below: +The `expvar` package exposes these metrics over HTTP and is very easy to use. +Consider this minimal sample below: -``` +```go package main import ( @@ -49,18 +51,23 @@ func main() { } ``` -When imported this way, the `expvar` package registers a HTTP handler at `/debug/vars` that exposes Go runtime's memory statistics in JSON format. You can inspect the output by opening the URL in your browser (or by using `wget` or `curl`). Sample output: +When imported this way, the `expvar` package registers a HTTP handler at `/debug/vars` that +exposes Go runtime's memory statistics in JSON format. You can inspect the output by opening +the URL in your browser (or by using `wget` or `curl`). -``` +Sample output: + +```json { "cmdline": ["./expvar-demo-binary"], "memstats": {"Alloc":630856,"TotalAlloc":630856,"Sys":3346432,"Lookups":27, <ommited for brevity>} } ``` -You can of course expose and monitor your own variables as well. Here is a sample Go application that exposes a few custom variables: +You can of course expose and monitor your own variables as well. +Here is a sample Go application that exposes a few custom variables: -``` +```go package main import ( @@ -91,13 +98,17 @@ func main() { } ``` -Apart from the runtime memory stats, this application publishes two counters and the number of currently running Goroutines and updates these stats every second. +Apart from the runtime memory stats, this application publishes two counters and the +number of currently running Goroutines and updates these stats every second. -In the next section, we will cover how to monitor and chart these exposed stats with the use of `netdata`s ```go_expvar``` module. +In the next section, we will cover how to monitor and chart these exposed stats with +the use of `netdata`s ```go_expvar``` module. ### Using netdata go_expvar module -The `go_expvar` module is disabled by default. To enable it, edit [`python.d.conf`](https://github.com/netdata/netdata/blob/master/conf.d/python.d.conf) (to edit it on your system run `/etc/netdata/edit-config python.d.conf`), and change the `go_expvar` variable to `yes`: +The `go_expvar` module is disabled by default. To enable it, edit [`python.d.conf`](../python.d.conf) +(to edit it on your system run `/etc/netdata/edit-config python.d.conf`), and change the `go_expvar` +variable to `yes`: ``` # Enable / Disable python.d.plugin modules @@ -113,7 +124,10 @@ go_expvar: yes ... ``` -Next, we need to edit the module configuration file (found at [`/etc/netdata/python.d/go_expvar.conf`](https://github.com/netdata/netdata/blob/master/conf.d/python.d/go_expvar.conf) by default) (to edit it on your system run `/etc/netdata/edit-config python.d/go_expvar.conf`). The module configuration consists of jobs, where each job can be used to monitor a separate Go application. Let's see a sample job configuration: +Next, we need to edit the module configuration file (found at [`/etc/netdata/python.d/go_expvar.conf`](go_expvar.conf) by default) +(to edit it on your system run `/etc/netdata/edit-config python.d/go_expvar.conf`). +The module configuration consists of jobs, where each job can be used to monitor a separate Go application. +Let's see a sample job configuration: ``` # /etc/netdata/python.d/go_expvar.conf @@ -129,23 +143,29 @@ Let's go over each of the defined options: name: 'app1' -This is the job name that will appear at the netdata dashboard. If not defined, the job_name (top level key) will be used. +This is the job name that will appear at the netdata dashboard. +If not defined, the job_name (top level key) will be used. url: 'http://127.0.0.1:8080/debug/vars' -This is the URL of the expvar endpoint. As the expvar handler can be installed in a custom path, the whole URL has to be specified. This value is mandatory. +This is the URL of the expvar endpoint. As the expvar handler can be installed +in a custom path, the whole URL has to be specified. This value is mandatory. collect_memstats: true -Whether to enable collecting stats about Go runtime's memory. You can find more information about the exposed values at the [runtime package docs](https://golang.org/pkg/runtime/#MemStats). +Whether to enable collecting stats about Go runtime's memory. You can find more +information about the exposed values at the [runtime package docs](https://golang.org/pkg/runtime/#MemStats). extra_charts: {} -Enables the user to specify custom expvars to monitor and chart. Will be explained in more detail below. +Enables the user to specify custom expvars to monitor and chart. +Will be explained in more detail below. -**Note: if `collect_memstats` is disabled and no `extra_charts` are defined, the plugin will disable itself, as there will be no data to collect!** +**Note: if `collect_memstats` is disabled and no `extra_charts` are defined, the plugin will +disable itself, as there will be no data to collect!** -Apart from these options, each job supports options inherited from netdata's `python.d.plugin` and its base `UrlService` class. These are: +Apart from these options, each job supports options inherited from netdata's `python.d.plugin` +and its base `UrlService` class. These are: update_every: 1 # the job's data collection frequency priority: 60000 # the job's order on the dashboard @@ -155,24 +175,30 @@ Apart from these options, each job supports options inherited from netdata's `py ### Monitoring custom vars with go_expvar -Now, memory stats might be useful, but what if you want netdata to monitor some custom values that your Go application exposes? The `go_expvar` module can do that as well with the use of the `extra_charts` configuration variable. +Now, memory stats might be useful, but what if you want netdata to monitor some custom values +that your Go application exposes? The `go_expvar` module can do that as well with the use of +the `extra_charts` configuration variable. -The `extra_charts` variable is a YaML list of netdata chart definitions. Each chart definition has the following keys: +The `extra_charts` variable is a YaML list of netdata chart definitions. +Each chart definition has the following keys: id: netdata chart ID options: a key-value mapping of chart options lines: a list of line definitions -**Note: please do not use dots in the chart or line ID field. See [this issue](https://github.com/netdata/netdata/pull/1902#issuecomment-284494195) for explanation.** +**Note: please do not use dots in the chart or line ID field. +See [this issue](https://github.com/netdata/netdata/pull/1902#issuecomment-284494195) for explanation.** Please see these two links to the official netdata documentation for more information about the values: -- [External plugins - charts](https://github.com/netdata/netdata/wiki/External-Plugins#chart) +- [External plugins - charts](../../plugins.d/#chart) - [Chart variables](https://github.com/netdata/netdata/wiki/How-to-write-new-module#global-variables-order-and-chart) **Line definitions** -Each chart can define multiple lines (dimensions). A line definition is a key-value mapping of line options. Each line can have the following options: +Each chart can define multiple lines (dimensions). +A line definition is a key-value mapping of line options. +Each line can have the following options: # mandatory expvar_key: the name of the expvar as present in the JSON output of /debug/vars endpoint @@ -187,9 +213,11 @@ Each chart can define multiple lines (dimensions). A line definition is a key-va hidden: False Please see the following link for more information about the options and their default values: -[External plugins - dimensions](https://github.com/netdata/netdata/wiki/External-Plugins#dimension) +[External plugins - dimensions](../../plugins.d/#dimension) -Apart from top-level expvars, this plugin can also parse expvars stored in a multi-level map; All dicts in the resulting JSON document are then flattened to one level. Expvar names are joined together with '.' when flattening. +Apart from top-level expvars, this plugin can also parse expvars stored in a multi-level map; +All dicts in the resulting JSON document are then flattened to one level. +Expvar names are joined together with '.' when flattening. Example: ``` @@ -199,11 +227,15 @@ Example: } ``` -In the above case, the exported variables will be available under `runtime.goroutines`, `counters.cnt1` and `counters.cnt2` expvar_keys. If the flattening results in a key collision, the first defined key wins and all subsequent keys with the same name are ignored. +In the above case, the exported variables will be available under `runtime.goroutines`, +`counters.cnt1` and `counters.cnt2` expvar_keys. If the flattening results in a key collision, +the first defined key wins and all subsequent keys with the same name are ignored. **Configuration example** -The configuration below matches the second Go application described above. Netdata will monitor and chart memory stats for the application, as well as a custom chart of running goroutines and two dummy counters. +The configuration below matches the second Go application described above. +Netdata will monitor and chart memory stats for the application, as well as a custom chart of +running goroutines and two dummy counters. ``` app1: |