diff options
26 files changed, 447 insertions, 45 deletions
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7b05bab2ea..fa06565269 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -37,9 +37,9 @@ Most of our documentation is in markdown (.md) files inside the netdata GitHub p ## Developers -We expect most contributions to be for new data collection plugins. You can read about how external plugins work [here](collectors/plugins.d/). Additional instructions are available for [Node.js plugins](collectors/node.d/) and [Python plugis](collectors/python.d/). +We expect most contributions to be for new data collection plugins. You can read about how external plugins work [here](collectors/plugins.d/). Additional instructions are available for [Node.js plugins](collectors/node.d.plugin) and [Python plugis](collectors/python.d.plugin). -Of course we appreciate contributions for any other part of the NetData agent, including the [deamon](deamon/), [backends for long term archiving](backends/), innovative ways of using the [REST API](web/api) to create cool [Custom Dashboards](web/gui/custom/) or to include NetData charts in other applications, similarly to what can be done with [Confluence](web/gui/confluence/). +Of course we appreciate contributions for any other part of the NetData agent, including the [daemon](daemon), [backends for long term archiving](backends/), innovative ways of using the [REST API](web/api) to create cool [Custom Dashboards](web/gui/custom/) or to include NetData charts in other applications, similarly to what can be done with [Confluence](web/gui/confluence/). ### Contributions Ground Rules @@ -184,7 +184,7 @@ Function|Description|Documentation **Check**|A lockless independent watchdog is evaluating **health checks** on the collected metrics, triggers alarms, maintains a health transaction log and dispatches alarm notifications.|[`health`](health/#health-monitoring) **Stream**|An lockless independent worker is streaming metrics, in full detail and in real-time, to remote netdata servers, as soon as they are collected.|[`streaming`](streaming/#metrics-streaming) **Archive**|A lockless independent worker is down-sampling the metrics and pushes them to **backend** time-series databases.|[`backends`](backends/) -**Query**|Multiple independent workers are attached to the [internal web server](server/#netdata-web-server), servicing API requests, including [data queries](web/api/queries/#database-queries).|[`web/api`](web/api/#api) +**Query**|Multiple independent workers are attached to the [internal web server](web/server/#netdata-web-server), servicing API requests, including [data queries](web/api/queries/#database-queries).|[`web/api`](web/api/#api) The result is a highly efficient, low latency system, supporting multiple readers and one writer on each metric. @@ -357,7 +357,7 @@ Its [Plugin API](collectors/plugins.d/) supports all programing languages (anyth #### Web Servers - **[Apache and lighttpd](collectors/python.d.plugin/apache/)** - `mod-status` (v2.2, v2.4) and cache log statistics, for multiple servers. -- **[IPFS](python.d.plugin/ipfs/)** - bandwidth, peers. +- **[IPFS](collectors/python.d.plugin/ipfs/)** - bandwidth, peers. - **[LiteSpeed](collectors/python.d.plugin/litespeed/)** - reads the litespeed rtreport files to collect metrics. - **[Nginx](collectors/python.d.plugin/nginx/)** - `stub-status`, for multiple servers. - **[Nginx+](collectors/python.d.plugin/nginx_plus/)** - connects to multiple nginx_plus servers (local or remote) to collect real-time performance metrics. @@ -411,12 +411,12 @@ Its [Plugin API](collectors/plugins.d/) supports all programing languages (anyth #### Hardware Sensors - **[IPMI](collectors/freeipmi.plugin/)** - enterprise hardware sensors and events. - **[lm-sensors](collectors/python.d.plugin/sensors/)** - temperature, voltage, fans, power, humidity, etc. -- **[Nvidia](collectors/charts.d.plugin/nvidia_smi/)** - collects information for Nvidia GPUs. +- **[Nvidia](collectors/python.d.plugin/nvidia_smi/)** - collects information for Nvidia GPUs. - **[RPi](collectors/charts.d.plugin/sensors/)** - Raspberry Pi temperature sensors. - **[w1sensor](collectors/python.d.plugin/w1sensor/)** - collects data from connected 1-Wire sensors. #### UPSes -- **[apcupsd](charts.d.plugin/apcupsd/)** - load, charge, battery voltage, temperature, utility metrics, output metrics +- **[apcupsd](collectors/charts.d.plugin/apcupsd/)** - load, charge, battery voltage, temperature, utility metrics, output metrics - **[NUT](collectors/charts.d.plugin/nut/)** - load, charge, battery voltage, temperature, utility metrics, output metrics - **[Linux Power Supply](collectors/python.d.plugin/linux_power_supply/)** - collects metrics reported by power supply drivers on Linux. diff --git a/backends/README.md b/backends/README.md index 97e9db9b60..cc943d4d71 100644 --- a/backends/README.md +++ b/backends/README.md @@ -110,7 +110,7 @@ of `netdata.conf` from your netdata): When multiple servers are defined, netdata will try the next one when the first one fails. This allows you to load-balance different servers: give your backend servers in different order on each netdata. - netdata also ships [`nc-backend.sh`](https://github.com/netdata/netdata/tree/master/backends/nc-backend.sh), + netdata also ships [`nc-backend.sh`](nc-backend.sh), a script that can be used as a fallback backend to save the metrics to disk and push them to the time-series database when it becomes available again. It can also be used to monitor / trace / debug the metrics netdata generates. @@ -185,7 +185,7 @@ netdata provides 5 charts: ## alarms -The latest version of the alarms configuration for monitoring the backend is [here](https://github.com/netdata/netdata/tree/master/health/health.d/backend.conf) +The latest version of the alarms configuration for monitoring the backend is [here](../health/health.d/backend.conf) netdata adds 4 alarms: diff --git a/collectors/apps.plugin/README.md b/collectors/apps.plugin/README.md index cbee77ed5a..d1ca8114c1 100644 --- a/collectors/apps.plugin/README.md +++ b/collectors/apps.plugin/README.md @@ -7,7 +7,7 @@ for every process found running. Since netdata needs to present this information in charts and track them through time, instead of presenting a `top` like list, `apps.plugin` uses a pre-defined list of **process groups** -to which it assigns all running processes. This list is [customizable](https://github.com/netdata/netdata/tree/master/collectors/apps.plugin/apps_groups.conf) and netdata +to which it assigns all running processes. This list is [customizable](apps_groups.conf) and netdata ships with a good default for most cases (to edit it on your system run `/etc/netdata/edit-config apps_groups.conf`). So, `apps.plugin` builds a process tree (much like `ps fax` does in Linux), and groups @@ -15,7 +15,7 @@ processes together (evaluating both child and parent processes) so that the resu a predefined set of members (of course, only process groups found running are reported). > If you find that `apps.plugin` categorizes standard applications as `other`, we would be -> glad to accept pull requests improving the [defaults](https://github.com/netdata/netdata/tree/master/collectors/apps.plugin/apps_groups.conf) shipped with netdata. +> glad to accept pull requests improving the [defaults](apps_groups.conf) shipped with netdata. Unlike traditional process monitoring tools (like `top`), `apps.plugin` is able to account the resource utilization of exit processes. Their utilization is accounted at their currently running parents. @@ -55,7 +55,7 @@ Each of these sections provides the same number of charts: The above are reported: -- For **Applications** per [target configured](https://github.com/netdata/netdata/tree/master/collectors/apps.plugin/apps_groups.conf). +- For **Applications** per [target configured](apps_groups.conf). - For **Users** per username or UID (when the username is not available). - For **User Groups** per groupname or GID (when groupname is not available). @@ -85,7 +85,7 @@ its CPU resources will be cut in half, and data collection will be once every 2 ## Configuration -The configuration file is `/etc/netdata/apps_groups.conf` (the default is [here](https://github.com/netdata/netdata/tree/master/collectors/apps.plugin/apps_groups.conf)). +The configuration file is `/etc/netdata/apps_groups.conf` (the default is [here](apps_groups.conf)). To edit it on your system run `/etc/netdata/edit-config apps_groups.conf`. The configuration file works accepts multiple lines, each having this format: @@ -341,7 +341,7 @@ So, the `ssh` session is using 95% CPU time. Why `ssh`? `apps.plugin` groups all processes based on its configuration file -[`/etc/netdata/apps_groups.conf`](https://github.com/netdata/netdata/tree/master/collectors/apps.plugin/apps_groups.conf) +[`/etc/netdata/apps_groups.conf`](apps_groups.conf) (to edit it on your system run `/etc/netdata/edit-config apps_groups.conf`). The default configuration has nothing for `bash`, but it has for `sshd`, so netdata accumulates all ssh sessions to a dimension on the charts, called `ssh`. This includes all the processes in diff --git a/collectors/cgroups.plugin/README.md b/collectors/cgroups.plugin/README.md index 8c66ba20f2..47eeebc538 100644 --- a/collectors/cgroups.plugin/README.md +++ b/collectors/cgroups.plugin/README.md @@ -87,7 +87,7 @@ For this mapping netdata provides 2 configuration options: The whole point for the additional pattern list, is to limit the number of times the script will be called. Without this pattern list, the script might be called thousands of times, depending on the number of cgroups available in the system. -The above pattern list is matched against the path of the cgroup. For matched cgroups, netdata calls the script [cgroup-name.sh](https://github.com/netdata/netdata/blob/master/collectors/cgroups.plugin/cgroup-name.sh.in) to get its name. This script queries `docker`, or applies heuristics to find give a name for the cgroup. +The above pattern list is matched against the path of the cgroup. For matched cgroups, netdata calls the script [cgroup-name.sh](cgroup-name.sh.in) to get its name. This script queries `docker`, or applies heuristics to find give a name for the cgroup. ## Monitoring systemd services diff --git a/collectors/diskspace.plugin/README.md b/collectors/diskspace.plugin/README.md index 74d6cde3c9..f7d0e7b497 100644 --- a/collectors/diskspace.plugin/README.md +++ b/collectors/diskspace.plugin/README.md @@ -1,5 +1,6 @@ -> for disks performance monitoring, see the `proc` plugin, [here](../proc.plugin/#monitoring-disks-performance-with-netdata) - # diskspace.plugin This plugin monitors the disk space usage of mounted disks, under Linux. + +> for disks performance monitoring, see the `proc` plugin, [here](../proc.plugin/#monitoring-disks) + diff --git a/collectors/fping.plugin/README.md b/collectors/fping.plugin/README.md index 0554a7edcf..a83b7912ce 100644 --- a/collectors/fping.plugin/README.md +++ b/collectors/fping.plugin/README.md @@ -37,7 +37,7 @@ fping_opts="-R -b 56 -i 1 -r 0 -t 5000" ## alarms netdata will automatically attach a few alarms for each host. -Check the [latest versions of the fping alarms](https://github.com/netdata/netdata/blob/master/health/health.d/fping.conf) +Check the [latest versions of the fping alarms](../../health/health.d/fping.conf) ## Additional Tips diff --git a/collectors/node.d.plugin/README.md b/collectors/node.d.plugin/README.md index fe85fbe4ce..af8708c7b9 100644 --- a/collectors/node.d.plugin/README.md +++ b/collectors/node.d.plugin/README.md @@ -45,7 +45,7 @@ For more information check the **[[Installation]]** guide. ## configuring `node.d.plugin` `node.d.plugin` can work even without any configuration. Its default configuration file is -[/etc/netdata/node.d.conf](https://github.com/netdata/netdata/tree/master/collectors/node.d.plugin/node.d.conf) (to edit it on your system run `/etc/netdata/edit-config node.d.conf`). +[/etc/netdata/node.d.conf](node.d.conf) (to edit it on your system run `/etc/netdata/edit-config node.d.conf`). ## configuring `node.d.plugin` modules @@ -229,4 +229,4 @@ The `service` object defines a set of functions to allow you send information to --- *FIXME: document an operational node.d.plugin data collector - the best example is the -[snmp collector](https://github.com/netdata/netdata/tree/master/collectors/node.d.plugin/snmp/snmp.node.js)* +[snmp collector](snmp/snmp.node.js)* diff --git a/collectors/python.d.plugin/go_expvar/README.md b/collectors/python.d.plugin/go_expvar/README.md index 070d6d50f0..e3356e1f1c 100644 --- a/collectors/python.d.plugin/go_expvar/README.md +++ b/collectors/python.d.plugin/go_expvar/README.md @@ -106,7 +106,7 @@ the use of `netdata`s ```go_expvar``` module. ### Using netdata go_expvar module -The `go_expvar` module is disabled by default. To enable it, edit [`python.d.conf`](https://github.com/netdata/netdata/tree/master/collectors/python.d.plugin/python.d.conf) +The `go_expvar` module is disabled by default. To enable it, edit [`python.d.conf`](../python.d.conf) (to edit it on your system run `/etc/netdata/edit-config python.d.conf`), and change the `go_expvar` variable to `yes`: @@ -124,7 +124,7 @@ go_expvar: yes ... ``` -Next, we need to edit the module configuration file (found at [`/etc/netdata/python.d/go_expvar.conf`](https://github.com/netdata/netdata/tree/master/collectors/python.d.plugin/go_expvar/go_expvar.conf) by default) +Next, we need to edit the module configuration file (found at [`/etc/netdata/python.d/go_expvar.conf`](go_expvar.conf) by default) (to edit it on your system run `/etc/netdata/edit-config python.d/go_expvar.conf`). The module configuration consists of jobs, where each job can be used to monitor a separate Go application. Let's see a sample job configuration: diff --git a/collectors/python.d.plugin/sensors/README.md b/collectors/python.d.plugin/sensors/README.md index 482ce60fb3..eb1642d90c 100644 --- a/collectors/python.d.plugin/sensors/README.md +++ b/collectors/python.d.plugin/sensors/README.md @@ -6,7 +6,7 @@ Charts are created dynamically. ### configuration -For detailed configuration information please read [`sensors.conf`](https://github.com/netdata/netdata/tree/master/collectors/python.d.plugin/sensors/sensors.conf) file. +For detailed configuration information please read [`sensors.conf`](sensors.conf) file. ### possible issues diff --git a/collectors/python.d.plugin/springboot/README.md b/collectors/python.d.plugin/springboot/README.md index 8a368c0947..a1817cc2bf 100644 --- a/collectors/python.d.plugin/springboot/README.md +++ b/collectors/python.d.plugin/springboot/README.md @@ -119,4 +119,4 @@ You can disable the default charts by set `defaults.<chart-id>: false`. The dimension name of extras charts should replace `.` to `_`. -Please check [springboot.conf](https://github.com/netdata/netdata/tree/master/collectors/python.d.plugin/springboot/springboot.conf) for more examples. +Please check [springboot.conf](springboot.conf) for more examples. diff --git a/collectors/python.d.plugin/w1sensor/README.md b/collectors/python.d.plugin/w1sensor/README.md index 9df9719e57..b18f083510 100644 --- a/collectors/python.d.plugin/w1sensor/README.md +++ b/collectors/python.d.plugin/w1sensor/README.md @@ -8,6 +8,6 @@ Charts are created dynamically based on the number of detected sensors. ### configuration -For detailed configuration information please read [`w1sensor.conf`](https://github.com/netdata/netdata/tree/master/collectors/python.d.plugin/w1sensor/w1sensor.conf) file. +For detailed configuration information please read [`w1sensor.conf`](w1sensor.conf) file. --- diff --git a/collectors/python.d.plugin/web_log/README.md b/collectors/python.d.plugin/web_log/README.md index 60ca41640f..e25a03fb32 100644 --- a/collectors/python.d.plugin/web_log/README.md +++ b/collectors/python.d.plugin/web_log/README.md @@ -197,5 +197,5 @@ alarm|description|minimum<br/>requests|warning|critical The column `minimum requests` state the minimum number of requests required for the alarm to be evaluated. We found that when the site is receiving requests above this rate, these alarms are pretty accurate (i.e. no false-positives). -[**netdata**](https://my-netdata.io/) alarms are [user configurable](../../../health/health.d). So, even [`web_log` alarms can be adapted to your needs](../../../health/health.d/web_log.conf). +[**netdata**](https://my-netdata.io/) alarms are user configurable. Sample config files can be found under directory `health/health.d` of the netdata github repository. So, even [`web_log` alarms can be adapted to your needs](../../../health/health.d/web_log.conf). diff --git a/collectors/tc.plugin/README.md b/collectors/tc.plugin/README.md index 74b856e827..a8b151de3c 100644 --- a/collectors/tc.plugin/README.md +++ b/collectors/tc.plugin/README.md @@ -9,10 +9,10 @@ Netdata monitors `tc` QoS classes for all interfaces. If you also use [FireQOS](http://firehol.org/tutorial/fireqos-new-user/) it will collect interface and class names. -There is a [shell helper](https://github.com/netdata/netdata/tree/master/collectors/tc.plugin/tc-qos-helper.sh.in) for this (all parsing is done by the plugin +There is a [shell helper](tc-qos-helper.sh.in) for this (all parsing is done by the plugin in `C` code - this shell script is just a configuration for the command to run to get `tc` output). -The source of the tc plugin is [here](https://github.com/netdata/netdata/tree/master/collectors/tc.plugin/plugin_tc.c). It is somewhat complex, because a state +The source of the tc plugin is [here](plugin_tc.c). It is somewhat complex, because a state machine was needed to keep track of all the `tc` classes, including the pseudo classes tc dynamically creates. diff --git a/daemon/README.md b/daemon/README.md index e593581334..305fc961d9 100644 --- a/daemon/README.md +++ b/daemon/README.md @@ -468,7 +468,7 @@ When you compile netdata with debugging: 1. compiler optimizations for your CPU are disabled (netdata will run somewhat slower) -2. a lot of code is added all over netdata, to log debug messages to `/var/log/netdata/debug.log`. However, nothing is printed by default. netdata allows you to select which sections of netdata you want to trace. Tracing is activated via the config option `debug flags`. It accepts a hex number, to enable or disable specific sections. You can find the options supported at [log.h](https://github.com/netdata/netdata/blob/master/libnetdata/log/log.h). They are the `D_*` defines. The value `0xffffffffffffffff` will enable all possible debug flags. +2. a lot of code is added all over netdata, to log debug messages to `/var/log/netdata/debug.log`. However, nothing is printed by default. netdata allows you to select which sections of netdata you want to trace. Tracing is activated via the config option `debug flags`. It accepts a hex number, to enable or disable specific sections. You can find the options supported at [log.h](../libnetdata/log/log.h). They are the `D_*` defines. The value `0xffffffffffffffff` will enable all possible debug flags. Once netdata is compiled with debugging and tracing is enabled for a few sections, the file `/var/log/netdata/debug.log` will contain the messages. diff --git a/doc/netdata-security.md b/doc/netdata-security.md index 44e3acdbe4..79858656be 100644 --- a/doc/netdata-security.md +++ b/doc/netdata-security.md @@ -89,7 +89,7 @@ In netdata v1.9+ there is also access list support, like this: #### use an authenticating web server in proxy mode -Use **one nginx** (or one apache) server to provide authentication in front of **all your netdata servers**. So, you will be accessing all your netdata with URLs like `http://nginx.host/netdata/{NETDATA_HOSTNAME}/` and authentication will be shared among all of them (you will sign-in once for all your servers). Check [this wiki page for more information on configuring nginx for such a setup](https://github.com/netdata/netdata/blob/master/doc/Running-behind-nginx.md#netdata-via-nginx). +Use **one nginx** (or one apache) server to provide authentication in front of **all your netdata servers**. So, you will be accessing all your netdata with URLs like `http://nginx.host/netdata/{NETDATA_HOSTNAME}/` and authentication will be shared among all of them (you will sign-in once for all your servers). Check [this wiki page for more information on configuring nginx for such a setup](Running-behind-nginx.md#netdata-via-nginx). To use this method, you should firewall protect all your netdata servers, so that only the nginx IP will allowed to directly access netdata. To do this, run this on each of your servers (or use your firewall manager): @@ -151,7 +151,7 @@ Of course, there are many more methods you could use to protect netdata: ## registry or how to not send any information to a third party server -The default configuration uses a public registry under registry.my-netdata.io (more information about the registry here: [mynetdata-menu-item](https://github.com/netdata/netdata/tree/master/registry) ). Please be aware that if you use that public registry, you submit at least the following information to a third party server, which might violate your security policies: +The default configuration uses a public registry under registry.my-netdata.io (more information about the registry here: [mynetdata-menu-item](../registry/) ). Please be aware that if you use that public registry, you submit at least the following information to a third party server, which might violate your security policies: - Your public ip where the browser runs - The url where you open the web-ui in the browser (via http request referer) - The hostnames of the netdata servers diff --git a/health/README.md b/health/README.md index bf89100570..5d68d752af 100644 --- a/health/README.md +++ b/health/README.md @@ -485,8 +485,7 @@ The external script will be called for all status changes. ## Examples - -Check the **[health.d directory](health.d/)** for all alarms shipped with netdata. +Check the `health/health.d/` directory for all alarms shipped with netdata. Here are a few examples: diff --git a/health/notifications/README.md b/health/notifications/README.md index 6804424784..c06638adee 100644 --- a/health/notifications/README.md +++ b/health/notifications/README.md @@ -1,7 +1,7 @@ # Netdata alarm notifications The `exec` line in health configuration defines an external script that will be called once -the alarm is triggered. The default script is **[alarm-notify.sh](https://github.com/netdata/netdata/tree/master/health/notifications/alarm-notify.sh.in)**. +the alarm is triggered. The default script is **[alarm-notify.sh](alarm-notify.sh.in)**. You can change the default script globally by editing `/etc/netdata/netdata.conf`. @@ -15,7 +15,7 @@ It uses **roles**. For example `sysadmin`, `webmaster`, `dba`, etc. Each alarm is assigned to one or more roles, using the `to` line of the alarm configuration. Then `alarm-notify.sh` uses its own configuration file `/etc/netdata/health_alarm_notify.conf` -the default is [here](https://github.com/netdata/netdata/tree/master/health/notifications/health_alarm_notify.conf) +the default is [here](health_alarm_notify.conf) (to edit it on your system run `/etc/netdata/edit-config health_alarm_notify.conf`) to find the destination address of the notification for each method. @@ -31,7 +31,7 @@ So, for example the `sysadmin` role may send: ## Configuration -Edit [`/etc/netdata/health_alarm_notify.conf`](https://github.com/netdata/netdata/tree/master/health/notifications/health_alarm_notify.conf) +Edit [`/etc/netdata/health_alarm_notify.conf`](health_alarm_notify.conf) by running `/etc/netdata/edit-config health_alarm_notify.conf`: - settings per notification method: diff --git a/htmldoc/buildhtml.sh b/htmldoc/buildhtml.sh index 87354fe371..8a41f454fb 100755 --- a/htmldoc/buildhtml.sh +++ b/htmldoc/buildhtml.sh @@ -6,16 +6,28 @@ # Assumes that the script is executed from the root netdata folder, by calling htmldoc/buildhtml.sh # Copy all netdata .md files to htmldoc/src. Exclude htmldoc itself and also the directory node_modules generated by Netlify +echo "Copying files" rm -rf htmldoc/src find . -type d \( -path ./htmldoc -o -path ./node_modules \) -prune -o -name "*.md" -print | cpio -pd htmldoc/src # Modify the first line of the main README.md, to enable proper static html generation sed -i '0,/# netdata /s//# Introducing NetData\n\n/' htmldoc/src/README.md +echo "Creating mkdocs.yaml" + # Generate mkdocs.yaml htmldoc/buildyaml.sh > htmldoc/mkdocs.yml +echo "Fixing links" + +# Fix links (recursively, all types, executing replacements) +htmldoc/checklinks.sh -rax +if [ $? -eq 1 ] ; then exit 1 ; fi + +echo "Calling mkdocs" + # Build html docs mkdocs build --config-file=htmldoc/mkdocs.yml +echo "Finished" diff --git a/htmldoc/checklinks.sh b/htmldoc/checklinks.sh new file mode 100755 index 0000000000..856af80682 --- /dev/null +++ b/htmldoc/checklinks.sh @@ -0,0 +1,390 @@ +#!/bin/bash + +# Doc link checker +# Validates and tries to fix all links that will cause issues either in the repo, or in the html site + +dbg () { + if [ $VERBOSE -eq 1 ] ; then printf "%s\n" "${1}" ; fi +} + +printhelp () { + echo "Usage: htmldoc/checklinks.sh [-r OR -f <fname>] [OPTIONS] + -r Recursively check all mds in all child directories, except htmldoc and node_modules (which is generated by netlify) + -f Just check the passed md file + General Options: + -x Execute commands. By default the script runs in test mode with no files changed by the script (results and fixes are just shown). Use -x to have it apply the changes. + -u trys to follow URLs using curl + -v Outputs debugging messages + By default, nothing is actually checked. The following options tell it what to check: + -a Check all link types + -w Check wiki links (and just warn if you see one) + -b Check absolute links to the netdata repo (and change them to relative). Only checks links to https://github.com/netdata/netdata/????/master* + -l Check relative links to the netdata repo (and replace them with links that the html static site can live with, under htmldoc/src only) + -e Check external links, outside the wiki or the repo (useless without adding the -u option, to verify that they're not broken) + " +} + +fix () { + if [ $EXECUTE -eq 0 ] ; then + echo "-- SHOULD EXECUTE: $1" + else + dbg "-- EXECUTING: $1" + eval "$1" + fi +} + +ck_netdata_absolute () { + f=$1 + alnk=$2 + lnkinfile=$3 + testURL $alnk + + if [[ $f =~ ^(.*)/([^/]*)$ ]] ; then + fpath="${BASH_REMATCH[1]}" + dbg "-- Current file is at $fpath" + fi + + if [ $? -eq 0 ] ; then + rlnk=$(echo $alnk | sed 's/https:\/\/github.com\/netdata\/netdata\/....\/master\///g') + case $rlnk in + \#* ) dbg "-- (#somelink)" ;; + */ ) dbg "-- # (path/)" ;; + */#* ) dbg "-- # (path/#somelink)" ;; + */*.md ) dbg "-- # (path/filename.md)" ;; + */*.md#* ) dbg "-- # (path/filename.md#somelink)" ;; + *#* ) + dbg "-- # (path#somelink) -> (path/#somelink)" + if [[ $rlnk =~ ^(.*)#(.*)$ ]] ; then + dbg "-- $rlnk -> ${BASH_REMATCH[1]}/#${BASH_REMATCH[2]}" + rlnk="${BASH_REMATCH[1]}/#${BASH_REMATCH[2]}" + fi + ;; + * ) + if [ -f "$rlnk" ] ; then + dbg "-- # (path/someotherfile) $rlnk" + else + if [ -d "$rlnk" ] ; then + dbg "-- # (path) -> (path/)" + rlnk="$rlnk/" + else + echo "-- ERROR: $f - $alnk is neither a file nor a directory. Giving up!" + EXITCODE=1 + return + fi + fi + ;; + esac + + if [[ $rlnk =~ ^(.*)/([^/]*)$ ]] ; then + abspath="${BASH_REMATCH[1]}" + rest="${BASH_REMATCH[2]}" + dbg "-- Target file is at $abspath" + fi + relativelink=$(realpath --relative-to=$fpath $abspath) + if [ $? -eq 0 ] ; then + srch=$(echo $lnkinfile | sed 's/\//\\\//g') + if [ $relativelink = "." ] ; then + rplc=$(echo $rest | sed 's/\//\\\//g') + else + rplc=$(echo $relativelink/$rest | sed 's/\//\\\//g') + fi + fix "sed -i 's/($srch)/($rplc)/g' $f" + else + echo "-- ERROR: $f - Can't determine relative path of $alnk" + fi + else + echo "-- ERROR: $f - $alnk is a broken link" + EXITCODE=1 + return + fi +} + +testURL () { + if [ $TESTURLS -eq 0 ] ; then return 0 ; fi + dbg "-- Testing URL $1" + curl -sS $1 > /dev/null + if [ $? -gt 0 ] ; then + return 1 + fi + return 0 +} + +testinternal () { + # Check if the header referred to by the internal link exists in the same file + ifile=${1} + ilnk=${2} + header=${ilnk//-/} + dbg "-- Searching for \"$header\" in $ifile" + tr -d ',_.:? `'< $ifile | sed 's/-//g' | grep -i "^\#*$header\$" >/dev/null + if [ $? -eq 0 ] ; then + dbg "-- $ilnk found in $ifile" + return 0 + else + echo "-- ERROR: $ifile - $ilnk header not found in the file" + EXITCODE=1 + return 1 + fi +} + +testf () { + sf=$1 + tf=$2 + + if [ -f "$tf" ] ; then + dbg "-- $tf exists" + return 0 + else + echo "-- ERROR: $sf - $tf does not exist" + EXITCODE=1 + return 1 + fi +} + +ck_netdata_relative () { + f=${1} + rlnk=${2} + dbg "-- Checking relative link $rlnk" + fpath="." + fname="$f" + # First ensure that the link works in the repo, then try to fix it in htmldocs + if [[ $f =~ ^(.*)/([^/]*)$ ]] ; then + fpath="${BASH_REMATCH[1]}" + fname="${BASH_REMATCH[2]}" + dbg "-- Current file is at $fpath" + else + dbg "-- Current file is at root directory" + fi + # Cases to handle: + # (#somelink) + # (path/) + # (path/#somelink) + # (path/filename.md) -> htmldoc (path/filename/) + # (path/filename.md#somelink) -> htmldoc (path/filename/#somelink) + # (path#somelink) -> htmldoc (path/#somelink) + # (path/someotherfile) -> htmldoc (absolutelink) + # (path) -> htmldoc (path/) + + TRGT="" + s="" + + case "$rlnk" in + \#* ) + dbg "-- # (#somelink)" + testinternal $f $rlnk + ;; + */ ) + dbg "-- # (path/)" + TRGT="$fpath/${rlnk}README.md" + testf $f $TRGT + if [ $? -eq 0 ] ; then + if [ $fname != "README.md" ] ; then s="../$rlnk"; fi + fi + ;; + */#* ) + dbg "-- # (path/#somelink)" + if [[ $rlnk =~ ^(.*)/#(.*)$ ]] ; then + TRGT="$fpath/${BASH_REMATCH[1]}/README.md" + LNK="#${BASH_REMATCH[2]}" + dbg "-- Look for $LNK in $TRGT" + testf $f $TRGT + if [ $? -eq 0 ] ; then + testinternal $TRGT $LNK + if [ $? -eq 0 ] ; then + if [ $fname != "README.md" ] ; then s="../$rlnk"; fi + fi + fi + fi + ;; + *.md ) + dbg "-- # (path/filename.md) -> htmldoc (path/filename/)" + testf $f "$fpath/$rlnk" + if [ $? -eq 0 ] ; then + if [[ $rlnk =~ ^(.*)/(.*).md$ ]] ; then + if [ "${BASH_REMATCH[2]}" = "README" ] ; then + s="../${BASH_REMATCH[1]}/" + else + s="../${BASH_REMATCH[1]}/${BASH_REMATCH[2]}/" + fi + if [ $fname != "README.md" ] ; then s="../$s"; fi + fi + fi + ;; + *.md#* )</ |