diff options
71 files changed, 5441 insertions, 312 deletions
diff --git a/.gitignore b/.gitignore index c3f327b37a..c64d759547 100644 --- a/.gitignore +++ b/.gitignore @@ -143,3 +143,8 @@ sitespeed-result/ # tests and temp files python.d/python-modules-installer.sh + +# documentation generated files +htmldoc/src +htmldoc/build +htmldoc/mkdocs.yml diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000000..7b05bab2ea --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,92 @@ +# Contributing + +Thank you for considering contributing to Netdata. + +We love to receive contributions. Maintaining a platform for monitoring everything imaginable requires a broad understanding of a plethora of technologies, systems and applications. We rely on community contributions and user feedback to continue providing the best monitoring solution out there. + +There are many ways to contribute, with varying requirements of skills: + +## All NetData Users + +### Give Netdata a GitHub star + +This is the minimum open-source users should contribute back to the projects they use. Github stars help the project gain visibility, stand out. So, if you use Netdata, consider pressing that button. **It really matters**. + +### Spread the word + +Community growth allows the project to attract new talent willing to contribute. This talent is then developing new features and improves the project. These new features and improvements attract more users and so on. It is a loop. So, post about netdata, present it to local meetups you attend, let your online social network or twitter, facebook, reddit, etc. know you are using it. **The more people involved, the faster the project evolves**. + +### Provide feedback + +Is there anything that bothers you about netdata? Did you experience an issue while installing it or using it? Would you like to see it evolve to you need? Let us know. [Open a github issue](https://github.com/netdata/netdata/issues) to discuss it. Feedback is very important for open-source projects. We can't commit we will do everything, but your feedback influences our road-map significantly. **We rely on your feedback to make Netdata better**. + +#### Help the developers understand what they have to do + +NetData is all about simplicity and meaningful presentation. It's impossible for a handful of people to know which metrics really matter when monitoring a particular software or hardware component you are interested in. Be specific about what should be collected, how the information should be presented in the dashboard and which alarms make sense in most situations. + +## Experienced Users + +### Help other users + +As the project grows, an increasing share of our time is spent on supporting this community of users in terms of answering questions, of helping users understand how netdata works and find their way with it. Helping other users is crucial. It allows the developers and maintainers of the project to focus on improving it. + +### Improve documentation + +Most of our documentation is in markdown (.md) files inside the netdata GitHub project. What remains in our Wiki will soon be moved in there as well. Don't be afraid to edit any of these documents and submit a GitHub Pull Request with your corrections/additions. + + +## Developers + +We expect most contributions to be for new data collection plugins. You can read about how external plugins work [here](collectors/plugins.d/). Additional instructions are available for [Node.js plugins](collectors/node.d/) and [Python plugis](collectors/python.d/). + +Of course we appreciate contributions for any other part of the NetData agent, including the [deamon](deamon/), [backends for long term archiving](backends/), innovative ways of using the [REST API](web/api) to create cool [Custom Dashboards](web/gui/custom/) or to include NetData charts in other applications, similarly to what can be done with [Confluence](web/gui/confluence/). + + +### Contributions Ground Rules + +#### Code of Conduct and CLA + +We expect all contributors to abide by the [Contributor Covenant Code of Conduct](CODE_OF_CONDUCT.md). For a pull request to be accepted, you will also need to accept the [netdata contributors license agreement](CONTRIBUTORS.md), as part of the PR process. + +#### Performance and efficiency + +Everything on Netdata is about efficiency. We need netdata to always be the most lightweight monitoring solution available. We will reject to merge PRs that are not optimal in resource utilization and efficiency. + +Of course there are cases that such technical excellence is either not reasonable or not feasible. In these cases, we may require the feature or code submitted to be by disabled by default. + +#### Meaningful metrics + +Unlike other monitoring solutions, Netdata requires all metrics collected to have some structure attached to them. So, Netdata metrics have a name, units, belong to a chart that has a title, a family, a context, belong to an application, etc. + +This structure is what makes netdata different. Most other monitoring solution collect bulk metrics in terms of name-value pairs and then expect their users to give meaning to these metrics during visualization. This does not work. It is neither practical nor reasonable to give to someone 2000 metrics and let him/her visualize them in a meaningful way. + +So, netdata requires all metrics to have a meaning at the time they are collected. We will reject to merge PRs that loosely collect just a "bunch of metrics", but we are very keen to help you fix this. + +#### Automated Testing + +Netdata is a very large application to have automated testing end-to-end. But automated testing is required for crucial functions of it. + +Generally, all pull requests should be coupled with automated testing scenarios. However since we do not currently have a framework in place for testing everything little bit of it, we currently require automated tests for parts of Netdata that seem too risky to be changed without automated testing. + +Of course, manual testing is always required. + +#### Netdata is a distributed application + +Netdata is a distributed monitoring application. A few basic features can become quite complicated for such applications. We may reject features that alter or influence the nature of netdata, though we usually discuss the requirements with contributors and help them adapt their code to be better suited for Netdata. + +#### Operating systems supported + +Netdata should be running everywhere, on every production system out there. + +Although we focus on **supported operating systems**, we still want Netdata to run even on non-supported systems. This, of course, may require some more manual work from the users (to prepare their environment, or enable certain flags, etc). + +If your contributions limit the number of operating systems supported we will request from you to improve it. + +#### Documentation + +Your contributions should be bundled with related documentation to help users understand how to use the features you introduce. + +#### Maintenance + +When you contribute code to Netdata, you are automatically accepting that you will be responsible for maintaining that code in the future. So, if users need help, or report bugs, we will invite you to the related github issues to help them or fix the issues or bugs of your contributions. + diff --git a/Makefile.am b/Makefile.am index ddb9a699a1..c90db5ca3c 100644 --- a/Makefile.am +++ b/Makefile.am @@ -44,6 +44,7 @@ EXTRA_DIST = \ CODE_OF_CONDUCT.md \ LICENSE \ REDISTRIBUTED.md \ + CONTRIBUTING.md \ $(NULL) SUBDIRS = \ @@ -61,6 +62,27 @@ dist_noinst_DATA= \ netdata.cppcheck \ netdata.spec \ package.json \ + doc/Add-more-charts-to-netdata.md \ + doc/Demo-Sites.md \ + doc/Donations-netdata-has-received.md \ + doc/Netdata-Security-and-Disclosure-Information.md \ + doc/Performance.md \ + doc/Running-behind-apache.md \ + doc/Running-behind-caddy.md \ + doc/Running-behind-lighttpd.md \ + doc/Running-behind-nginx.md \ + doc/Third-Party-Plugins.md \ + doc/a-github-star-is-important.md \ + doc/high-performance-netdata.md \ + doc/netdata-for-IoT.md \ + doc/netdata-security.md \ + doc/Why-Netdata.md \ + htmldoc/themes/material/partials/footer.html \ + installer/README.md \ + installer/UNINSTALL.md \ + installer/UPDATE.md \ + requirements.txt \ + runtime.txt \ $(NULL) # until integrated within build @@ -71,6 +93,8 @@ dist_noinst_SCRIPTS= \ kickstart-static64.sh \ netdata-installer.sh \ installer/functions.sh \ + htmldoc/buildhtml.sh \ + htmldoc/buildyaml.sh \ $(NULL) # ----------------------------------------------------------------------------- @@ -1,4 +1,4 @@ -# netdata [](https://travis-ci.com/netdata/netdata) [](https://bestpractices.coreinfrastructure.org/projects/2231) [](https://www.gnu.org/licenses/gpl-3.0) []() +# netdata [](https://travis-ci.com/netdata/netdata) [](https://bestpractices.coreinfrastructure.org/projects/2231) [](https://www.gnu.org/licenses/gpl-3.0) []() [](https://codeclimate.com/github/netdata/netdata) [](https://www.codacy.com/app/netdata/netdata?utm_source=github.com&utm_medium=referral&utm_content=netdata/netdata&utm_campaign=Badge_Grade) [](https://lgtm.com/projects/g/netdata/netdata/context:cpp) [](https://lgtm.com/projects/g/netdata/netdata/context:javascript) [](https://lgtm.com/projects/g/netdata/netdata/context:python) @@ -52,7 +52,7 @@ The following animated image, shows the top part of a typical netdata dashboard. *A typical netdata dashboard, in 1:1 timing. Charts can be panned by dragging them, zoomed in/out with `SHIFT` + `mouse wheel`, an area can be selected for zoom-in with `SHIFT` + `mouse selection`. Netdata is highly interactive and **real-time**, optimized to get the work done!* -> *We have a few online demos to see it in action: [http://my-netdata.io](http://my-netdata.io)* +> *We have a few online demos to check: [https://my-netdata.io](https://my-netdata.io)* ## User base @@ -73,7 +73,7 @@ When you install multiple netdata, they are integrated into **one distributed ap [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) -*in the last 24 hours:*<br/> [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) +*in the last 24 hours:*<br/> [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) ## Quick Start @@ -86,7 +86,7 @@ bash # install netdata, directly from github sources bash <(curl -Ss https://my-netdata.io/kickstart.sh) ``` -  +  The above command will: @@ -130,7 +130,6 @@ Netdata is **open-source**, **free**, super **fast**, very **easy**, completely It has been designed by **SysAdmins**, **DevOps** and **Developers** for troubleshooting performance problems, not just visualize metrics. - ## News `Nov 6th, 2018` - **[netdata v1.11.0 released!](https://github.com/netdata/netdata/releases)** @@ -256,13 +255,13 @@ To improve visual anomaly detection across charts, the user can highlight a time ## What does it monitor Netdata data collection is **extensible** - you can monitor anything you can get a metric for. -Its [Plugin API](collectors/plugins.d) supports all programing languages (anything can be a netdata plugin, BASH, python, perl, node.js, java, Go, ruby, etc). +Its [Plugin API](collectors/plugins.d/) supports all programing languages (anything can be a netdata plugin, BASH, python, perl, node.js, java, Go, ruby, etc). - For better performance, most system related plugins (cpu, memory, disks, filesystems, networking, etc) have been written in `C`. - For faster development and easier contributions, most application related plugins (databases, web servers, etc) have been written in `python`. #### APM (Application Performance Monitoring) -- **statsd** - [netdata is a fully featured statsd server](collectors/statsd.plugin). +- **statsd** - [netdata is a fully featured statsd server](collectors/statsd.plugin/). - **go_expvar** - collects metrics exposed by applications written in the Go programming language using the expvar package. - **Spring Boot** - monitors running Java Spring Boot applications that expose their metrics with the use of the Spring Boot Actuator included in Spring Boot library. diff --git a/backends/Makefile.am b/backends/Makefile.am index 268259edd7..b8daefc592 100644 --- a/backends/Makefile.am +++ b/backends/Makefile.am @@ -12,6 +12,7 @@ SUBDIRS = \ dist_noinst_DATA = \ README.md \ + WALKTHROUGH.md \ $(NULL) dist_noinst_SCRIPTS = \ diff --git a/backends/README.md b/backends/README.md index b449f060ff..97e9db9b60 100644 --- a/backends/README.md +++ b/backends/README.md @@ -1,4 +1,3 @@ - # Metrics Long Term Archiving netdata supports backends for archiving the metrics, or providing long term dashboards, @@ -111,7 +110,7 @@ of `netdata.conf` from your netdata): When multiple servers are defined, netdata will try the next one when the first one fails. This allows you to load-balance different servers: give your backend servers in different order on each netdata. - netdata also ships [`nc-backend.sh`](nc-backend.sh), + netdata also ships [`nc-backend.sh`](https://github.com/netdata/netdata/tree/master/backends/nc-backend.sh), a script that can be used as a fallback backend to save the metrics to disk and push them to the time-series database when it becomes available again. It can also be used to monitor / trace / debug the metrics netdata generates. @@ -186,7 +185,7 @@ netdata provides 5 charts: ## alarms -The latest version of the alarms configuration for monitoring the backend is [here](../health/health.d/backend.conf) +The latest version of the alarms configuration for monitoring the backend is [here](https://github.com/netdata/netdata/tree/master/health/health.d/backend.conf) netdata adds 4 alarms: diff --git a/backends/WALKTHROUGH.md b/backends/WALKTHROUGH.md new file mode 100644 index 0000000000..ac5020af53 --- /dev/null +++ b/backends/WALKTHROUGH.md @@ -0,0 +1,294 @@ +# Netdata, Prometheus, Grafana stack + +## Intro +In this article I will walk you through the basics of getting Netdata, +Prometheus and Grafana all working together and monitoring your application +servers. This article will be using docker on your local workstation. We will be +working with docker in an ad-hoc way, launching containers that run ‘/bin/bash’ +and attaching a TTY to them. I use docker here in a purely academic fashion and +do not condone running Netdata in a container. I pick this method so individuals +without cloud accounts or access to VMs can try this out and for it’s speed of +deployment. + +## Why Netdata, Prometheus, and Grafana +Some time ago I was introduced to Netdata by a coworker. We were attempting to +troubleshoot python code which seemed to be bottlenecked. I was instantly +impressed by the amount of metrics Netdata exposes to you. I quickly added +Netdata to my set of go-to tools when troubleshooting systems performance. + +Some time ago, even later, I was introduced to Prometheus. Prometheus is a +monitoring application which flips the normal architecture around and polls +rest endpoints for its metrics. This architectural change greatly simplifies +and decreases the time necessary to begin monitoring your applications. +Compared to current monitoring solutions the time spent on designing the +infrastructure is greatly reduced. Running a single Prometheus server per +application becomes feasible with the help of Grafana. + +Grafana has been the go to graphing tool for… some time now. It’s awesome, +anyone that has used it knows it’s awesome |