diff options
| author | Markos Fountoulakis <44345837+mfundul@users.noreply.github.com> | 2019-06-21 13:31:03 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2019-06-21 13:31:03 +0300 |
| commit | 965ae842f3bfb093349b0dfe26e34a1f05622ebb (patch) | |
| tree | 5d9dac4ee07850bb11e419497344ba701b2e5ab4 /database | |
| parent | 6312080b697dd3b3415a504dd59c351815155e47 (diff) | |
Update documentation about dbengine (#6264)
* Update README.md with more up-to-date information regarding dbengine
Diffstat (limited to 'database')
| -rw-r--r-- | database/README.md | 10 | ||||
| -rw-r--r-- | database/engine/README.md | 37 |
2 files changed, 44 insertions, 3 deletions
diff --git a/database/README.md b/database/README.md index dc40a3e407..de0aa9b53b 100644 --- a/database/README.md +++ b/database/README.md @@ -87,7 +87,9 @@ server that will maintain the entire database for all nodes, and will also run h for all nodes. For this central netdata, memory size can be a problem. Fortunately, netdata supports several -memory modes. One interesting option for this setup is `memory mode = map`. +memory modes. **One interesting option** for this setup is `memory mode = map`. + +### map In this mode, the database of netdata is stored in memory mapped files. netdata continues to read and write the database in memory, but the kernel automatically loads and saves memory pages from/to @@ -162,8 +164,10 @@ vm.dirty_ratio = 90 vm.dirty_writeback_centisecs = 0 ``` -There is another memory mode to help overcome the memory size problem. What is most interesting -for this setup is `memory mode = dbengine`. +There is another memory mode to help overcome the memory size problem. What is **most interesting +for this setup** is `memory mode = dbengine`. + +### dbengine In this mode, the database of netdata is stored in database files. The [Database Engine](engine/) works like a traditional database. There is some amount of RAM dedicated to data caching and diff --git a/database/engine/README.md b/database/engine/README.md index fcb313d539..334f5046c7 100644 --- a/database/engine/README.md +++ b/database/engine/README.md @@ -106,4 +106,41 @@ There are explicit memory requirements **per** DB engine **instance**, meaning * An important observation is that RAM usage depends on both the `page cache size` and the `dbengine disk space` options. +## File descriptor requirements + +The Database Engine may keep a **significant** amount of files open per instance (e.g. per streaming +slave or master server). When configuring your system you should make sure there are at least 50 +file descriptors available per `dbengine` instance. + +Netdata allocates 25% of the available file descriptors to its Database Engine instances. This means that only 25% +of the file descriptors that are available to the Netdata service are accessible by dbengine instances. +You should take that into account when configuring your service +or system-wide file descriptor limits. You can roughly estimate that the netdata service needs 2048 file +descriptors for every 10 streaming slave hosts when streaming is configured to use `memory mode = dbengine`. + +If for example one wants to allocate 65536 file descriptors to the netdata service on a systemd system +one needs to edit the netdata systemd file `netdata.service` in `/lib/systemd/system` or `/usr/lib/systemd/system` +and add the line: + +``` +LimitNOFILE=65536 +``` + +in the `[Service]` section. + +For other types of services one can add the line: +``` +ulimit -n 65536 +``` +at the beginning of the service file. Alternatively you can change the system-wide limits of the kernel by changing `/etc/sysctl.conf`. For linux that would be: +``` +fs.file-max = 65536 +``` +In FreeBSD and OS X you change the lines like this: +``` +kern.maxfilesperproc=65536 +kern.maxfiles=65536 +``` +You can apply the settings by running `sysctl -p` or by rebooting. + []() |