From 3a5f9a1e1cec75e395115567acf17ddce2f6de59 Mon Sep 17 00:00:00 2001 From: PradeepKiruvale Date: Wed, 6 Apr 2022 20:27:20 +0530 Subject: [953] tedge watchdog (#1026) * [953] tedge-systemd-watchdog * update service file to watchdog * Closes #953 thin-edge watchdog * health check for all the services * update document --- docs/src/SUMMARY.md | 1 + .../021_enable_tedge_watchdog_using_systemd.md | 59 ++++++++++++++++++++++ docs/src/howto-guides/README.md | 3 +- 3 files changed, 62 insertions(+), 1 deletion(-) create mode 100644 docs/src/howto-guides/021_enable_tedge_watchdog_using_systemd.md (limited to 'docs') diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index 3f504210..4f608c57 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -34,6 +34,7 @@ - [How to use apama software management plugin](./howto-guides/017_apama_software_management_plugin.md) - [How to change temp path](./howto-guides/018_change_temp_path.md) - [How to use thin-edge.io with your preferred init system](./howto-guides/019_how_to_use_preferred_init_system.md) + - [How to enable watchdog using systemd?](./howto-guides/021_enable_tedge_watchdog_using_systemd.md) - [Developer Documentation](dev_doc.md) - [Architecture](architecture/README.md) diff --git a/docs/src/howto-guides/021_enable_tedge_watchdog_using_systemd.md b/docs/src/howto-guides/021_enable_tedge_watchdog_using_systemd.md new file mode 100644 index 00000000..aedaa335 --- /dev/null +++ b/docs/src/howto-guides/021_enable_tedge_watchdog_using_systemd.md @@ -0,0 +1,59 @@ +# Watchdog feature using `systemd` in `thin-edge.io` + +## Introduction + +`Watchdog` feature is used to check the health of a service/process by constantly exchanging +the message between the watchdog process and the service/process that is being monitored. +When the process does not update its health status within a specified `time` period, then the watchdog process will +restart the service that is being monitored. + +This document shows how `thin-edge.io` services are managed using the systemd, then one can use `systemd` +feature to check the health of these services as well. This document provides +information about how to use `systemd` for checking the health of the services. + +## Enabling the `watchdog` feature in `systemd` + +Enabling the `watchdog` feature in systemd for a `thin-edge.io` service (tedge_agent, tedge_mapper_c8y/az/collectd) +using the `systemd` is a two-step process. + +### Step 1: Enable the `watchdog` feature in the `systemd` service file +For example to enable the `watchdog` feature for `tedge-mapper-c8y` service, update systemd service file as shown below. + +Add `tedge_watchdog.service` in `After` under `[Unit]` section. +Add `WatchdogSec=5` under `[Service]` section. + +The sample service file after updating looks as below. + +```shell +[Unit] +Description=tedge-mapper-c8y converts Thin Edge JSON measurements to Cumulocity JSON format. +After=syslog.target network.target mosquitto.service tedge_watchdog.service + +[Service] +User=tedge-mapper +ExecStart=/usr/bin/tedge_mapper c8y +Restart=on-failure +RestartPreventExitStatus=255 +WatchdogSec=5 +``` + +> Note: The systemd service file usually present in `/lib/systemd/system/tedge-mapper-c8y.service`. + +### Step 2: Start the `tedge-watchdog` service + +Start the `watchdog` service as below. +```shell +systemctl start tedge-watchdog.service +``` + +Now the `tedge-watchdog` service will be keep sending health check messages for every `WatchdogSec/2` seconds. +Once the response is received from the particular service, the `watchdog` service will send the notification +to the systemd on behalf of the service. + +## Debugging +One can observe the message exchange between the `service` and the `watchdog` by subscribing to `tedge/health/#` and `tedge/health-check/#` topics. +For more info check [here](./020_monitor_tedge_health) + +> Note: If the watchdog service did not send the notification to the systemd within `WatchdogSec`, then the systemd will kill the existing service process and restarts it. + +> Note: [Here](https://www.medo64.com/2019/01/systemd-watchdog-for-any-service/) is an example about using `systemd watchdog` feature. \ No newline at end of file diff --git a/docs/src/howto-guides/README.md b/docs/src/howto-guides/README.md index bcc3ce42..e8c6b8b4 100644 --- a/docs/src/howto-guides/README.md +++ b/docs/src/howto-guides/README.md @@ -14,4 +14,5 @@ 12. [How to connect an external device?](./013_connect_external_device.md) 13. [How to access the logs on the device?](./014_thin_edge_logs.md) 14. [How to install thin-edge.io on any Linux OS (no deb support)?](./015_installation_without_deb_support.md) -16. [How to manage apama software artefatcs with apama plugin?](./017_apama_software_management_plugin.md) \ No newline at end of file +16. [How to manage apama software artefatcs with apama plugin?](./017_apama_software_management_plugin.md) +17. [How to enable watchdog using systemd?](./021_enable_tedge_watchdog_using_systemd.md) \ No newline at end of file -- cgit v1.2.3