From 2249b8bbb9f4233d1f4a2f8f3987ad9c8ee968fd Mon Sep 17 00:00:00 2001 From: Didier Wenzek Date: Thu, 16 Jun 2022 19:31:40 +0100 Subject: Document the decision to refactor thin-edge using rust components Signed-off-by: Didier Wenzek --- design/decisions/0001.md | 97 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 97 insertions(+) create mode 100644 design/decisions/0001.md (limited to 'design') diff --git a/design/decisions/0001.md b/design/decisions/0001.md new file mode 100644 index 00000000..3fccdef6 --- /dev/null +++ b/design/decisions/0001.md @@ -0,0 +1,97 @@ +# Re-organization of the rust code base using components + +* Date: __2022-06-13__ +* Status: __Proposed__ + +## Motivations + +The version 0.7 of thin-edge makes more difficult than expected one of its main goal: +that is to "give developers tools to build IIoT agents from software components provided by independent vendors". + +To reach this goal, thin-edge has been designed around the idea of +software components implemented as daemon processes that interact over MQTT using JSON messages. +Two main flaws of the current implementation impair the expected freedom of extension by independent contributors : + +1. Some APIs have not been exposed over MQTT but only through the file system and via command line tools + - preventing components that run in a container or on a child device to leverage all the features of thin-edge. +2. The Rust code-base has been produced by and for a team of contributors working closely, + - making difficult for independent teams to contribute with loose communication. + +The first point is critical but will be addressed independently. +The focus is set here on the second point. + +* The aim is to __organize the rust code-base so independent contributors can implement in rust their own features with minimal conflicts__. +* Contributors who add a new feature, operation or mapper should not be forced as today + * to either work closely with the core team to update the `tedge` command and associated daemons, + * or to implement their features as independent daemons interacting with thin-edge over MQTT, + gaining coordination flexibility but as the price of fewer opportunities to reduce resource consumption. +* Agent developers who package device software on top of thin-edge should not be forced as today + * to incorporate all the features added by independent contributors in the `tedge` command and associated daemons, + * *i.e.* to include all the cloud connectors, features and protocol extensions when only some of them are actually used. + +## Proposal + +The code of thin-edge is organized around : +* external MQTT components that are daemon processes interacting over MQTT, +* internal Rust components that are rust crates interacting over in-memory channels, +* tools to build an external MQTT component from a cherry-picked subset of internal Rust components, +* batteries-included thin-edge daemons, built from stable internal Rust components. + +For that to work, thin-edge defines: +* a Rust API that rules the interaction between internal Rust components, +* an MQTT API that rules the interaction between external MQTT components, +* conversion rules between Rust in-memory messages and MQTT messages. + +The crate hierarchy of the project reflects this organization: + +``` +crates/ +├─ core/ +│ ├─ # The foundations +│ ├─ tedge_api +│ ├─ tedge_mqtt_api +│ ├─ tedge_runtime +│ +├─ rust_plugins/ +│ ├─ # The building blocks +│ ├─ apama_plugin +│ ├─ apt_plugin +│ ├─ c8y_conf_plugin +│ ├─ c8y_log_plugin +│ ├─ c8y_plugin +│ ├─ collectd_plugin +│ ├─ mqtt_plugin +│ ├─ sm_plugin +│ ├─ ... +│ ├─ # Added by independent contributors +│ +├─ rust_based_mqtt_components/ +│ ├─ # Assemblage of Rust plugins into MQTT components +│ ├─ c8y_conf_manager +│ ├─ c8y_log_manager +│ ├─ tedge_agent +│ ├─ tedge_mapper +│ ├─ ... +│ ├─ # Added by independent contributors +│ +mqtt_components/ +│ ├─ # Non Rust MQTT components +│ ├─ ... +│ ├─ # Added by independent contributors +``` + +## Consequences + +Benefits: +* Ease collaboration. A feature implemented independently of the core team can be integrated in a thin-edge daemon. +* Ease optimisation. Cherry-picked component assemblages give opportunities to share resources and to remove unnecessary overhead. +* Ease packaging. A rust component can be packaged in various ways and in combination with various components. +* Ease security. A cherry-picked assemblage of components reduce the attack surface. +* Ease testing. An internal Rust component can be tested in isolation. +* Ease QA. A cherry-picked assemblage of components is not impacted by updates of independent vendors' components. +* Ease project governance. Only the core needs strong consensus. The rust components can follow a lighter integration process. + +Drawbacks: +* This implies a major refactoring of the `0.7` code base. +* There are risks, notably because the Rust API is still under design. +* This makes things a bit more complicated to explain with two kinds of plugins / components, internal and external. \ No newline at end of file -- cgit v1.2.3