summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorMatthias Beyer <matthias.beyer@ifm.com>2022-07-15 17:36:23 +0200
committerMatthias Beyer <matthias.beyer@ifm.com>2022-08-30 13:52:36 +0200
commit08a32f3294d65b4011e54838fc67c792cf8f3ff0 (patch)
tree8360aa6821b7d1d5efe5b91c4089c9dc149e4a67
parent49aac61a251a95db7bc8e8b3589caa1a99db682d (diff)
parent06d1f29576c52566bd3ea8715c02025832de06a6 (diff)
Merge branch 'feature/add_tedge_api_impl' into integrate-feature-apiintegrate-feature-api
This merge now introduces the "core" library, a CLI implementation, a helper library for simplifying some plugin implementation as well as a number of example plugins. The following message can help you understanding the ideas and concepts that are introduced with this merge as well as help to answer some questions that might arise when reviewing the changeset. In the previous merge we introduced the API. This API does _only_ define an interface how thin-edge.io plugins can be written in a way so they can be used within the thin-edge.io ecosystem. For more information, have a look at the merge commit where the API was introduced. This merge now introduces a list of crates of different importance. I will quickly introduce them here, with only some short notes. More is written in the `README.md` file of the respective crate. * `tedge_core` - The implementation of the part of thin-edge.io, that runs the plugins which implement the `tedge_api` * Ensures composability via configuration with fail-early mechanisms if configuration is faulty * Runs plugins concurrently on multiple cores without spending much time itself * Implements high-performance asynchronous and typed message passing without serialization/deserialization overhead * Ensures plugin lifecycle with panic-catching in all phases of the lifecycle of a plugin * Features fine-grained `tracing` output to be able to make use of the excellent `tracing` ecosystem and enabling detailed performance analytics * Ensures proper plugin shutdown in case of application shutdown * If a plugin does not shutdown after a certain configurable timeout, the plugin will be killed * High testability! The core is tested for several things, for example: * whether a plugin does panic in any phase of it's lifecycle, so that the core does not crash if it happens * whether a plugin does shutdown cleanly * whether application startup is aborted properly if plugins are wired up that are not compatible (`Plugin A` wants to send `Message M` to `Plugin B`, but `Plugin B` only supports `Message X`) * `tedge-cli` - one possible implementation of the CLI for starting a thin-edge.io process. This implementation is not really big, which is definitively intended * Implements easy to use CLI interface * Setup of stdout/chrome/Tracy tracing output * Ties together the core and the plugins * `tedge_lib` - A helper library for helping plugin authors with some recurring aspects/patterns in a plugin implementation * Mainloop abstraction helper for "ticking" mainloop * Mainloop abstraction for stateful `loop{}`-style mainloops * Defines common message types, e.g.: * `Measurement` (just a mockup for now, details have to discussed later) * `Notification` (just a mockup for now, details have to discussed later) * Defines config helper types for more typing in the configuration * Defines address helper types for more convenience over the basics of message passing * A list of example plugin implementations of different complexity * `plugin_avg` - A plugin that calculates an average value from a stream of measurements sent to it. This is a good example plugin for learning how a plugin implementation could look like * `plugin_log` - A plugin that logs all messages sent to it. This is intentionally minimal and does not serve a real purpose * `plugin_httpstop` - A plugin that tells thin-edge.io to stop if a web request was made to a certain endpoint. In short: `curl localhost:8080` to shutdown thin-edge.io. This would also be a good example for reviewing how a basic plugin implementation would look like * `plugin_inotify` - A plugin that watches some files with inotify and sends notification messages if inotify raises an event. This is a good example of a bit more involved plugin * `plugin_measurement_filter` - A plugin to filter `Measurement` messages by a user-defined predicate. Think "Ignore all measurements below value `12`" * `plugin_mqtt` - A way for connecting thin-edge.io with an MQTT broker. This plugin does not parse MQTT payloads, which is done by another plugin, depending on the usecase that one wants to cover. Instead it only contains the MQTT handling, so that other plugins do not have to care about the transport mechanisms their payloads are sent/received with * `plugin_mqtt_measurement_bridge` - A plugin that receives `Measurement` objects, serializes them to JSON and sends them to an MQTT Plugin. This is another building block in an MQTT setup and makes the serialization method decoupled from the business logic that emits `Measurement` objects. * `plugin_sysstat` - A plugin that uses the `sysinfo` crate to get system statistics and sends them as `Measurement` messages. This is a really involved plugin (code-size wise) and might require some more effort to understand. We did not yet introduce plugins for things like collectd, systemd, apt, and so on in this work, as these are clearly _usecases_ and we want to keep the plugin implementations we provide in this PR in the "this is an example" space. The work merged with this commit does not claim to provide the end-result of anything! This is code, it can and most certainly will be changed after this is merged to the `main` branch. The most stable part of this PR is the `tedge_core` crate. The plugins are only examples! There are also some types defined for representing a `Measurement` for example. This is not final by any means and its purpose is to be able to implement example plugins! Of course we have to iterate on how a Measurement (e.g.) should and can look like! The existing types for representing a `Measurement` were not used in this PR because of development-workflow reasons. All changes that are merged with this commit are made under the DCO and _not_ under any CLA. DCO: Developer Certificate of Origin Version 1.1 Copyright (C) 2004, 2006 The Linux Foundation and its contributors. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Developer's Certificate of Origin 1.1 By making a contribution to this project, I certify that: (a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or (b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or (c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it. (d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. Signed-off-by: Marcel Müller <m.mueller@ifm.com> Signed-off-by: Matthias Beyer <matthias.beyer@ifm.com>
-rw-r--r--Cargo.lock668
-rw-r--r--Cargo.toml13
-rw-r--r--crates/core/tedge_api/Cargo.toml4
-rw-r--r--crates/core/tedge_api/README.md2
-rw-r--r--crates/core/tedge_api/examples/heartbeat.rs6
-rw-r--r--crates/core/tedge_api/examples/print_config.rs150
-rw-r--r--crates/core/tedge_api/src/address.rs6
-rw-r--r--crates/core/tedge_api/src/config.rs196
-rw-r--r--crates/core/tedge_api/src/error.rs3
-rw-r--r--crates/core/tedge_api/src/lib.rs4
-rw-r--r--crates/core/tedge_api/src/message.rs4
-rw-r--r--crates/core/tedge_api/src/plugin.rs20
-rw-r--r--crates/core/tedge_api/src/util.rs36
-rw-r--r--crates/core/tedge_api/tedge_config_derive/src/lib.rs4
-rw-r--r--crates/core/tedge_core/Cargo.toml39
-rw-r--r--crates/core/tedge_core/README.md65
-rw-r--r--crates/core/tedge_core/benches/simple_throughput.rs272
-rw-r--r--crates/core/tedge_core/benches/simple_throughput.toml14
-rw-r--r--crates/core/tedge_core/src/communication.rs410
-rw-r--r--crates/core/tedge_core/src/configuration.rs175
-rw-r--r--crates/core/tedge_core/src/core_task.rs38
-rw-r--r--crates/core/tedge_core/src/errors.rs249
-rw-r--r--crates/core/tedge_core/src/lib.rs266
-rw-r--r--crates/core/tedge_core/src/message_handler.rs99
-rw-r--r--crates/core/tedge_core/src/plugin_task.rs242
-rw-r--r--crates/core/tedge_core/src/reactor.rs391
-rw-r--r--crates/core/tedge_core/src/utils.rs18
-rw-r--r--crates/core/tedge_core/tests/check_concurrent_messages.rs211
-rw-r--r--crates/core/tedge_core/tests/check_concurrent_messages.toml10
-rw-r--r--crates/core/tedge_core/tests/plugin_does_not_shutdown.rs144
-rw-r--r--crates/core/tedge_core/tests/plugin_does_not_shutdown.toml8
-rw-r--r--crates/core/tedge_core/tests/plugin_does_not_support_message.rs196
-rw-r--r--crates/core/tedge_core/tests/plugin_does_not_support_message.toml12
-rw-r--r--crates/core/tedge_core/tests/plugin_panic_lifecycle.rs178
-rw-r--r--crates/core/tedge_core/tests/plugin_panic_lifecycle.toml15
-rw-r--r--crates/core/tedge_core/tests/plugin_panics_in_msg_handler.rs170
-rw-r--r--crates/core/tedge_core/tests/plugin_panics_in_msg_handler.toml9
-rw-r--r--crates/core/tedge_core/tests/plugin_send_to_self_name.rs180
-rw-r--r--crates/core/tedge_core/tests/plugin_send_to_self_name.toml6
-rw-r--r--crates/core/tedge_core/tests/verify_configuration_fails.rs107
-rw-r--r--crates/core/tedge_core/tests/verify_configuration_fails.toml9
-rw-r--r--crates/core/tedge_lib/Cargo.toml30
-rw-r--r--crates/core/tedge_lib/src/address.rs108
-rw-r--r--crates/core/tedge_lib/src/config/address.rs64
-rw-r--r--crates/core/tedge_lib/src/config/humantime.rs56
-rw-r--r--crates/core/tedge_lib/src/config/mod.rs59
-rw-r--r--crates/core/tedge_lib/src/config/one_or_many.rs81
-rw-r--r--crates/core/tedge_lib/src/config/port.rs24
-rw-r--r--crates/core/tedge_lib/src/config/socket_addr.rs40
-rw-r--r--crates/core/tedge_lib/src/iter/mod.rs5
-rw-r--r--crates/core/tedge_lib/src/iter/result.rs120
-rw-r--r--crates/core/tedge_lib/src/iter/send_all.rs121
-rw-r--r--crates/core/tedge_lib/src/lib.rs7
-rw-r--r--crates/core/tedge_lib/src/mainloop/detach.rs19
-rw-r--r--crates/core/tedge_lib/src/mainloop/mod.rs45
-rw-r--r--crates/core/tedge_lib/src/mainloop/stopper.rs13
-rw-r--r--crates/core/tedge_lib/src/mainloop/ticking.rs74
-rw-r--r--crates/core/tedge_lib/src/measurement.rs38
-rw-r--r--crates/core/tedge_lib/src/notification.rs24
-rw-r--r--crates/core/tedge_lib/src/pubsub.rs86
-rw-r--r--crates/core/tedge_lib/tests/test_pubsub.rs300
-rw-r--r--crates/core/tedge_lib/tests/test_pubsub.toml15
-rw-r--r--crates/core/tedge_lib/tests/test_send_group.rs278
-rw-r--r--crates/core/tedge_lib/tests/test_send_group.toml21
-rw-r--r--plugins/plugin_avg/Cargo.toml20
-rw-r--r--plugins/plugin_avg/README.md34
-rw-r--r--plugins/plugin_avg/src/lib.rs212
-rw-r--r--plugins/plugin_httpstop/Cargo.toml20
-rw-r--r--plugins/plugin_httpstop/README.md15
-rw-r--r--plugins/plugin_httpstop/src/lib.rs146
-rw-r--r--plugins/plugin_inotify/Cargo.toml23
-rw-r--r--plugins/plugin_inotify/README.md197
-rw-r--r--plugins/plugin_inotify/src/config.rs169
-rw-r--r--plugins/plugin_inotify/src/lib.rs228
-rw-r--r--plugins/plugin_log/Cargo.toml19
-rw-r--r--plugins/plugin_log/README.md24
-rw-r--r--plugins/plugin_log/src/lib.rs207
-rw-r--r--plugins/plugin_measurement_filter/Cargo.toml24
-rw-r--r--plugins/plugin_measurement_filter/README.md55
-rw-r--r--plugins/plugin_measurement_filter/src/builder.rs74
-rw-r--r--plugins/plugin_measurement_filter/src/config.rs43
-rw-r--r--plugins/plugin_measurement_filter/src/error.rs5
-rw-r--r--plugins/plugin_measurement_filter/src/extractor.rs181
-rw-r--r--plugins/plugin_measurement_filter/src/filter.rs231
-rw-r--r--plugins/plugin_measurement_filter/src/lib.rs9
-rw-r--r--plugins/plugin_measurement_filter/src/plugin.rs66
-rw-r--r--plugins/plugin_mqtt/Cargo.toml25
-rw-r--r--plugins/plugin_mqtt/src/builder.rs67
-rw-r--r--plugins/plugin_mqtt/src/config.rs90
-rw-r--r--plugins/plugin_mqtt/src/error.rs14
-rw-r--r--plugins/plugin_mqtt/src/lib.rs10
-rw-r--r--plugins/plugin_mqtt/src/message.rs64
-rw-r--r--plugins/plugin_mqtt/src/plugin.rs239
-rw-r--r--plugins/plugin_mqtt_measurement_bridge/Cargo.toml24
-rw-r--r--plugins/plugin_mqtt_measurement_bridge/README.md5
-rw-r--r--plugins/plugin_mqtt_measurement_bridge/src/lib.rs142
-rw-r--r--