diff options
author | Hugo Landau <hlandau@openssl.org> | 2024-02-05 17:48:59 +0000 |
---|---|---|
committer | Hugo Landau <hlandau@openssl.org> | 2024-04-19 09:31:06 +0100 |
commit | 3f592185a07b0bca4c1e871bd9d4f90c76bc8e8c (patch) | |
tree | bed01618b7c34166c8dd7b5323cf92737111ee67 | |
parent | a72832cd604907266f8c036c8685b893d6ec248e (diff) |
QUIC RADIX: Add README
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/23487)
-rw-r--r-- | test/radix/README.md | 107 |
1 files changed, 107 insertions, 0 deletions
diff --git a/test/radix/README.md b/test/radix/README.md new file mode 100644 index 0000000000..1352a413b9 --- /dev/null +++ b/test/radix/README.md @@ -0,0 +1,107 @@ +RADIX Test Framework +==================== + +Purpose +------- + +This directory contains the RADIX test framework, which is a six-dimension +script-driven facility intended to facilitate execution of + +- multi-stream +- multi-client +- multi-server +- multi-thread +- multi-process (in future) +- multi-node (in future) + +test vignettes for network protocol testing applications. While it is currently +used for QUIC, it has been designed to be agnostic so that it can be adapted to +other protocols in future if desired. + +In particular, unilke the older multistream test framework, it does not assume a +single client and a single server. Examples of vignettes designed to be +supported by the RADIX test framework in future include: + +- single client ↔ single server +- multiple clients ↔ single server +- single client ↔ multiple servers +- multiple clients ↔ multiple servers + +“Multi-process” and “multi-node” means there has been some consideration +given to support of multi-process and multi-node testing in the future, though +this is not currently supported. + +Differences to `quic_multistream_test` +-------------------------------------- + +The RADIX test features the following improvements relative to the +`quic_multistream_test` framework: + +- Due to usage of the new QUIC server API, opcodes are no longer duplicated + into “C” and “S” variants. There is symmetry between all endpoints in this + regard. The legacy `QUIC_TSERVER` facility is not used and will eventually + be retired once other test suites no longer rely on it. + +- The framework is not limited to two objects (client and server), + but can instead instantiate an arbitrary number of SSL objects and pass these + to operations. Other kinds of object could be supported in future if needed. + +- Scripts are now generated dynamically at launch time by functions rather + than hardcoding them into the executable. This has the advantage that scripts + can be generated dynamically. In particular, this allows very long + procedurally generated test scripts which would be impractical to write + by hand. This can be used for stress testing, for example. + +- As a result of the fact that scripts are now generated dynamically, the + in-memory representation of script operations has been improved as the ability + to write a script operation as an initializer list is no longer required. The + new representation is simpler, more compact, and more flexible. A stack-based + approach allows arbitrary argument types to be passed as operands to an + operation, rather than having a fixed number of operands of fixed type for all + script operations. + +- Scripts are now named, rather than numbered, giving more useful debug output, + and hopefully reducing the frequency of merge conflicts. + +- Debug logging has in general been significantly improved and has been designed + to be accessible and useful. + +- Logging of child threads is now buffered and printed after a test is complete, + which avoids non-deterministic interleaving of test output. + +- The number of core opcodes for the interpreter has been dramatically reduced + and now the vast majority of test operations are performed via `OP_FUNC`, + which is similar to `OP_CHECK`. This change is largely transparent to the + test developer. + +- In the future, multi-process or multi-node testing will be supported. + The expectation is that multi-node testing will be facilitated by having the + master process invoke a hook shell script which is responsible for propping up + and tearing down the additional nodes. Writing a suitable hook script will be + left as an exercise to the test infrastructure maintainer. This abstracts the + test framework from the details of a specific infrastructure environment and + its management tools (VMs, containers, etc.). + +- The core test interpreter is designed to be agnostic to QUIC and could be + used for testing other protocols in the future. There is a clean, layered + design which draws a clear distinction between the core interpreter, + protocol-specific bindings and support code, test operation definitions, and + script definitions. + +Architecture +------------ + +The RADIX test suite framework is built in four layers: + +- **TERP** ([terp.c](./terp.c)), a protocol-agnostic stack-based script + interpreter for interruptible execution of test vignettes; + +- the **QUIC bindings** ([quic_bindings.c](./quic_bindings.c)), which defines + QUIC-specific test framework; + +- the **QUIC operations** ([quic_ops.c](./quic_ops.c)), which define specific + test operations for TERP which can be invoked by QUIC unit tests on top of the + basic infrastructure defined by the QUIC bindings; + +- the QUIC unit tests ([quic_tests.c](./quic_tests.c)), which use the above + QUIC bindings. |