summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorHugo Landau <hlandau@openssl.org>2024-02-05 17:48:59 +0000
committerHugo Landau <hlandau@openssl.org>2024-04-19 09:31:06 +0100
commit3f592185a07b0bca4c1e871bd9d4f90c76bc8e8c (patch)
treebed01618b7c34166c8dd7b5323cf92737111ee67
parenta72832cd604907266f8c036c8685b893d6ec248e (diff)
QUIC RADIX: Add README
Reviewed-by: Neil Horman <nhorman@openssl.org> Reviewed-by: Tomas Mraz <tomas@openssl.org> (Merged from https://github.com/openssl/openssl/pull/23487)
-rw-r--r--test/radix/README.md107
1 files changed, 107 insertions, 0 deletions
diff --git a/test/radix/README.md b/test/radix/README.md
new file mode 100644
index 0000000000..1352a413b9
--- /dev/null
+++ b/test/radix/README.md
@@ -0,0 +1,107 @@
+RADIX Test Framework
+====================
+
+Purpose
+-------
+
+This directory contains the RADIX test framework, which is a six-dimension
+script-driven facility intended to facilitate execution of
+
+- multi-stream
+- multi-client
+- multi-server
+- multi-thread
+- multi-process (in future)
+- multi-node (in future)
+
+test vignettes for network protocol testing applications. While it is currently
+used for QUIC, it has been designed to be agnostic so that it can be adapted to
+other protocols in future if desired.
+
+In particular, unilke the older multistream test framework, it does not assume a
+single client and a single server. Examples of vignettes designed to be
+supported by the RADIX test framework in future include:
+
+- single client ↔ single server
+- multiple clients ↔ single server
+- single client ↔ multiple servers
+- multiple clients ↔ multiple servers
+
+“Multi-process” and “multi-node” means there has been some consideration
+given to support of multi-process and multi-node testing in the future, though
+this is not currently supported.
+
+Differences to `quic_multistream_test`
+--------------------------------------
+
+The RADIX test features the following improvements relative to the
+`quic_multistream_test` framework:
+
+- Due to usage of the new QUIC server API, opcodes are no longer duplicated
+ into “C” and “S” variants. There is symmetry between all endpoints in this
+ regard. The legacy `QUIC_TSERVER` facility is not used and will eventually
+ be retired once other test suites no longer rely on it.
+
+- The framework is not limited to two objects (client and server),
+ but can instead instantiate an arbitrary number of SSL objects and pass these
+ to operations. Other kinds of object could be supported in future if needed.
+
+- Scripts are now generated dynamically at launch time by functions rather
+ than hardcoding them into the executable. This has the advantage that scripts
+ can be generated dynamically. In particular, this allows very long
+ procedurally generated test scripts which would be impractical to write
+ by hand. This can be used for stress testing, for example.
+
+- As a result of the fact that scripts are now generated dynamically, the
+ in-memory representation of script operations has been improved as the ability
+ to write a script operation as an initializer list is no longer required. The
+ new representation is simpler, more compact, and more flexible. A stack-based
+ approach allows arbitrary argument types to be passed as operands to an
+ operation, rather than having a fixed number of operands of fixed type for all
+ script operations.
+
+- Scripts are now named, rather than numbered, giving more useful debug output,
+ and hopefully reducing the frequency of merge conflicts.
+
+- Debug logging has in general been significantly improved and has been designed
+ to be accessible and useful.
+
+- Logging of child threads is now buffered and printed after a test is complete,
+ which avoids non-deterministic interleaving of test output.
+
+- The number of core opcodes for the interpreter has been dramatically reduced
+ and now the vast majority of test operations are performed via `OP_FUNC`,
+ which is similar to `OP_CHECK`. This change is largely transparent to the
+ test developer.
+
+- In the future, multi-process or multi-node testing will be supported.
+ The expectation is that multi-node testing will be facilitated by having the
+ master process invoke a hook shell script which is responsible for propping up
+ and tearing down the additional nodes. Writing a suitable hook script will be
+ left as an exercise to the test infrastructure maintainer. This abstracts the
+ test framework from the details of a specific infrastructure environment and
+ its management tools (VMs, containers, etc.).
+
+- The core test interpreter is designed to be agnostic to QUIC and could be
+ used for testing other protocols in the future. There is a clean, layered
+ design which draws a clear distinction between the core interpreter,
+ protocol-specific bindings and support code, test operation definitions, and
+ script definitions.
+
+Architecture
+------------
+
+The RADIX test suite framework is built in four layers:
+
+- **TERP** ([terp.c](./terp.c)), a protocol-agnostic stack-based script
+ interpreter for interruptible execution of test vignettes;
+
+- the **QUIC bindings** ([quic_bindings.c](./quic_bindings.c)), which defines
+ QUIC-specific test framework;
+
+- the **QUIC operations** ([quic_ops.c](./quic_ops.c)), which define specific
+ test operations for TERP which can be invoked by QUIC unit tests on top of the
+ basic infrastructure defined by the QUIC bindings;
+
+- the QUIC unit tests ([quic_tests.c](./quic_tests.c)), which use the above
+ QUIC bindings.