QUIC DDD: Final report

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21715)

2 files changed, 343 insertions, 0 deletions

diff --git a/doc/designs/ddd/README.md b/doc/designs/ddd/README.md
index 99d3832bfc..3f430de24a 100644
--- a/doc/designs/ddd/README.md
+++ b/doc/designs/ddd/README.md
@@ -50,6 +50,9 @@ certificates or other TLS functionality, the use of QUIC is unlikely to have
 implications for these APIs and demos demonstrating such functionality are
 therefore out of scope.
 
+[A report is available](REPORT.md) on the results of the DDD process following
+the completion of the development of the QUIC MVP.
+
 Background
 ----------
 
diff --git a/doc/designs/ddd/REPORT.md b/doc/designs/ddd/REPORT.md
new file mode 100644
index 0000000000..ce742507ce
--- /dev/null
+++ b/doc/designs/ddd/REPORT.md
@@ -0,0 +1,340 @@
+Report on the Conclusions of the QUIC DDD Process
+=================================================
+
+The [QUIC Demo-Driven Design process](README.md) was undertaken to meet the OMC
+requirement to develop a QUIC API that required only minimal changes to existing
+applications to be able to adapt their code to use QUIC. The demo-driven design
+process developed a set of representative demos modelling a variety of common
+OpenSSL usage patterns based on analysis of a broad spectrum of open source
+software projects using OpenSSL.
+
+As part of this process, a set of proposed diffs were produced. These proposed
+diffs were the expected changes which would be needed to the baseline demos to
+support QUIC based on theoretical analysis of the minimum requirements to be
+able to support QUIC. This analysis concluded that the changes needed to
+applications could be kept very small in many circumstances, with only minimal
+diff sizes to the baseline demos.
+
+Following the development of QUIC MVP, these demos have been revisited and the
+correspondence of our actual final API and usage patterns with the planned diffs
+have been reviewed.
+
+This document discusses the planned changes and the actual changes for each demo
+and draws conclusions on the level of disparity.
+
+Since tracking a set of diffs separately is unwieldy, both the planned and
+unplanned changes have been folded into the original baseline demo files guarded
+with `#ifdef USE_QUIC`. Viewing these files therefore is informative to
+application writers as it provides a clear view of what is different when using
+QUIC. (The originally planned changes, and the final changes, are added in
+separate, clearly-labelled commits; to view the originally planned changes only,
+view the commit history for a given demo file.)
+
+ddd-01-conn-blocking
+--------------------
+
+This demo exists to demonstrate the simplest possible usage of OpenSSL, whether
+with TLS or QUIC.
+
+### Originally planned changes
+
+The originally planned change to enable applications for QUIC amounted to just a
+single line:
+
+```diff
++    ctx = SSL_CTX_new(QUIC_client_method());
+-    ctx = SSL_CTX_new(TLS_client_method());
+```
+
+### Actual changes
+
+The following additional changes needed to be made:
+
+- `QUIC_client_method` was renamed to `OSSL_QUIC_client_method` for namespacing
+  reasons.
+
+- A call to `SSL_set_alpn_protos` to configure ALPN was added. This is necessary
+  because QUIC mandates the use of ALPN, and this was not noted during the
+  DDD process.
+
+ddd-02-conn-nonblocking
+-----------------------
+
+This demo exists to demonstrate simple non-blocking usage. As with
+ddd-01-conn-blocking, the name resolution process is managed by `BIO_s_connect`.
+
+It also arbitrarily adds a `BIO_f_buffer` pushed onto the BIO stack
+as this is a common application usage pattern.
+
+### Originally planned changes
+
+The originally planned changes to enable applications for QUIC amounted to:
+
+- Change of method (as for ddd-01-conn-blocking);
+
+- Use of a `BIO_f_dgram_buffer` BIO method instead of a `BIO_f_buffer`;
+
+- Use of a `BIO_get_poll_fd` function to get the FD to poll rather than
+  `BIO_get_fd`;
+
+- A change to how the `POLLIN`/`POLLOUT`/`POLLERR` flags to pass to poll(2)
+  need to be determined.
+
+- Additional functions in application code to determine event handling
+  timeouts related to QUIC (`get_conn_pump_timeout`) and to pump
+  the QUIC event loop (`pump`).
+
+- Timeout computation code which involves merging and comparing different
+  timeouts and calling `pump` as needed, based on deadlines reported
+  by libssl.
+
+Note that some of these changes are unnecessary when using the thread assisted
+mode (see the variant ddd-02-conn-nonblocking-threads below).
+
+### Actual changes
+
+The following additional changes needed to be made:
+
+- Change of method name (as for ddd-01-conn-blocking);
+
+- Use of ALPN (as for ddd-01-conn-blocking);
+
+- The strategy for how to expose pollable OS resource handles
+  to applications to determine I/O readiness has changed substantially since the
+  original DDD process. As such, applications now use `BIO_get_rpoll_descriptor`
+  and `BIO_get_wpoll_descriptor` to determine I/O readiness, rather than the
+  originally hypothesised `SSL_get_poll_fd`.
+
+- The strategy for how to determine when to poll for `POLLIN`, when to
+  poll for `POLLOUT`, etc. has changed since the original DDD process.
+  This information is now exposed via `SSL_net_read_desired` and
+  `SSL_net_write_desired`.
+
+- The API to expose the event handling deadline for the QUIC engine
+  has evolved since the original DDD process. The new API
+  `SSL_get_event_timeout` is used, rather than the originally hypothesised
+  `BIO_get_timeout`/`SSL_get_timeout`.
+
+- The API to perform QUIC event processing has been renamed to be
+  more descriptive. It is now called `SSL_handle_events` rather than
+  the originally hypothesised `BIO_pump`/`SSL_pump`.
+
+The following changes were foreseen to be necessary, but turned out to actually
+not be necessary:
+
+- The need to change code which pushes a `BIO_f_buffer()` after a SSL BIO
+  was foreseen as use of buffering on the network side is unworkable with
+  QUIC. This turned out not to be necessary since we can just reject the
+  BIO_push() call. The buffer should still be freed eventually when the
+  SSL BIO is freed. The buffer is not used and is unnecessary, so it is
+  still desirable for applications to remove this code.
+
+ddd-02-conn-nonblocking-threads
+-------------------------------
+
+This is a variant of the ddd-02-conn-nonblocking demo. The base is the same, but
+the changes made are different. The use of thread-assisted mode, in which an
+internal assist thread is used to perform QUIC event handling, enables an
+application to make fewer changes than are needed in the ddd-02-conn-nonblocking
+demo.
+
+### Originally planned changes
+
+The originally planned changes to enable applications for QUIC amounted to:
+
+- Change of method, this time using method `QUIC_client_thread_method` rather
+  than `QUIC_client_method`;
+
+- Use of a `BIO_get_poll_fd` function to get the FD to poll rather than
+  `BIO_get_fd`;
+
+- A change to how the `POLLIN`/`POLLOUT`/`POLLERR` flags to pass to poll(2)
+  need to be determined.
+
+  Note that this is a subtantially smaller list of changes than for
+  ddd-02-conn-nonblocking.
+
+### Actual changes
+
+The following additional changes needed to be made:
+
+- Change of method name (`QUIC_client_thread_method` was renamed to
+  `OSSL_QUIC_client_thread_method` for namespacing reasons);
+
+- Use of ALPN (as for ddd-01-conn-blocking);
+
+- Use of `BIO_get_rpoll_descriptor` rather than `BIO_get_poll_fd` (as for
+  ddd-02-conn-nonblocking).
+
+- Use of `SSL_net_read_desired` and `SSL_net_write_desired` (as for
+  ddd-02-conn-nonblocking).
+
+ddd-03-fd-blocking
+------------------
+
+This demo is similar to ddd-01-conn-blocking but uses a file descriptor passed
+directly by the application rather than BIO_s_connect.
+
+### Originally planned changes
+
+- Change of method (as for ddd-01-conn-blocking);
+
+- The arguments to the `socket(2)` call are changed from `(AF_INET, SOCK_STREAM,
+  IPPROTO_TCP)` to `(AF_INET, SOCK_DGRAM, IPPROTO_UDP)`.
+
+### Actual changes
+
+The following additional changes needed to be made:
+
+- Change of method name (as for ddd-01-conn-blocking);
+
+- Use of ALPN (as for ddd-01-conn-blocking).
+
+ddd-04-fd-nonblocking
+---------------------
+
+This demo is similar to ddd-01-conn-nonblocking but uses a file descriptor
+passed directly by the application rather than BIO_s_connect.
+
+### Originally planned changes
+
+- Change of method (as for ddd-01-conn-blocking);
+
+- The arguments to the `socket(2)` call are changed from `(AF_INET, SOCK_STREAM,
+  IPPROTO_TCP)` to `(AF_INET, SOCK_DGRAM, IPPROTO_UDP)`;
+
+- A change to how the `POLLIN`/`POLLOUT`/`POLLERR` flags to pass to poll(2)
+  need to be determined.
+
+- Additional functions in application code to determine event handling
+  timeouts related to QUIC (`get_conn_pump_timeout`) and to pump
+  the QUIC event loop (`pump`).
+
+- Timeout computation code which involves merging and comparing different
+  timeouts and calling `pump` as needed, based on deadlines reported
+  by libssl.
+
+### Actual changes
+
+The following additional changes needed to be made:
+
+- Change of method name (as for ddd-01-conn-blocking);
+
+- Use of ALPN (as for ddd-01-conn-blocking);
+
+- `SSL_get_timeout` replaced with `SSL_get_event_timeout` (as for
+  ddd-02-conn-nonblocking);
+
+- `SSL_pump` renamed to `SSL_handle_events` (as for ddd-02-conn-nonblocking);
+
+- The strategy for how to determine when to poll for `POLLIN`, when to
+  poll for `POLLOUT`, etc. has changed since the original DDD process.
+  This information is now exposed via `SSL_net_read_desired` and
+  `SSL_net_write_desired` (as for ddd-02-conn-nonblocking).
+
+ddd-05-mem-nonblocking
+----------------------
+
+This demo is more elaborate. It uses memory buffers created and managed by an
+application as an intermediary between libssl and the network, which is a common
+usage pattern for applications. Managing this pattern for QUIC is more elaborate
+since datagram semantics on the network channel need to be maintained.
+
+### Originally planned changes
+
+- Change of method (as for ddd-01-conn-blocking);
+
+- Call to `BIO_new_bio_pair` is changed to `BIO_new_dgram_pair`, which
+  provides a bidirectional memory buffer BIO with datagram semantics.
+
+- A change to how the `POLLIN`/`POLLOUT`/`POLLERR` flags to pass to poll(2)
+  need to be determined.
+
+- Potential changes to buffer sizes used by applications to buffer
+  datagrams, if those buffers are smaller than 1472 bytes.
+
+- The arguments to the `socket(2)` call are changed from `(AF_INET, SOCK_STREAM,
+  IPPROTO_TCP)` to `(AF_INET, SOCK_DGRAM, IPPROTO_UDP)`;
+
+### Actual changes
+
+The following additional changes needed to be made:
+
+- Change of method name (as for ddd-01-conn-blocking);
+
+- Use of ALPN (as for ddd-01-conn-blocking);
+
+- The API to construct a `BIO_s_dgram_pair` ended up being named
+  `BIO_new_bio_dgram_pair` rather than `BIO_new_dgram_pair`;
+
+- Use of `SSL_net_read_desired` and `SSL_net_write_desired` (as for
+  ddd-02-conn-nonblocking).
+
+ddd-06-mem-uv
+-------------
+
+This demo is the most elaborate of the set. It uses a real-world asynchronous
+I/O reactor, namely libuv (the engine used by Node.js). In doing so it seeks to
+demonstrate and prove the viability of our API design with a real-world
+asynchronous I/O system. It operates wholly in non-blocking mode and uses memory
+buffers on either side of the QUIC stack to feed data to and from the
+application and the network.
+
+### Originally planned changes
+
+- Change of method (as for ddd-01-conn-blocking);
+
+- Various changes to use of libuv needed to switch to using UDP;
+
+- Additional use of libuv to configure a timer event;
+
+- Call to `BIO_new_bio_pair` is changed to `BIO_new_dgram_pair`
+  (as for ddd-05-mem-nonblocking);
+
+- Some reordering of code required by the design of libuv.
+
+### Actual changes
+
+The following additional changes needed to be made:
+
+- Change of method name (as for ddd-01-conn-blocking);
+
+- Use of ALPN (as for ddd-01-conn-blocking);
+
+- `BIO_new_dgram_pair` renamed to `BIO_new_bio_dgram_pair` (as for
+  ddd-05-mem-nonblocking);
+
+- `SSL_get_timeout` replaced with `SSL_get_event_timeout` (as for
+  ddd-02-conn-nonblocking);
+
+- `SSL_pump` renamed to `SSL_handle_events` (as for ddd-02-conn-nonblocking);
+
+- Fixes to use of libuv based on a corrected understanding
+  of its operation, and changes that necessarily ensue.
+
+Conclusions
+-----------
+
+The DDD process has successfully delivered on the objective of delivering a QUIC
+API which can be used with only minimal API changes. The additional changes on
+top of those originally planned which were required to successfully execute the
+demos using QUIC were highly limited in scope and mostly constituted only minor
+changes. The sum total of the changes required for each demo (both planned and
+additional), as denoted in each DDD demo file under `#ifdef USE_QUIC` guards,
+are both minimal and limited in scope.
+
+“Minimal” and “limited” are distinct criteria. If inexorable technical
+requirements dictate, an enormous set of changes to an application could be
+considered “minimal”. The changes required to representative applications, as
+demonstrated by the DDD demos, are not merely minimal but also limited.
+
+For example, while the extent of these necessary changes varies by the
+sophistication of each demo and the kind of application usage pattern it
+represents, some demos in particular demonstrate exceptionally small changesets;
+for example, ddd-01-conn-blocking and ddd-02-conn-nonblocking-threads, with
+ddd-01-conn-blocking literally being enabled by a single line change assuming
+ALPN is already configured.
+
+This report concludes the DDD process for the single-stream QUIC client API
+design process, which sought to validate our API design and API ease of use for
+existing applications seeking to adopt QUIC.