Add initial demo-driven design demos

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17991)

9 files changed, 2187 insertions, 0 deletions

diff --git a/doc/designs/ddd/Makefile b/doc/designs/ddd/Makefile
new file mode 100644
index 0000000000..0671f4ed73
--- /dev/null
+++ b/doc/designs/ddd/Makefile
@@ -0,0 +1,24 @@
+#
+# To run the demos when linked with a shared library (default):
+#
+#    LD_LIBRARY_PATH=../.. make test
+
+TESTS=ddd-01-conn-blocking ddd-02-conn-nonblocking ddd-03-fd-blocking ddd-04-fd-nonblocking ddd-05-mem-nonblocking ddd-06-mem-uv
+
+CFLAGS = -I../include -O3 -g -Wall
+LDFLAGS = -L..
+LDLIBS = -lcrypto -lssl
+
+all: $(TESTS)
+
+clean:
+	rm -f $(TESTS) *.o
+
+test: all
+	for x in $(TESTS); do echo "$$x"; LD_LIBRARY_PATH="$$(pwd)/.." ./$$x | grep -q '</html>' || { echo >&2 'Error'; exit 1; }; done
+
+ddd-06-mem-uv: ddd-06-mem-uv.c
+	$(CC) $(CFLAGS) $(LDFLAGS) -o "$@" "$<" $(LDLIBS) -luv
+
+ddd-%: ddd-%.c
+	$(CC) $(CFLAGS) $(LDFLAGS) -o "$@" "$<" $(LDLIBS)
diff --git a/doc/designs/ddd/README.md b/doc/designs/ddd/README.md
new file mode 100644
index 0000000000..61b74e8ceb
--- /dev/null
+++ b/doc/designs/ddd/README.md
@@ -0,0 +1,115 @@
+Demo-Driven Design
+==================
+
+The OpenSSL project from time to time must evolve its public API surface in
+order to support new functionality and deprecate old functionality. When this
+occurs, the changes to OpenSSL's public API must be planned, discussed and
+agreed. One significant dimension which must be considered when considering any
+proposed API change is how a broad spectrum of real-world OpenSSL applications
+uses the APIs which exist today, as this determines the ways in which those
+applications will be affected by any proposed changes, the extent to which they
+will be affected, and the extent of any changes which will need to be made by
+codebases using OpenSSL to remain current with best practices for OpenSSL API
+usage.
+
+As such, it is useful for the OpenSSL project to have a good understanding of
+the usage patterns common in codebases which use OpenSSL, so that it can
+anticipate the impact of any evolution of its API on those codebases. This
+directory seeks to maintain a set of **API usage demos** which demonstrate a
+full spectrum of ways in which real-world applications use the OpenSSL APIs.
+This allows the project to discuss any proposed API changes in terms of the
+changes that would need to be made to each demo. Since the demos are
+representative of a broad spectrum of real-world OpenSSL-based applications,
+this ensures that API evolution is made both with reference to real-world API
+usage patterns and with reference to the impact on existing applications.
+
+As such, these demos are maintained in the OpenSSL repository because they are
+useful both to current and any future proposed API changes. The set of demos may
+be expanded over time, and the demos in this directory at any one time consitute
+a present body of understanding of API usage patterns, which can be used to plan
+API changes.
+
+For further background information on the premise of this approach, see [API
+long-term evolution](https://github.com/openssl/openssl/issues/17939).
+
+Scope
+-----
+
+The current emphasis is on client demos. Server support for QUIC is deferred to
+subsequent OpenSSL releases, and therefore is (currently) out of scope for this
+design exercise.
+
+The demos also deliberately focus on aspects of libssl usage which are likely to
+be relevant to QUIC and require changes; for example, how varied applications
+have libssl perform network I/O, and how varied applications create sockets and
+connections for use with libssl. The libssl API as a whole has a much larger
+scope and includes innumerate functions and myriad features; the intention is
+not to demonstrate all of these, because most of them will not be touched by
+QUIC. For example, while many users of OpenSSL may make use of APIs for client
+certificates or other TLS functionality, the use of QUIC is unlikely to have
+implications for these APIs and demos demonstrating such functionality are
+therefore out of scope.
+
+Background
+----------
+
+These demos were developed after analysis of the following open source
+applications to determine libssl API usage patterns. The modally occuring usage
+patterns were determined and used to determine categories into which to classify
+the applications:
+
+|                  | Blk? | FD |
+|------------------|------|----|
+| mutt             | S |      AOSF  |
+| vsftpd           | S |      AOSF  |
+| exim             | S |      AOSFx |
+| wget             | S |      AOSF  |
+| Fossil           | S |      BIOc  |
+| librabbitmq      | A |      BIOx  |
+| ngircd           | A |      AOSF  |
+| stunnel          | A |      AOSFx |
+| Postfix          | A |      AOSF  |
+| socat            | A |      AOSF  |
+| HAProxy          | A |      BIOx  |
+| Dovecot          | A |      BIOm  |
+| Apache httpd     | A |      BIOx  |
+| UnrealIRCd       | A |      AOSF  |
+| wpa_supplicant   | A |      BIOm  |
+| icecast          | A |      AOSF  |
+| nginx            | A |      AOSF  |
+| curl             | A |      AOSF  |
+| Asterisk         | A |      AOSF  |
+| Asterisk (DTLS)  | A |      BIOm/x |
+| pgbouncer        | A |      AOSF, BIOc  |
+
+* Blk: Whether the application uses blocking or non-blocking I/O.
+  * S: Blocking
+  * A: Nonblocking
+* FD: Whether the application creates and owns its own FD.
+  * AOSF: Application owns, calls SSL_set_fd.
+  * AOSFx: Application owns, calls SSL_set_[rw]fd, different FDs for read/write.
+  * BIOs: Application creates a socket/FD BIO and calls SSL_set_bio.
+    Application created the connection.
+  * BIOx: Application creates a BIO with a custom BIO method and calls SSL_set_bio.
+  * BIOm: Application creates a memory BIO and does its own
+    pumping to/from actual socket, treating libssl as a pure state machine which
+    does no I/O itself.
+  * BIOc: Application uses BIO_s_connect-based methods such as BIO_new_ssl_connect
+    and leaves connection establishment to OpenSSL.
+
+Demos
+-----
+
+The demos found in this directory are:
+
+|                 | Type  | Description |
+|-----------------|-------|-------------|
+| [ddd-01-conn-blocking](ddd-01-conn-blocking.c) | S-BIOc | A `BIO_s_connect`-based blocking example demonstrating exemplary OpenSSL API usage |
+| [ddd-02-conn-nonblocking](ddd-02-conn-nonblocking.c) | A-BIOc | A `BIO_s_connect`-based nonblocking example demonstrating exemplary OpenSSL API usage, with use of a buffering BIO |
+| [ddd-03-fd-blocking](ddd-03-fd-blocking.c) | S-AOSF | A `SSL_set_fd`-based blocking example demonstrating real-world OpenSSL API usage (corresponding to S-AOSF applications above) |
+| [ddd-04-fd-nonblocking](ddd-04-fd-nonblocking.c) | A-AOSF | A `SSL_set_fd`-based non-blocking example demonstrating real-world OpenSSL API usage (corresponding to A-AOSF applications above) |
+| [ddd-05-mem-nonblocking](ddd-05-mem-nonblocking.c) | A-BIOm | A non-blocking example based on use of a memory buffer to feed OpenSSL encrypted data (corresponding to A-BIOm applications above) |
+| [ddd-06-mem-uv](ddd-06-mem-uv.c) | A-BIOm | A non-blocking example based on use of a memory buffer to feed OpenSSL encrypted data; uses libuv, a real-world async I/O library |
+
+Availability of a default certificate store is assumed. `SSL_CERT_DIR` may be
+set when running the demos if necessary.
diff --git a/doc/designs/ddd/WINDOWS.md b/doc/designs/ddd/WINDOWS.md
new file mode 100644
index 0000000000..2c96a9542e
--- /dev/null
+++ b/doc/designs/ddd/WINDOWS.md
@@ -0,0 +1,80 @@
+Windows-related issues
+======================
+
+Supporting Windows introduces some complications due to some "fun" peculiarities
+of Windows's socket API.
+
+In general, Windows does not provide a poll(2) call. WSAPoll(2) was introduced
+in Vista and supposed to bring this functionality, but it had a bug in it which
+Microsoft refused to fix, making it rather pointless. However Microsoft has now
+finally fixed this bug in a build of Windows 10. So WSAPoll(2) is a viable
+method, but only on fairly new versions of Windows.
+
+Traditionally, polling has been done on windows using select(). However, this
+call works a little differently than on POSIX platforms. Whereas on POSIX
+platforms select() accepts a bitmask of FDs, on Windows select() accepts a
+structure which embeds a fixed-length array of socket handles. This is necessary
+because sockets are NT kernel handles on Windows and thus are not allocated
+contiguously like FDs. As such, Windows select() is actually very similar to
+POSIX poll(), making select() a viable option for polling on Windows.
+
+Neither select() nor poll() are, of course, high performance polling options.
+Windows does not provide anything like epoll or kqueue. For high performance
+network I/O, you are expected to use a Windows API called I/O Completion Ports
+(IOCP).
+
+Supporting these is a pain for applications designed around polling. The reason
+is that IOCPs are a higher-level interface; it is easy to build an IOCP-like
+interface on top of polling, but it is not really possible to build a
+polling-like interface on top of IOCPs.
+
+For this reason it's actually common for asynchronous I/O libraries to basically
+contain two separate implementations of their APIs internally, or at least a
+substantial chunk of their code (e.g. libuv, nanomsg). It turns out to be easier
+just to write a poll-based implementation of an I/O reactor and an IOCP-based
+implementation than try to overcome the impedence discontinuities.
+
+The difference between polling and IOCPs is that polling reports *readiness*
+whereas IOCPs report *completion of an operation*. For example, in the IOCP
+model, you make a read or write on a socket and an event is posted to the IOCP
+when the read or write is complete. This is a fundamentally different model and
+actually more similar to a high-level asynchronous I/O library such as libuv or
+so on.
+
+Evaluation of the existing demos and their applicability to Windows IOCP:
+
+- ddd-01-conn-blocking: Blocking example, use of IOCP is not applicable.
+
+- ddd-02-conn-nonblocking: Socket is managed by OpenSSL, and IOCP is not
+  supported.
+
+- ddd-03-fd-blocking: Blocking example, use of IOCP is not applicable.
+
+- ddd-04-fd-nonblocking: libssl is passed an FD with BIO_set_fd.
+
+  BIO_s_sock doesn't appear to support overlapped (that is, IOCP-based) I/O
+  as this requires use of special WSASend() and WSARecv() functions, rather
+  than standard send()/recv().
+
+  Since libssl already doesn't support IOCP for use of BIO_s_sock,
+  we might say here that any existing application using BIO_s_sock
+  obviously isn't trying to use IOCP, and therefore we don't need to
+  worry about the adapability of this example to IOCP.
+
+- ddd-05-mem-nonblocking: Since the application is in full control of passing
+  data from the memory BIO to the network, or vice versa, the application
+  can use IOCP if it wishes.
+
+  This is demonstrated in the following demo:
+
+- ddd-06-mem-uv: This demo uses a memory BIO and libuv. Since libuv supports
+  IOCP, it proves that a memory BIO can be used to support IOCP-based usage.
+
+Further, a cursory examination of code on GitHub seems to suggest that when
+people do use IOCP with libssl, they do it using memory BIOs passed to libssl.
+So ddd-05 and ddd-06 essentially demonstate this use case, especially ddd-06 as
+it uses IOCP internally on Windows.
+
+My conclusion here is that since libssl does not support IOCP in the first
+place, we don't need to be particularly worried about this. But in the worst
+case there are always workable solutions, as in demos 5 and 6.
diff --git a/doc/designs/ddd/ddd-01-conn-blocking.c b/doc/designs/ddd/ddd-01-conn-blocking.c
new file mode 100644
index 0000000000..8ed79e5147
--- /dev/null
+++ b/doc/designs/ddd/ddd-01-conn-blocking.c
@@ -0,0 +1,161 @@
+#include <openssl/ssl.h>
+
+/*
+ * Demo 1: Client — Managed Connection — Blocking
+ * ==============================================
+ *
+ * This is an example of (part of) an application which uses libssl in a simple,
+ * synchronous, blocking fashion. The functions show all interactions with
+ * libssl the application makes, and would hypothetically be linked into a
+ * larger application.
+ */
+
+/*
+ * The application is initializing and wants an SSL_CTX which it will use for
+ * some number of outgoing connections, which it creates in subsequent calls to
+ * new_conn. The application may also call this function multiple times to
+ * create multiple SSL_CTX.
+ */
+SSL_CTX *create_ssl_ctx(void)
+{
+    SSL_CTX *ctx;
+
+    ctx = SSL_CTX_new(TLS_client_method());
+    if (ctx == NULL)
+        return NULL;
+
+    /* Enable trust chain verification. */
+    SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, NULL);
+
+    /* Load default root CA store. */
+    if (SSL_CTX_set_default_verify_paths(ctx) == 0) {
+        SSL_CTX_free(ctx);
+        return NULL;
+    }
+
+    return ctx;
+}
+
+/*
+ * The application wants to create a new outgoing connection using a given
+ * SSL_CTX.
+ *
+ * hostname is a string like "openssl.org:443" or "[::1]:443".
+ */
+BIO *new_conn(SSL_CTX *ctx, const char *hostname)
+{
+    BIO *out;
+    SSL *ssl = NULL;
+    const char *bare_hostname;
+
+    out = BIO_new_ssl_connect(ctx);
+    if (out == NULL)
+        return NULL;
+
+    if (BIO_get_ssl(out, &ssl) == 0) {
+        BIO_free_all(out);
+        return NULL;
+    }
+
+    if (BIO_set_conn_hostname(out, hostname) == 0) {
+        BIO_free_all(out);
+        return NULL;
+    }
+
+    /* Returns the parsed hostname extracted from the hostname:port string. */
+    bare_hostname = BIO_get_conn_hostname(out);
+    if (bare_hostname == NULL) {
+        BIO_free_all(out);
+        return NULL;
+    }
+
+    /* Tell the SSL object the hostname to check certificates against. */
+    if (SSL_set1_host(ssl, bare_hostname) <= 0) {
+        BIO_free_all(out);
+        return NULL;
+    }
+
+    return out;
+}
+
+/*
+ * The application wants to send some block of data to the peer.
+ * This is a blocking call.
+ */
+int tx(BIO *bio, const void *buf, int buf_len)
+{
+    return BIO_write(bio, buf, buf_len);
+}
+
+/*
+ * The application wants to receive some block of data from
+ * the peer. This is a blocking call.
+ */
+int rx(BIO *bio, void *buf, int buf_len)
+{
+    return BIO_read(bio, buf, buf_len);
+}
+
+/*
+ * The application wants to close the connection and free bookkeeping
+ * structures.
+ */
+void teardown(BIO *bio)
+{
+    BIO_free_all(bio);
+}
+
+/*
+ * The application is shutting down and wants to free a previously
+ * created SSL_CTX.
+ */
+void teardown_ctx(SSL_CTX *ctx)
+{
+    SSL_CTX_free(ctx);
+}
+
+/*
+ * ============================================================================
+ * Example driver for the above code. This is just to demonstrate that the code
+ * works and is not intended to be representative of a real application.
+ */
+int main(int argc, char **argv)
+{
+    const char msg[] = "GET / HTTP/1.0\r\nHost: www.openssl.org\r\n\r\n";
+    SSL_CTX *ctx = NULL;
+    BIO *b = NULL;
+    char buf[2048];
+    int l, res = 1;
+
+    ctx = create_ssl_ctx();
+    if (ctx == NULL) {
+        fprintf(stderr, "could not create context\n");
+        goto fail;
+    }
+
+    b = new_conn(ctx, "www.openssl.org:443");
+    if (b == NULL) {
+        fprintf(stderr, "could not create conn\n");
+        goto fail;
+    }
+
+    if (tx(b, msg, sizeof(msg)) < sizeof(msg)) {
+        fprintf(stderr, "tx error\n");
+        goto fail;
+    }
+
+    for (;;) {
+        l = rx(b, buf, sizeof(buf));
+        if (l <= 0)
+            break;
+        fwrite(buf, 1, l, stdout);
+    }
+
+    res = 0;
+fail:
+    if (b != NULL)
+        teardown(b);
+    if (ctx != NULL)
+        teardown_ctx(ctx);
+    return res;
+}
diff --git a/doc/designs/ddd/ddd-02-conn-nonblocking.c b/doc/designs/ddd/ddd-02-conn-nonblocking.c
new file mode 100644
index 0000000000..fa508afc7c
--- /dev/null
+++ b/doc/designs/ddd/ddd-02-conn-nonblocking.c
@@ -0,0 +1,289 @@
+#include <sys/poll.h>
+#include <openssl/ssl.h>
+
+/*
+ * Demo 2: Client — Managed Connection — Asynchronous Nonblocking
+ * ==============================================================
+ *
+ * This is an example of (part of) an application which uses libssl in an
+ * asynchronous, nonblocking fashion. The functions show all interactions with
+ * libssl the application makes, and would hypothetically be linked into a
+ * larger application.
+ *
+ * In this example, libssl still makes syscalls directly using an fd, which is
+ * configured in nonblocking mode. As such, the application can still be
+ * abstracted from the details of what that fd is (is it a TCP socket? is it a
+ * UDP socket?); this code passes the application an fd and the application
+ * simply calls back into this code when poll()/etc. indicates it is ready.
+ */
+typedef struct app_conn_st {
+    SSL *ssl;
+    BIO *ssl_bio;
+    int rx_need_tx, tx_need_rx;
+} APP_CONN;
+
+/*
+ * The application is initializing and wants an SSL_CTX which it will use for
+ * some number of outgoing connections, which it creates in subsequent calls to
+ * new_conn. The application may also call this function multiple times to
+ * create multiple SSL_CTX.
+ */
+SSL_CTX *create_ssl_ctx(void)
+{
+    SSL_CTX *ctx;
+
+    ctx = SSL_CTX_new(TLS_client_method());
+    if (ctx == NULL)
+        return NULL;
+
+    /* Enable trust chain verification. */
+    SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, NULL);
+
+    /* Load default root CA store. */
+    if (SSL_CTX_set_default_verify_paths(ctx) == 0) {
+        SSL_CTX_free(ctx);
+        return NULL;
+    }
+
+    return ctx;
+}
+
+/*
+ * The application wants to create a new outgoing connection using a given
+ * SSL_CTX.
+ *
+ * hostname is a string like "openssl.org:443" or "[::1]:443".
+ */
+APP_CONN *new_conn(SSL_CTX *ctx, const char *hostname)
+{
+    APP_CONN *conn;
+    BIO *out, *buf;
+    SSL *ssl = NULL;
+    const char *bare_hostname;
+
+    conn = calloc(1, sizeof(APP_CONN));
+    if (conn == NULL)
+        return NULL;
+
+    out = BIO_new_ssl_connect(ctx);
+    if (out == NULL) {
+        free(conn);
+        return NULL;
+    }
+
+    if (BIO_get_ssl(out, &ssl) == 0) {
+        BIO_free_all(out);
+        free(conn);
+        return NULL;
+    }
+
+    buf = BIO_new(BIO_f_buffer());
+    if (buf == NULL) {
+        BIO_free_all(out);
+        free(conn);
+        return NULL;
+    }
+
+    BIO_push(out, buf);
+
+    if (BIO_set_conn_hostname(out, hostname) == 0) {
+        BIO_free_all(out);
+        free(conn);
+        return NULL;
+    }
+
+    /* Returns the parsed hostname extracted from the hostname:port string. */
+    bare_hostname = BIO_get_conn_hostname(out);
+    if (bare_hostname == NULL) {
+        BIO_free_all(out);
+        free(conn);
+        return NULL;
+    }
+
+    /* Tell the SSL object the hostname to check certificates against. */
+    if (SSL_set1_host(ssl, bare_hostname) <= 0) {
+        BIO_free_all(out);
+        free(conn);
+        return NULL;
+    }
+
+    /* Make the BIO nonblocking. */
+    BIO_set_nbio(out, 1);
+
+    conn->ssl_bio = out;
+    return conn;
+}
+
+/*
+ * Non-blocking transmission.
+ *
+ * Returns -1 on error. Returns -2 if the function would block (corresponds to
+ * EWOULDBLOCK).
+ */
+int tx(APP_CONN *conn, const void *buf, int buf_len)
+{
+    int l;
+
+    conn->tx_need_rx = 0;
+
+    l = BIO_write(conn->ssl_bio, buf, buf_len);
+    if (l <= 0) {
+        if (BIO_should_retry(conn->ssl_bio)) {
+            conn->tx_need_rx = BIO_should_read(conn->ssl_bio);
+            return -2;
+        } else {
+            return -1;
+        }
+    }
+
+    return l;
+}
+
+/*
+ * Non-blocking reception.
+ *
+ * Returns -1 on error. Returns -2 if the function would block (corresponds to
+ * EWOULDBLOCK).
+ */
+int rx(APP_CONN *conn, void *buf, int buf_len)
+{
+    int l;
+
+    conn->rx_need_tx = 0;
+
+    l = BIO_read(conn->ssl_bio, buf, buf_len);
+    if (l <= 0) {
+        if (BIO_should_retry(conn->ssl_bio)) {
+            conn->rx_need_tx = BIO_should_write(conn->ssl_bio);
+            return -2;
+        } else {
+            return -1;
+        }
+    }
+
+    return l;
+}
+
+/*
+ * The application wants to know a fd it can poll on to determine when the
+ * SSL state machine needs to be pumped.
+ */
+int get_conn_fd(APP_CONN *conn)
+{
+    return BIO_get_fd(conn->ssl_bio, NULL);
+}
+
+/*
+ * These functions returns zero or more of:
+ *
+ *   POLLIN:    The SSL state machine is interested in socket readability events.
+ *
+ *   POLLOUT:   The SSL state machine is interested in socket writeability events.
+ *
+ *   POLLERR:   The SSL state machine is interested in socket error events.
+ *
+ * get_conn_pending_tx returns events which may cause SSL_write to make
+ * progress and get_conn_pending_rx returns events which may cause SSL_read
+ * to make progress.
+ */
+int get_conn_pending_tx(APP_CONN *conn)
+{
+    return (conn->tx_need_rx ? POLLIN : 0) | POLLOUT | POLLERR;
+}
+
+int get_conn_pending_rx(APP_CONN *conn)
+{
+    return (conn->rx_need_tx ? POLLOUT : 0) | POLLIN | POLLERR;
+}
+
+/*
+ * The application wants to close the connection and free bookkeeping
+ * structures.
+ */
+void teardown(APP_CONN *conn)
+{
+    BIO_free_all(conn->ssl_bio);
+    free(conn);
+}
+
+/*
+ * The application is shutting down and wants to free a previously
+ * created SSL_CTX.
+ */
+void teardown_ctx(SSL_CTX *ctx)
+{
+    SSL_CTX_free(ctx);
+}
+
+/*
+ * ============================================================================
+ * Example driver for the above code. This is just to demonstrate that the code
+ * works and is not intended to be representative of a real application.
+ */
+int main(int argc, char **argv)
+{
+    const char tx_msg[] = "GET / HTTP/1.0\r\nHost: www.openssl.org\r\n\r\n";
+    const char *tx_p = tx_msg;
+    char rx_buf[2048];
+    int res = 1, l, tx_len = sizeof(tx_msg)-1;
+    int timeout = 2000 /* ms */;
+    APP_CONN *conn = NULL;
+    SSL_CTX *ctx;
+
+    ctx = create_ssl_ctx();
+    if (ctx == NULL) {
+        fprintf(stderr, "cannot create SSL context\n");
+        goto fail;
+    }
+
+    conn = new_conn(ctx, "www.openssl.org:443");
+    if (conn == NULL) {
+        fprintf(stderr, "cannot establish connection\n");
+        goto fail;
+    }
+
+    /* TX */
+    while (tx_len != 0) {
+        l = tx(conn, tx_p, tx_len);
+        if (l > 0) {
+            tx_p += l;
+            tx_len -= l;
+        } else if (l == -1) {
+            fprintf(stderr, "tx error\n");
+        } else if (l == -2) {
+            struct pollfd pfd = {0};
+            pfd.fd = get_conn_fd(conn);
+            pfd.events = get_conn_pending_tx(conn);
+            if (poll(&pfd, 1, timeout) == 0) {
+                fprintf(stderr, "tx timeout\n");
+                goto fail;
+            }
+        }
+    }
+
+    /* RX */
+    for (;;) {
+        l = rx(conn, rx_buf, sizeof(rx_buf));
+        if (l > 0) {
+            fwrite(rx_buf, 1, l, stdout);
+        } else if (l == -1) {
+            break;
+        } else if (l == -2) {
+            struct pollfd pfd = {0};
+            pfd.fd = get_conn_fd(conn);
+            pfd.events = get_conn_pending_rx(conn);
+            if (poll(&pfd, 1, timeout) == 0) {
+                fprintf(stderr, "rx timeout\n");
+                goto fail;
+            }
+        }
+    }
+
+    res = 0;
+fail:
+    if (conn != NULL)
+        teardown(conn);
+    if (ctx != NULL)
+        teardown_ctx(ctx);
+    return res;
+}
diff --git a/doc/designs/ddd/ddd-03-fd-blocking.c b/doc/designs/ddd/ddd-03-fd-blocking.c
new file mode 100644
index 0000000000..f4aaf35a1e
--- /dev/null
+++ b/doc/designs/ddd/ddd-03-fd-blocking.c
@@ -0,0 +1,188 @@
+#include <openssl/ssl.h>
+
+/*
+ * Demo 3: Client — Client Creates FD — Blocking
+ * =============================================
+ *
+ * This is an example of (part of) an application which uses libssl in a simple,
+ * synchronous, blocking fashion. The client is responsible for creating the
+ * socket and passing it to libssl. The functions show all interactions with
+ * libssl the application makes, and would hypothetically be linked into a
+ * larger application.
+ */
+
+/*
+ * The application is initializing and wants an SSL_CTX which it will use for
+ * some number of outgoing connections, which it creates in subsequent calls to
+ * new_conn. The application may also call this function multiple times to
+ * create multiple SSL_CTX.
+ */
+SSL_CTX *create_ssl_ctx(void)
+{
+    SSL_CTX *ctx;
+
+    ctx = SSL_CTX_new(TLS_client_method());
+    if (ctx == NULL)
+        return NULL;
+
+    /* Enable trust chain verification. */
+    SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, NULL);
+
+    /* Load default root CA store. */
+    if (SSL_CTX_set_default_verify_paths(ctx) == 0) {
+        SSL_CTX_free(ctx);
+        return NULL;
+    }
+
+    return ctx;
+}
+
+/*
+ * The application wants to create a new outgoing connection using a given
+ * SSL_CTX.
+ *
+ * hostname is a string like "openssl.org" used for certificate validation.
+ */
+SSL *new_conn(SSL_CTX *ctx, int fd, const char *bare_hostname)
+{
+    SSL *ssl;
+
+    ssl = SSL_new(ctx);
+    if (ssl == NULL)
+        return NULL;
+
+    SSL_set_connect_state(ssl); /* cannot fail */
+
+    if (SSL_set_fd(ssl, fd) <= 0) {
+        SSL_free(ssl);
+        return NULL;
+    }
+
+    if (SSL_set1_host(ssl, bare_hostname) <= 0) {
+        SSL_free(ssl);
+        return NULL;
+    }
+
+    if (SSL_set_tlsext_host_name(ssl, bare_hostname) <= 0) {
+        SSL_free(ssl);
+        return NULL;
+    }
+
+    return ssl;
+}
+
+/*
+ * The application wants to send some block of data to the peer.
+ * This is a blocking call.
+ */
+int tx(SSL *ssl, const void *buf, int buf_len)
+{
+    return SSL_write(ssl, buf, buf_len);
+}
+
+/*
+ * The application wants to receive some block of data from
+ * the peer. This is a blocking call.
+ */
+int rx(SSL *ssl, void *buf, int buf_len)
+{
+    return SSL_read(ssl, buf, buf_len);
+}
+
+/*
+ * The application wants to close the connection and free bookkeeping
+ * structures.
+ */
+void teardown(SSL *ssl)
+{
+    SSL_free(ssl);
+}
+
+/*
+ * The application is shutting down and wants to free a previously
+ * created SSL_CTX.
+ */
+void teardown_ctx(SSL_CTX *ctx)
+{
+    SSL_CTX_free(ctx);
+}
+
+/*
+ * ============================================================================
+ * Example driver for the above code. This is just to demonstrate that the code
+ * works and is not intended to be representative of a real application.
+ */
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/signal.h>
+#include <netdb.h>
+#include <unistd.h>
+
+int main(int argc, char **argv)
+{
+    int rc, fd = -1, l, res = 1;
+    const char msg[] = "GET / HTTP/1.0\r\nHost: www.openssl.org\r\n\r\n";
+    struct addrinfo hints = {0}, *result = NULL;
+    SSL *ssl = NULL;
+    SSL_CTX *ctx;
+    char buf[2048];
+
+    ctx = create_ssl_ctx();
+    if (ctx == NULL) {
+        fprintf(stderr, "cannot create context\n");
+        goto fail;
+    }
+
+    hints.ai_family     = AF_INET;
+    hints.ai_socktype   = SOCK_STREAM;
+    hints.ai_flags      = AI_PASSIVE;
+    rc = getaddrinfo("www.openssl.org", "443", &hints, &result);
+    if (rc < 0) {
+        fprintf(stderr, "cannot resolve\n");
+        goto fail;
+    }
+
+    signal(SIGPIPE, SIG_IGN);
+
+    fd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
+    if (fd < 0) {
+        fprintf(stderr, "cannot create socket\n");
+        goto fail;
+    }
+
+    rc = connect(fd, result->ai_addr, result->ai_addrlen);
+    if (rc < 0) {
+        fprintf(stderr, "cannot connect\n");
+        goto fail;
+    }
+
+    ssl = new_conn(ctx, fd, "www.openssl.org");
+    if (ssl == NULL) {
+        fprintf(stderr, "cannot create connection\n");
+        goto fail;
+    }
+
+    if (tx(ssl, msg, sizeof(msg)-1) < sizeof(msg)-1) {
+        fprintf(stderr, "tx error\n");
+        goto fail;
+    }
+
+    for (;;) {
+        l = rx(ssl, buf, sizeof(buf));
+        if (l <= 0)
+            break;
+        fwrite(buf, 1, l, stdout);
+    }
+
+    res = 0;
+fail:
+    if (ssl != NULL)
+        teardown(ssl);
+    if (ctx != NULL)
+        teardown_ctx(ctx);
+    if (fd >= 0)
+        close(fd);
+    if (result != NULL)
+        freeaddrinfo(result);
+    return res;
+}
diff --git a/doc/designs/ddd/ddd-04-fd-nonblocking.c b/doc/designs/ddd/ddd-04-fd-nonblocking.c
new file mode 100644
index 0000000000..2e9606b921
--- /dev/null
+++ b/doc/designs/ddd/ddd-04-fd-nonblocking.c
@@ -0,0 +1,323 @@
+#include <sys/poll.h>
+#include <openssl/ssl.h>
+
+/*
+ * Demo 4: Client — Client Creates FD — Nonblocking
+ * ================================================
+ *
+ * This is an example of (part of) an application which uses libssl in an
+ * asynchronous, nonblocking fashion. The client is responsible for creating the
+ * socket and passing it to libssl. The functions show all interactions with
+ * libssl the application makes, and wouldn hypothetically be linked into a
+ * larger application.
+ */
+typedef struct app_conn_st {
+    SSL *ssl;
+    int fd;
+    int rx_need_tx, tx_need_rx;
+} APP_CONN;
+
+/*
+ * The application is initializing and wants an SSL_CTX which it will use for
+ * some number of outgoing connections, which it creates in subsequent calls to
+ * new_conn. The application may also call this function multiple times to
+ * create multiple SSL_CTX.
+ */
+SSL_CTX *create_ssl_ctx(void)
+{
+    SSL_CTX *ctx;
+
+    ctx = SSL_CTX_new(TLS_client_method());
+    if (ctx == NULL)
+        return NULL;
+
+    /* Enable trust chain verification. */
+    SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, NULL);
+
+    /* Load default root CA store. */
+    if (SSL_CTX_set_default_verify_paths(ctx) == 0) {
+        SSL_CTX_free(ctx);
+        return NULL;
+    }
+
+    return ctx;
+}
+
+/*
+ * The application wants to create a new outgoing connection using a given
+ * SSL_CTX.
+ *
+ * hostname is a string like "openssl.org" used for certificate validation.
+ */
+APP_CONN *new_conn(SSL_CTX *ctx, int fd, const char *bare_hostname)
+{
+    APP_CONN *conn;
+    SSL *ssl;
+
+    conn = calloc(1, sizeof(APP_CONN));
+    if (conn == NULL)
+        return NULL;
+
+    ssl = conn->ssl = SSL_new(ctx);
+    if (ssl == NULL) {
+        free(conn);
+        return NULL;
+    }
+
+    SSL_set_connect_state(ssl); /* cannot fail */
+
+    if (SSL_set_fd(ssl, fd) <= 0) {
+        SSL_free(ssl);
+        free(conn);
+        return NULL;
+    }
+
+    if (SSL_set1_host(ssl, bare_hostname) <= 0) {
+        SSL_free(ssl);
+        free(conn);
+        return NULL;
+    }
+
+    if (SSL_set_tlsext_host_name(ssl, bare_hostname) <= 0) {
+        SSL_free(ssl);
+        free(conn);
+        return NULL;
+    }
+
+    conn->fd = fd;
+    return conn;<