Allow an endpoint to read the alert data before closing the socket

If an alert gets sent and then we close the connection immediately with
data still in the input buffer then a TCP-RST gets sent. Some OSs
immediately abandon data in their input buffer if a TCP-RST is received -
meaning the alert data itself gets ditched. Sending a TCP-FIN before the
TCP-RST seems to avoid this.

This was causing test failures in MSYS2 builds.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4333)

1 files changed, 17 insertions, 0 deletions

diff --git a/apps/s_socket.c b/apps/s_socket.c
index 3bdb587395..2f3e90bbf9 100644
--- a/apps/s_socket.c
+++ b/apps/s_socket.c
@@ -221,6 +221,23 @@ int do_server(int *accept_sock, const char *host, const char *port,
                 break;
             }
             i = (*cb)(sock, type, protocol, context);
+            /*
+             * If we ended with an alert being sent, but still with data in the
+             * network buffer to be read, then calling BIO_closesocket() will
+             * result in a TCP-RST being sent. On some platforms (notably
+             * Windows) then this will result in the peer immediately abandoning
+             * the connection including any buffered alert data before it has
+             * had a chance to be read. Shutting down the sending side first,
+             * and then closing the socket sends TCP-FIN first followed by
+             * TCP-RST. This seems to allow the peer to read the alert data.
+             */
+#ifdef _WIN32
+# ifdef SD_SEND
+            shutdown(sock, SD_SEND);
+# endif
+#elif defined(SHUT_WR)
+            shutdown(sock, SHUT_WR);
+#endif
             BIO_closesocket(sock);
         } else {
             i = (*cb)(asock, type, protocol, context);