Fix a DTLS server hangup due to TLS13_AD_MISSING_EXTENSION

This causes the DTLS server to enter an error state:

./openssl s_server -dtls
./openssl s_client -dtls -maxfraglen 512 -sess_out s1.txt
[...]
Q
./openssl s_client -dtls -sess_in s1.txt
CONNECTED(00000003)
^C
./openssl s_client -dtls
CONNECTED(00000003)
140335537067840:error:14102410:SSL routines:dtls1_read_bytes:sslv3 alert handshake failure:ssl/record/rec_layer_d1.c:614:SSL alert number 40

At this point the dtls server needs to be restarted,
because verify_cookie_callback always fails, because
the previous cookie is checked against the current one.
The reason for this is not fully understood.

In wireshark we see the following each time:
c->s Client Hello (without cookie)
s->c Hello Verify Request (with new cookie)
s->c Alert (Level: Fatal, Description: Handshake Failure)
c->s Client Hello (echoes new cookie)

The client gives up when the Alert arrives.
The Alert is triggered because the server calls
verify_cookie_callback with the previous cookie,
although it just sent the current cookie in the
Hello Verify Request.

However this does only happen because no Alert message
is sent when the client re-connects the session with
the missing -maxfraglen option.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18094)

2 files changed, 4 insertions, 0 deletions

diff --git a/ssl/s3_enc.c b/ssl/s3_enc.c
index 8a89f512fe..eb1f36ac7e 100644
--- a/ssl/s3_enc.c
+++ b/ssl/s3_enc.c
@@ -589,6 +589,8 @@ int ssl3_alert_code(int code)
         return TLS1_AD_NO_APPLICATION_PROTOCOL;
     case SSL_AD_CERTIFICATE_REQUIRED:
         return SSL_AD_HANDSHAKE_FAILURE;
+    case SSL_AD_MISSING_EXTENSION:
+        return SSL_AD_HANDSHAKE_FAILURE;
     default:
         return -1;
     }
diff --git a/ssl/t1_enc.c b/ssl/t1_enc.c
index c85c0b0310..2087b274d1 100644
--- a/ssl/t1_enc.c
+++ b/ssl/t1_enc.c
@@ -672,6 +672,8 @@ int tls1_alert_code(int code)
         return TLS1_AD_NO_APPLICATION_PROTOCOL;
     case SSL_AD_CERTIFICATE_REQUIRED:
         return SSL_AD_HANDSHAKE_FAILURE;
+    case SSL_AD_MISSING_EXTENSION:
+        return SSL_AD_HANDSHAKE_FAILURE;
     default:
         return -1;
     }