Rework the decrypt ticket callback

Don't call the decrypt ticket callback if we've already encountered a
fatal error. Do call it if we have an empty ticket present.

Change the return code to have 5 distinct returns codes and separate it
from the input status value.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6198)

1 files changed, 84 insertions, 47 deletions

diff --git a/doc/man3/SSL_CTX_set_session_ticket_cb.pod b/doc/man3/SSL_CTX_set_session_ticket_cb.pod
index 3066534223..8f98c6f1c9 100644
--- a/doc/man3/SSL_CTX_set_session_ticket_cb.pod
+++ b/doc/man3/SSL_CTX_set_session_ticket_cb.pod
@@ -16,7 +16,7 @@ SSL_CTX_decrypt_session_ticket_fn - manage session ticket application data
  typedef SSL_TICKET_RETURN (*SSL_CTX_decrypt_session_ticket_fn)(SSL *s, SSL_SESSION *ss,
                                                                 const unsigned char *keyname,
                                                                 size_t keyname_len,
-                                                                SSL_TICKET_RETURN retv,
+                                                                SSL_TICKET_STATUS status,
                                                                 void *arg);
  int SSL_CTX_set_session_ticket_cb(SSL_CTX *ctx,
                                    SSL_CTX_generate_session_ticket_fn gen_cb,
@@ -39,13 +39,13 @@ is the same as that given to SSL_CTX_set_session_ticket_cb(). The B<gen_cb>
 callback is defined as type B<SSL_CTX_generate_session_ticket_fn>.
 
 B<dec_cb> is the application defined callback invoked after session ticket
-decryption has been attempted and any session ticket application data is available.
-The application can call SSL_SESSION_get_ticket_appdata() at this time to retrieve
-the application data. The value of B<arg> is the same as that given to
-SSL_CTX_set_session_ticket_cb(). The B<retv> argument is the result of the ticket
-decryption. The B<keyname> and B<keyname_len> identify the key used to decrypt the
-session ticket. The B<dec_cb> callback is defined as type
-B<SSL_CTX_decrypt_session_ticket_fn>.
+decryption has been attempted and any session ticket application data is
+available. If ticket decryption was successful then the B<ss> argument contains
+the session data. The B<keyname> and B<keyname_len> arguments identify the key
+used to decrypt the session ticket. The B<status> argument is the result of the
+ticket decryption. See the L<NOTES> section below for further details. The value
+of B<arg> is the same as that given to SSL_CTX_set_session_ticket_cb(). The
+B<dec_cb> callback is defined as type B<SSL_CTX_decrypt_session_ticket_fn>.
 
 SSL_SESSION_set1_ticket_appdata() sets the application data specified by
 B<data> and B<len> into B<ss> which is then placed into any generated session
@@ -66,73 +66,110 @@ application that a session ticket has just been decrypted.
 =head1 NOTES
 
 When the B<dec_cb> callback is invoked, the SSL_SESSION B<ss> has not yet been
-assigned to the SSL B<s>. The B<retv> indicates the result of the ticket
-decryption which can be modified by the callback before being returned. The
-callback must check the B<retv> value before performing any action, as it's
-called even if ticket decryption fails.
+assigned to the SSL B<s>. The B<status> indicates the result of the ticket
+decryption. The callback must check the B<status> value before performing any
+action, as it is called even if ticket decryption fails.
 
 The B<keyname> and B<keyname_len> arguments to B<dec_cb> may be used to identify
 the key that was used to encrypt the session ticket.
 
-When the B<gen_cb> callback is invoked, the SSL_get_session() function can be
-used to retrieve the SSL_SESSION for SSL_SESSION_set1_ticket_appdata().
+The B<status> argument can be any of these values:
 
-By default, in TLSv1.2 and below, a new session ticket is not issued on a
-successful resumption and therefore B<gen_cb> will not be called. In TLSv1.3 the
-default behaviour is to always issue a new ticket on resumption. In both cases
-this behaviour can be changed if a ticket key callback is in use (see
-L<SSL_CTX_set_tlsext_ticket_key_cb(3)>).
+=over 4
 
-=head1 RETURN VALUES
+=item SSL_TICKET_EMPTY
 
-The SSL_CTX_set_session_ticket_cb(), SSL_SESSION_set1_ticket_appdata() and
-SSL_SESSION_get0_ticket_appdata() functions return 1 on success and 0 on
-failure.
+Empty ticket present. No ticket data will be used and a new ticket should be
+sent to the client. This only occurs in TLSv1.2 or below. In TLSv1.3 it is not
+valid for a client to send an empty ticket.
 
-The B<gen_cb> callback must return 1 to continue the connection. A return of 0
-will terminate the connection with an INTERNAL_ERROR alert.
+=item SSL_TICKET_NO_DECRYPT
 
-The B<dec_cb> callback must return one of the following B<SSL_TICKET_RETURN>
-values. Under normal circumstances the B<retv> value is returned unmodified,
-but the callback can change the behavior of the post-ticket decryption code
-by returning something different. The B<dec_cb> callback must check the B<retv>
-value before performing any action.
+The ticket couldn't be decrypted. No ticket data will be used and a new ticket
+should be sent to the client.
 
- typedef int SSL_TICKET_RETURN;
+=item SSL_TICKET_SUCCESS
 
-=over 4
+A ticket was successfully decrypted, any session ticket application data should
+be available. A new ticket should not be sent to the client.
 
-=item SSL_TICKET_FATAL_ERR_MALLOC
+=item SSL_TICKET_SUCCESS_RENEW
 
-Fatal error, malloc failure.
+Same as B<SSL_TICKET_SUCCESS>, but a new ticket should be sent to the client.
 
-=item SSL_TICKET_FATAL_ERR_OTHER
+=back
 
-Fatal error, either from parsing or decrypting the ticket.
+The return value can be any of these values:
 
-=item SSL_TICKET_NONE
+=over 4
 
-No ticket present.
+=item SSL_TICKET_RETURN_ABORT
 
-=item SSL_TICKET_EMPTY
+The handshake should be aborted, either because of an error or because of some
+policy. Note that in TLSv1.3 a client may send more than one ticket in a single
+handshake. Therefore just because one ticket is unacceptable it does not mean
+that all of them are. For this reason this option should be used with caution.
 
-Empty ticket present.
+=item SSL_TICKET_RETURN_IGNORE
 
-=item SSL_TICKET_NO_DECRYPT
+Do not use a ticket (if one was available). Do not send a renewed ticket to the
+client.
 
-The ticket couldn't be decrypted.
+=item SSL_TICKET_RETURN_IGNORE_RENEW
 
-=item SSL_TICKET_SUCCESS
+Do not use a ticket (if one was available). Send a renewed ticket to the client.
 
-A ticket was successfully decrypted, any session ticket application data should
-be available.
+If the callback does not wish to change the default ticket behaviour then it
+should return this value if B<status> is B<SSL_TICKET_EMPTY> or
+B<SSL_TICKET_NO_DECRYPT>.
 
-=item TICKET_SUCCESS_RENEW
+=item SSL_TICKET_RETURN_USE
 
-Same as B<TICKET_SUCCESS>, but the ticket needs to be renewed.
+Use the ticket. Do not send a renewed ticket to the client. It is an error for
+the callback to return this value if B<status> has a value other than
+B<SSL_TICKET_SUCCESS> or B<SSL_TICKET_SUCCESS_RENEW>.
+
+If the callback does not wish to change the default ticket behaviour then it
+should return this value if B<status> is B<SSL_TICKET_SUCCESS>.
+
+=item SSL_TICKET_RETURN_USE_RENEW
+
+Use the ticket. Send a renewed ticket to the client. It is an error for the
+callback to return this value if B<status> has a value other than
+B<SSL_TICKET_SUCCESS> or B<SSL_TICKET_SUCCESS_RENEW>.
+
+If the callback does not wish to change the default ticket behaviour then it
+should return this value if B<status> is B<SSL_TICKET_SUCCESS_RENEW>.
 
 =back
 
+If B<status> has the value B<SSL_TICKET_EMPTY> or B<SSL_TICKET_NO_DECRYPT> then
+no session data will be available and the callback must not use the B<ss>
+argument. If B<status> has the value B<SSL_TICKET_SUCCESS> or
+B<SSL_TICKET_SUCCESS_RENEW> then the application can call
+SSL_SESSION_get0_ticket_appdata() using the session provided in the B<ss>
+argument to retrieve the application data.
+
+When the B<gen_cb> callback is invoked, the SSL_get_session() function can be
+used to retrieve the SSL_SESSION for SSL_SESSION_set1_ticket_appdata().
+
+By default, in TLSv1.2 and below, a new session ticket is not issued on a
+successful resumption and therefore B<gen_cb> will not be called. In TLSv1.3 the
+default behaviour is to always issue a new ticket on resumption. In both cases
+this behaviour can be changed if a ticket key callback is in use (see
+L<SSL_CTX_set_tlsext_ticket_key_cb(3)>).
+
+=head1 RETURN VALUES
+
+The SSL_CTX_set_session_ticket_cb(), SSL_SESSION_set1_ticket_appdata() and
+SSL_SESSION_get0_ticket_appdata() functions return 1 on success and 0 on
+failure.
+
+The B<gen_cb> callback must return 1 to continue the connection. A return of 0
+will terminate the connection with an INTERNAL_ERROR alert.
+
+The B<dec_cb> callback must return a value as described in L<NOTES> above.
+
 =head1 SEE ALSO
 
 L<ssl(7)>,