Avoid errors with a priori inapplicable protocol bounds

The 'MinProtocol' and 'MaxProtocol' configuration commands now silently
ignore TLS protocol version bounds when configurign DTLS-based contexts,
and conversely, silently ignore DTLS protocol version bounds when
configuring TLS-based contexts.  The commands can be repeated to set
bounds of both types.  The same applies with the corresponding
"min_protocol" and "max_protocol" command-line switches, in case some
application uses both TLS and DTLS.

SSL_CTX instances that are created for a fixed protocol version (e.g.
TLSv1_server_method()) also silently ignore version bounds.  Previously
attempts to apply bounds to these protocol versions would result in an
error.  Now only the "version-flexible" SSL_CTX instances are subject to
limits in configuration files in command-line options.

Expected to resolve #12394

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>

GH: #12507

2 files changed, 29 insertions, 15 deletions

diff --git a/doc/man3/SSL_CONF_cmd.pod b/doc/man3/SSL_CONF_cmd.pod
index 7f2449e379..c5fed8e1e0 100644
--- a/doc/man3/SSL_CONF_cmd.pod
+++ b/doc/man3/SSL_CONF_cmd.pod
@@ -147,13 +147,16 @@ B<SSL_OP_NO_RENEGOTIATION>.
 =item B<-min_protocol>, B<-max_protocol>
 
 Sets the minimum and maximum supported protocol.
-Currently supported protocol values are B<SSLv3>, B<TLSv1>,
-B<TLSv1.1>, B<TLSv1.2>, B<TLSv1.3> for TLS and B<DTLSv1>, B<DTLSv1.2> for DTLS,
-and B<None> for no limit.
-If either bound is not specified then only the other bound applies,
-if specified.
-To restrict the supported protocol versions use these commands rather
-than the deprecated alternative commands below.
+Currently supported protocol values are B<SSLv3>, B<TLSv1>, B<TLSv1.1>,
+B<TLSv1.2>, B<TLSv1.3> for TLS; B<DTLSv1>, B<DTLSv1.2> for DTLS, and B<None>
+for no limit.
+If either the lower or upper bound is not specified then only the other bound
+applies, if specified.
+If your application supports both TLS and DTLS you can specify any of these
+options twice, once with a bound for TLS and again with an appropriate bound
+for DTLS.
+To restrict the supported protocol versions use these commands rather than the
+deprecated alternative commands below.
 
 =item B<-no_ssl3>, B<-no_tls1>, B<-no_tls1_1>, B<-no_tls1_2>, B<-no_tls1_3>
 
@@ -370,7 +373,11 @@ This sets the minimum supported SSL, TLS or DTLS version.
 
 Currently supported protocol values are B<SSLv3>, B<TLSv1>, B<TLSv1.1>,
 B<TLSv1.2>, B<TLSv1.3>, B<DTLSv1> and B<DTLSv1.2>.
-The value B<None> will disable the limit.
+The SSL and TLS bounds apply only to TLS-based contexts, while the DTLS bounds
+apply only to DTLS-based contexts.
+The command can be repeated with one instance setting a TLS bound, and the
+other setting a DTLS bound.
+The value B<None> applies to both types of contexts and disables the limits.
 
 =item B<MaxProtocol>
 
@@ -378,7 +385,11 @@ This sets the maximum supported SSL, TLS or DTLS version.
 
 Currently supported protocol values are B<SSLv3>, B<TLSv1>, B<TLSv1.1>,
 B<TLSv1.2>, B<TLSv1.3>, B<DTLSv1> and B<DTLSv1.2>.
-The value B<None> will disable the limit.
+The SSL and TLS bounds apply only to TLS-based contexts, while the DTLS bounds
+apply only to DTLS-based contexts.
+The command can be repeated with one instance setting a TLS bound, and the
+other setting a DTLS bound.
+The value B<None> applies to both types of contexts and disables the limits.
 
 =item B<Protocol>
 
diff --git a/doc/man5/config.pod b/doc/man5/config.pod
index 7b50b09919..7a0459d993 100644
--- a/doc/man5/config.pod
+++ b/doc/man5/config.pod
@@ -262,13 +262,11 @@ Example of a configuration with the system default:
  ssl_conf = ssl_sect
 
  [ssl_sect]
-
  system_default = system_default_sect
 
  [system_default_sect]
-
  MinProtocol = TLSv1.2
-
+ MinProtocol = DTLSv1.2
 
 =head1 NOTES
 
@@ -355,8 +353,8 @@ Simple OpenSSL library configuration example to enter FIPS mode:
 Note: in the above example you will get an error in non FIPS capable versions
 of OpenSSL.
 
-Simple OpenSSL library configuration to make TLS 1.3 the system-default
-minimum TLS version:
+Simple OpenSSL library configuration to make TLS 1.2 and DTLS 1.2 the
+system-default minimum TLS and DTLS versions, respectively:
 
  # Toplevel section for openssl (including libssl)
  openssl_conf = default_conf_section
@@ -369,7 +367,12 @@ minimum TLS version:
  system_default = system_default_section
 
  [system_default_section]
- MinProtocol = TLSv1.3
+ MinProtocol = TLSv1.2
+ MinProtocol = DTLSv1.2
+
+The minimum TLS protocol is applied to B<SSL_CTX> objects that are TLS-based,
+and the minimum DTLS protocol to those are DTLS-based.
+The same applies also to maximum versions set with B<MaxProtocol>.
 
 More complex OpenSSL library configuration. Add OID and don't enter FIPS mode: