rsa: add implicit rejection in PKCS#1 v1.5

The RSA decryption as implemented before required very careful handling
of both the exit code returned by OpenSSL and the potentially returned
ciphertext. Looking at the recent security vulnerabilities
(CVE-2020-25659 and CVE-2020-25657) it is unlikely that most users of
OpenSSL do it correctly.

Given that correct code requires side channel secure programming in
application code, we can classify the existing RSA decryption methods
as CWE-676, which in turn likely causes CWE-208 and CWE-385 in
application code.

To prevent that, we can use a technique called "implicit rejection".
For that we generate a random message to be returned in case the
padding check fails. We generate the message based on static secret
data (the private exponent) and the provided ciphertext (so that the
attacker cannot determine that the returned value is randomly generated
instead of result of decryption and de-padding). We return it in case
any part of padding check fails.

The upshot of this approach is that then not only is the length of the
returned message useless as the Bleichenbacher oracle, so are the
actual bytes of the returned message. So application code doesn't have
to perform any operations on the returned message in side-channel free
way to remain secure against Bleichenbacher attacks.

Note: this patch implements a specific algorithm, shared with Mozilla
NSS, so that the attacker cannot use one library as an oracle against the
other in heterogeneous environments.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13817)

2 files changed, 10 insertions, 0 deletions

diff --git a/doc/man1/openssl-pkeyutl.pod.in b/doc/man1/openssl-pkeyutl.pod.in
index b0054ead66..b7c45caa23 100644
--- a/doc/man1/openssl-pkeyutl.pod.in
+++ b/doc/man1/openssl-pkeyutl.pod.in
@@ -240,6 +240,11 @@ signed or verified directly instead of using a B<DigestInfo> structure. If a
 digest is set then the a B<DigestInfo> structure is used and its the length
 must correspond to the digest type.
 
+Note, for B<pkcs1> padding, as a protection against Bleichenbacher attack,
+the decryption will not fail in case of padding check failures. Use B<none>
+and manual inspection of the decrypted message to verify if the decrypted
+value has correct PKCS#1 v1.5 padding.
+
 For B<oaep> mode only encryption and decryption is supported.
 
 For B<x931> if the digest type is set it is used to format the block data
diff --git a/doc/man1/openssl-rsautl.pod.in b/doc/man1/openssl-rsautl.pod.in
index 186e49e5e4..eab34979de 100644
--- a/doc/man1/openssl-rsautl.pod.in
+++ b/doc/man1/openssl-rsautl.pod.in
@@ -105,6 +105,11 @@ The padding to use: PKCS#1 v1.5 (the default), PKCS#1 OAEP,
 ANSI X9.31, or no padding, respectively.
 For signatures, only B<-pkcs> and B<-raw> can be used.
 
+Note: because of protection against Bleichenbacher attacks, decryption
+using PKCS#1 v1.5 mode will not return errors in case padding check failed.
+Use B<-raw> and inspect the returned value manually to check if the
+padding is correct.
+
 =item B<-hexdump>
 
 Hex dump the output data.