Correctly calculate the length of SM2 plaintext given the ciphertext

Previously the length of the SM2 plaintext could be incorrectly calculated.
The plaintext length was calculated by taking the ciphertext length and
taking off an "overhead" value.

The overhead value was assumed to have a "fixed" element of 10 bytes.
This is incorrect since in some circumstances it can be more than 10 bytes.
Additionally the overhead included the length of two integers C1x and C1y,
which were assumed to be the same length as the field size (32 bytes for
the SM2 curve). However in some cases these integers can have an additional
padding byte when the msb is set, to disambiguate them from negative
integers. Additionally the integers can also be less than 32 bytes in
length in some cases.

If the calculated overhead is incorrect and larger than the actual value
this can result in the calculated plaintext length being too small.
Applications are likely to allocate buffer sizes based on this and therefore
a buffer overrun can occur.

CVE-2021-3711

Issue reported by John Ouyang.

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>

2 files changed, 8 insertions, 17 deletions

diff --git a/crypto/sm2/sm2_crypt.c b/crypto/sm2/sm2_crypt.c
index ef505f6441..1188abfc6b 100644
--- a/crypto/sm2/sm2_crypt.c
+++ b/crypto/sm2/sm2_crypt.c
@@ -61,29 +61,20 @@ static size_t ec_field_size(const EC_GROUP *group)
     return field_size;
 }
 
-int sm2_plaintext_size(const EC_KEY *key, const EVP_MD *digest, size_t msg_len,
-                       size_t *pt_size)
+int sm2_plaintext_size(const unsigned char *ct, size_t ct_size, size_t *pt_size)
 {
-    const size_t field_size = ec_field_size(EC_KEY_get0_group(key));
-    const int md_size = EVP_MD_size(digest);
-    size_t overhead;
+    struct SM2_Ciphertext_st *sm2_ctext = NULL;
 
-    if (md_size < 0) {
-        SM2err(SM2_F_SM2_PLAINTEXT_SIZE, SM2_R_INVALID_DIGEST);
-        return 0;
-    }
-    if (field_size == 0) {
-        SM2err(SM2_F_SM2_PLAINTEXT_SIZE, SM2_R_INVALID_FIELD);
-        return 0;
-    }
+    sm2_ctext = d2i_SM2_Ciphertext(NULL, &ct, ct_size);
 
-    overhead = 10 + 2 * field_size + (size_t)md_size;
-    if (msg_len <= overhead) {
+    if (sm2_ctext == NULL) {
         SM2err(SM2_F_SM2_PLAINTEXT_SIZE, SM2_R_INVALID_ENCODING);
         return 0;
     }
 
-    *pt_size = msg_len - overhead;
+    *pt_size = sm2_ctext->C2->length;
+    SM2_Ciphertext_free(sm2_ctext);
+
     return 1;
 }
 
diff --git a/crypto/sm2/sm2_pmeth.c b/crypto/sm2/sm2_pmeth.c
index b42a14c32f..27025fbf3a 100644
--- a/crypto/sm2/sm2_pmeth.c
+++ b/crypto/sm2/sm2_pmeth.c
@@ -151,7 +151,7 @@ static int pkey_sm2_decrypt(EVP_PKEY_CTX *ctx,
     const EVP_MD *md = (dctx->md == NULL) ? EVP_sm3() : dctx->md;
 
     if (out == NULL) {
-        if (!sm2_plaintext_size(ec, md, inlen, outlen))
+        if (!sm2_plaintext_size(in, inlen, outlen))
             return -1;
         else
             return 1;