Fix use of OPENSSL_realloc in provider

Fix OPENSSL_realloc failure case; `provider->operation_bits` memory
is lost when `OPENSSL_realloc()` returns NULL.

`operation_bits_sz` is never set to the length of the allocated array.
This means that operation_bits is always reallocated in
`ossl_provider_set_operation_bit()`, possibly shrinking the array.
In addition, it means that the `memset()` always zeros out the
whole reallocated array, not just the new part. Also, because
`operation_bits_sz` is always zero, the value of `*result` in
`ossl_provider_test_operation_bit()` will always be zero.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/12760)

1 files changed, 6 insertions, 3 deletions

diff --git a/crypto/provider_core.c b/crypto/provider_core.c
index a714a71681..f282071e2d 100644
--- a/crypto/provider_core.c
+++ b/crypto/provider_core.c
@@ -875,14 +875,17 @@ int ossl_provider_set_operation_bit(OSSL_PROVIDER *provider, size_t bitnum)
     unsigned char bit = (1 << (bitnum % 8)) & 0xFF;
 
     if (provider->operation_bits_sz <= byte) {
-        provider->operation_bits = OPENSSL_realloc(provider->operation_bits,
-                                                   byte + 1);
-        if (provider->operation_bits == NULL) {
+        unsigned char *tmp = OPENSSL_realloc(provider->operation_bits,
+                                             byte + 1);
+
+        if (tmp == NULL) {
             ERR_raise(ERR_LIB_CRYPTO, ERR_R_MALLOC_FAILURE);
             return 0;
         }
+        provider->operation_bits = tmp;
         memset(provider->operation_bits + provider->operation_bits_sz,
                '\0', byte + 1 - provider->operation_bits_sz);
+        provider->operation_bits_sz = byte + 1;
     }
     provider->operation_bits[byte] |= bit;
     return 1;