Admit unknown pkey types at security level 0

The check_key_level() function currently fails when the public key
cannot be extracted from the certificate because its algorithm is not
supported.  However, the public key is not needed for the last
certificate in the chain.

This change moves the check for level 0 before the check for a
non-NULL public key.

For background, this is the TPM 1.2 endorsement key certificate.
I.e., this is a real application with millions of certificates issued.
The key is an RSA-2048 key.

The TCG (for a while) specified

     Public Key Algorithm: rsaesOaep

rather than the commonly used

     Public Key Algorithm: rsaEncryption

because the key is an encryption key rather than a signing key.
The X509 certificate parser fails to get the public key.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7906)

1 files changed, 9 insertions, 2 deletions

diff --git a/crypto/x509/x509_vfy.c b/crypto/x509/x509_vfy.c
index 61e81922b4..4ced716e36 100644
--- a/crypto/x509/x509_vfy.c
+++ b/crypto/x509/x509_vfy.c
@@ -3232,12 +3232,19 @@ static int check_key_level(X509_STORE_CTX *ctx, X509 *cert)
     EVP_PKEY *pkey = X509_get0_pubkey(cert);
     int level = ctx->param->auth_level;
 
+    /*
+     * At security level zero, return without checking for a supported public
+     * key type.  Some engines support key types not understood outside the
+     * engine, and we only need to understand the key when enforcing a security
+     * floor.
+     */
+    if (level <= 0)
+        return 1;
+
     /* Unsupported or malformed keys are not secure */
     if (pkey == NULL)
         return 0;
 
-    if (level <= 0)
-        return 1;
     if (level > NUM_AUTH_LEVELS)
         level = NUM_AUTH_LEVELS;