Allow to continue on UNABLE_TO_VERIFY_LEAF_SIGNATURE

This unifies the behaviour of a single certificate with
an unknown CA certificate with a self-signed certificate.
The user callback can mask that error to retrieve additional
error information. So the user application can decide to
abort the connection instead to be forced by openssl.

This change in behaviour is backward compatible as user callbacks
who don't want to ignore UNABLE_TO_VERIFY_LEAF_SIGNATURE will
still abort the connection by default.

CLA: trivial
Fixes #11297

Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11359)

1 files changed, 9 insertions, 3 deletions

diff --git a/crypto/x509/x509_vfy.c b/crypto/x509/x509_vfy.c
index 801055f5a0..ffa8d637ff 100644
--- a/crypto/x509/x509_vfy.c
+++ b/crypto/x509/x509_vfy.c
@@ -1750,9 +1750,15 @@ static int internal_verify(X509_STORE_CTX *ctx)
             xs = xi;
             goto check_cert;
         }
-        if (n <= 0)
-            return verify_cb_cert(ctx, xi, 0,
-                                  X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE);
+        if (n <= 0) {
+            if (!verify_cb_cert(ctx, xi, 0,
+                                X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE))
+                return 0;
+
+            xs = xi;
+            goto check_cert;
+        }
+
         n--;
         ctx->error_depth = n;
         xs = sk_X509_value(ctx->chain, n);