EVP: add EVP_PKEY_is_a() and EVP_PKEY_can_sign()

EVP_PKEY_is_a() is the provider side key checking function corresponding
to checking EVP_PKEY_id() or an EVP_PKEY against macros like EVP_PKEY_EC.
It also works with legacy internal keys.

We also add a warning indoc/man3/EVP_PKEY_set1_RSA.pod regarding the
reliability of certain functions that only understand legacy keys.

Finally, we take the opportunity to clean up doc/man3/EVP_PKEY_set1_RSA.pod
to better conform with man-page layout norms, see man-pages(7) on Linux.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11358)

1 files changed, 72 insertions, 0 deletions

diff --git a/doc/man3/EVP_PKEY_is_a.pod b/doc/man3/EVP_PKEY_is_a.pod
new file mode 100644
index 0000000000..cfce3de5da
--- /dev/null
+++ b/doc/man3/EVP_PKEY_is_a.pod
@@ -0,0 +1,72 @@
+=pod
+
+=head1 NAME
+
+EVP_PKEY_is_a, EVP_PKEY_can_sign
+- key type and capabilities functions
+
+=head1 SYNOPSIS
+
+ #include <openssl/evp.h>
+
+ int EVP_PKEY_is_a(const EVP_PKEY *pkey, const char *name);
+ int EVP_PKEY_can_sign(const EVP_PKEY *pkey);
+
+=head1 DESCRIPTION
+
+EVP_PKEY_is_a() checks if the key type of I<pkey> is I<name>.
+
+EVP_PKEY_can_sign() checks if the functionality for the key type of
+I<pkey> supports signing.  No other check is done, such as whether
+I<pkey> contains a private key.
+
+=head1 RETURN VALUES
+
+EVP_PKEY_is_a() returns 1 if I<pkey> has the key type I<name>,
+otherwise 0.
+
+EVP_PKEY_can_sign() returns 1 if the I<pkey> key type functionality
+supports signing, otherwise 0.
+
+=head1 EXAMPLES
+
+=head2 EVP_PKEY_is_a()
+
+The loaded providers and what key types they support will ultimately
+determine what I<name> is possible to use with EVP_PKEY_is_a().  We do know
+that the default provider supports RSA, DH, DSA and EC keys, so we can use
+this as an crude example:
+
+ #include <openssl/evp.h>
+
+ ...
+     /* |pkey| is an EVP_PKEY* */
+     if (EVP_PKEY_is_a(pkey, "RSA")) {
+         BIGNUM *modulus = NULL;
+         if (EVP_PKEY_get_bn_param(pkey, "n", &modulus))
+             /* do whatever with the modulus */
+         BN_free(modulus);
+     }
+
+=head2 EVP_PKEY_can_sign()
+
+ #include <openssl/evp.h>
+
+ ...
+     /* |pkey| is an EVP_PKEY* */
+     if (!EVP_PKEY_can_sign(pkey)) {
+         fprintf(stderr, "Not a signing key!");
+         exit(1);
+     }
+     /* Sign something... */
+
+=head1 COPYRIGHT
+
+Copyright 2020 The OpenSSL Project Authors. All Rights Reserved.
+
+Licensed under the Apache License 2.0 (the "License").  You may not use
+this file except in compliance with the License.  You can obtain a copy
+in the file LICENSE in the source distribution or at
+L<https://www.openssl.org/source/license.html>.
+
+=cut