Update the EVP_PKEY_get_id documentation

The documentation didn't mention the development where EVP_PKEY_get_id()
returns a negative value for provider-only implementations, and the
migration guide didn't mention how to cope with that.

Fixes #20497

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/20501)

3 files changed, 33 insertions, 5 deletions

diff --git a/doc/man3/EVP_PKEY_set1_RSA.pod b/doc/man3/EVP_PKEY_set1_RSA.pod
index c0366d34fc..f2bdede46f 100644
--- a/doc/man3/EVP_PKEY_set1_RSA.pod
+++ b/doc/man3/EVP_PKEY_set1_RSA.pod
@@ -62,13 +62,16 @@ see L<openssl_user_macros(7)>:
 EVP_PKEY_get_base_id() returns the type of I<pkey>. For example
 an RSA key will return B<EVP_PKEY_RSA>.
 
-EVP_PKEY_get_id() returns the actual OID associated with I<pkey>.
-Historically keys using the same algorithm could use different OIDs.
-For example an RSA key could use the OIDs corresponding to
+EVP_PKEY_get_id() returns the actual NID associated with I<pkey>
+only if the I<pkey> type isn't implemented just in a L<provider(7)>.
+Historically keys using the same algorithm could use different NIDs.
+For example an RSA key could use the NIDs corresponding to
 the NIDs B<NID_rsaEncryption> (equivalent to B<EVP_PKEY_RSA>) or
 B<NID_rsa> (equivalent to B<EVP_PKEY_RSA2>). The use of
-alternative non-standard OIDs is now rare so B<EVP_PKEY_RSA2> et al are not
+alternative non-standard NIDs is now rare so B<EVP_PKEY_RSA2> et al are not
 often seen in practice.
+EVP_PKEY_get_id() returns -1 (B<EVP_PKEY_KEYMGMT>) if the I<pkey> is
+only implemented in a L<provider(7)>.
 
 EVP_PKEY_type() returns the underlying type of the NID I<type>. For example
 EVP_PKEY_type(EVP_PKEY_RSA2) will return B<EVP_PKEY_RSA>.
@@ -142,6 +145,9 @@ EVP_PKEY_get_id(), EVP_PKEY_get_base_id(), EVP_PKEY_type()
 
 For EVP_PKEY key type checking purposes, L<EVP_PKEY_is_a(3)> is more generic.
 
+For purposes of retrieving the name of the B<EVP_PKEY> the function
+L<EVP_PKEY_get0_type_name(3)> is more generally useful.
+
 The keys returned from the functions EVP_PKEY_get0_RSA(), EVP_PKEY_get0_DSA(),
 EVP_PKEY_get0_DH() and EVP_PKEY_get0_EC_KEY() were changed to have a "const"
 return type in OpenSSL 3.0. As described above the keys returned may be cached
diff --git a/doc/man7/migration_guide.pod b/doc/man7/migration_guide.pod
index bad1eab873..5294f57225 100644
--- a/doc/man7/migration_guide.pod
+++ b/doc/man7/migration_guide.pod
@@ -2242,6 +2242,28 @@ Use L<X509_load_http(3)> and L<X509_CRL_load_http(3)> instead.
 
 =back
 
+=head3 NID handling for provided keys and algorithms
+
+The following functions for NID (numeric id) handling have changed semantics.
+
+=over 4
+
+=item *
+
+EVP_PKEY_id(), EVP_PKEY_get_id()
+
+This function was previously used to reliably return the NID of
+an EVP_PKEY object, e.g., to look up the name of the algorithm of
+such EVP_PKEY by calling L<OBJ_nid2sn(3)>. With the introduction
+of L<provider(7)>s EVP_PKEY_id() or its new equivalent
+L<EVP_PKEY_get_id(3)> might now also return the value -1
+(B<EVP_PKEY_KEYMGMT>) indicating the use of a provider to
+implement the EVP_PKEY object. Therefore, the use of
+L<EVP_PKEY_get0_type_name(3)> is recommended for retrieving
+the name of the EVP_PKEY algorithm.
+
+=back
+
 =head2 Using the FIPS Module in applications
 
 See L<fips_module(7)> and L<OSSL_PROVIDER-FIPS(7)> for details.
diff --git a/ssl/t1_lib.c b/ssl/t1_lib.c
index 7ec8be4c27..e528467dd9 100644
--- a/ssl/t1_lib.c
+++ b/ssl/t1_lib.c
@@ -1855,7 +1855,7 @@ int tls12_check_peer_sigalg(SSL_CONNECTION *s, uint16_t sig, EVP_PKEY *pkey)
     }
     lu = tls1_lookup_sigalg(s, sig);
     /* if this sigalg is loaded, set so far unknown pkeyid to its sig NID */
-    if ((pkeyid == -1) && (lu != NULL))
+    if ((pkeyid == EVP_PKEY_KEYMGMT) && (lu != NULL))
         pkeyid = lu->sig;
 
     /* Should never happen */