Honor OSSL_PKEY_PARAM_EC_POINT_CONVERSION_FORMAT as set and default to UNCOMPRESSED

Originally the code to im/export the EC pubkey was meant to be consumed
only by the im/export functions when crossing the provider boundary.
Having our providers exporting to a COMPRESSED format octet string made
sense to avoid memory waste, as it wasn't exposed outside the provider
API, and providers had all tools available to convert across the three
formats.

Later on, with #13139 deprecating the `EC_KEY_*` functions, more state
was added among the params imported/exported on an EC provider-native
key (including `OSSL_PKEY_PARAM_EC_POINT_CONVERSION_FORMAT`, although it
did not affect the format used to export `OSSL_PKEY_PARAM_PUB_KEY`).

Finally, in #14800, `EVP_PKEY_todata()` was introduced and prominently
exposed directly to users outside the provider API, and the choice of
COMPRESSED over UNCOMPRESSED as the default became less sensible in
light of usability, given the latter is more often needed by
applications and protocols.

This commit fixes it, by using `EC_KEY_get_conv_form()` to get the
point format from the internal state (an `EC_KEY` under the hood) of the
provider-side object, and using it on
`EVP_PKEY_export()`/`EVP_PKEY_todata()` to format
`OSSL_PKEY_PARAM_PUB_KEY`.
The default for an `EC_KEY` was already UNCOMPRESSED, and it is altered
if the user sets `OSSL_PKEY_PARAM_EC_POINT_CONVERSION_FORMAT` via
`EVP_PKEY_fromdata()`, `EVP_PKEY_set_params()`, or one of the
more specialized methods.

For symmetry, this commit also alters `ec_pkey_export_to()` in
`crypto/ec/ec_ameth.c`, part of the `EVP_PKEY_ASN1_METHOD` for legacy EC
keys: it exclusively used COMPRESSED format, and now it honors the
conversion format specified in the EC_KEY object being exported to a
provider when this function is called.

Expand documentation about `OSSL_PKEY_PARAM_PUB_KEY` and mention the
change in behavior for our providers.

Fixes #16595

(cherry picked from commit 926db476bc669fdcc4c4d2f1cb547060bdbfa153)

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19901)

1 files changed, 17 insertions, 7 deletions

diff --git a/doc/man7/EVP_PKEY-EC.pod b/doc/man7/EVP_PKEY-EC.pod
index d9b1666b2e..668a024014 100644
--- a/doc/man7/EVP_PKEY-EC.pod
+++ b/doc/man7/EVP_PKEY-EC.pod
@@ -79,6 +79,10 @@ Enable Cofactor DH (ECC CDH) if this value is 1, otherwise it uses normal EC DH
 if the value is zero. The cofactor variant multiplies the shared secret by the
 EC curve's cofactor (note for some curves the cofactor is 1).
 
+See also L<EVP_KEYEXCH-ECDH(7)> for the related
+B<OSSL_EXCHANGE_PARAM_EC_ECDH_COFACTOR_MODE> parameter that can be set on a
+per-operation basis.
+
 =item "encoding" (B<OSSL_PKEY_PARAM_EC_ENCODING>) <UTF8 string>
 
 Set the format used for serializing the EC group parameters.
@@ -104,15 +108,21 @@ but is equivalent to "named-nist" for the OpenSSL FIPS provider.
 Setting this value to 0 indicates that the public key should not be included when
 encoding the private key. The default value of 1 will include the public key.
 
-See also L<EVP_KEYEXCH-ECDH(7)> for the related
-B<OSSL_EXCHANGE_PARAM_EC_ECDH_COFACTOR_MODE> parameter that can be set on a
-per-operation basis.
-
 =item "pub" (B<OSSL_PKEY_PARAM_PUB_KEY>) <octet string>
 
-The public key value in encoded EC point format. This parameter is used
-when importing or exporting the public key value with the EVP_PKEY_fromdata()
-and EVP_PKEY_todata() functions.
+The public key value in encoded EC point format conforming to Sec. 2.3.3 and
+2.3.4 of the SECG SEC 1 ("Elliptic Curve Cryptography") standard.
+This parameter is used when importing or exporting the public key value with the
+EVP_PKEY_fromdata() and EVP_PKEY_todata() functions.
+
+Note, in particular, that the choice of point compression format used for
+encoding the exported value via EVP_PKEY_todata() depends on the underlying
+provider implementation.
+Before OpenSSL 3.0.8, the implementation of providers included with OpenSSL always
+opted for an encoding in compressed format, unconditionally.
+Since OpenSSL 3.0.8, the implementation has been changed to honor the
+B<OSSL_PKEY_PARAM_EC_POINT_CONVERSION_FORMAT> parameter, if set, or to default
+to uncompressed format.
 
 =item "priv" (B<OSSL_PKEY_PARAM_PRIV_KEY>) <unsigned integer>