SERIALIZER: add support for serializing EVP_PKEYs

The following public functions is added:

- OSSL_SERIALIZER_CTX_new_by_EVP_PKEY()
- OSSL_SERIALIZER_CTX_set_cipher()
- OSSL_SERIALIZER_CTX_set_passphrase()
- OSSL_SERIALIZER_CTX_set_passphrase_cb()
- OSSL_SERIALIZER_CTX_set_passphrase_ui()

OSSL_SERIALIZER_CTX_new_by_EVP_PKEY() selects a suitable serializer
for the given EVP_PKEY, and sets up the OSSL_SERIALIZER_CTX to
function together with OSSL_SERIALIZER_to_bio() and
OSSL_SERIALIZER_to_fp().

OSSL_SERIALIZER_CTX_set_cipher() indicates what cipher should be used
to produce an encrypted serialization of the EVP_PKEY.  This is passed
directly to the provider using OSSL_SERIALIZER_CTX_set_params().

OSSL_SERIALIZER_CTX_set_passphrase() can be used to set a pass phrase
to be used for the encryption.  This is passed directly to the
provider using OSSL_SERIALIZER_CTX_set_params().

OSSL_SERIALIZER_CTX_set_passphrase_cb() and
OSSL_SERIALIZER_CTX_set_passphrase_ui() sets up a callback to be used
to prompt for a passphrase.  This is stored in the context, and is
called via an internal intermediary at the time of serialization.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10394)

3 files changed, 184 insertions, 1 deletions

diff --git a/doc/man3/OSSL_SERIALIZER.pod b/doc/man3/OSSL_SERIALIZER.pod
index bf6ef3431c..05b889bf60 100644
--- a/doc/man3/OSSL_SERIALIZER.pod
+++ b/doc/man3/OSSL_SERIALIZER.pod
@@ -111,7 +111,7 @@ OSSL_SERIALIZER_number() returns an integer.
 =head1 SEE ALSO
 
 L<provider(7)>, L<OSSL_SERIALIZER_CTX(3)>, L<OSSL_SERIALIZER_to_bio(3)>,
-L<OPENSSL_CTX(3)>
+L<OSSL_SERIALIZER_CTX_new_by_EVP_PKEY(3)>, L<OPENSSL_CTX(3)>
 
 =head1 HISTORY
 
diff --git a/doc/man3/OSSL_SERIALIZER_CTX_new_by_EVP_PKEY.pod b/doc/man3/OSSL_SERIALIZER_CTX_new_by_EVP_PKEY.pod
new file mode 100644
index 0000000000..caa9294bcc
--- /dev/null
+++ b/doc/man3/OSSL_SERIALIZER_CTX_new_by_EVP_PKEY.pod
@@ -0,0 +1,134 @@
+=pod
+
+=head1 NAME
+
+OSSL_SERIALIZER_CTX_new_by_EVP_PKEY,
+OSSL_SERIALIZER_CTX_set_cipher,
+OSSL_SERIALIZER_CTX_set_passphrase,
+OSSL_SERIALIZER_CTX_set_passphrase_cb,
+OSSL_SERIALIZER_CTX_set_passphrase_ui,
+OSSL_SERIALIZER_PUBKEY_TO_PEM_PQ,
+OSSL_SERIALIZER_PrivateKey_TO_PEM_PQ,
+OSSL_SERIALIZER_Parameters_TO_PEM_PQ,
+OSSL_SERIALIZER_PUBKEY_TO_TEXT_PQ,
+OSSL_SERIALIZER_PrivateKey_TO_TEXT_PQ,
+OSSL_SERIALIZER_Parameters_TO_TEXT_PQ
+- Serializer routines to serialize EVP_PKEYs
+
+=head1 SYNOPSIS
+
+ #include <openssl/serializer.h>
+
+ OSSL_SERIALIZER_CTX *OSSL_SERIALIZER_CTX_new_by_EVP_PKEY(const EVP_PKEY *pkey,
+                                                          const char *propquery);
+
+ int OSSL_SERIALIZER_CTX_set_cipher(OSSL_SERIALIZER_CTX *ctx,
+                                    const char *cipher_name,
+                                    const char *propquery);
+ int OSSL_SERIALIZER_CTX_set_passphrase(OSSL_SERIALIZER_CTX *ctx,
+                                        const unsigned char *kstr,
+                                        size_t klen);
+ int OSSL_SERIALIZER_CTX_set_passphrase_cb(OSSL_SERIALIZER_CTX *ctx, int enc,
+                                           pem_password_cb *cb, void *cbarg);
+ int OSSL_SERIALIZER_CTX_set_passphrase_ui(OSSL_SERIALIZER_CTX *ctx,
+                                           const UI_METHOD *ui_method,
+                                           void *ui_data);
+
+ #define OSSL_SERIALIZER_PUBKEY_TO_PEM_PQ "format=pem,type=public"
+ #define OSSL_SERIALIZER_PrivateKey_TO_PEM_PQ "format=pem,type=private"
+ #define OSSL_SERIALIZER_Parameters_TO_PEM_PQ "format=pem,type=domainparams"
+
+ #define OSSL_SERIALIZER_PUBKEY_TO_TEXT_PQ "format=text,type=public"
+ #define OSSL_SERIALIZER_PrivateKey_TO_TEXT_PQ "format=text,type=private"
+ #define OSSL_SERIALIZER_Parameters_TO_TEXT_PQ "format=text,type=domainparams"
+
+=head1 DESCRIPTION
+
+OSSL_SERIALIZER_CTX_new_by_EVP_PKEY() creates a B<OSSL_SERIALIZER_CTX>
+with a suitable attached output routine for B<EVP_PKEY>s.  It will
+search for a serializer implementation that matches the algorithm of
+the B<EVP_PKEY> and the property query given with I<propquery>.  It
+will prefer to find a serializer from the same provider as the key
+data of the B<EVP_PKEY> itself, but failing that, it will choose the
+first serializer that supplies a generic serializing function.
+
+If no suitable serializer was found, OSSL_SERIALIZER_CTX_new_by_EVP_PKEY()
+still creates a B<OSSL_SERIALIZER_CTX>, but with no associated
+serializer (L<OSSL_SERIALIZER_CTX_get_serializer(3)> returns NULL).
+This helps the caller distinguish between an error when creating
+the B<OSSL_SERIALIZER_CTX>, and the lack the serializer support and
+act accordingly.
+
+OSSL_SERIALIZER_CTX_set_cipher() tells the implementation what cipher
+should be used to encrypt serialized keys.  The cipher is given by
+name I<cipher_name>.  The interpretation of that I<cipher_name> is
+implementation dependent.  The implementation may implement the digest
+directly itself or by other implementations, or it may choose to fetch
+it.  If the implementation supports fetching the cipher, then it may
+use I<propquery> as properties to be queried for when fetching.
+I<cipher_name> may also be NULL, which will result in unencrypted
+serialization.
+
+OSSL_SERIALIZER_CTX_set_passphrase() gives the implementation a
+pass phrase to use when encrypting the serialized private key.
+Alternatively, a pass phrase callback may be specified with the
+following functions.
+
+OSSL_SERIALIZER_CTX_set_passphrase_cb() and
+OSSL_SERIALIZER_CTX_set_passphrase_ui() sets up a callback method that
+the implementation can use to prompt for a pass phrase.
+
+=for comment Note that the callback method is called indirectly,
+through an internal B<OSSL_PASSPHRASE_CALLBACK> function.
+
+The macros B<OSSL_SERIALIZER_PUBKEY_TO_PEM_PQ>,
+B<OSSL_SERIALIZER_PrivateKey_TO_PEM_PQ>,
+B<OSSL_SERIALIZER_Parameters_TO_PEM_PQ>,
+B<OSSL_SERIALIZER_PUBKEY_TO_TEXT_PQ>,
+B<OSSL_SERIALIZER_PrivateKey_TO_TEXT_PQ>,
+B<OSSL_SERIALIZER_Parameters_TO_TEXT_PQ> are convenience macros with
+property queries to serialize the B<EVP_PKEY> as a public key, private
+key or parameters to B<PEM>, or to text.
+
+=head1 RETURN VALUES
+
+OSSL_SERIALIZER_CTX_new_by_EVP_PKEY() returns a pointer to a
+B<OSSL_SERIALIZER_CTX>, or NULL if it couldn't be created.
+
+OSSL_SERIALIZER_CTX_set_cipher(),
+OSSL_SERIALIZER_CTX_set_passphrase(),
+OSSL_SERIALIZER_CTX_set_passphrase_cb(), and
+OSSL_SERIALIZER_CTX_set_passphrase_ui() all return 1 on success, or 0
+on failure.
+
+=head1 NOTES
+
+Parts of the function and macro names are made to match already
+existing OpenSSL names.
+
+B<EVP_PKEY> in OSSL_SERIALIZER_CTX_new_by_EVP_PKEY() matches the type
+name, thus making for the naming pattern
+B<OSSL_SERIALIZER_CTX_new_by_I<TYPE>>() when new types are handled.
+
+B<PUBKEY>, B<PrivateKey> and B<Parameters> in the macro names match
+the B<I<TYPE>> part of of B<PEM_write_bio_I<TYPE>> functions as well
+as B<i2d_I<TYPE>_bio> functions.
+
+=head1 SEE ALSO
+
+L<provider(7)>, L<OSSL_SERIALIZER(3)>, L<OSSL_SERIALIZER_CTX(3)>
+
+=head1 HISTORY
+
+The functions described here were added in OpenSSL 3.0.
+
+=head1 COPYRIGHT
+
+Copyright 2019 The OpenSSL Project Authors. All Rights Reserved.
+
+Licensed under the Apache License 2.0 (the "License").  You may not use
+this file except in compliance with the License.  You can obtain a copy
+in the file LICENSE in the source distribution or at
+L<https://www.openssl.org/source/license.html>.
+
+=cut
diff --git a/doc/man7/provider-serializer.pod b/doc/man7/provider-serializer.pod
index e43e293d60..b23a8b64e2 100644
--- a/doc/man7/provider-serializer.pod
+++ b/doc/man7/provider-serializer.pod
@@ -193,6 +193,55 @@ Both serialization functions also take an B<OSSL_PASSPHRASE_CALLBACK>
 function pointer along with a pointer to application data I<cbarg>,
 which should be used when a pass phrase prompt is needed.
 
+=head2 Serializer parameters
+
+Parameters currently recognised by built-in serializers are as
+follows:
+
+=over 4
+
+=item "cipher" (B<OSSL_SERIALIZER_PARAM_CIPHER>) <UTF8 string>
+
+The name of the encryption cipher to be used when generating encrypted
+serialization.  This is used when serializing private keys, as well as
+other objects that need protection.
+
+If this name is invalid for the serialization implementation, the
+implementation should refuse to perform the serialization, i.e.
+OP_serializer_serialize_data() and OP_serializer_serialize_object()
+should return an error.
+
+=item "properties" (B<OSSL_SERIALIZER_PARAM_PROPERTIES>) <UTF8 string>
+
+The properties to be queried when trying to fetch the algorithm given
+with the "cipher" parameter.
+This must be given together with the "cipher" parameter to be
+considered valid.
+
+The serialization implementation isn't obligated to use this value.
+However, it is recommended that implementations that do not handle
+property strings return an error on receiving this parameter unless
+its value NULL or the empty string.
+
+=item "passphrase" (B<OSSL_SERIALIZER_PARAM_PASS>) <octet string>
+
+A pass phrase provided by the application.  When this is given, the
+built-in serializers will not attempt to use the passphrase callback.
+
+=back
+
+Parameters currently recognised by the built-in pass phrase callback:
+
+=over 4
+
+=item "info" (B<OSSL_PASSPHRASE_PARAM_INFO>) <UTF8 string>
+
+A string of information that will become part of the pass phrase
+prompt.  This could be used to give the user information on what kind
+of object it's being prompted for.
+
+=back
+
 =head1 RETURN VALUES
 
 OP_serializer_newctx() returns a pointer to a context, or NULL on