Redesign the KEYMGMT libcrypto <-> provider interface - the basics

The KEYMGMT libcrypto <-> provider interface currently makes a few
assumptions:

1.  provider side domain parameters and key data isn't mutable. In
    other words, as soon as a key has been created in any (loaded,
    imported data, ...), it's set in stone.
2.  provider side domain parameters can be strictly separated from the
    key data.

This does work for the most part, but there are places where that's a
bit too rigid for the functionality that the EVP_PKEY API delivers.
Key data needs to be mutable to allow the flexibility that functions
like EVP_PKEY_copy_parameters promise, as well as to provide the
combinations of data that an EVP_PKEY is generally assumed to be able
to hold:

- domain parameters only
- public key only
- public key + private key
- domain parameters + public key
- domain parameters + public key + private key

To remedy all this, we:

1.  let go of the distinction between domain parameters and key
    material proper in the libcrypto <-> provider interface.

    As a consequence, functions that still need it gain a selection
    argument, which is a set of bits that indicate what parts of the
    key object are to be considered in a specific call.  This allows
    a reduction of very similar functions into one.

2.  Rework the libcrypto <-> provider interface so provider side key
    objects are created and destructed with a separate function, and
    get their data filled and extracted in through import and export.

(future work will see other key object constructors and other
functions to fill them with data)

Fixes #10979

squash! Redesign the KEYMGMT libcrypto <-> provider interface - the basics

Remedy 1 needs a rewrite:

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11006)

2 files changed, 163 insertions, 138 deletions

diff --git a/doc/man7/provider-keymgmt.pod b/doc/man7/provider-keymgmt.pod
index 94427de89f..e9e4de5622 100644
--- a/doc/man7/provider-keymgmt.pod
+++ b/doc/man7/provider-keymgmt.pod
@@ -14,47 +14,29 @@ provider-keymgmt - The KEYMGMT library E<lt>-E<gt> provider functions
   * pointers in OSSL_DISPATCH arrays.
   */
 
- /* Key domain parameter creation and destruction */
- void *OP_keymgmt_importdomparams(void *provctx, const OSSL_PARAM params[]);
- void *OP_keymgmt_gendomparams(void *provctx, const OSSL_PARAM params[]);
- void OP_keymgmt_freedomparams(void *domparams);
+ /* Key object (keydata) creation and destruction */
+ void *OP_keymgmt_new(void *provctx);
+ void OP_keymgmt_free(void *keydata);
 
- /* Key domain parameter export */
- int OP_keymgmt_exportdomparams(void *domparams, OSSL_PARAM params[]);
+ /* Key object information */
+ int OP_keymgmt_get_params(void *keydata, OSSL_PARAM params[]);
+ const OSSL_PARAM *OP_keymgmt_gettable_params(void);
 
- /* Key domain parameter discovery */
- const OSSL_PARAM *OP_keymgmt_importdomparam_types(void);
- const OSSL_PARAM *OP_keymgmt_exportdomparam_types(void);
+ /* Key object content checks */
+ int OP_keymgmt_has(void *keydata, int selection);
 
- /* Key domain parameter information */
- int OP_keymgmt_get_domparam_params(void *domparams, OSSL_PARAM params[]);
- const OSSL_PARAM *OP_keymgmt_gettable_domparam_params(void);
-
- /* Key domain parameter validation */
- int OP_keymgmt_validate_domparams(void *key);
-
- /* Key creation and destruction */
- void *OP_keymgmt_importkey(void *provctx, const OSSL_PARAM params[]);
- void *OP_keymgmt_genkey(void *provctx,
-                         void *domparams, const OSSL_PARAM genkeyparams[]);
- void *OP_keymgmt_loadkey(void *provctx, void *id, size_t idlen);
- void OP_keymgmt_freekey(void *key);
-
- /* Key export */
- int OP_keymgmt_exportkey(void *key, OSSL_PARAM params[]);
-
- /* Key discovery */
- const OSSL_PARAM *OP_keymgmt_importkey_types(void);
- const OSSL_PARAM *OP_keymgmt_exportkey_types(void);
+ /* Discovery of supported operations */
+ const char *OP_keymgmt_query_operation_name(int operation_id);
 
- /* Key information */
- int OP_keymgmt_get_key_params(void *key, OSSL_PARAM params[]);
- const OSSL_PARAM *OP_keymgmt_gettable_key_params(void);
+ /* Key object import and export functions */
+ int OP_keymgmt_import(int selection, void *keydata, const OSSL_PARAM params[]);
+ const OSSL_PARAM *OP_keymgmt_import_types, (int selection);
+ int OP_keymgmt_export(int selection, void *keydata,
+                       OSSL_CALLBACK *param_cb, void *cbarg);
+ const OSSL_PARAM *OP_keymgmt_export_types(int selection);
 
- /* Key validation */
- int OP_keymgmt_validate_public(void *key);
- int OP_keymgmt_validate_private(void *key);
- int OP_keymgmt_validate_pairwise(void *key);
+ /* Key object validation */
+ int OP_keymgmt_validate(void *keydata, int selection);
 
  /* Discovery of supported operations */
  const char *OP_keymgmt_query_operation_name(int operation_id);
@@ -70,8 +52,7 @@ works with in tandem, they must belong to the same provider.
 The OpenSSL libraries will ensure that they do.
 
 The primary responsibility of the KEYMGMT operation is to hold the
-provider side domain parameters and keys for the OpenSSL library
-EVP_PKEY structure.
+provider side key data for the OpenSSL library EVP_PKEY structure.
 
 All "functions" mentioned here are passed as function pointers between
 F<libcrypto> and the provider in B<OSSL_DISPATCH> arrays via
@@ -83,133 +64,147 @@ All these "functions" have a corresponding function type definition
 named B<OSSL_{name}_fn>, and a helper function to retrieve the
 function pointer from a B<OSSL_DISPATCH> element named
 B<OSSL_get_{name}>.
-For example, the "function" OP_keymgmt_importdomparams() has these:
+For example, the "function" OP_keymgmt_new() has these:
 
- typedef void *
-     (OSSL_OP_keymgmt_importdomparams_fn)(void *provctx,
-                                          const OSSL_PARAM params[]);
- static ossl_inline OSSL_OP_keymgmt_importdomparams_fn
-     OSSL_get_OP_keymgmt_importdomparams(const OSSL_DISPATCH *opf);
+ typedef void *(OSSL_OP_keymgmt_new_fn)(void *provctx);
+ static ossl_inline OSSL_OP_keymgmt_new_fn
+     OSSL_get_OP_keymgmt_new(const OSSL_DISPATCH *opf);
 
 B<OSSL_DISPATCH> arrays are indexed by numbers that are provided as
 macros in L<openssl-core_numbers.h(7)>, as follows:
 
- OP_keymgmt_importdomparams      OSSL_FUNC_KEYMGMT_IMPORTDOMPARAMS
- OP_keymgmt_gendomparams         OSSL_FUNC_KEYMGMT_GENDOMPARAMS
- OP_keymgmt_freedomparams        OSSL_FUNC_KEYMGMT_FREEDOMPARAMS
- OP_keymgmt_exportdomparams      OSSL_FUNC_KEYMGMT_EXPORTDOMPARAMS
- OP_keymgmt_importdomparam_types OSSL_FUNC_KEYMGMT_IMPORTDOMPARAM_TYPES
- OP_keymgmt_exportdomparam_types OSSL_FUNC_KEYMGMT_EXPORTDOMPARAM_TYPES
- OP_keymgmt_get_domparam_params  OSSL_FUNC_KEYMGMT_GET_DOMPARAM_PARAMS
- OP_keymgmt_gettable_domparam_params
-                                 OSSL_FUNC_KEYMGMT_GETTABLE_DOMPARAM_PARAMS
-
- OP_keymgmt_importkey            OSSL_FUNC_KEYMGMT_IMPORTKEY
- OP_keymgmt_genkey               OSSL_FUNC_KEYMGMT_GENKEY
- OP_keymgmt_loadkey              OSSL_FUNC_KEYMGMT_LOADKEY
- OP_keymgmt_freekey              OSSL_FUNC_KEYMGMT_FREEKEY
- OP_keymgmt_exportkey            OSSL_FUNC_KEYMGMT_EXPORTKEY
- OP_keymgmt_importkey_types      OSSL_FUNC_KEYMGMT_IMPORTKEY_TYPES
- OP_keymgmt_exportkey_types      OSSL_FUNC_KEYMGMT_EXPORTKEY_TYPES
- OP_keymgmt_get_key_params       OSSL_FUNC_KEYMGMT_GET_KEY_PARAMS
- OP_keymgmt_gettable_key_params  OSSL_FUNC_KEYMGMT_GETTABLE_KEY_PARAMS
+ OP_keymgmt_new                  OSSL_FUNC_KEYMGMT_NEW
+ OP_keymgmt_free                 OSSL_FUNC_KEYMGMT_FREE
+
+ OP_keymgmt_get_params           OSSL_FUNC_KEYMGMT_GET_PARAMS
+ OP_keymgmt_gettable_params      OSSL_FUNC_KEYMGMT_GETTABLE_PARAMS
 
  OP_keymgmt_query_operation_name OSSL_FUNC_KEYMGMT_QUERY_OPERATION_NAME
 
- OP_keymgmt_validate_domparams   OSSL_FUNC_KEYMGMT_VALIDATE_DOMPARAMS
- OP_keymgmt_validate_public      OSSL_FUNC_KEYMGMT_VALIDATE_PUBLIC
- OP_keymgmt_validate_private     OSSL_FUNC_KEYMGMT_VALIDATE_PRIVATE
- OP_keymgmt_validate_pairwise    OSSL_FUNC_KEYMGMT_VALIDATE_PAIRWISE
+ OP_keymgmt_has                  OSSL_FUNC_KEYMGMT_HAS
+ OP_keymgmt_validate             OSSL_FUNC_KEYMGMT_VALIDATE
 
-=head2 Domain Parameter Functions
+ OP_keymgmt_import               OSSL_FUNC_KEYMGMT_IMPORT
+ OP_keymgmt_import_types         OSSL_FUNC_KEYMGMT_IMPORT_TYPES
+ OP_keymgmt_export               OSSL_FUNC_KEYMGMT_EXPORT
+ OP_keymgmt_export_types         OSSL_FUNC_KEYMGMT_EXPORT_TYPES
 
-OP_keymgmt_importdomparams() should create a provider side structure
-for domain parameters, with values taken from the passed B<OSSL_PARAM>
-array I<params>.
 
-OP_keymgmt_gendomparams() should generate domain parameters and create
-a provider side structure for them.
-Values of the passed B<OSSL_PARAM> array I<params> should be used as
-input for parameter generation.
+=head2 Key Objects
 
-OP_keymgmt_freedomparams() should free the passed provider side domain
-parameter structure I<domparams>.
+A key object is a collection of data for an asymmetric key, and is
+represented as I<keydata> in this manual.
 
-OP_keymgmt_exportdomparams() should extract values from the passed
-provider side domain parameter structure I<domparams> into the passed
-B<OSSL_PARAM> I<params>.
-Only the values specified in I<params> should be extracted.
+The exact contents of a key object are defined by the provider, and it
+is assumed that different operations in one and the same provider use
+the exact same structure to represent this collection of data, so that
+for example, a key object that has been created using the KEYMGMT
+interface that we document here can be passed as is to other provider
+operations, such as OP_signature_sign_init() (see
+L<provider-signature(7)>).
 
-OP_keymgmt_importdomparam_types() should return a constant array of
-descriptor B<OSSL_PARAM>, for parameters that OP_keymgmt_importdomparams()
-can handle.
+With some of the KEYMGMT functions, it's possible to select a specific
+subset of data to handle, governed by the bits in a I<selection>
+indicator.  The bits are:
+
+=over 4
 
-OP_keymgmt_exportdomparam_types() should return a constant array of
-descriptor B<OSSL_PARAM>, for parameters that can be exported with
-OP_keymgmt_exportdomparams().
+=item B<OSSL_KEYMGMT_SELECT_PRIVATE_KEY>
 
-OP_keymgmt_get_domparam_params() should extract information data
-associated with the given I<domparams>,
-see L</Information Parameters>.
+Indicating that the private key data in a key object should be
+considered.
 
-OP_keymgmt_gettable_domparam_params() should return a constant array
-of descriptor B<OSSL_PARAM>, for parameters that
-OP_keymgmt_get_domparam_params() can handle.
+=item B<OSSL_KEYMGMT_SELECT_PUBLIC_KEY>
 
-OP_keymgmt_validate_domparams() should return a value of 1 if the
-domain parameters are valid, or 0 for invalid.
+Indicating that the public key data in a key object should be
+considered.
 
-=head2 Key functions
+=item B<OSSL_KEYMGMT_SELECT_DOMAIN_PARAMETERS>
 
-OP_keymgmt_importkey() should create a provider side structure
-for keys, with values taken from the passed B<OSSL_PARAM> array
-I<params>.
+Indicating that the domain parameters in a key object should be
+considered.
 
-OP_keymgmt_genkey() should generate keys and create a provider side
-structure for them.
-Values from the passed domain parameters I<domparams> as well as from
-the passed B<OSSL_PARAM> array I<params> should be used as input for
-key generation.
+=item B<OSSL_KEYMGMT_SELECT_OTHER_PARAMETERS>
 
-OP_keymgmt_loadkey() should return a provider side key structure with
-a key loaded from a location known only to the provider, identitified
-with the identity I<id> of size I<idlen>.
-This identity is internal to the provider and is retrieved from the
-provider through other means.
+Indicating that other parameters in a key object should be
+considered.
 
-=for comment Right now, OP_keymgmt_loadkey is useless, but will be
-useful as soon as we have a OSSL_STORE interface
+Other parameters are key parameters that don't fit any other
+classification.  In other words, this particular selector bit works as
+a last resort bit bucket selector.
 
-OP_keymgmt_freekey() should free the passed I<key>.
+=back
 
-OP_keymgmt_exportkey() should extract values from the passed
-provider side key I<key> into the passed B<OSSL_PARAM> I<params>.
-Only the values specified in I<params> should be extracted.
+Some selector bits have also been combined for easier use:
 
-OP_keymgmt_importkey_types() should return a constant array of
-descriptor B<OSSL_PARAM>, for parameters that OP_keymgmt_importkey()
-can handle.
+=over 4
+
+=item B<OSSL_KEYMGMT_SELECT_ALL_PARAMETERS>
+
+Indicating that all key object parameters should be considered,
+regardless of their more granular classification.
 
-OP_keymgmt_exportkey_types() should return a constant array of
-descriptor B<OSSL_PARAM>, for parameters that can be exported with
-OP_keymgmt_exportkeys().
+=for comment This should used by EVP functions such as
+EVP_PKEY_copy_parameters() and EVP_PKEY_cmp_parameters()
 
-OP_keymgmt_get_key_params() should extract information data associated
-with the given I<key>, see L</Information Parameters>.
+This is a combination of B<OSSL_KEYMGMT_SELECT_DOMAIN_PARAMETERS> and
+B<OSSL_KEYMGMT_SELECT_OTHER_PARAMETERS>.
 
-OP_keymgmt_gettable_key_params() should return a constant array of
-descriptor B<OSSL_PARAM>, for parameters that
-OP_keymgmt_get_key_params() can handle.
+=for comment If more parameter categories are added, they should be
+mentioned here too.
 
-OP_keymgmt_validate_public() should return 1 if the public component of the
-key is valid, or 0 if invalid.
-OP_keymgmt_validate_private() should return 1 if the private component of the
-key is valid, or 0 if invalid.
-OP_keymgmt_validate_pairwise() should return 1 if the the pairwise consistency
-of the key is valid, or 0 if invalid.
+=item B<OSSL_KEYMGMT_SELECT_KEYPAIR>
 
+Indicating that both the whole key pair in a key object should be
+considered, i.e. the combination of public and private key.
+
+This is a combination of B<OSSL_KEYMGMT_SELECT_PRIVATE_KEY> and
+B<OSSL_KEYMGMT_SELECT_PUBLIC_KEY>.
+
+=item B<OSSL_KEYMGMT_SELECT_ALL>
+
+Indicating that everything in a key object should be considered.
+
+=back
 
-=head2 Supported operations
+The exact interpretation of those bits or how they combine is left to
+each function where you can specify a selector.
+
+=for comment One might think that a combination of bits means that all
+the selected data subsets must be considered, but then you have to
+consider that when comparing key objects (future function), an
+implementation might opt to not compare the private key if it has
+compared the public key, since a match of one half implies a match of
+the other half.
+
+=head2 Constructing and Destructing Functions
+
+OP_keymgmt_new() should create a provider side key object.  The
+provider context I<provctx> is passed and may be incorporated in the
+key object, but that is not mandatory.
+
+OP_keymgmt_free() should free the passed I<keydata>.
+
+The constructor and destructor are mandatory, a KEYMGMT implementation
+without them will not be accepted.
+
+=for comment when new constructors appear, it's sufficient if only one
+of them is present.  The remark above will have to change to reflect
+that.
+
+=head2 Key Object Information Functions
+
+OP_keymgmt_get_params() should extract information data associated
+with the given I<keydata>, see L</Information Parameters>.
+
+OP_keymgmt_gettable_params() should return a constant array of
+descriptor B<OSSL_PARAM>, for parameters that OP_keymgmt_get_params()
+can handle.
+
+If OP_keymgmt_gettable_params() is present, OP_keymgmt_get_params()
+must also be present.
+
+=head2 Key Object Checking Functions
 
 OP_keymgmt_query_operation_name() should return the name of the
 supported algorithm for the operation I<operation_id>.  This is
@@ -219,13 +214,43 @@ returns NULL, the caller is free to assume that there's an algorithm
 from the same provider, of the same name as the one used to fetch the
 keymgmt and try to use that.
 
+OP_keymgmt_has() should check whether the given I<keydata> the subsets
+of data indicated by the I<selector>.  A combination of several
+selector bits must consider all those subsets, not just one.  An
+implementation is, however, free to consider an empty subset of data
+to still be a valid subset.
+
+OP_keymgmt_validate() should check if the I<keydata> contains valid
+data subsets indicated by I<selection>.  Some combined selections of
+data subsets may cause validation of the combined data.
+For example, the combination of B<OSSL_KEYMGMT_SELECT_PRIVATE_KEY> and
+B<OSSL_KEYMGMT_SELECT_PUBLIC_KEY> (or B<OSSL_KEYMGMT_SELECT_KEYPAIR>
+for short) is expected to check that the pairwise consistency of
+I<keydata> is valid.
+
+=head2 Key Object Import and Export Functions
+
+OP_keymgmt_import() should import data indicated by I<selection> into
+I<keydata> with values taken from the B<OSSL_PARAM> array I<params>.
+
+OP_keymgmt_export() should extract values indicated by I<selection>
+from I<keydata>, create an B<OSSL_PARAM> array with them and call
+I<param_cb> with that array as well as the given I<cbarg>.
+
+OP_keymgmt_import_types() should return a constant array of descriptor
+B<OSSL_PARAM> for data indicated by I<selection>, for parameters that
+OP_keymgmt_import() can handle.
+
+OP_keymgmt_export_types() should return a constant array of descriptor
+B<OSSL_PARAM> for data indicated by I<selection>, that the
+OP_keymgmt_export() callback can expect to receive.
+
 =head2 Information Parameters
 
 See L<OSSL_PARAM(3)> for further details on the parameters structure.
 
 Parameters currently recognised by built-in keymgmt algorithms'
-OP_keymgmt_get_domparams_params() and OP_keymgmt_get_key_params()
-are:
+OP_keymgmt_get_params:
 
 =over 4
 
diff --git a/doc/man7/provider-serializer.pod b/doc/man7/provider-serializer.pod
index b23a8b64e2..121eb336e0 100644
--- a/doc/man7/provider-serializer.pod
+++ b/doc/man7/provider-serializer.pod
@@ -127,7 +127,7 @@ formatted data.
 
 With objects that have multiple purposes, this can be used to specify
 the purpose type.  The currently known use cases are asymmetric keys
-and domain parameters, where the type can be one of:
+and key parameters, where the type can be one of:
 
 =over 4
 
@@ -141,9 +141,9 @@ key.
 An implementation with that format property value outputs a public
 key.
 
-=item domainparams
+=item parameters
 
-An implementation with that format property value outputs domain
+An implementation with that format property value outputs key
 parameters.
 
 =back