From 0666d5318e5b71869b461fa5aae38ed44cd7e4ab Mon Sep 17 00:00:00 2001 From: Matt Caswell Date: Wed, 24 Mar 2021 17:41:01 +0000 Subject: Expand the libcrypto documentation Reviewed-by: Paul Dale (Merged from https://github.com/openssl/openssl/pull/1487) --- doc/man7/crypto.pod | 463 +++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 440 insertions(+), 23 deletions(-) diff --git a/doc/man7/crypto.pod b/doc/man7/crypto.pod index aa6b6405ba..cf2c3f01d7 100644 --- a/doc/man7/crypto.pod +++ b/doc/man7/crypto.pod @@ -12,41 +12,458 @@ See the individual manual pages for details. The OpenSSL crypto library (C) implements a wide range of cryptographic algorithms used in various Internet standards. The services -provided by this library are used by the OpenSSL implementations of SSL, TLS -and S/MIME, and they have also been used to implement SSH, OpenPGP, and -other cryptographic standards. +provided by this library are used by the OpenSSL implementations of TLS and +CMS, and they have also been used to implement many other third party products +and protocols. -C consists of a number of sub-libraries that implement the -individual algorithms. +The functionality includes symmetric encryption, public key cryptography, key +agreement, certificate handling, cryptographic hash functions, cryptographic +pseudo-random number generators, message authentication codes (MACs), key +derivation functions (KDFs), and various utilities. -The functionality includes symmetric encryption, public key -cryptography and key agreement, certificate handling, cryptographic -hash functions, cryptographic pseudo-random number generator, and -various utilities. +=head2 Algorithms -=head1 NOTES +Cryptographic primitives such as the SHA256 digest, or AES encryption are +referred to in OpenSSL as "algorithms". Each algorithm may have multiple +implementations available for use. For example the RSA algorithm is available as +a "default" implementation suitable for general use, and a "fips" implementation +which has been validated to FIPS standards for situations where that is +important. It is also possible that a third party could add additional +implementations such as in a hardware security module (HSM). -Some of the newer functions follow a naming convention using the numbers -B<0> and B<1>. For example the functions: +=head2 Operations - int X509_CRL_add0_revoked(X509_CRL *crl, X509_REVOKED *rev); - int X509_add1_trust_object(X509 *x, const ASN1_OBJECT *obj); +Different algorithms can be grouped together by their purpose. For example there +are algorithms for encryption, and different algorithms for digesting data. +These different groups are known as "operations" in OpenSSL. Each operation +has a different set of functions associated with it. For example to perform an +encryption operation using AES (or any other encryption algorithm) you would use +the encryption functions detailed on the L page. Or to +perform a digest operation using SHA256 then you would use the digesting +functions on the L page. -The B<0> version uses the supplied structure pointer directly -in the parent and it will be freed up when the parent is freed. -In the above example I would be freed but I would not. +=head2 Providers -The B<1> function uses a copy of the supplied structure pointer -(or in some cases increases its link count) in the parent and -so both (I and I above) should be freed up. +A provider in OpenSSL is a component that collects together algorithm +implementations. In order to use an algorithm you must have at least one +provider loaded that contains an implementation of it. OpenSSL comes with a +number of providers and they may also be obtained from third parties. If you +don't load a provider explicitly (either in program code or via config) then the +OpenSSL built-in "default" provider will be automatically loaded. -=head1 RETURN VALUES +=head2 Library contexts -See the individual manual pages for details. +A library context can be thought of as a "scope" within which configuration +options take effect. When a provider is loaded, it is only loaded within the +scope of a given library context. In this way it is possible for different +components of a complex application to each use a different library context and +have different providers loaded with different configuration settings. + +If an application does not explicitly create a library context then the +"default" library context will be used. + +Library contexts are represented by the B type. Many OpenSSL API +functions take a library context as a parameter. Applications can always pass +B for this parameter to just use the default library context. + +The default library context is automatically created the first time it is +needed. This will automatically load any available configuration file and will +initialise OpenSSL for use. Unlike in earlier versions of OpenSSL (prior to +1.1.0) no explicit initialisation steps need to be taken. + +Similarly when the application exits the default library context is +automatically destroyed. No explicit de-initialisation steps need to be taken. + +See L for more information about library contexts. + +=head2 Multi-threaded applications + +As long as OpenSSL has been built with support for threads (the default case +on most platforms) then most OpenSSL I are thread-safe in the sense +that it is safe to call the same function from multiple threads at the same +time. However most OpenSSL I are not thread-safe. For example +the L and L functions are thread safe. However it +would not be thread safe to call BIO_write() from one thread while calling +BIO_read() in another where both functions are passed the same B object +since both of them may attempt to make changes to the same B object. + +There are exceptions to these rules. A small number of functions are not thread +safe at all. Where this is the case this restriction should be noted in the +documentation for the function. Similarly some data structures may be partially +or fully thread safe. For example it is safe to use an B in +multiple threads. + +See L for a more detailed discussion on OpenSSL threading +support. + +=head1 ALGORITHM FETCHING + +In order to use an algorithm an implementation for it must first be "fetched". +Fetching is the process of looking through the available implementations, +applying selection criteria (via a property query string), and finally choosing +the implementation that will be used. + +Two types of fetching are supported by OpenSSL - explicit fetching and implicit +fetching. + +=head2 Property query strings + +When fetching an algorithm it is possible to specify a property query string to +guide the selection process. For example a property query string of +"provider=default" could be used to force the selection to only consider +algorithm implementations in the default provider. + +Property query strings can be specified explicitly as an argument to a function. +It is also possible to specify a default property query string for the whole +library context using the L function. Where both +default properties and function specific properties are specified then they are +combined. Function specific properties will override default properties where +there is a conflict. + +See L for more information about properties. + +=head2 Explicit fetching + +Users of the OpenSSL libraries never query a provider directly for an algorithm +implementation. Instead, the diverse OpenSSL APIs often have explicit fetching +functions that do the work, and they return an appropriate algorithm object back +to the user. These functions usually have the name C, where +C is the name of the operation. For example L can +be used to explicitly fetch a digest algorithm implementation. The user is +responsible for freeing the object returned from the C function +using C when it is no longer needed. + +These fetching functions follow a fairly common pattern, where three +arguments are passed: + +=over 4 + +=item The library context + +See L for a more detailed description. +This may be NULL to signify the default (global) library context, or a +context created by the user. Only providers loaded in this library context (see +L) will be considered by the fetching function. In case +no provider has been loaded in this library context then the default provider +will be loaded as a fallback (see L). + +=item An identifier + +For all currently implemented fetching functions this is the algorithm name. + +=item A property query string + +The property query string used to guide selection of the algorithm +implementation. + +=back + +The algorithm implementation that is fetched can then be used with other diverse +functions that use them. For example the L function takes +as a parameter an B object which may have been returned from an earlier +call to L. + +=head2 Implicit fetch + +OpenSSL has a number of functions that return an algorithm object with no +associated implementation, such as L, +L or L. These are present for +compatibility with OpenSSL before version 3.0 where explicit fetching was not +available. + +When they are used with functions like L or +L, the actual implementation to be used is +fetched implicitly using default search criteria. + +In some cases implicit fetching can also occur when a NULL algorithm parameter +is supplied. In this case an algorithm implementation is implicitly fetched +using default search criteria and an algorithm name that is consistent with +the context in which it is being used. + +=head1 FETCHING EXAMPLES + +The following section provides a series of examples of fetching algorithm +implementations. + +Fetch any available implementation of SHA2-256 in the default context. Note +that some algorithms have aliases. So "SHA256" and "SHA2-256" are synonymous: + + EVP_MD *md = EVP_MD_fetch(NULL, "SHA2-256", NULL); + ... + EVP_MD_free(md); + +Fetch any available implementation of AES-128-CBC in the default context: + + EVP_CIPHER *cipher = EVP_CIPHER_fetch(NULL, "AES-128-CBC", NULL); + ... + EVP_CIPHER_free(cipher); + +Fetch an implementation of SHA2-256 from the default provider in the default +context: + + EVP_MD *md = EVP_MD_fetch(NULL, "SHA2-256", "provider=default"); + ... + EVP_MD_free(md); + +Fetch an implementation of SHA2-256 that is not from the default provider in the +default context: + + EVP_MD *md = EVP_MD_fetch(NULL, "SHA2-256", "provider!=default"); + ... + EVP_MD_free(md); + +Fetch an implementation of SHA2-256 from the default provider in the specified +context: + + EVP_MD *md = EVP_MD_fetch(ctx, "SHA2-256", "provider=default"); + ... + EVP_MD_free(md); + +Load the legacy provider into the default context and then fetch an +implementation of WHIRLPOOL from it: + + /* This only needs to be done once - usually at application start up */ + OSSL_PROVIDER *legacy = OSSL_PROVIDER_load(NULL, "legacy"); + + EVP_MD *md = EVP_MD_fetch(NULL, "WHIRLPOOL", "provider=legacy"); + ... + EVP_MD_free(md); + +Note that in the above example the property string "provider=legacy" is optional +since, assuming no other providers have been loaded, the only implementation of +the "whirlpool" algorithm is in the "legacy" provider. Also note that the +default provider should be explicitly loaded if it is required in addition to +other providers: + + /* This only needs to be done once - usually at application start up */ + OSSL_PROVIDER *legacy = OSSL_PROVIDER_load(NULL, "legacy"); + OSSL_PROVIDER *default = OSSL_PROVIDER_load(NULL, "default"); + + EVP_MD *md_whirlpool = EVP_MD_fetch(NULL, "whirlpool", NULL); + EVP_MD *md_sha256 = EVP_MD_fetch(NULL, "SHA2-256", NULL); + ... + EVP_MD_free(md_whirlpool); + EVP_MD_free(md_sha256); + +=head1 OPENSSL PROVIDERS + +OpenSSL comes with a set of providers. + +The algorithms available in each of these providers may vary due to build time +configuration options. The L command can be used to list the +currently available algorithms. + +The names of the algorithms shown from L can be used as an +algorithm identifier to the appropriate fetching function. Also see the provider +specific manual pages linked below for further details about using the +algorithms available in each of the providers. + +As well as the OpenSSL providers third parties can also implemment providers. +For information on writing a provider see L. + +=head2 Default provider + +The default provider is built in as part of the F library and +contains all of the most commonly used algorithm implementations. Should it be +needed (if other providers are loaded and offer implementations of the same +algorithms), the property query string "provider=default" can be used as a +search criterion for these implementations. The default provider includes all +of the functionality in the base provider below. + +If you don't load any providers at all then the "default" provider will be +automatically loaded. If you explicitly load any provider then the "default" +provider would also need to be explicitly loaded if it is required. + +See L. + +=head2 Base provider + +The base provider is built in as part of the F library and contains +algorithm implementations for encoding and decoding for OpenSSL keys. +Should it be needed (if other providers are loaded and offer +implementations of the same algorithms), the property query string +"provider=base" can be used as a search criterion for these implementations. +Some encoding and decoding algorithm implementations are not FIPS algorithm +implementations in themselves but support algorithms from the FIPS provider and +are allowed for use in "FIPS mode". The property query string "fips=yes" can be +used to select such algorithms. + +See L. + +=head2 FIPS provider + +The FIPS provider is a dynamically loadable module, and must therefore +be loaded explicitly, either in code or through OpenSSL configuration +(see L). It contains algorithm implementations that have been +validated according to the FIPS 140-2 standard. Should it be needed (if other +providers are loaded and offer implementations of the same algorithms), the +property query string "provider=fips" can be used as a search criterion for +these implementations. All approved algorithm implementations in the FIPS +provider can also be selected with the property "fips=yes". The FIPS provider +may also contain non-approved algorithm implementations and these can be +selected with the property "fips=no". + +See L. + +=head2 Legacy provider + +The legacy provider is a dynamically loadable module, and must therefore +be loaded explicitly, either in code or through OpenSSL configuration +(see L). It contains algorithm implementations that are considered +insecure, or are no longer in common use such as MD2 or RC4. Should it be needed +(if other providers are loaded and offer implementations of the same algorithms), +the property "provider=legacy" can be used as a search criterion for these +implementations. + +See L. + +=head2 Null provider + +The null provider is built in as part of the F library. It contains +no algorithms in it at all. When fetching algorithms the default provider will +be automatically loaded if no other provider has been explicitly loaded. To +prevent that from happening you can explicitly load the null provider. + +See L. + +=head1 USING ALGORITHMS IN APPLICATIONS + +Cryptographic algorithms are made available to applications through use of the +"EVP" APIs. Each of the various operations such as encryption, digesting, +message authentication codes, etc., have a set of EVP function calls that can +be invoked to use them. See the L page for further details. + +Most of these follow a common pattern. A "context" object is first created. For +example for a digest operation you would use an B, and for an +encryption/decryption operation you would use an B. The +operation is then initialised ready for use via an "init" function - optionally +passing in a set of parameters (using the B type) to configure how +the operation should behave. Next data is fed into the operation in a series of +"update" calls. The operation is finalised using a "final" call which will +typically provide some kind of output. Finally the context is cleaned up and +freed. + +The following shows a complete example for doing this process for digesting +data using SHA256. The process is similar for other operations such as +encryption/decryption, signatures, message authentication codes, etc. + + #include + #include + #include + + int main(void) + { + EVP_MD_CTX *ctx = NULL; + EVP_MD *sha256 = NULL; + const unsigned char msg[] = { + 0x00, 0x01, 0x02, 0x03 + }; + unsigned int len = 0; + unsigned char *outdigest = NULL; + + /* Create a context for the digest operation */ + ctx = EVP_MD_CTX_new(); + if (ctx == NULL) + goto err; + + /* + * Fetch the SHA256 algorithm implementation for doing the digest. We're + * using the "default" library context here (first NULL parameter), and + * we're not supplying any particular search criteria for our SHA256 + * implementation (second NULL parameter). Any SHA256 implementation will + * do. + */ + sha256 = EVP_MD_fetch(NULL, "SHA256", NULL); + if (sha256 == NULL) + goto err; + + /* Initialise the digest operation */ + if (!EVP_DigestInit_ex(ctx, sha256, NULL)) + goto err; + + /* + * Pass the message to be digested. This can be passed in over multiple + * EVP_DigestUpdate calls if necessary + */ + if (!EVP_DigestUpdate(ctx, msg, sizeof(msg))) + goto err; + + /* Allocate the output buffer */ + outdigest = OPENSSL_malloc(EVP_MD_size(sha256)); + if (outdigest == NULL) + goto err; + + /* Now calculate the digest itself */ + if (!EVP_DigestFinal_ex(ctx, outdigest, &len)) + goto err; + + /* Print out the digest result */ + BIO_dump_fp(stdout, outdigest, len); + + err: + /* Clean up all the resources we allocated */ + OPENSSL_free(outdigest); + EVP_MD_free(sha256); + EVP_MD_CTX_free(ctx); + } + +=head1 CONFIGURATION + +By default OpenSSL will load a configuration file when it is first used. This +will set up various configuration settings within the default library context. +Applications that create their own library contexts may optionally configure +them with a config file using the L function. + +The configuration file can be used to automatically load providers and set up +default property query strings. + +For information on the OpenSSL configuration file format see L. + +=head1 ENCODING AND DECODING KEYS + +Many algorithms require the use of a key. Keys can be generated dynamically +using the EVP APIs (for example see L). However it is often +necessary to save or load keys (or their associated parameters) to or from some +external format such as PEM or DER (see L). OpenSSL uses +encoders and decoders to perform this task. + +Encoders and decoders are just algorithm implementations in the same way as +any other algorithm implementation in OpenSSL. They are implemented by +providers. The OpenSSL encoders and decoders are available in the default +provider. They are also duplicated in the base provider. + +For information about encoders see L. For +information about decoders see L. + +=head1 LIBRARY CONVENTIONS + +Many OpenSSL functions that "get" or "set" a value follow a naming convention +using the numbers B<0> and B<1>, i.e. "get0", "get1", "set0" and "set1". This +can also apply to some functions that "add" a value to an existing set, i.e. +"add0" and "add1". + +For example the functions: + + int X509_CRL_add0_revoked(X509_CRL *crl, X509_REVOKED *rev); + int X509_add1_trust_object(X509 *x, const ASN1_OBJECT *obj); + +In the B<0> version the ownership of the object is passed to (for an add or set) +or retained by (for a get) the parent object. For example after calling the +X509_CRL_add0_revoked() function above, ownership of the I object is passed +to the I object. Therefore, after calling this function I should not +be freed directly. It will be freed implicitly when I is freed. + +In the B<1> version the ownership of the object is not passed to or retained by +the parent object. Instead a copy or "up ref" of the object is performed. So +after calling the X509_add1_trust_object() function above the application will +still be responsible for freeing the I value where appropriate. =head1 SEE ALSO -L, L +L, L, L, L, L, +L, L, L, +L, L, L, +L, L =head1 COPYRIGHT -- cgit v1.2.3