Pick a q size consistent with the digest for DSA param generation

There are two undocumented DSA parameter generation options available in
the genpkey command line app:
dsa_paramgen_md and dsa_paramgen_q_bits.

These can also be accessed via the EVP API but only by using
EVP_PKEY_CTX_ctrl() or EVP_PKEY_CTX_ctrl_str() directly. There are no
helper macros for these options.

dsa_paramgen_q_bits sets the length of q in bits (default 160 bits).
dsa_paramgen_md sets the digest that is used during the parameter
generation (default SHA1). In particular the output length of the digest
used must be equal to or greater than the number of bits in q because of
this code:

            if (!EVP_Digest(seed, qsize, md, NULL, evpmd, NULL))
                goto err;
            if (!EVP_Digest(buf, qsize, buf2, NULL, evpmd, NULL))
                goto err;
            for (i = 0; i < qsize; i++)
                md[i] ^= buf2[i];

            /* step 3 */
            md[0] |= 0x80;
            md[qsize - 1] |= 0x01;
            if (!BN_bin2bn(md, qsize, q))
                goto err;

qsize here is the number of bits in q and evpmd is the digest set via
dsa_paramgen_md. md and buf2 are buffers of length SHA256_DIGEST_LENGTH.
buf2 has been filled with qsize bits of random seed data, and md is
uninitialised.

If the output size of evpmd is less than qsize then the line "md[i] ^=
buf2[i]" will be xoring an uninitialised value and the random seed data
together to form the least significant bits of q (and not using the
output of the digest at all for those bits) - which is probably not what
was intended. The same seed is then used as an input to generating p. If
the uninitialised data is actually all zeros (as seems quite likely)
then the least significant bits of q will exactly match the least
significant bits of the seed.

This problem only occurs if you use these undocumented and difficult to
find options and you set the size of q to be greater than the message
digest output size. This is for parameter generation only not key
generation. This scenario is considered highly unlikely and
therefore the security risk of this is considered negligible.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5800)

1 files changed, 1 insertions, 0 deletions

diff --git a/include/openssl/dsaerr.h b/include/openssl/dsaerr.h
index a5248ab700..d94f97bba8 100644
--- a/include/openssl/dsaerr.h
+++ b/include/openssl/dsaerr.h
@@ -44,6 +44,7 @@ int ERR_load_DSA_strings(void);
 #  define DSA_F_DSA_SIG_NEW                                102
 #  define DSA_F_OLD_DSA_PRIV_DECODE                        122
 #  define DSA_F_PKEY_DSA_CTRL                              120
+#  define DSA_F_PKEY_DSA_CTRL_STR                          104
 #  define DSA_F_PKEY_DSA_KEYGEN                            121
 
 /*