RT992: RSA_check_key should have a callback arg

The original RT request included a patch.  By the time
we got around to doing it, however, the callback scheme
had changed. So I wrote a new function RSA_check_key_ex()
that uses the BN_GENCB callback.  But thanks very much
to Vinet Sharma <vineet.sharma@gmail.com> for the
initial implementation.

Reviewed-by: Dr. Stephen Henson <steve@openssl.org>

2 files changed, 24 insertions, 14 deletions

diff --git a/doc/crypto/BN_generate_prime.pod b/doc/crypto/BN_generate_prime.pod
index f1f2265a86..4522fa9bdb 100644
--- a/doc/crypto/BN_generate_prime.pod
+++ b/doc/crypto/BN_generate_prime.pod
@@ -104,10 +104,10 @@ programs should prefer the "new" style, whilst the "old" style is provided
 for backwards compatibility purposes.
 
 For "new" style callbacks a BN_GENCB structure should be initialised with a
-call to BN_GENCB_set, where B<gencb> is a B<BN_GENCB *>, B<callback> is of
+call to BN_GENCB_set(), where B<gencb> is a B<BN_GENCB *>, B<callback> is of
 type B<int (*callback)(int, int, BN_GENCB *)> and B<cb_arg> is a B<void *>.
 "Old" style callbacks are the same except they are initialised with a call
-to BN_GENCB_set_old and B<callback> is of type
+to BN_GENCB_set_old() and B<callback> is of type
 B<void (*callback)(int, int, void *)>.
 
 A callback is invoked through a call to B<BN_GENCB_call>. This will check
diff --git a/doc/crypto/RSA_check_key.pod b/doc/crypto/RSA_check_key.pod
index a5198f3db5..522a6c2b5e 100644
--- a/doc/crypto/RSA_check_key.pod
+++ b/doc/crypto/RSA_check_key.pod
@@ -8,35 +8,42 @@ RSA_check_key - validate private RSA keys
 
  #include <openssl/rsa.h>
 
+ int RSA_check_key_ex(RSA *rsa, BN_GENCB *cb);
+
  int RSA_check_key(RSA *rsa);
 
 =head1 DESCRIPTION
 
-This function validates RSA keys. It checks that B<p> and B<q> are
+RSA_check_key_ex() function validates RSA keys.
+It checks that B<p> and B<q> are
 in fact prime, and that B<n = p*q>.
 
+It does not work on RSA public keys that have only the modulus
+and public exponent elements populated.
 It also checks that B<d*e = 1 mod (p-1*q-1)>,
 and that B<dmp1>, B<dmq1> and B<iqmp> are set correctly or are B<NULL>.
+It performs integrity checks on all
+the RSA key material, so the RSA key structure must contain all the private
+key data too.
+Therefore, it cannot be used with any arbitrary RSA key object,
+even if it is otherwise fit for regular RSA operation.
+
+The B<cb> parameter is a callback that will be invoked in the same
+manner as L<BN_is_prime_ex(3)|BN_is_prime_ex(3)>.
 
-As such, this function can not be used with any arbitrary RSA key object,
-even if it is otherwise fit for regular RSA operation. See B<NOTES> for more
-information.
+RSA_check_key() is equivalent to RSA_check_key_ex() with a NULL B<cb>.
 
 =head1 RETURN VALUE
 
-RSA_check_key() returns 1 if B<rsa> is a valid RSA key, and 0 otherwise.
--1 is returned if an error occurs while checking the key.
+RSA_check_key_ex() and RSA_check_key()
+return 1 if B<rsa> is a valid RSA key, and 0 otherwise.
+They return -1 if an error occurs while checking the key.
 
 If the key is invalid or an error occurred, the reason code can be
 obtained using L<ERR_get_error(3)|ERR_get_error(3)>.
 
 =head1 NOTES
 
-This function does not work on RSA public keys that have only the modulus
-and public exponent elements populated. It performs integrity checks on all
-the RSA key material, so the RSA key structure must contain all the private
-key data too.
-
 Unlike most other RSA functions, this function does B<not> work
 transparently with any underlying ENGINE implementation because it uses the
 key data in the RSA structure directly. An ENGINE implementation can
@@ -58,10 +65,13 @@ provide their own verifiers.
 
 =head1 SEE ALSO
 
-L<rsa(3)|rsa(3)>, L<ERR_get_error(3)|ERR_get_error(3)>
+L<BN_is_prime_ex(3)|BN_is_prime_ex(3)>,
+L<rsa(3)|rsa(3)>,
+L<ERR_get_error(3)|ERR_get_error(3)>
 
 =head1 HISTORY
 
 RSA_check_key() appeared in OpenSSL 0.9.4.
+RSA_check_key_ex() appeared after OpenSSL 1.0.2.
 
 =cut