This is a first-cut at improving the callback mechanisms used in

key-generation and prime-checking functions. Rather than explicitly passing
callback functions and caller-defined context data for the callbacks, a new
structure BN_GENCB is defined that encapsulates this; a pointer to the
structure is passed to all such functions instead.

This wrapper structure allows the encapsulation of "old" and "new" style
callbacks - "new" callbacks return a boolean result on the understanding
that returning FALSE should terminate keygen/primality processing.  The
BN_GENCB abstraction will allow future callback modifications without
needing to break binary compatibility nor change the API function
prototypes. The new API functions have been given names ending in "_ex" and
the old functions are implemented as wrappers to the new ones.  The
OPENSSL_NO_DEPRECATED symbol has been introduced so that, if defined,
declaration of the older functions will be skipped. NB: Some
openssl-internal code will stick with the older callbacks for now, so
appropriate "#undef" logic will be put in place - this is in case the user
is *building* openssl (rather than *including* its headers) with this
symbol defined.

There is another change in the new _ex functions; the key-generation
functions do not return key structures but operate on structures passed by
the caller, the return value is a boolean. This will allow for a smoother
transition to having key-generation as "virtual function" in the various
***_METHOD tables.

1 files changed, 2 insertions, 2 deletions

diff --git a/crypto/dh/dh_check.c b/crypto/dh/dh_check.c
index f0373f7d68..bfc9c3ad76 100644
--- a/crypto/dh/dh_check.c
+++ b/crypto/dh/dh_check.c
@@ -104,12 +104,12 @@ int DH_check(const DH *dh, int *ret)
 	else
 		*ret|=DH_UNABLE_TO_CHECK_GENERATOR;
 
-	if (!BN_is_prime(dh->p,BN_prime_checks,NULL,ctx,NULL))
+	if (!BN_is_prime_ex(dh->p,BN_prime_checks,ctx,NULL))
 		*ret|=DH_CHECK_P_NOT_PRIME;
 	else
 		{
 		if (!BN_rshift1(q,dh->p)) goto err;
-		if (!BN_is_prime(q,BN_prime_checks,NULL,ctx,NULL))
+		if (!BN_is_prime_ex(q,BN_prime_checks,ctx,NULL))
 			*ret|=DH_CHECK_P_NOT_SAFE_PRIME;
 		}
 	ok=1;