Fix issues in ia32 RDRAND asm leading to reduced entropy

This patch fixes two issues in the ia32 RDRAND assembly code that result in a
(possibly significant) loss of entropy.

The first, less significant, issue is that, by returning success as 0 from
OPENSSL_ia32_rdrand() and OPENSSL_ia32_rdseed(), a subtle bias was introduced.
Specifically, because the assembly routine copied the remaining number of
retries over the result when RDRAND/RDSEED returned 'successful but zero', a
bias towards values 1-8 (primarily 8) was introduced.

The second, more worrying issue was that, due to a mixup in registers, when a
buffer that was not size 0 or 1 mod 8 was passed to OPENSSL_ia32_rdrand_bytes
or OPENSSL_ia32_rdseed_bytes, the last (n mod 8) bytes were all the same value.
This issue impacts only the 64-bit variant of the assembly.

This change fixes both issues by first eliminating the only use of
OPENSSL_ia32_rdrand, replacing it with OPENSSL_ia32_rdrand_bytes, and fixes the
register mixup in OPENSSL_ia32_rdrand_bytes. It also adds a sanity test for
OPENSSL_ia32_rdrand_bytes and OPENSSL_ia32_rdseed_bytes to help catch problems
of this nature in the future.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5342)

1 files changed, 5 insertions, 18 deletions

diff --git a/crypto/engine/eng_rdrand.c b/crypto/engine/eng_rdrand.c
index 7be64e3fd4..261e5debbf 100644
--- a/crypto/engine/eng_rdrand.c
+++ b/crypto/engine/eng_rdrand.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2011-2016 The OpenSSL Project Authors. All Rights Reserved.
+ * Copyright 2011-2018 The OpenSSL Project Authors. All Rights Reserved.
  *
  * Licensed under the OpenSSL license (the "License").  You may not use
  * this file except in compliance with the License.  You can obtain a copy
@@ -20,28 +20,15 @@
      defined(__x86_64) || defined(__x86_64__) || \
      defined(_M_AMD64) || defined (_M_X64)) && defined(OPENSSL_CPUID_OBJ)
 
-size_t OPENSSL_ia32_rdrand(void);
+size_t OPENSSL_ia32_rdrand_bytes(unsigned char *buf, size_t len);
 
 static int get_random_bytes(unsigned char *buf, int num)
 {
-    size_t rnd;
-
-    while (num >= (int)sizeof(size_t)) {
-        if ((rnd = OPENSSL_ia32_rdrand()) == 0)
-            return 0;
-
-        *((size_t *)buf) = rnd;
-        buf += sizeof(size_t);
-        num -= sizeof(size_t);
-    }
-    if (num) {
-        if ((rnd = OPENSSL_ia32_rdrand()) == 0)
-            return 0;
-
-        memcpy(buf, &rnd, num);
+    if (num < 0) {
+        return 0;
     }
 
-    return 1;
+    return (size_t)num == OPENSSL_ia32_rdrand_bytes(buf, (size_t)num);
 }
 
 static int random_status(void)