Change default directory for storing the .rnd file on Windows

Previously we would try %RANDFILE%, then %HOME% and finally "C:".
Unfortunately this often ends up being "C:" which the user may not
have write permission for.

Now we try %RANDFILE% first, and then the same set of environment vars
as GetTempFile() uses, i.e. %TMP%, then %TEMP%, %USERPROFILE% and
%SYSTEMROOT%. If all else fails we fall back to %HOME% and only then "C:".

Reviewed-by: Rich Salz <rsalz@openssl.org>

3 files changed, 30 insertions, 2 deletions

diff --git a/CHANGES b/CHANGES
index ef01b27e48..8fa6f44e0f 100644
--- a/CHANGES
+++ b/CHANGES
@@ -4,6 +4,12 @@
 
  Changes between 1.0.2h and 1.1.0  [xx XXX 2016]
 
+  *) The method for finding the storage location for the Windows RAND seed file
+     has changed. First we check %RANDFILE%. If that is not set then we check
+     the directories %TMP%, %TEMP%, %USERPROFILE%, %SYSTEMROOT% and %HOME% in
+     that order. If all else fails we fall back to "C:".
+     [Matt Caswell]
+
   *) The EVP_EncryptUpdate() function has had its return type changed from void
      to int. A return of 0 indicates and error while a return of 1 indicates
      success.
diff --git a/crypto/rand/randfile.c b/crypto/rand/randfile.c
index 49f5405dbc..19cce2c7a8 100644
--- a/crypto/rand/randfile.c
+++ b/crypto/rand/randfile.c
@@ -286,8 +286,22 @@ const char *RAND_file_name(char *buf, size_t size)
         if (OPENSSL_strlcpy(buf, s, size) >= size)
             return NULL;
     } else {
+#ifdef OPENSSL_SYS_WINDOWS
+        /*
+         * We use the same env variables as GetTempFile() - but that function
+         * uses TCHARs, but getenv() gives us chars so its easier to do it this
+         * way
+         */
+        if ((s = getenv("TMP")) == NULL
+            && (s = getenv("TEMP")) == NULL
+            && (s = getenv("USERPROFILE")) == NULL
+            && (s = getenv("SYSTEMROOT")) == NULL) {
+            s = getenv("HOME");
+        }
+#else
         if (OPENSSL_issetugid() == 0)
             s = getenv("HOME");
+#endif
 #ifdef DEFAULT_HOME
         if (s == NULL) {
             s = DEFAULT_HOME;
diff --git a/doc/crypto/RAND_load_file.pod b/doc/crypto/RAND_load_file.pod
index 133b8d2311..dd79af28ae 100644
--- a/doc/crypto/RAND_load_file.pod
+++ b/doc/crypto/RAND_load_file.pod
@@ -18,8 +18,16 @@ RAND_load_file, RAND_write_file, RAND_file_name - PRNG seed file
 
 RAND_file_name() generates a default path for the random seed
 file. B<buf> points to a buffer of size B<num> in which to store the
-filename. The seed file is $RANDFILE if that environment variable is
-set, $HOME/.rnd otherwise. If $HOME is not set either, or B<num> is
+filename.
+
+On Windows the seed file is %RANDFILE% if that environment variable is set.
+Otherwise the file is called ".rnd" in one of the following locations (in order
+of preference): %TMP%, %TEMP%, %USERPROFILE%, %SYSTEMROOT%, %HOME%, "C:".
+
+On all other systems the seed file is $RANDFILE if that environment variable is
+set, $HOME/.rnd otherwise.
+
+If $HOME (on non Windows systems) is not set either, or B<num> is
 too small for the path name, an error occurs.
 
 RAND_load_file() reads a number of bytes from file B<filename> and