diff options
| author | Pauli <paul.dale@oracle.com> | 2020-12-12 21:38:17 +1000 |
|---|---|---|
| committer | Pauli <paul.dale@oracle.com> | 2020-12-15 20:03:07 +1000 |
| commit | c678f68a19638c1e2bbfee6a7a1d8d728976ce66 (patch) | |
| tree | 915e8c52e7e1d621ba2222f955323ac017d23553 | |
| parent | a21a1c23c97163d341d1caf07c2160fcb68506b0 (diff) | |
test: document the random test ordering env variable
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13672)
| -rw-r--r-- | test/README.md | 20 |
1 files changed, 20 insertions, 0 deletions
diff --git a/test/README.md b/test/README.md index 43f8471120..9a5c91953d 100644 --- a/test/README.md +++ b/test/README.md @@ -148,3 +148,23 @@ for `TAP::Harness` to know more. To run up to four tests in parallel at any given time: $ make HARNESS_JOBS=4 test + +Randomisation of Test Ordering +------------------------------ + +By default, the test harness will execute tests in the order they were added. +By setting the `OPENSSL_TEST_RAND_ORDER` environment variable to zero, the +test ordering will be randomised. If a randomly ordered test fails, the +seed value used will be reported. Setting the `OPENSSL_TEST_RAND_ORDER` +environment variable to this value will rerun the tests in the same +order. This assures repeatability of randomly ordered test runs. +This repeatability is independent of the operating system, processor or +platform used. + +To randomise the test ordering: + + $ make OPENSSL_TEST_RAND_ORDER=0 test + +To run the tests using the order defined by the random seed `42`: + + $ make OPENSSL_TEST_RAND_ORDER=42 test |