From 3a0b3cc9050c3dfb0b45bfc0eba3a4e53e801217 Mon Sep 17 00:00:00 2001 From: "Dr. David von Oheimb" Date: Mon, 22 Jun 2020 19:47:50 +0200 Subject: Move test-related info from INSTALL.md to new test/README.md, updating references Reviewed-by: Paul Dale Reviewed-by: Nicola Tuveri (Merged from https://github.com/openssl/openssl/pull/12232) --- test/README.md | 130 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 130 insertions(+) create mode 100644 test/README.md (limited to 'test') diff --git a/test/README.md b/test/README.md new file mode 100644 index 0000000000..4db26bd047 --- /dev/null +++ b/test/README.md @@ -0,0 +1,130 @@ +Test OpenSSL +============ + +After a successful build, and before installing, the libraries should be tested. +Run: + + $ make test # Unix + $ mms test ! OpenVMS + $ nmake test # Windows + +**Warning:** you MUST run the tests from an unprivileged account +(or disable your privileges temporarily if your platform allows it). + +If some tests fail, take a look at the section Test Failures below. + +Test Failures +------------- + +If some tests fail, look at the output. There may be reasons for the failure +that isn't a problem in OpenSSL itself (like an OS malfunction or a Perl issue). +You may want increased verbosity, that can be accomplished like this: + +Full verbosity, showing full output of all successful and failed test cases +(`make` macro `VERBOSE` or `V`): + + $ make V=1 test # Unix + $ mms /macro=(V=1) test ! OpenVMS + $ nmake V=1 test # Windows + +Verbosity on test failure (`VERBOSE_FAILURE` or `VF`, Unix example shown): + + $ make test VF=1 + +Verbosity on failed (sub-)tests only (`VERBOSE_FAILURES_ONLY` or `VFO`): + + $ make test VFO=1 + +Verbosity on failed (sub-)tests, in addition progress on succeeded (sub-)tests +(`VERBOSE_FAILURES_PROGRESS` or `VFP`): + + $ make test VFP=1 + +If you want to run just one or a few specific tests, you can use +the `make` variable `TESTS` to specify them, like this: + + $ make TESTS='test_rsa test_dsa' test # Unix + $ mms/macro="TESTS=test_rsa test_dsa" test ! OpenVMS + $ nmake TESTS='test_rsa test_dsa' test # Windows + +And of course, you can combine (Unix examples shown): + + $ make test TESTS='test_rsa test_dsa' VF=1 + $ make test TESTS="test_cmp_*" VFO=1 + +You can find the list of available tests like this: + + $ make list-tests # Unix + $ mms list-tests ! OpenVMS + $ nmake list-tests # Windows + +Have a look at the manual for the perl module Test::Harness to +see what other HARNESS_* variables there are. + +To report a bug please open an issue on GitHub, at +. + +For more details on how the `make` variables `TESTS` can be used, +see section Running Selected Tests below. + +Running Selected Tests +---------------------- + +The `make` variable `TESTS` supports a versatile set of space separated tokens +with which you can specify a set of tests to be performed. With a "current +set of tests" in mind, initially being empty, here are the possible tokens: + + alltests The current set of tests becomes the whole set of available + tests (as listed when you do 'make list-tests' or similar). + + xxx Adds the test 'xxx' to the current set of tests. + + -xxx Removes 'xxx' from the current set of tests. If this is the + first token in the list, the current set of tests is first + assigned the whole set of available tests, effectively making + this token equivalent to TESTS="alltests -xxx". + + nn Adds the test group 'nn' (which is a number) to the current + set of tests. + + -nn Removes the test group 'nn' from the current set of tests. + If this is the first token in the list, the current set of + tests is first assigned the whole set of available tests, + effectively making this token equivalent to + TESTS="alltests -xxx". + +Also, all tokens except for "alltests" may have wildcards, such as *. +(on Unix and Windows, BSD style wildcards are supported, while on VMS, +it's VMS style wildcards) + +### Examples + +Run all tests except for the fuzz tests: + + $ make TESTS=-test_fuzz test + +or, if you want to be explicit: + + $ make TESTS='alltests -test_fuzz' test + +Run all tests that have a name starting with "test_ssl" but not those +starting with "test_ssl_": + + $ make TESTS='test_ssl* -test_ssl_*' test + +Run only test group 10: + + $ make TESTS='10' + +Run all tests except the slow group (group 99): + + $ make TESTS='-99' + +Run all tests in test groups 80 to 99 except for tests in group 90: + + $ make TESTS='[89]? -90' + +To stochastically verify that the algorithm that produces uniformly distributed +random numbers is operating correctly (with a false positive rate of 0.01%): + + $ ./util/wrap.sh test/bntest -stochastic -- cgit v1.2.3