summaryrefslogtreecommitdiffstats
path: root/test/README-dev.md
diff options
context:
space:
mode:
authorDr. David von Oheimb <David.von.Oheimb@siemens.com>2020-06-10 14:15:28 +0200
committerDr. David von Oheimb <David.von.Oheimb@siemens.com>2020-07-05 11:29:43 +0200
commit036cbb6bbf30955abdcffaf6e52cd926d8d8ee75 (patch)
tree1929b9d33c7041858cbbed980f8c981d8eb77c3c /test/README-dev.md
parent915e7e75a49343ff5ddd23a54219eb32f57aa01c (diff)
Rename NOTES*, README*, VERSION, HACKING, LICENSE to .md or .txt
Reviewed-by: Tim Hudson <tjh@openssl.org> (Merged from https://github.com/openssl/openssl/pull/12109)
Diffstat (limited to 'test/README-dev.md')
-rw-r--r--test/README-dev.md156
1 files changed, 156 insertions, 0 deletions
diff --git a/test/README-dev.md b/test/README-dev.md
new file mode 100644
index 0000000000..56114fdc34
--- /dev/null
+++ b/test/README-dev.md
@@ -0,0 +1,156 @@
+Guidelines for test developers
+==============================
+
+How to add recipes
+------------------
+
+For any test that you want to perform, you write a script located in
+`test/recipes/`, named `{nn}-test_{name}.t`,
+where `{nn}` is a two digit number and
+`{name}` is a unique name of your choice.
+
+Please note that if a test involves a new testing executable, you will need to
+do some additions in test/build.info. Please refer to the section
+["Changes to test/build.info"](README.md#changes-to-testbuildinfo) below.
+
+Naming conventions
+------------------
+
+A test executable is named `test/{name}test.c`
+
+A test recipe is named `test/recipes/{nn}-test_{name}.t`, where `{nn}` is a two
+digit number and `{name}` is a unique name of your choice.
+
+The number `{nn}` is (somewhat loosely) grouped as follows:
+
+ 00-04 sanity, internal and essential API tests
+ 05-09 individual symmetric cipher algorithms
+ 10-14 math (bignum)
+ 15-19 individual asymmetric cipher algorithms
+ 20-24 openssl commands (some otherwise not tested)
+ 25-29 certificate forms, generation and verification
+ 30-35 engine and evp
+ 60-79 APIs:
+ 60 X509 subsystem
+ 61 BIO subsystem
+ 65 CMP subsystem
+ 70 PACKET layer
+ 80-89 "larger" protocols (CA, CMS, OCSP, SSL, TSA)
+ 90-98 misc
+ 99 most time consuming tests [such as test_fuzz]
+
+A recipe that just runs a test executable
+-----------------------------------------
+
+A script that just runs a program looks like this:
+
+ #! /usr/bin/perl
+
+ use OpenSSL::Test::Simple;
+
+ simple_test("test_{name}", "{name}test", "{name}");
+
+`{name}` is the unique name you have chosen for your test.
+
+The second argument to `simple_test` is the test executable, and `simple_test`
+expects it to be located in `test/`
+
+For documentation on `OpenSSL::Test::Simple`,
+do `perldoc util/perl/OpenSSL/Test/Simple.pm`.
+
+A recipe that runs a more complex test
+--------------------------------------
+
+For more complex tests, you will need to read up on Test::More and
+OpenSSL::Test. Test::More is normally preinstalled, do `man Test::More` for
+documentation. For OpenSSL::Test, do `perldoc util/perl/OpenSSL/Test.pm`.
+
+A script to start from could be this:
+
+ #! /usr/bin/perl
+
+ use strict;
+ use warnings;
+ use OpenSSL::Test;
+
+ setup("test_{name}");
+
+ plan tests => 2; # The number of tests being performed
+
+ ok(test1, "test1");
+ ok(test2, "test1");
+
+ sub test1
+ {
+ # test feature 1
+ }
+
+ sub test2
+ {
+ # test feature 2
+ }
+
+Changes to test/build.info
+--------------------------
+
+Whenever a new test involves a new test executable you need to do the
+following (at all times, replace {NAME} and {name} with the name of your
+test):
+
+ * add `{name}` to the list of programs under `PROGRAMS_NO_INST`
+
+ * create a three line description of how to build the test, you will have
+ to modify the include paths and source files if you don't want to use the
+ basic test framework:
+
+ SOURCE[{name}]={name}.c
+ INCLUDE[{name}]=.. ../include ../apps/include
+ DEPEND[{name}]=../libcrypto libtestutil.a
+
+Generic form of C test executables
+----------------------------------
+
+ #include "testutil.h"
+
+ static int my_test(void)
+ {
+ int testresult = 0; /* Assume the test will fail */
+ int observed;
+
+ observed = function(); /* Call the code under test */
+ if (!TEST_int_eq(observed, 2)) /* Check the result is correct */
+ goto end; /* Exit on failure - optional */
+
+ testresult = 1; /* Mark the test case a success */
+ end:
+ cleanup(); /* Any cleanup you require */
+ return testresult;
+ }
+
+ int setup_tests(void)
+ {
+ ADD_TEST(my_test); /* Add each test separately */
+ return 1; /* Indicate success */
+ }
+
+You should use the `TEST_xxx` macros provided by `testutil.h` to test all failure
+conditions. These macros produce an error message in a standard format if the
+condition is not met (and nothing if the condition is met). Additional
+information can be presented with the `TEST_info` macro that takes a `printf`
+format string and arguments. `TEST_error` is useful for complicated conditions,
+it also takes a `printf` format string and argument. In all cases the `TEST_xxx`
+macros are guaranteed to evaluate their arguments exactly once. This means
+that expressions with side effects are allowed as parameters. Thus,
+
+ if (!TEST_ptr(ptr = OPENSSL_malloc(..)))
+
+works fine and can be used in place of:
+
+ ptr = OPENSSL_malloc(..);
+ if (!TEST_ptr(ptr))
+
+The former produces a more meaningful message on failure than the latter.
+
+Note that the test infrastructure automatically sets up all required environment
+variables (such as `OPENSSL_MODULES`, `OPENSSL_CONF`, etc.) for the tests.
+Individual tests may choose to override the default settings as required.