unified build scheme: a first introduction

The "unified" build scheme revolves around small information files,
build.info, which each describe their own bit of everything that needs
to be built, using a mini-language described in Configurations/README.

The information in build.info file contain references to source files
and final result.  Object files are not mentioned at all, they are
simply from source files.  Because of this, all the *_obj items in
Configurations/*.conf are renamed to *_asm_src and the files listed
in the values are change from object files to their corresponding
source files.  For the sake of the other build schemes, Configure
generates corresponding *_obj entries in %target.

Furthermore, the "unified" build scheme supports having a build
directory tree separate from the source directry tree.

All paths in a build.info file is assumed to be relative to its
location, either within the source tree or within the build tree.

Reviewed-by: Andy Polyakov <appro@openssl.org>

1 files changed, 178 insertions, 28 deletions

diff --git a/Configurations/README b/Configurations/README
index 5e544b194d..ecf2b7908b 100644
--- a/Configurations/README
+++ b/Configurations/README
@@ -105,8 +105,11 @@ In each table entry, the following keys are significant:
                            string in the list is the name of the build
                            scheme.
                            Currently recognised build schemes are
-                           "mk1mf" and "unixmake".  Others may appear
-                           in the future.
+                           "mk1mf" and "unixmake" and "unified".
+                           For the "unified" build scheme, this item
+                           *must* be an array with the first being the
+                           word "unified" and the second being a word
+                           to identify the platform family.
 
         multilib        => On systems that support having multiple
                            implementations of a library (typically a
@@ -146,47 +149,48 @@ In each table entry, the following keys are significant:
                                                 export vars as
                                                 accessor functions.
 
-        cpuid_obj       => assembler implementation of cpuid code as
+        cpuid_asm_src   => assembler implementation of cpuid code as
                            well as OPENSSL_cleanse().
-                           Default to mem_clr.o
-        bn_obj          => assembler implementation of core bignum
+                           Default to mem_clr.c
+        bn_asm_src      => Assembler implementation of core bignum
                            functions.
-                           Defaults to bn_asm.o
-        ec_obj          => assembler implementation of core EC
+                           Defaults to bn_asm.c
+        ec_asm_src      => Assembler implementation of core EC
                            functions.
-        des_obj         => assembler implementation of core DES
+        des_asm_src     => Assembler implementation of core DES
                            encryption functions.
-                           Defaults to 'des_enc.o fcrypt_b.o'
-        aes_obj         => assembler implementation of core AES
+                           Defaults to 'des_enc.c fcrypt_b.c'
+        aes_asm_src     => Assembler implementation of core AES
                            functions.
-                           Defaults to 'aes_core.o aes_cbc.o'
-        bf_obj          => assembler implementation of core BF
+                           Defaults to 'aes_core.c aes_cbc.c'
+        bf_asm_src      => Assembler implementation of core BlowFish
                            functions.
-                           Defaults to 'bf_enc.o'
-        md5_obj         => assembler implementation of core MD5
+                           Defaults to 'bf_enc.c'
+        md5_asm_src     => Assembler implementation of core MD5
                            functions.
-        sha1_obj        => assembler implementation of core SHA1,
+        sha1_asm_src    => Assembler implementation of core SHA1,
                            functions, and also possibly SHA256 and
                            SHA512 ones.
-        cast_obj        => assembler implementation of core BF
+        cast_asm_src    => Assembler implementation of core CAST
                            functions.
-                           Defaults to 'c_enc.o'
-        rc4_obj         => assembler implementation of core BF
+                           Defaults to 'c_enc.c'
+        rc4_asm_src     => Assembler implementation of core RC4
                            functions.
-                           Defaults to 'rc4_enc.o rc4_skey.o'
-        rmd160_obj      => assembler implementation of core RMD160
+                           Defaults to 'rc4_enc.c rc4_skey.c'
+        rmd160_asm_src  => Assembler implementation of core RMD160
                            functions.
-        rc5_obj         => assembler implementation of core RC4
+        rc5_asm_src     => Assembler implementation of core RC5
                            functions.
-                           Defaults to 'rc5_enc.o'
-        wp_obj          => assembler implementation of core WHIRLPOOL
+                           Defaults to 'rc5_enc.c'
+        wp_asm_src      => Assembler implementation of core WHIRLPOOL
                            functions.
-        cmll_obj        => assembler implementation of core CAMELLIA
+        cmll_asm_src    => Assembler implementation of core CAMELLIA
                            functions.
-                           Defaults to 'camellia.o cmll_misc.o cmll_cbc.o'
-        modes_obj       => assembler implementation of the
-                           functions gcm_gmult_4bit and gcm_ghash_4bit.
-        padlock_obj     => assembler implementation of core parts of
+                           Defaults to 'camellia.c cmll_misc.c cmll_cbc.c'
+        modes_asm_src   => Assembler implementation of cipher modes,
+                           currently the functions gcm_gmult_4bit and
+                           gcm_ghash_4bit.
+        padlock_asm_src => Assembler implementation of core parts of
                            the padlock engine.  This is mandatory on
                            any platform where the padlock engine might
                            actually be built.
@@ -255,3 +259,149 @@ values separated by colons.  This use is deprecated.  The string form
 looked like this:
 
    "target" => "{cc}:{cflags}:{unistd}:{thread_cflag}:{sys_id}:{lflags}:{bn_ops}:{cpuid_obj}:{bn_obj}:{ec_obj}:{des_obj}:{aes_obj}:{bf_obj}:{md5_obj}:{sha1_obj}:{cast_obj}:{rc4_obj}:{rmd160_obj}:{rc5_obj}:{wp_obj}:{cmll_obj}:{modes_obj}:{padlock_obj}:{perlasm_scheme}:{dso_scheme}:{shared_target}:{shared_cflag}:{shared_ldflag}:{shared_extension}:{ranlib}:{arflags}:{multilib}"
+
+
+Build info files
+================
+
+The build.info files that are spread over the source tree contain the
+minimum information needed to build and distribute OpenSSL.  It uses a
+simple and yet fairly powerful language to determine what needs to be
+built, from what sources, and other relationships between files.
+
+For every build.info file, all file references are relative to the
+directory of the build.info file for source files, and the
+corresponding build directory for built files if the build tree
+differs from the source tree.
+
+When processed, every line is processed with the perl module
+Text::Template, using the delimiters "{-" and "-}".  The hashes
+%config and %target are passed to the perl fragments, along with
+$sourcedir and $builddir, which are the locations of the source
+directory for the current build.info file and the corresponding build
+directory, all relative to the top of the build tree.
+
+To begin with, things to be built are declared by setting specific
+variables:
+
+    PROGRAMS=foo bar
+    LIBS=libsomething
+    ENGINES=libeng
+    SCRIPTS=myhack
+    EXTRA=file1 file2
+
+Note that the files mentioned for PROGRAMS, LIBS and ENGINES *must* be
+without extensions.  The build file templates will figure them out.
+
+For each thing to be built, it is then possible to say what sources
+they are built from:
+
+    PROGRAMS=foo bar
+    SOURCE[foo]=foo.c common.c
+    SOURCE[bar]=bar.c extra.c common.c
+
+It's also possible to tell some other dependencies:
+
+    DEPEND[foo]=libsomething
+    DEPEND[libbar]=libsomethingelse
+
+(it could be argued that 'libsomething' and 'libsomethingelse' are
+source as well.  However, the files given through SOURCE are expected
+to be located in the source tree while files given through DEPEND are
+expected to be located in the build tree)
+
+For some libraries, we maintain files with public symbols and their
+slot in a transfer vector (important on some platforms).  It can be
+declared like this:
+
+    ORDINALS[libcrypto]=crypto
+
+The value is not the name of the file in question, but rather the
+argument to util/mkdef.pl that indicates which file to use.
+
+One some platforms, shared libraries come with a name that's different
+from their static counterpart.  That's declared as follows:
+
+    SHARED_NAME[libfoo]=cygfoo-{- $config{shlibver} -}
+
+The example is from Cygwin, which has a required naming convention.
+
+Sometimes, it makes sense to rename an output file, for example a
+library:
+
+    RENAME[libfoo]=libbar
+
+That lines has "libfoo" get renamed to "libbar".  While it makes no
+sense at all to just have a rename like that (why not just use
+"libbar" everywhere?), it does make sense when it can be used
+conditionally.  See a little further below for an example.
+
+For any file to be built, it's also possible to tell what extra
+include paths the build of their source files should use:
+
+    INCLUDE[foo]=include
+
+It's possible to have raw build file lines, between BEGINRAW and
+ENDRAW lines as follows:
+
+    BEGINRAW[Makefile(unix)]
+    haha.h: {- $builddir -}/Makefile
+        echo "/* haha */" > haha.h
+    ENDRAW[Makefile(unix)]
+
+The word withing square brackets is the build_file configuration item
+or the build_file configuration item followed by the second word in the
+build_scheme configuration item for the configured target within
+parenthesis as shown above.  For example, with the following relevant
+configuration items:
+
+   build_file   => "build.ninja"
+   build_scheme => [ "unified", "unix" ]
+
+... these lines will be considered:
+
+   BEGINRAW[build.ninja]
+   build haha.h: echo "/* haha */" > haha.h
+   ENDRAW[build.ninja]
+
+   BEGINRAW[build.ninja(unix)]
+   build hoho.h: echo "/* hoho */" > hoho.h
+   ENDRAW[build.ninja(unix)]
+
+See the documentation further up for more information on configuration
+items.
+
+Finally, you can have some simple conditional use of the build.info
+information, looking like this:
+
+    IF[1]
+     something
+    ELSIF[2]
+     something other
+    ELSE
+     something else
+    ENDIF
+
+The expression in square brackets is interpreted as a string in perl,
+and will be seen as true if perl thinks it is, otherwise false.  For
+example, the above would have "something" used, since 1 is true.
+
+Together with the use of Text::Template, this can be used as
+conditions based on something in the passed variables, for example:
+
+    IF[{- $config{no_shared} -}]
+      LIBS=libcrypto
+      SOURCE[libcrypto]=...
+    ELSE
+      LIBS=libfoo
+      SOURCE[libfoo]=...
+    ENDIF
+
+or:
+
+    # VMS has a cultural standard where all libraries are prefixed.
+    # For OpenSSL, the choice is 'ossl_'
+    IF[{- $config{target} =~ /^vms/ -}]
+     RENAME[libcrypto]=ossl_libcrypto
+     RENAME[libssl]=ossl_libssl
+    ENDIF