unified build scheme: add and document the "unified" driving engine

common.tmpl will be used together with the template build file, and is
the engine that connects the information gathered from all the
build.info files with making the build file itself.

This file expects there to be a template section in the build file
template that defines a number perl functions designed to return
strings with appropriate lines for the build system at hand.  The
exact functions, what they can expect as arguments and what output
they're expected to produce is documented in Configurations/README.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

2 files changed, 290 insertions, 1 deletions

diff --git a/Configurations/README b/Configurations/README
index fb94aa723e..75907a6d5c 100644
--- a/Configurations/README
+++ b/Configurations/README
@@ -1,5 +1,5 @@
 Configurations of OpenSSL target platforms
-------------------------------------------
+==========================================
 
 Target configurations are a collection of facts that we know about
 different platforms and their capabilities.  We organise them in a
@@ -427,3 +427,174 @@ or:
      RENAME[libcrypto]=ossl_libcrypto
      RENAME[libssl]=ossl_libssl
     ENDIF
+
+
+Build-file programming with the "unified" build system
+======================================================
+
+"Build files" are called "Makefile" on Unix-like operating systems,
+"descrip.mms" for MMS on VMS, "makefile" for nmake on Windows, etc.
+
+To use the "unified" build system, the target configuration needs to
+set the three items 'build_scheme', 'build_file' and 'build_command'.
+In the rest of this section, we will assume that 'build_scheme' is set
+to "unified" (see the configurations documentation above for the
+details).
+
+For any name given by 'build_file', the "unified" system expects a
+template file in Configurations/ named like the build file, with
+".tmpl" appended, or in case of possible ambiguity, a combination of
+the second 'build_scheme' list item and the 'build_file' name.  For
+example, if 'build_file' is set to "Makefile", the template could be
+Configurations/Makefile.tmpl or Configurations/unix-Makefile.tmpl.
+In case both Configurations/unix-Makefile.tmpl and
+Configurations/Makefile.tmpl are present, the former takes
+precedence.
+
+The build-file template is processed with the perl module
+Text::Template, using "{-" and "-}" as delimiters that enclose the
+perl code fragments that generate configuration-dependent content.
+Those perl fragments have access to all the hash variables from
+configdata.pem.
+
+The build-file template is expected to define at least the following
+perl functions in a perl code fragment enclosed with "{-" and "-}".
+They are all expected to return a string with the lines they produce.
+
+    src2dep     - function that produces build file lines to get the
+                  dependencies for an object file into a dependency
+                  file.
+
+                  It's called like this:
+
+                        src2dep(obj => "PATH/TO/objectfile",
+                                srcs => [ "PATH/TO/sourcefile", ... ],
+                                incs => [ "INCL/PATH", ... ]);
+
+                  'obj' has the dependent object file as well as
+                  object file the dependencies are for; it's *without*
+                  extension, src2dep() is expected to add that.
+                  'srcs' has the list of source files to build the
+                  object file, with the first item being the source
+                  file that directly corresponds to the object file.
+                  'incs' is a list of include file directories.
+
+    src2obj     - function that produces build file lines to build an
+                  object file from source files and associated data.
+
+                  It's called like this:
+
+                        src2obj(obj => "PATH/TO/objectfile",
+                                srcs => [ "PATH/TO/sourcefile", ... ],
+                                deps => [ "dep1", ... ],
+                                incs => [ "INCL/PATH", ... ]);
+
+                  'obj' has the intended object file *without*
+                  extension, src2obj() is expected to add that.
+                  'srcs' has the list of source files to build the
+                  object file, with the first item being the source
+                  file that directly corresponds to the object file.
+                  'deps' is a list of dependencies.  'incs' is a list
+                  of include file directories.
+
+    obj2lib     - function that produces build file lines to build a
+                  static library file ("libfoo.a" in Unix terms) from
+                  object files.
+
+                  called like this:
+
+                        obj2lib(lib => "PATH/TO/libfile",
+                                objs => [ "PATH/TO/objectfile", ... ]);
+
+                  'lib' has the intended library file name *without*
+                  extension, obj2lib is expected to add that.  'objs'
+                  has the list of object files (also *without*
+                  extension) to build this library.
+
+    libobj2shlib - function that produces build file lines to build a
+                  shareable object library file ("libfoo.so" in Unix
+                  terms) from the corresponding static library file
+                  or object files.
+
+                  called like this:
+
+                        libobj2shlib(shlib => "PATH/TO/shlibfile",
+                                     lib => "PATH/TO/libfile",
+                                     objs => [ "PATH/TO/objectfile", ... ],
+                                     deps => [ "PATH/TO/otherlibfile", ... ],
+                                     ordinals => [ "word", "/PATH/TO/ordfile" ]);
+
+                  'lib' has the intended library file name *without*
+                  extension, libobj2shlib is expected to add that.
+                  'shlib' has the correcponding shared library name
+                  *without* extension.  'deps' has the list of other
+                  libraries (also *without* extension) this library
+                  needs to be linked with.  'objs' has the list of
+                  object files (also *without* extension) to build
+                  this library.  'ordinals' MAY be present, and when
+                  it is, its value is an array where the word is
+                  "crypto" or "ssl" and the file is one of the ordinal
+                  files util/libeay.num or util/ssleay.num in the
+                  source directory.
+
+                  This function has a choice; it can use the
+                  corresponding static library as input to make the
+                  shared library, or the list of object files.
+
+    obj2dynlib  - function that produces build file lines to build a
+                  dynamically loadable library file ("libfoo.so" on
+                  Unix) from object files.
+
+                  called like this:
+
+                        obj2dynlib(lib => "PATH/TO/libfile",
+                                   objs => [ "PATH/TO/objectfile", ... ],
+                                   deps => [ "PATH/TO/otherlibfile",
+                                   ... ]);
+
+                  This is almost the same as libobj2shlib, but the
+                  intent is to build a shareable library that can be
+                  loaded in runtime (a "plugin"...).  The differences
+                  are subtle, one of the most visible ones is that the
+                  resulting shareable library is produced from object
+                  files only.
+
+    obj2bin     - function that produces build file lines to build an
+                  executable file from object files.
+
+                  called like this:
+
+                        obj2bin(bin => "PATH/TO/binfile",
+                                objs => [ "PATH/TO/objectfile", ... ],
+                                deps => [ "PATH/TO/libfile", ... ]);
+
+                  'bin' has the intended executable file name
+                  *without* extension, obj2bin is expected to add
+                  that.  'objs' has the list of object files (also
+                  *without* extension) to build this library.  'deps'
+                  has the list of library files (also *without*
+                  extension) that the programs needs to be linked
+                  with.
+
+    in2script   - function that produces build file lines to build a
+                  script file from some input.
+
+                  called like this:
+
+                        in2script(script => "PATH/TO/scriptfile",
+                                  sources => [ "PATH/TO/infile", ... ]);
+
+                  'script' has the intended script file name.
+                  'sources' has the list of source files to build the
+                  resulting script from.
+
+In all cases, file file paths are relative to the build tree top, and
+the build file actions run with the build tree top as current working
+directory.
+
+Make sure to end the section with these functions with a string that
+you thing is apropriate for the resulting build file.  If nothing
+else, end it like this:
+
+      "";       # Make sure no lingering values end up in the Makefile
+    -}
diff --git a/Configurations/common.tmpl b/Configurations/common.tmpl
new file mode 100644
index 0000000000..a750e21a33
--- /dev/null
+++ b/Configurations/common.tmpl
@@ -0,0 +1,118 @@
+{- # -*- Mode: perl -*-
+
+     my $a;
+
+ # resolvedepends and reducedepends work in tandem to make sure
+ # there are no duplicate dependencies and that they are in the
+ # right order.  This is especially used to sort the list of
+ # libraries that a build depends on.
+ sub resolvedepends {
+     my $thing = shift;
+     my @listsofar = @_;    # to check if we're looping
+     my @list = @{$unified_info{depends}->{$thing}};
+     my @newlist = ();
+     if (scalar @list) {
+         foreach my $item (@list) {
+             # It's time to break off when the dependency list starts looping
+             next if grep { $_ eq $item } @listsofar;
+             push @newlist, $item, resolvedepends($item, @listsofar, $item);
+         }
+     }
+     @newlist;
+ }
+ sub reducedepends {
+     my @list = @_;
+     my @newlist = ();
+     while (@list) {
+         my $item = shift @list;
+         push @newlist, $item
+             unless grep { $item eq $_ } @list;
+     }
+     @newlist;
+ }
+
+ # doobj is responsible for producing all the recipes that build
+ # object files as well as dependency files.
+ sub doobj {
+     my $obj = shift;
+     (my $obj_no_o = $obj) =~ s|\.o$||;
+     my $bin = shift;
+     if (@{$unified_info{sources}->{$obj}}) {
+         $OUT .= src2obj(obj => $obj_no_o,
+                         srcs => $unified_info{sources}->{$obj},
+                         deps => [ reducedepends(resolvedepends($obj)) ],
+                         incs => [ @{$unified_info{includes}->{$bin}},
+                                   @{$unified_info{includes}->{$obj}} ]);
+         $OUT .= src2dep(obj => $obj_no_o,
+                         srcs => $unified_info{sources}->{$obj},
+                         incs => [ @{$unified_info{includes}->{$bin}},
+                                   @{$unified_info{includes}->{$obj}} ]);
+     }
+ }
+
+ # dolib is responsible for building libraries.  It will call
+ # libobj2shlib is shared libraries are produced, and obj2lib in all
+ # cases.  It also makes sure all object files for the library are
+ # built.
+ sub dolib {
+     my $lib = shift;
+     if (!$config{no_shared}) {
+         my %ordinals =
+             $unified_info{ordinals}->{$lib}
+             ? (ordinals => $unified_info{ordinals}->{$lib}) : ();
+         $OUT .= libobj2shlib(shlib => $unified_info{sharednames}->{$lib},
+                              lib => $lib,
+                              objs => [ map { (my $x = $_) =~ s|\.o$||; $x }
+                                        @{$unified_info{sources}->{$lib}} ],
+                              deps => [ reducedepends(resolvedepends($lib)) ],
+                              %ordinals);
+     }
+     $OUT .= obj2lib(lib => $lib,
+                     objs => [ map { (my $x = $_) =~ s|\.o$||; $x }
+                               @{$unified_info{sources}->{$lib}} ]);
+     map { doobj($_, $lib, intent => "lib") } @{$unified_info{sources}->{$lib}};
+ }
+
+ # doengine is responsible for building engines.  It will call
+ # obj2dynlib, and also makes sure all object files for the library
+ # are built.
+ sub doengine {
+     my $lib = shift;
+     $OUT .= obj2dynlib(lib => $lib,
+                        objs => [ map { (my $x = $_) =~ s|\.o$||; $x }
+                                  @{$unified_info{sources}->{$lib}} ],
+                        deps => [ resolvedepends($lib) ]);
+     map { doobj($_, $lib, intent => "lib") } @{$unified_info{sources}->{$lib}};
+ }
+
+ # dobin is responsible for building programs.  It will call obj2bin,
+ # and also makes sure all object files for the library are built.
+ sub dobin {
+     my $bin = shift;
+     my $deps = [ reducedepends(resolvedepends($bin)) ];
+     $OUT .= obj2bin(bin => $bin,
+                     objs => [ map { (my $x = $_) =~ s|\.o$||; $x }
+                               @{$unified_info{sources}->{$bin}} ],
+                     deps => $deps);
+     map { doobj($_, $bin, intent => "bin") } @{$unified_info{sources}->{$bin}};
+ }
+
+ # dobin is responsible for building scripts from templates.  It will
+ # call in2script.
+ sub doscript {
+     my $script = shift;
+     $OUT .= in2script(script => $script,
+                       sources => $unified_info{sources}->{$script});
+ }
+
+ # Build all known libraries, engines, programs and scripts.
+ # Everything else will be handled as a consequence.
+ map { dolib($_) } @{$unified_info{libraries}};
+ map { doengine($_) } @{$unified_info{engines}};
+ map { dobin($_) } @{$unified_info{programs}};
+ map { doscript($_) } @{$unified_info{scripts}};
+
+ # Finally, should there be any applicable BEGINRAW/ENDRAW sections,
+ # they are added here.
+ $OUT .= $_."\n" foreach(@{$unified_info{rawlines}});
+-}