summaryrefslogtreecommitdiffstats
path: root/util/perl
AgeCommit message (Collapse)Author
2020-10-12Make OpenSSL::ParseC and OpenSSL::Ordinals treat deprecation consistentlyRichard Levitte
The triggering macro that decides if a symbol is to be considered deprecated is OPENSSL_NO_DEPRECATEDIN_x_y[_z]. OpenSSL::ParseC renames any OPENSSL_NO_DEPRECATED_x_y[_z] by inserting "IN". Reviewed-by: Paul Dale <paul.dale@oracle.com> (Merged from https://github.com/openssl/openssl/pull/13074)
2020-10-09OpenSSL::Ordinals: Add options for the writing functionsRichard Levitte
OpenSSL::Ordinals::rewrite() and OpenSSL::Ordinals::write() now take options, that are simply passed to OpenSSL::Ordinals::items(). The 'sort' option is forbidden, though, since write() already uses it, but that means it's possible to filter the output. Reviewed-by: Paul Dale <paul.dale@oracle.com> (Merged from https://github.com/openssl/openssl/pull/13092)
2020-10-06Test.pm: Add result_dir and export both result_dir and result_fileDr. David von Oheimb
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org> (Merged from https://github.com/openssl/openssl/pull/12934)
2020-10-03Configuration: add initial NonStop values in OpenSSL::configRichard Levitte
This makes Configure work it's automatic config detection, at least for the simple straightforward cases. Fixes #12972 Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com> (Merged from https://github.com/openssl/openssl/pull/12973)
2020-09-24Test.pm: Some clarifications added to the documentationDr. David von Oheimb
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org> Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/12893)
2020-09-18Fix some doc-nits and make update errorsMatt Caswell
The new lhash changes have confused some of the perl scripts so we add some fixes. Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/12860)
2020-09-18Provide basis for fixing lhash codeMatt Caswell
Following on from the earlier safestack work we provide the basis for fixing the lhash code such that unused static inline functions do not cause linker errors for applications including those headers. This brings the lhash code into line with the safestack code. Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/12860)
2020-09-16OpenSSL::ParseC: recognise inline function bodiesRichard Levitte
Function bodies in headers weren't a thing when OpenSSL::ParseC was created, at least not as clearly as they are nowadays. This module must evolve to recognise them (and promptly ignore them). Reviewed-by: Matt Caswell <matt@openssl.org> Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org> (Merged from https://github.com/openssl/openssl/pull/12882)
2020-09-13Streamline the safestack generated codeMatt Caswell
The safestack code generation was generating a little too much. Some of it could be done with a normal macro. Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: Paul Dale <paul.dale@oracle.com> (Merged from https://github.com/openssl/openssl/pull/12781)
2020-09-13Remove some safestack things that are no longer neededMatt Caswell
... and add SKM_DEFINE_STACK_OF_INTERNAL Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: Paul Dale <paul.dale@oracle.com> (Merged from https://github.com/openssl/openssl/pull/12781)
2020-09-13Fix stacks of OPENSSL_STRING, OPENSSL_CSTRING and OPENSSL_BLOCKMatt Caswell
Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: Paul Dale <paul.dale@oracle.com> (Merged from https://github.com/openssl/openssl/pull/12781)
2020-09-13Fix safestack issues in ssl.hMatt Caswell
We fix 3 problems with safestack: - Including an openssl header file without linking against libcrypto can cause compilation failures (even if the app does not otherwise need to link against libcrypto). See issue #8102 - Recent changes means that applications in no-deprecated builds will need to include additional macro calls in the source code for all stacks that they need to use - which is an API break. This changes avoids that necessity. - It is not possible to write code using stacks that works in both a no-deprecated and a normal build of OpenSSL. See issue #12707. Fixes #12707 Contains a partial fix for #8102. A similar PR will be needed for hash to fully fix. Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: Paul Dale <paul.dale@oracle.com> (Merged from https://github.com/openssl/openssl/pull/12781)
2020-08-27Ignore vendor name in Clang version number.Jung-uk Kim
For example, FreeBSD prepends "FreeBSD" to version string, e.g., FreeBSD clang version 11.0.0 (git@github.com:llvm/llvm-project.git llvmorg-11.0.0-rc2-0-g414f32a9e86) Target: x86_64-unknown-freebsd13.0 Thread model: posix InstalledDir: /usr/bin This prevented us from properly detecting AVX support, etc. CLA: trivial Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: Paul Dale <paul.dale@oracle.com> Reviewed-by: Ben Kaduk <kaduk@mit.edu> (Merged from https://github.com/openssl/openssl/pull/12725)
2020-07-04util/perl/OpenSSL/config.pm: Fix /armv[7-9].*-.*-linux2/Richard Levitte
This entry added the macro B_ENDIAN when it shouldn't have. Fixes #12332 Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/12335)
2020-07-04util/perl/OpenSSL/config.pm: move misplaced Windows and VMS entriesRichard Levitte
OpenSSL::config::guess_system() is supposed to return system triplets. However, for Windows and VMS, it returned the final OpenSSL config target instead. We move the entries for them to the table that OpenSSL::config::map_guess() uses, so it can properly convert the input triplet to an OpenSSL config target. Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/12339)
2020-06-28util/perl/OpenSSL/config.pm: refactor guess_system()Richard Levitte
There's no reason to have two different tables, when we can simply detect if the tuple elements are code or scalar. Furthermore, order is important in some cases, and that order is harder not to say impossible when maintaining two tables. Reviewed-by: Tim Hudson <tjh@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11230)
2020-06-28util/perl/OpenSSL/config.pm: remove expand() and use evalRichard Levitte
The strings we expand contain other variable references than just ${MACHINE}. Instead of having to remember what to expand, we simply evaluate the string as a, well, string. Reviewed-by: Tim Hudson <tjh@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11230)
2020-06-28util/perl/OpenSSL/config.pm: refactor map_guess()Richard Levitte
map_guess() is now table driven, just like get_system(). Additionally, it now takes a config hash table and returns one of its own. This way, 'Configure' can pass whatever it has already found to OpenSSL::config::get_platform(), and easily merge the returned hash table into its %config. This also gets rid of variables that we no longer need. That includes $PERL and all the $__CNF_ environment variables. Reviewed-by: Tim Hudson <tjh@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11230)
2020-06-28util/perl/OpenSSL/config.pm, Configure: move check of target with compilerRichard Levitte
Previously, ./config would check if "$target-$CC", then "$target" exists and choose the one that does. This is now moved to Configure. Reviewed-by: Tim Hudson <tjh@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11230)
2020-06-28util/perl/OpenSSL/config.pm: Rework determining compiler informationRichard Levitte
determine_compiler_settings() has been refactored to: - find a compiler if none has been given by the user - allow platform specific overrides, but only when the user didn't already specify a desired compiler - figure out the compiler vendor and version, making sure that the version number is deterministic - gather platform specific compiler information Reviewed-by: Tim Hudson <tjh@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11230)
2020-06-28Remove OpenSSL::config::main(), it's not necessaryRichard Levitte
This also remove all option parsing. We leave that to Configure. Reviewed-by: Tim Hudson <tjh@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11230)
2020-06-28util/perl/OpenSSL/config.pm: Prefer POSIX::uname() over piping the commandRichard Levitte
POSIX::uname() has the advantage to work on non-POSIX systems as well, such as the Windows command prompt and VMS. Reviewed-by: Tim Hudson <tjh@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11230)
2020-06-28util/perl/OpenSSL/config.pm: Don't detect removed directories inRichard Levitte
This is much better handled in Configure. [There's another PR moving this to Configure, so this commit should eventually disappear because rebase] Reviewed-by: Tim Hudson <tjh@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11230)
2020-06-28Initial rewrite of config as a Perl moduleRich Salz
- Use $^X; to find perl. - Big re-ordering: Put all variables at the top, move most inline code into functions. The heart of the script now basically just calls functions to do its work. - Unify warning text, add -w option - Don't use needless (subshells) - Ensure Windows gets a VC-xxx option - Make config a perl module - Top-level "config" command-line is a dummy that just calls the module. Added module stuff so that it can be called from Configure. Reviewed-by: Tim Hudson <tjh@openssl.org> Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11230)
2020-06-10run_tests.pl: Add options for focussing output on failed (sub-)testsDr. David von Oheimb
VERBOSE_FAILURES_ONLY (VFO): verbose output only of failed (sub-)tests VERBOSE_FAILURES_PROGRESS (VFP): in addition summary for passed tests This adds a workaroud for TAP::Parser not coping well with indentation. Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/12088)
2020-05-15Update copyright yearMatt Caswell
Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11839)
2020-05-03util/perl/OpenSSL/OID.pm: remove the included unit testRichard Levitte
The unit test uses features that appeared in perl 5.12, and is therefore a source of trouble when building. Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11704)
2020-04-24In OpenSSL builds, declare STACK for datatypes ...Rich Salz
... and only *define* them in the source files that need them. Use DEFINE_OR_DECLARE which is set appropriately for internal builds and not non-deprecated builds. Deprecate stack-of-block Better documentation Move some ASN1 struct typedefs to types.h Update ParseC to handle this. Most of all, ParseC needed to be more consistent. The handlers are "recursive", in so far that they are called again and again until they terminate, which depends entirely on what the "massager" returns. There's a comment at the beginning of ParseC that explains how that works. {Richard Levtte} Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com> Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org> (Merged from https://github.com/openssl/openssl/pull/10669)
2020-04-23Update copyright yearMatt Caswell
Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11616)
2020-04-11Rework util/find-doc-nits to distinguish internal documentationRichard Levitte
We didn't really distinguish internal and public documentation, or matched that with the state of the documented symbols. we therefore needed to rework the logic to account for the state of each symbol. To simplify things, and make them consistent, we load all of util/*.num, util/*.syms and util/missing*.txt unconditionally. Also, we rework the reading of the manuals to happen only once (or well, not quite, Pod::Checker reads from file too, but at the very least, our script isn't reading the same file multiple times). Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11476)
2020-04-09OpenSSL::OID: Don't use List::UtilRichard Levitte
It turns out that the pairwise functions of List::Util came into perl far later than 5.10.0. We can't use that under those conditions, so must revert to a quick internal implementation of the functions we're after. Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org> (Merged from https://github.com/openssl/openssl/pull/11503)
2020-04-07Add perl support to parse and DER encode ASN.1 OID specsRichard Levitte
We have an old OID database that's not as readable as would be desired, and we have spots with hand coded DER for well known OIDs. The perl modules added here give enough support that we can parse OBJECT IDENTIFIER definitions and encode them as DER. OpenSSL::OID is a general OID parsing and encoding of ASN.1 definitions, and supports enough of the X.680 syntax to understand what we find in RFCs and similar documents and produce the DER encoding for them. oids_to_c is a specialized module to convert the DER encoding from OpenSSL::OID to C code. This is primarily useful in file templates that are processed with util/dofile.pl. Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11450)
2020-02-27TEST: add util/wrap.pl and use itRichard Levitte
util/wrap.pl is a script that defines the environment variables OPENSSL_ENGINES and OPENSSL_MODULES, then calls the command line that's given as its arguments. On a POSIX platform, the command line call is done via util/shlib_wrap.sh to ensure that the shared library paths are correct. For other platforms, util/wrap.pl currently assumes that similar things are already in place through other means. Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com> (Merged from https://github.com/openssl/openssl/pull/11110)
2020-02-18TEST: Optionally silence OpenSSL::Test::setup()Richard Levitte
test/generate_ssl_tests.pl uses OpenSSL::Test to get to some of its practical location functions. A recent note in the setup() code made its result not quite match the original (we do check that), so there's a need to silence setup(), which we do with a simple optional argument. Reviewed-by: Paul Dale <paul.dale@oracle.com> (Merged from https://github.com/openssl/openssl/pull/11080)
2020-02-18TEST: Create test specific output directoriesRichard Levitte
We had all tests run with test/test-runs/ as working directory, and tests cleaned up after themselves... which is well and good, until you want to have a look at what went wrong when a complex test fails, and you have to recreate everything it does manually. To remedy this, we have OpenSSL::Test create the result directory dynamically (and cleaning it up first if it's already there) and let the test recipe have that as working directory. Test recipes are now encouraged to name their diverse output files uniquely, and not to clean them up, to allow a developer to have a look at the files that were produced. With continuous integration that allows this, the result directories could also be archived and be left as a build artifact. Reviewed-by: Paul Dale <paul.dale@oracle.com> (Merged from https://github.com/openssl/openssl/pull/11080)
2020-01-27OpenSSL::Test: bring back the relative pathsRichard Levitte
Because there was a bug in File::Spec::Unix' abs2rel when it was given relative paths as both PATH and BASE arguments, the directories we deal with were made to be all absolute. Unfortunately, this meant getting paths in our verbose test output which are difficult to use anywhere else (such as a separate test build made for comparison), due to the constant need to edit all the paths all the time. We're therefore getting back the relative paths, by doing an extra abs2rel() in __srctop_file, __srctop_dir, __bldtop_file and __bldtop_dir, with a 'Cwd::getcwd' call as BASE argument. Fixes #10628 Reviewed-by: Matt Caswell <matt@openssl.org> Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com> (Merged from https://github.com/openssl/openssl/pull/10913)
2019-12-21OpenSSL::Util::extract_pod_info(): Allow invisible namesRichard Levitte
This should be very unusual, but we do have a case of a name we don't want to display. Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/10621)
2019-12-21OpenSSL::Util::extract_pod_info(): Read the POD one paragraph at a timeRichard Levitte
POD files should always be treated this way Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/10621)
2019-12-21perl: OpenSSL::Util::Pod::extract_pod_info() now saves the file contentsRichard Levitte
Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/10621)
2019-12-11Fix some typosVeres Lajos
Reported-by: misspell-fixer <https://github.com/vlajos/misspell-fixer> CLA: trivial Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com> Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/10544)
2019-12-10Remove handling of outdated macro'sRich Salz
DECLARE_STACK_OF was renamed to DEFINE_STACK_OF in commit 8588571. Expanded the only use of TYPEDEF_{D2I,I2D,D2I2D}_OF, so that they can easily be removed in a future release Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com> (Merged from https://github.com/openssl/openssl/pull/10464)
2019-11-12util/perl/OpenSSL/ParseC.pm: Fix 'extern "C" DEPRECATEDIN_x_y(...)'Richard Levitte
The parser expected an 'extern "C"' followed by a single declaration to always end with a semicolon. Then came along something like this: extern "C" DEPRECATEDIN_3_0(int ERR_load_KDF_strings(void)) This change adjusts the detector of 'extern "C"' to also take in accound a declaration that ends with a parenthesis. Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org> (Merged from https://github.com/openssl/openssl/pull/10368)
2019-11-07Change the logic and behaviour surrounding '--api' and 'no-deprecated'Richard Levitte
At some point in time, there was a 'no-deprecated' configuration option, which had the effect of hiding all declarations of deprecated stuff, i.e. make the public API look like they were all removed. At some point in time, there was a '--api' configuration option, which had the effect of having the public API look like it did in the version given as value, on a best effort basis. In practice, this was used to get different implementations of BN_zero(), depending on the desired API compatibility level. At some later point in time, '--api' was changed to mean the same as 'no-deprecated', but only for the deprecations up to and including the desired API compatibility level. BN_zero() has been set to the pre-1.0.0 implementation ever since, unless 'no-deprecation' has been given. This change turns these options back to their original meaning, but with the slight twist that when combined, i.e. both '--api' and 'no-deprecated' is given, the declarations that are marked deprecated up to an including the desired API compatibility level are hidden, simulating that they have been removed. If no desired API compatibility level has been given, then configuration sets the current OpenSSL version by default. Furthermore, the macro OPENSSL_API_LEVEL is now used exclusively to check what API compatibility level is desired. For checking in code if `no-deprecated` has been configured for the desired API compatibility level, macros for each supported level is generated, such as OPENSSL_NO_DEPRECATED_1_1_1, corresponding to the use of DEPRECATEDIN_ macros, such as DEPRECATEDIN_1_1_1(). Just like before, to set an API compatibility level when building an application, define OPENSSL_API_COMPAT with an appropriate value. If it's desirable to hide deprecated functions up to and including that level, additionally define OPENSSL_NO_DEPRECATED (the value is ignored). Reviewed-by: Tim Hudson <tjh@openssl.org> (Merged from https://github.com/openssl/openssl/pull/10364)
2019-11-05OpenSSL::Ordinals: when validating, collect statistics on unassigned symsRichard Levitte
If a script wants to display how many symbols have assigned numbers and how many don't, this gives them those numbers. Reviewed-by: Matt Caswell <matt@openssl.org> Reviewed-by: Paul Dale <paul.dale@oracle.com> (Merged from https://github.com/openssl/openssl/pull/10348)
2019-11-05OpenSSL::Ordinals: add a renumber() function, to assign unassigned symbolsRichard Levitte
This should be used when it's time to assign constant numbers to the unassigned symbols. Reviewed-by: Matt Caswell <matt@openssl.org> Reviewed-by: Paul Dale <paul.dale@oracle.com> (Merged from https://github.com/openssl/openssl/pull/10348)
2019-11-05OpenSSL::Ordinals: Handle symbols with unassigned ordinal numbersRichard Levitte
We preserve the number or '?' or '?+', but assign numbers internally on the latter, to ensure we keep the order of the input. Reviewed-by: Matt Caswell <matt@openssl.org> Reviewed-by: Paul Dale <paul.dale@oracle.com> (Merged from https://github.com/openssl/openssl/pull/10348)
2019-09-12util/dofile.pl, util/perl/OpenSSL/Template.pm: move parts of dofile.plRichard Levitte
We make a module OpenSSL::Template from the central parts of util/dofile.pl, and also reduce the amount of ugly code with more proper use of Text::Template. OpenSSL::Template is a simply subclass of Text::Template. Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/9693)
2019-09-12util/perl/OpenSSL/Test.pm: Disable stdout/stderr redirection on non-verbosityRichard Levitte
... except on VMS, where output from executed programs doesn't seem to be captured properly by Test::Harness or TAP::Harness. Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/9862)
2019-09-12Rework the perl fallback functionalityRichard Levitte
The module with_fallback.pm was kind of clunky and required a transfer module. This change replaces if with a much more generic pragma type module, which simply appends given directories to @INC (as opposed to the 'lib' pragma, which prepends the directories to @INC). This also supports having a file MODULES.txt with sub-directories to modules. This ensures that we don't have to spray individual module paths throughout our perl code, but can have them collected in one place. (do note that there is a 'fallback' module on CPAN. However, it isn't part of the core perl, and it has no support the any MODULES.txt kind of construct) Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/9826)
2019-09-06Teach TLSProxy how to parse CertificateRequest messagesMatt Caswell
We also use this in test_tls13messages to check that the extensions we expect to see in a CertificateRequest are there. Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org> (Merged from https://github.com/openssl/openssl/pull/9780)