summaryrefslogtreecommitdiffstats
path: root/crypto/chacha/asm
AgeCommit message (Collapse)Author
2020-10-22Fix aarch64 static linking into shared libraries (see issue #10842 and pull ↵Romain Geissler
request #11464) This tries to fix the following link errors on aarch64 when using OpenSSL 3.0.0 alpha 6, compiling it with "no-shared" and -fPIC in CFLAGS, then trying to use the resulting OpenSSL static libraries in the build of elfutils, which embed libcrypto.a into libdebuginfo.so, which hides all symbols (except the libdebuginfod ones) by default: /opt/1A/toolchain/aarch64-v4.0.86/lib/gcc/aarch64-1a-linux-gnu/8.4.1/../../../../aarch64-1a-linux-gnu/bin/ld: /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-sha1-armv8.o): relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-sha1-armv8.o): in function `sha1_block_data_order': (.text+0x0): dangerous relocation: unsupported relocation /opt/1A/toolchain/aarch64-v4.0.86/lib/gcc/aarch64-1a-linux-gnu/8.4.1/../../../../aarch64-1a-linux-gnu/bin/ld: /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-chacha-armv8.o): relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-chacha-armv8.o): in function `ChaCha20_ctr32': (.text+0x6c): dangerous relocation: unsupported relocation /opt/1A/toolchain/aarch64-v4.0.86/lib/gcc/aarch64-1a-linux-gnu/8.4.1/../../../../aarch64-1a-linux-gnu/bin/ld: /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-sha256-armv8.o): relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-sha256-armv8.o): in function `sha256_block_data_order': (.text+0x0): dangerous relocation: unsupported relocation /opt/1A/toolchain/aarch64-v4.0.86/lib/gcc/aarch64-1a-linux-gnu/8.4.1/../../../../aarch64-1a-linux-gnu/bin/ld: /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-sha512-armv8.o): relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-sha512-armv8.o): in function `sha512_block_data_order': (.text+0x0): dangerous relocation: unsupported relocation /opt/1A/toolchain/aarch64-v4.0.86/lib/gcc/aarch64-1a-linux-gnu/8.4.1/../../../../aarch64-1a-linux-gnu/bin/ld: /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-poly1305-armv8.o): relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-poly1305-armv8.o): in function `poly1305_init': (.text+0x14): dangerous relocation: unsupported relocation /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-poly1305-armv8.o): in function `poly1305_emit_neon': (.text+0x8e4): relocation truncated to fit: R_AARCH64_CONDBR19 against symbol `poly1305_emit' defined in .text section in /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-poly1305-armv8.o) In poly1305-armv8.pl, hide symbols the same way they are hidden in poly1305-x86_64.pl. Reviewed-by: Kurt Roeckx <kurt@roeckx.be> Reviewed-by: Paul Dale <paul.dale@oracle.com> (Merged from https://github.com/openssl/openssl/pull/13056)
2020-08-27Ignore vendor name in Clang version number.Jung-uk Kim
For example, FreeBSD prepends "FreeBSD" to version string, e.g., FreeBSD clang version 11.0.0 (git@github.com:llvm/llvm-project.git llvmorg-11.0.0-rc2-0-g414f32a9e86) Target: x86_64-unknown-freebsd13.0 Thread model: posix InstalledDir: /usr/bin This prevented us from properly detecting AVX support, etc. CLA: trivial Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: Paul Dale <paul.dale@oracle.com> Reviewed-by: Ben Kaduk <kaduk@mit.edu> (Merged from https://github.com/openssl/openssl/pull/12725)
2020-04-23Update copyright yearMatt Caswell
Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/11616)
2020-02-17Also check for errors in x86_64-xlate.pl.David Benjamin
In https://github.com/openssl/openssl/pull/10883, I'd meant to exclude the perlasm drivers since they aren't opening pipes and do not particularly need it, but I only noticed x86_64-xlate.pl, so arm-xlate.pl and ppc-xlate.pl got the change. That seems to have been fine, so be consistent and also apply the change to x86_64-xlate.pl. Checking for errors is generally a good idea. Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: David Benjamin <davidben@google.com> (Merged from https://github.com/openssl/openssl/pull/10930)
2020-01-22Do not silently truncate files on perlasm errorsDavid Benjamin
If one of the perlasm xlate drivers crashes, OpenSSL's build will currently swallow the error and silently truncate the output to however far the driver got. This will hopefully fail to build, but better to check such things. Handle this by checking for errors when closing STDOUT (which is a pipe to the xlate driver). Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: Tim Hudson <tjh@openssl.org> Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org> (Merged from https://github.com/openssl/openssl/pull/10883)
2020-01-17For all assembler scripts where it matters, recognise clang > 9.xRichard Levitte
Fixes #10853 Reviewed-by: Paul Dale <paul.dale@oracle.com> (Merged from https://github.com/openssl/openssl/pull/10855)
2019-11-28chacha/asm/chacha-armv8.pl: preserve FP registers d8 and d9 correctlyArd Biesheuvel
Depending on the size of the input, we may take different paths through the accelerated arm64 ChaCha20 routines, each of which use a different subset of the FP registers, some of which need to be preserved and restored, as required by the AArch64 calling convention (AAPCS64) In some cases, (e.g., when the input size is 640 bytes), we call the 512 byte NEON path followed directly by the scalar path, and in this case, we preserve and restore d8 and d9, only to clobber them again immediately before handing over to the scalar path which does not touch the FP registers at all, and hence does not restore them either. Fix this by moving the restoration of d8 and d9 to a later stage in the 512 byte routine, either before calling the scalar path, or when exiting the function. Fixes #10470 CLA: trivial Reviewed-by: Paul Dale <paul.dale@oracle.com> Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/10497)
2019-11-05s390x assembly pack: perlasm module updatePatrick Steuer
- add instructions: clfi, stck, stckf, kdsa - clfi and clgfi belong to extended-immediate (not long-displacement) - some cleanup Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com> Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/10346)
2019-09-16Unify all assembler file generatorsRichard Levitte
They now generally conform to the following argument sequence: script.pl "$(PERLASM_SCHEME)" [ C preprocessor arguments ... ] \ $(PROCESSOR) <output file> However, in the spirit of being able to use these scripts manually, they also allow for no argument, or for only the flavour, or for only the output file. This is done by only using the last argument as output file if it's a file (it has an extension), and only using the first argument as flavour if it isn't a file (it doesn't have an extension). While we're at it, we make all $xlate calls the same, i.e. the $output argument is always quoted, and we always die on error when trying to start $xlate. There's a perl lesson in this, regarding operator priority... This will always succeed, even when it fails: open FOO, "something" || die "ERR: $!"; The reason is that '||' has higher priority than list operators (a function is essentially a list operator and gobbles up everything following it that isn't lower priority), and since a non-empty string is always true, so that ends up being exactly the same as: open FOO, "something"; This, however, will fail if "something" can't be opened: open FOO, "something" or die "ERR: $!"; The reason is that 'or' has lower priority that list operators, i.e. it's performed after the 'open' call. Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/9884)
2019-04-25s390x assembly pack: remove chacha20 dependency on non-base memnonicsPatrick Steuer
Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com> Reviewed-by: Paul Dale <paul.dale@oracle.com> Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/8181)
2019-04-17chacha/asm/chacha-armv8.pl: replace 3+1 code paths with 4+1.Andy Polyakov
The change is triggered by ThunderX2 where 3+1 was slower than scalar code path, but it helps all processors [to handle <512 inputs]. Reviewed-by: Tim Hudson <tjh@openssl.org> Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/8776)
2019-04-17ARM64 assembly pack: add ThunderX2 results.Andy Polyakov
Reviewed-by: Tim Hudson <tjh@openssl.org> Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/8776)
2019-03-29IA64 assembly pack: add {chacha|poly1305}-ia64 modules.Andy Polyakov
Reviewed-by: Paul Dale <paul.dale@oracle.com> Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/8540)
2019-03-11s390x assembly pack: import chacha from cryptogams repoPatrick Steuer
featuring 6x"horizontal" code path which is up to 25% faster than present 4x"vertical" for larger blocks. Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com> Reviewed-by: Matt Caswell <matt@openssl.org> Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/8287)
2019-02-16ARM64 assembly pack: make it Windows-friendly.Andy Polyakov
"Windows friendliness" means a) unified PIC-ification, unified across all platforms; b) unified commantary delimiter; c) explicit ldur/stur, as Visual Studio assembler can't automatically encode ldr/str as ldur/stur when needed. Reviewed-by: Paul Dale <paul.dale@oracle.com> Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/8256)
2019-02-16ARM assembly pack: make it Windows-friendly.Andy Polyakov
"Windows friendliness" means a) flipping .thumb and .text directives, b) always generate Thumb-2 code when asked(*); c) Windows-specific references to external OPENSSL_armcap_P. (*) so far *some* modules were compiled as .code 32 even if Thumb-2 was targeted. It works at hardware level because processor can alternate between the modes with no overhead. But clang --target=arm-windows's builtin assembler just refuses to compile .code 32... Reviewed-by: Paul Dale <paul.dale@oracle.com> Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/8252)
2019-02-16s390x assembly pack: fix formal interface bug in chacha modulePatrick Steuer
Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com> Reviewed-by: Tim Hudson <tjh@openssl.org> Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/8257)
2019-02-12AArch64 assembly pack: authenticate return addresses.Andy Polyakov
ARMv8.3 adds pointer authentication extension, which in this case allows to ensure that, when offloaded to stack, return address is same at return as at entry to the subroutine. The new instructions are nops on processors that don't implement the extension, so that the vetification is backward compatible. Reviewed-by: Kurt Roeckx <kurt@roeckx.be> Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/8205)
2019-01-05crypto/chacha/asm/chacha-s390x.pl: add vx code path.Patrick Steuer
Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com> Reviewed-by: Tim Hudson <tjh@openssl.org> Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/6919)
2018-12-06Following the license change, modify the boilerplates in crypto/chacha/Richard Levitte
[skip ci] Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/7781)
2018-07-03chacha/asm/chacha-x86_64.pl: add dedicated path for 128-byte inputs.Andy Polyakov
The 128-byte vectors are extensively used in chacha20_poly1305_tls_cipher and dedicated code path is ~30-50% faster on most platforms. Reviewed-by: Rich Salz <rsalz@openssl.org> (Merged from https://github.com/openssl/openssl/pull/6626)
2018-06-20Update copyright yearMatt Caswell
Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/6538)
2018-06-18{chacha|poly1305}/asm/*-x64.pl: harmonize clang version detection.Andy Polyakov
Reviewed-by: Rich Salz <rsalz@openssl.org> (Merged from https://github.com/openssl/openssl/pull/6499)
2018-06-08chacha/asm/chacha-ppc.pl: fix big-endian build.Andy Polyakov
It's kind of a "brown-bag" bug, as I did recognize the problem and verified an ad-hoc solution, but failed to follow up with cross-checks prior filing previous merge request. Reviewed-by: Rich Salz <rsalz@openssl.org> (Merged from https://github.com/openssl/openssl/pull/6435)
2018-06-06chacha/asm/chacha-ppc.pl: improve performance by 40/80% on POWER8/9.Andy Polyakov
Reviewed-by: Rich Salz <rsalz@openssl.org> (Merged from https://github.com/openssl/openssl/pull/6419)
2018-06-03chacha/asm/chacha-ppc.pl: improve POWER8 performance by 15%.Andy Polyakov
This comes at cost of minor 2.5% regression on G4, which is reasonable trade-off. [Further improve compliance with ABI requirements.] Reviewed-by: Rich Salz <rsalz@openssl.org> (Merged from https://github.com/openssl/openssl/pull/6406)
2018-06-03PPC assembly pack: correct POWER9 results.Andy Polyakov
As it turns out originally published results were skewed by "turbo" mode. VM apparently remains oblivious to dynamic frequency scaling, and reports that processor operates at "base" frequency at all times. While actual frequency gets increased under load. Reviewed-by: Rich Salz <rsalz@openssl.org> (Merged from https://github.com/openssl/openssl/pull/6406)
2018-05-30chacha/asm/chacha-ppc.pl: optimize AltiVec/VMX code path.Andy Polyakov
32-bit vector rotate instruction was defined from beginning, it not being used from the start must be a brain-slip... Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de> Reviewed-by: Rich Salz <rsalz@openssl.org> (Merged from https://github.com/openssl/openssl/pull/6363)
2018-05-29Update copyright yearMatt Caswell
Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/6371)
2018-05-10PPC assembly pack: add POWER9 results.Andy Polyakov
Reviewed-by: Rich Salz <rsalz@openssl.org>
2017-12-08chacha/asm/chacha-x86_64.pl: add AVX512VL code path.Andy Polyakov
256-bit AVX512VL was estimated to deliver ~50% improvement over AVX2 and it did live up to the expectations. Reviewed-by: Rich Salz <rsalz@openssl.org> (Merged from https://github.com/openssl/openssl/pull/4838)
2017-11-28Fix chacha-armv4.pl with clang -fno-integrated-as.David Benjamin
The __clang__-guarded #defines cause gas to complain if clang is passed -fno-integrated-as. Emitting .syntax unified when those are used fixes this. This matches the change made to ghash-armv4.pl in 6cf412c473d8145562b76219ce3da73b201b3255. Reviewed-by: Andy Polyakov <appro@openssl.org> Reviewed-by: Kurt Roeckx <kurt@roeckx.be> (Merged from https://github.com/openssl/openssl/pull/3694)
2017-11-25chacha/asm/chacha-x86_64.pl: fix sporadic crash in AVX512 code path.Andy Polyakov
Only chacha_internal_test is affected, since this path is not used from EVP. Reviewed-by: Rich Salz <rsalz@openssl.org> (Merged from https://github.com/openssl/openssl/pull/4758)
2017-11-13ARMv8 assembly pack: add Qualcomm Kryo results.Andy Polyakov
[skip ci] Reviewed-by: Tim Hudson <tjh@openssl.org>
2017-11-11Many spelling fixes/typo's corrected.Josh Soref
Around 138 distinct errors found and fixed; thanks! Reviewed-by: Kurt Roeckx <kurt@roeckx.be> Reviewed-by: Tim Hudson <tjh@openssl.org> Reviewed-by: Rich Salz <rsalz@openssl.org> (Merged from https://github.com/openssl/openssl/pull/3459)
2017-07-21x86_64 assembly pack: "optimize" for Knights Landing, add AVX-512 results.Andy Polyakov
"Optimize" is in quotes because it's rather a "salvage operation" for now. Idea is to identify processor capability flags that drive Knights Landing to suboptimial code paths and mask them. Two flags were identified, XSAVE and ADCX/ADOX. Former affects choice of AES-NI code path specific for Silvermont (Knights Landing is of Silvermont "ancestry"). And 64-bit ADCX/ADOX instructions are effectively mishandled at decode time. In both cases we are looking at ~2x improvement. AVX-512 results cover even Skylake-X :-) Hardware used for benchmarking courtesy of Atos, experiments run by Romain Dolbeau <romain.dolbeau@atos.net>. Kudos! Reviewed-by: Rich Salz <rsalz@openssl.org>
2017-07-03x86_64 assembly pack: fill some blanks in Ryzen results.Andy Polyakov
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
2017-05-11Remove filename argument to x86 asm_init.David Benjamin
The assembler already knows the actual path to the generated file and, in other perlasm architectures, is left to manage debug symbols itself. Notably, in OpenSSL 1.1.x's new build system, which allows a separate build directory, converting .pl to .s as the scripts currently do result in the wrong paths. This also avoids inconsistencies from some of the files using $0 and some passing in the filename. Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: Andy Polyakov <appro@openssl.org> (Merged from https://github.com/openssl/openssl/pull/3431)
2017-03-22x86_64 assembly pack: add some Ryzen performance results.Andy Polyakov
Reviewed-by: Tim Hudson <tjh@openssl.org>
2017-02-26chacha/asm/chacha-x86_64.pl: add CFI annotations.Andy Polyakov
Reviewed-by: Rich Salz <rsalz@openssl.org>
2017-02-06x86_64 assembly pack: Win64 SEH face-lift.Andy Polyakov
- harmonize handlers with guidelines and themselves; - fix some bugs in handlers; - add missing handlers in chacha and ecp_nistz256 modules; Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-12-25chacha/asm/chacha-x86_64.pl: add AVX512 path optimized for shorter inputs.Andy Polyakov
Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-12-19x86 assembly pack: update performance results.Andy Polyakov
Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-12-15poly1305/asm/poly1305-x86_64.pl: allow nasm to assemble AVX512 code.Andy Polyakov
chacha/asm/chacha-x86_64.pl: refine nasm version detection logic. Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-12-12x86_64 assembly pack: add AVX512 ChaCha20 and Poly1305 code paths.Andy Polyakov
Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-11-11chacha/asm/chacha-x86.pl: improve [backward] portability.Andy Polyakov
In order to minimize dependency on assembler version a number of post-SSE2 instructions are encoded manually. But in order to simplify the procedure only register operands are considered. Non-register operands are passed down to assembler. Module in question uses pshufb with memory operands, and old [GNU] assembler can't handle it. Fortunately in this case it's possible skip just the problematic segment without skipping SSSE3 support altogether. Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-11-11PPC assembler pack: add some PPC970/G5 performance data.Andy Polyakov
Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-10-24x86_64 assembly pack: add Goldmont performance results.Andy Polyakov
Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-10-10Remove trailing whitespace from some files.David Benjamin
The prevailing style seems to not have trailing whitespace, but a few lines do. This is mostly in the perlasm files, but a few C files got them after the reformat. This is the result of: find . -name '*.pl' | xargs sed -E -i '' -e 's/( |'$'\t'')*$//' find . -name '*.c' | xargs sed -E -i '' -e 's/( |'$'\t'')*$//' find . -name '*.h' | xargs sed -E -i '' -e 's/( |'$'\t'')*$//' Then bn_prime.h was excluded since this is a generated file. Note mkerr.pl has some changes in a heredoc for some help output, but other lines there lack trailing whitespace too. Reviewed-by: Kurt Roeckx <kurt@openssl.org> Reviewed-by: Matt Caswell <matt@openssl.org>
2016-09-08chacha/asm/chacha-ppc.pl: add missing .text directive.Andy Polyakov
RT#4667 Reviewed-by: Rich Salz <rsalz@openssl.org>