summaryrefslogtreecommitdiffstats
path: root/src/nix-prefetch-url/nix-prefetch-url.cc
AgeCommit message (Collapse)Author
2021-01-07Add 'nix store prefetch-{file,tarball}'Eelco Dolstra
These replace nix-prefetch-url and nix-prefetch-url --unpack, respectively.
2020-10-06Remove static variable name clashesEelco Dolstra
This was useful for an experiment with building Nix as a single compilation unit. It's not very useful otherwise but also doesn't hurt...
2020-09-18nix-prefetch-url: Add --executable flagBryan Richter
pkgs.fetchurl supports an executable argument, which is especially nice when downloading a large executable. This patch adds the same option to nix-prefetch-url. I have tested this to work on the simple case of prefetching a little executable: 1. nix-prefetch-url --executable https://my/little/script 2. Paste the hash into a pkgs.fetchurl-based package, script-pkg.nix 3. Delete the output from the store to avoid any misidentified artifacts 4. Realise the package script-pkg.nix 5. Run the executable I repeated the above while using --name, as well. I suspect --executable would have no meaningful effect if combined with --unpack, but I have not tried it.
2020-07-13Merge branch 'hash-always-has-type' into better-ca-parse-errorsJohn Ericson
2020-07-13Merge remote-tracking branch 'upstream/master' into hash-always-has-typeJohn Ericson
2020-07-10Factor out commonality between nix-prefetch-url and nix-store --add-fixedEelco Dolstra
2020-07-10nix-prefetch-url: Run in constant memory when using RemoteStoreEelco Dolstra
Fixes #3684.
2020-07-01Rename two hash constructors to proper functionsCarlo Nucera
2020-06-19WIP: Make Hash always store a valid hash typeJohn Ericson
2020-06-18Revert the `enum struct` changeJohn Ericson
Not a regular git revert as there have been many merges and things.
2020-06-18Merge branch 'enum-class' into no-hash-type-unknownJohn Ericson
2020-06-18Merge remote-tracking branch 'upstream/master' into enum-classJohn Ericson
2020-06-11Merge remote-tracking branch 'upstream/master' into errors-phase-2Ben Burdette
2020-06-08Move progress-bar.cc to libmainregnat
Needed so that we can include it as a logger in loggers.cc without adding a dependency on nix This also requires moving names.hh to libutil to prevent a circular dependency between libmain and libexpr
2020-06-02Remove `HashType::Unknown`John Ericson
Instead, `Hash` uses `std::optional<HashType>`. In the future, we may also make `Hash` itself require a known hash type, encoraging people to use `std::optional<Hash>` instead.
2020-05-29Merge remote-tracking branch 'upstream/master' into errors-phase-2Ben Burdette
2020-05-28Merge branch 'master' of github.com:NixOS/nix into enum-classCarlo Nucera
2020-05-26Merge remote-tracking branch 'origin/master' into enum-FileIngestionMethodCarlo Nucera
2020-05-11Merge branch 'master' into errors-phase-2Ben Burdette
2020-04-21remove 'format' from Error constructor callsBen Burdette
2020-04-08datatransfer.{cc,hh} -> filetransfer.{cc,hh}Nikola Knezevic
2020-04-08DataTransfer -> FileTransferNikola Knezevic
2020-04-08{get,make,new}Downloader -> DataTransferNikola Knezevic
2020-04-08Rename src/lib/download.* to src/lib/datatransfer.*Nikola Knezevic
2020-04-08DownloadRequest -> DataTransferRequestNikola Knezevic
2020-03-30Merge remote-tracking branch 'upstream/master' into enum-FileIngestionMethodJohn Ericson
2020-03-30Remove global -I flagsEelco Dolstra
(cherry picked from commit 2c692a3b144523bca68dd6de618124ba6c9bb332)
2020-03-29Replace some `bool recursive` with a new `FileIngestionMethod` enumJohn Ericson
2020-03-29Use `enum struct` and drop prefixesJohn Ericson
This does a few enums; the rest will be gotten in subsequent commits.
2020-03-24findAlongAttrPath(): Return positionEelco Dolstra
(cherry picked from commit 0b013a54dc570395bed887369f8dd622b8ce337b)
2020-01-05build: fix sandboxing on darwinDaiderd Jordan
Starting ba87b08f8529e4d9f8c58d8c625152058ceadb75 getEnv now returns an std::optional which means these getEnv() != "" conditions no longer happen if the variables are not defined.
2019-12-19Merge branch 'libarchive' of https://github.com/yorickvP/nixEelco Dolstra
2019-12-10Make the Store API more type-safeEelco Dolstra
Most functions now take a StorePath argument rather than a Path (which is just an alias for std::string). The StorePath constructor ensures that the path is syntactically correct (i.e. it looks like <store-dir>/<base32-hash>-<name>). Similarly, functions like buildPaths() now take a StorePathWithOutputs, rather than abusing Path by adding a '!<outputs>' suffix. Note that the StorePath type is implemented in Rust. This involves some hackery to allow Rust values to be used directly in C++, via a helper type whose destructor calls the Rust type's drop() function. The main issue is the dynamic nature of C++ move semantics: after we have moved a Rust value, we should not call the drop function on the original value. So when we move a value, we set the original value to bitwise zero, and the destructor only calls drop() if the value is not bitwise zero. This should be sufficient for most types. Also lots of minor cleanups to the C++ API to make it more modern (e.g. using std::optional and std::string_view in some places).
2019-12-07libarchive proof of conceptYorick van Pelt
2019-11-26Drop remaining uses of external "tar"Eelco Dolstra
Also, fetchGit now runs in O(1) memory since we pipe the output of 'git archive' directly into unpackTarball() (rather than first reading it all into memory).
2018-11-09nix-prefetch-url: Stop progress bar before printing resultsEelco Dolstra
2018-10-26nix-prefetch-url: progressbarWill Dietz
2018-10-26Merge all nix-* binaries into nixEelco Dolstra
These are all symlinks to 'nix' now, reducing the installed size by about ~1.7 MiB.
2018-07-12nix-prefetch-url: Download file in constant memoryEelco Dolstra
Before: $ command time nix-prefetch-url https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.6.tar.xz 1.19user 1.02system 0:41.96elapsed 5%CPU (0avgtext+0avgdata 182720maxresident)k After: 1.38user 1.05system 0:39.73elapsed 6%CPU (0avgtext+0avgdata 16204maxresident)k Note however that addToStore() can still take a lot of memory (e.g. RemoteStore::addToStore() is constant space, but LocalStore::addToStore() isn't; that's fixed by https://github.com/edolstra/nix/commit/c94b4fc7ee0c7b322a5f3c7ee784063b47a11d98 though). Fixes #1400.
2018-06-12Move EvalState from the stack to the heapEelco Dolstra
EvalState contains a few counters (e.g. nrValues) that increase quickly enough that they end up being interpreted as pointers by the garbage collector. Moving it to the heap makes them invisible to the garbage collector. This reduces the max RSS doing 100 evaluations of nixos.tests.firefox.x86_64-linux.drvPath from 455 MiB to 292 MiB. Note: ideally, allocations would be much further up in the 64-bit address space to reduce the odds of an integer being misinterpreted as a pointer. Maybe we can use some linker magic to move the .bss segment to a higher address.
2018-02-08Add plugins to make Nix more extensible.Shea Levy
All plugins in plugin-files will be dlopened, allowing them to statically construct instances of the various Register* types Nix supports.
2017-10-24nix: Respect -I, --arg, --argstrEelco Dolstra
Also, random cleanup to argument handling.
2017-09-18Don't remove Content-Encoding in fetchurl / nix-prefetch-urlEelco Dolstra
Fixes #1568.
2017-07-30Replace Unicode quotes in user-facing strings by ASCIIJörg Thalheim
Relevant RFC: NixOS/rfcs#4 $ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-07-04Support base-64 hashesEelco Dolstra
Also simplify the Hash API. Fixes #1437.
2017-03-15runProgram(): Distinguish between empty input and no inputEelco Dolstra
For example, if we call brotli with an empty input, it shouldn't read from the caller's stdin.
2016-11-26Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25Get rid of unicode quotes (#1140)Guillaume Maudoux
2016-09-21printMsg(lvlError, ...) -> printError(...) etc.Eelco Dolstra
2016-09-14Enable HTTP/2 supportEelco Dolstra
The binary cache store can now use HTTP/2 to do lookups. This is much more efficient than HTTP/1.1 due to multiplexing: we can issue many requests in parallel over a single TCP connection. Thus it's no longer necessary to use a bunch of concurrent TCP connections (25 by default). For example, downloading 802 .narinfo files from https://cache.nixos.org/, using a single TCP connection, takes 11.8s with HTTP/1.1, but only 0.61s with HTTP/2. This did require a fairly substantial rewrite of the Downloader class to use the curl multi interface, because otherwise curl wouldn't be able to do multiplexing for us. As a bonus, we get connection reuse even with HTTP/1.1. All downloads are now handled by a single worker thread. Clients call Downloader::enqueueDownload() to tell the worker thread to start the download, getting a std::future to the result.