| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
See also:
https://rustsec.org/advisories/RUSTSEC-2021-0037.html
https://github.com/diesel-rs/diesel/pull/2663
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
(cherry picked from commit 38b6ded40c6d840c9d3951f0a082cd55b1b0109c)
|
|
See:
https://rustsec.org/advisories/RUSTSEC-2018-0002.html
https://github.com/alexcrichton/tar-rs/pull/156
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
(cherry picked from commit c75d620e5bc6e454f8c6cecf1e440d430dac97a8)
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This is because the "sha1" crate seems to be unmaintained (no release in 3
years) but the sha-1 crate seems to be actively maintained in the context of a
bigger rust-crypto-functions project.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
Tested-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
|
|
|
|
This patch rewrites patch post-processing with a new setter in the config-rs
crate:
Config::set_once()
That is introduced on a PR I've written for the crate.
The PR: https://github.com/mehcode/config-rs/pull/172
The new function is more likely to land on the config-rs master branch than the
old one, as it does not introduce more exported types from the config-rs crate.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
Tested-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch ensures that pathes for patches are always relative to the repository
root, when loading the packages layered from the repository.
The problem is well described in the comment in the code, so this commit message
ends here.
This patch was written using an proposed feature for the config-rs dependency:
https://github.com/mehcode/config-rs/pull/165
This PR added a `Config::get_value_mut()` function and exported the
`config::value::ValueKind` type in config-rs public API.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
Tested-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
We need the "time" feature in tokio, because we want to implement pinging with a
wait time between the pings.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
This forces the transitive dependency of "bitvec", named "funty", to 1.1.0.
Rationale:
https://github.com/bitvecto-rs/bitvec/issues/105#issuecomment-778570981
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch reimplements the package orchestration functionality to rely
on a DAG rather than a tree.
A
/ \
B E
/ \ \
C D F
Before this change, the structure the packages were organized in for a
build was a tree.
That did work reasonable well for initial development of butido, because
this is a simple case and the implementation is rather simple, too.
But, packages and their dependencies are not always organized in a tree.
Most of the time, they are organized in a DAG:
.-> C -,
/ \
D > A
\ /
`-> B -ยด
This is a real-world example: A could be a common crypto-library that I
do not want to name here.
B and C could be libraries that use the said crypto-library and D could
be a program that use B and C.
Because said crypto-library builds rather long, building it twice and
throwing one result away is a no-go.
A DAG as organizational structure makes that issue go away entirely.
Also, we can later implement checks whether the DAG contains multiple
versions of the same library, if that is undesireable.
The change itself is rather big, frankly because it is a non-trivial
change the replace the whole data structure and its handling in the
orchestrator code.
First of all, we introduce the "daggy" library, which provides the DAG
implementation on top of the popular "petgraph" library.
The package `Tree` datastructure was replaced by a package `Dag`
datastructure. This type implements the heavy-lifting that is needed to
load a package and all its dependencies from the `Repository` object.
The `JobTree` was also reimplemented, but as `daggy::Dag` provides a
convenient `map()` function, its implementation which transforms the
package `Dag` into a job `Dag` is rather trivial.
`crate::job::Dag` then provides the convenience `iter()` function to
iterate over all elements in the DAG and providing a `JobDefinition`
object for each node.
The topology in which we traverse the DAG is not an issue, as we need to
create tasks for all `JobDefinition`s anyways, so we do not care about
traversal topology at all.
The `crate::package::Package` type got an `Hash` implementation, which
is necessary to keep track of the mappings while reading the DAG from
the repository.
The implementation does not create the edges between the nodes in the
DAG right when inserting, but afterwards.
To keep track of the `daggy::NodeIndex`es, it keeps a mapping
Package -> NodeIndex
in a Hashmap. Thus, `Package` must implement `std::hash::Hash`
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
Tested-by: Matthias Beyer <mail@beyermatthias.de>
squash! Reimplement as DAG
|
|
Because tokio 1.0 does not ship with the Stream trait, this patch also
introduces tokio_stream as new dependency.
For more information, look here:
https://docs.rs/tokio/1.0.3/tokio/stream/index.html
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
We don't need resiter::Map here anymore because itertools 0.10 provides
a map_ok() extension.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
MIT licensed documentation helper crate for drawing nice graphes.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch reimplements the tree-printing using the 'ptree' crate.
Because ptree wants the tree item to implement `Clone`, a wrapper type is
added which then implements `Clone` and `ptree::TreeItem`
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
This patch adds a compatibility setting in the configuration file, so that
butido does not fail late when parsing configuration parameters or package
definition files, but early, when the configured compatilibty does not match the
version of butido itself.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This reverts commit c3fc1281142ec10414197a31070cc45930a859e3
and commit 18b256e040881ac674463913b2a7e290125ea738.
The problem here is that we introduced "ring" to our dependencies, which has a
unappropriate license (ISC and openssl thing).
We should not depend on such a library, because it may conflict with our own EPL
2.0 (IANAL).
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
|
|
|
|
This patch re-implements hashing using the "ring" crypto library and
implementing a streaming hashing with it.
This way, we stream the file to the hasher rather than reading the full file to
memory.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch changes the (possibly heavy) File reading from disk to be executed on
a blocking tokio thread.
This way, we ensure we do not block until the `read()` operation is finished and
can continue executing async tasks on the runtime.
Because IO might be expensive (think of a 5GB file that is read to memory), this
is a good optimization.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
With this patch, we pin shiplift to a revision.
That is because shiplift may break from one revision to another on master (which
just happened and which is what we fixed in
cc8d3a7912203c90a6591e0d178c80f620bd61d0 ("Fix: Add type annotation").
We expect shiplift to hit tokio 1.0 anytime now and because of that we might
want to do a manual update of shiplift and tokio rather than an imperative
`cargo update` which may (short-term) break our codebase.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
We actually need to make an ugly (and very bad performing) hack here, to
get the `unindent` crate to actually unindent here.
text = format!("\n{}", text).unindent(),
the format!() call is necessary to tell `unindent::Unindent::unindent()`
that the intended indentation is actually zero.
If we do not add that `\n` here, the first line of the to-be-unindented
indented text stays unintentionally indented.
For example, unindenting the intentionally indented string:
" intentionally indented
intentionally indented
"
results unintentionally in the partially indented and partially
unindented text:
" intentionally indented
intentionally indented
"
Which is not our intention of course, although it seems to be the
intention of the unindent crate, as documented.
But with the intentionally added `\n` we unindent the indentation of all
indented "intentionally indented" lines.
Signed-off-by: Christoph Prokop <christoph.prokop@atos.net>
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
Tested-by: Christoph Prokop <christoph.prokop@atos.net>
Tested-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Because the scripts we run should not be able to access the logger
during handlbars rendering.
This is due to the fact that we need to have control over stdout/stderr
because of the progress bar printing, but also because there's
absolutely no need for the user to log during the handlebars rendering
process.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
This removes the ContainerError type for simplicity. We can call
anyhow!() and just work with that, no need to pull in thiserror just for
one type.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Sort and make indention nice.
Also dev-dependencies where removed because indoc is a normal dependency
anyways.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
This patch adds script linting via a configurable script.
With this patch, the configuration can point to a script or program that
gets the packaging script on STDIN and might exit with a nonzero value
on error.
The simplest script I can think of that adds value is:
#!/bin/bash
shellcheck -
to run the packaging script through shellcheck.
stdout and stderr of the linting script are printed in case of non-zero
return value.
Linting can be disabled by not setting the script in the config or by
passing a commandline flag to skip linting (a warning will be printed in
this case).
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
The licensing issue fixed by this patch is described in the comment in
the code.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
This patch updates the ascii_table dependency with a requirement to be
3.0.2 or newer.
This is because ascii_table was licensed as GPLv3 before that version,
and got relicensed to MIT in 3.0.2.
Because we do not want to depend on a library that is GPLv3, we asked
the author to relicense their crate, which is what they did for us.
This patch thus ensures that we do not link GPLv3 code.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
