| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Because it makes logically more sense to implement Into<> here than implementing
From<> for a foreign type (Vec<>).
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
clippy complained that we cannot have a derived PartialEq and a handwritten Hash
implementation.
HashMap does not implement Hash. After some research we found that BTreeMap
implements Hash, so we can derive Hash if we use BTreeMap as a key-value type
for the `env_eq` member.
Logically they are the same and this reduces code size, so go for it.
This has the nice benefit, that we do not need to implement PartialOrd and Ord
either.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch adds the condition-filtering support in the DAG-building code for
conditional dependencies.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
We need to refactor because we want, in the next step, add filters in the
dependency-processing mechanism, where we filter out dependencies by their
condition.
Thus, using the Package::get_self_packaged_dependences() helper fn does not work
anymore and we remove it with this patch.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch implements the matching of the condition(s) against the provided
data.
The patch includes tests for all three supported conditions.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch extends the interface for building the package DAG with a parameter
for the data that is required to check conditions for conditional dependencies.
A "DTO" type is added for this data.
The actual condition-checking is not implemented in this patch.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch adds a conditional dependency variant to the build/runtime dependency
deserialization type(s).
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch adds the first implementation for conditional dependencies in the
dependency fields (that is the "build" and "runtime" keys in the package
defintion).
This is only the deserialization-interface, not the actual condition resolving
code.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch reimplements the Dependency type as an enum.
This is required because later we want to be able to use an "if" expression in
the build dependencies, for example (not decided yet) something like
runtime = [
{
d = "libfoobar",
if = [ { has_env = [ "TARGET_RH7" ]; } ]
}
]
(no valid toml here, formatting for readability)
to make a dependency optional on a condition.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch reimplements the BuildDependency type as an enum.
This is required because later we want to be able to use an "if" expression in
the build dependencies, for example (not decided yet) something like
build = [
{
d = "libfoobar",
if = [ { has_env = [ "TARGET_RH7" ]; } ]
}
]
(no valid toml here, formatting for readability)
to make a dependency optional on a condition.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch fixes that punctuation (namingly dot and dash) are allowed in a
package name.
A testcase was added.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Fixes: f20bf09292739e1bdbba9c1f8235a35f7d2d7712 ("Make progress bar for loading DAG optional")
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch adds a possibility to print the Package object with more detailed
information.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch removes the `PackageFlags` type and the feature from the package
definition struct, so a user cannot longer use the package flags.
This feature was not used yet in our repository, so it seems that this is
nothing that is valuable.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This fixes a bug where dependencies were not found, but this was
ignored.
This happened even if there was a typo in the version of the
dependency, for example.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
Tested-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
This function is only needed in the PackageVersionConstraint::parser() function,
thus make it private here so it is not accidentally used somewhere else.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Use the TryFrom trait rather than a `::new()` constructor that can fail.
This is way more idomatic.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Rewrite this function to use the constructor or PackageVersionConstraint instead
of getting the parser and using it, because this shouldn't be allowed anyways.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This is because the "sha1" crate seems to be unmaintained (no release in 3
years) but the sha-1 crate seems to be actively maintained in the context of a
bigger rust-crypto-functions project.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
Tested-by: Matthias Beyer <mail@beyermatthias.de>
|
|
Because passing by value is simply not necessary here.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
This patch implements the find-artifact subcommand, which is the basis for
artifact finding.
Right now we have the problem that finding artifacts after the fact is non-trivial.
What we can do, though, is search for them in the database using the same inputs
as the ones that created the artifact.
That means that if we want to find an artifact or multiple artifacts for a
package P in version V, we have to search the database for jobs that built that
package in that version, with the same script and the same variables
(environment, used container image, etc).
So if a package was built with the same script, the same environment variables
and on an image that is, for example, not in the denylist of the package,
chances are good that the artifacts produced by the job for the package are the
ones we search for.
In the `crate::command::find_artifact()` function, results are sorted before
they are printed, so that we preferrably print results with a release date.
Env filtering is also implemented, so a user has to provide the appropriate
additional environment variables, as they were submitted for the `buid` command
when the artifact was built.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
Tested-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
This patch implements testing for the crate::package::Dag code.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
In a DAG, we can find a (packagename, packageversion) multiple times:
.-> C -.
/ \
D > A
\ /
`-> B -´
for example, in this scenario we would find "A" multiple times.
If we re-find a package that we already recursed for, do not recurse
again for it.
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
|
|
This patch reimplements the package orchestration functionality to rely
on a DAG rather than a tree.
A
/ \
B E
/ \ \
C D F
Before this change, the structure the packages were organized in for a
build was a tree.
That did work reasonable well for initial development of butido, because
this is a simple case and the implementation is rather simple, too.
But, packages and their dependencies are not always organized in a tree.
Most of the time, they are organized in a DAG:
.-> C -,
/ \
D > A
\ /
`-> B -´
This is a real-world example: A could be a common crypto-library that I
do not want to name here.
B and C could be libraries that use the said crypto-library and D could
be a program that use B and C.
Because said crypto-library builds rather long, building it twice and
throwing one result away is a no-go.
A DAG as organizational structure makes that issue go away entirely.
Also, we can later implement checks whether the DAG contains multiple
versions of the same library, if that is undesireable.
The change itself is rather big, frankly because it is a non-trivial
change the replace the whole data structure and its handling in the
orchestrator code.
First of all, we introduce the "daggy" library, which provides the DAG
implementation on top of the popular "petgraph" library.
The package `Tree` datastructure was replaced by a package `Dag`
datastructure. This type implements the heavy-lifting that is needed to
load a package and all its dependencies from the `Repository` object.
The `JobTree` was also reimplemented, but as `daggy::Dag` provides a
convenient `map()` function, its implementation which transforms the
package `Dag` into a job `Dag` is rather trivial.
`crate::job::Dag` then provides the convenience `iter()` function to
iterate over all elements in the DAG and providing a `JobDefinition`
object for each node.
The topology in which we traverse the DAG is not an issue, as we need to
create tasks for all `JobDefinition`s anyways, so we do not care about
traversal topology at all.
The `crate::package::Package` type got an `Hash` implementation, which
is necessary to keep track of the mappings while reading the DAG from
the repository.
The implementation does not create the edges between the nodes in the
DAG right when inserting, but afterwards.
To keep track of the `daggy::NodeIndex`es, it keeps a mapping
Package -> NodeIndex
in a Hashmap. Thus, `Package` must implement `std::hash::Hash`
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
Tested-by: Matthias Beyer <mail@beyermatthias.de>
squash! Reimplement as DAG
|
|
This seems strange, but it works. I don't know whether this is right, tho.
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
Tested-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
|
Signed-off-by: Matthias Beyer <matthias.beyer@atos.net>
|
