summaryrefslogtreecommitdiffstats
path: root/src
AgeCommit message (Collapse)Author
2020-11-16Return Job object when created in databaseMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-16Add passing of additional envMatthias Beyer
This patch adds the code to pass the additional environment to the container job. Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-16Add additional-env to database writingMatthias Beyer
This patch adds that the additionally passed environment is added to the database as soon as possible. Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-16Pass package environment and package variables (as environment variables) to ↵Matthias Beyer
container Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-15Print package name and version when listing jobs from DBMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-15Add URI reportingMatthias Beyer
This patch adds the URI to the container error error-message. The URI is required, of course, to connect to the remote docker container if there was an error. Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-15Implement error reporting of failed jobsMatthias Beyer
This patch implements error reporting if a container job did not end successfully. It does so by adding an error type `ContainerError`, which is either an error that describes that a container did not exit with success, or an anyhow::Error (that describes an error from the container management code). The algorithm of log-aggregation is now intercepted to catch any exit-state log items. If there is no exit-state from the container (No line with "#BUTIDO:STATE:..."), no error is assumed. Here could be a warning later on. The so aggregated state is then passed up to the orchestrator, which then collects the errors and prints them. If the implementation is correct (which is not tested yet, because this is rather difficult to test), all other containers should continue operation until they are ready, before the errors are handled. The code responsible for this (in the Orchestrator implementation) was adapted to not collect until the first error, but collect everything and then check for errors. Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-15Implement Displayfor ContainerHashMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-15Add error context messages for better error reportingMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-15Fix indentationMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-14Implement on-disk logsMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-14Make bar messages equivalent to finish messages (in format)Matthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-14Remove tick() because it does not seem necessaryMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-14Make multi bar available to scheduler when creating jobMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-14Rewrite to use tokio::sync::RwLockMatthias Beyer
This patch rewrites the codebase to not use std::sync::RwLock, but tokio::sync::RwLock. tokios RwLock is an async RwLock, which is what we want in an async-await context. The more I use tokio, the more I understand what you should do and what you shouldn't do. Some parts of this patch are a rewrite, for example, JobSet::into_runables() was completely rewritten. That was necessary because the function used inside is `Runnable::build_from_job()`, which uses an RwLock internally, thus, gets `async` in this patch. Because of this, `JobSet::into_runables()` needed a complete rewrite as well. Because it is way more difficult than transforming the function to return an iterator of futures, this patch simply rewrites it to return a `Result<Vec<RunnableJob>>` instead. Internally, tokio jobs are submitted via the `futures::stream::FuturesUnordered<_>` now. This is not the most performant implementation for the problem at hand, but it is a reasonable simple one. Optimization could happen here, of course. Also, the implementation of resource preparation inside `RunnableJob::build_from_job()` got a rewrite using the same technique. Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-14Add debug output, error context informationMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-14Move multibar creation back to orchestratorMatthias Beyer
This needs to be done to prepare for one thing: We need to be able to call `multibar.join()` (which blocks the current thread) _while_ running the jobs on the scheduler. That's ugly, but that's the way indicatif works. Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-14Make sure to drop bar as soon as possibleMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-14Adapt finish message to other message formatsMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-14Add trace!() outputMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-14Add scheduler shutdown function, to drop multibarMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-14Set job bar length by default to 100Matthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-14Pass ProgressBar by referenceMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Change from hiding the log via flag to showing the log via flagMatthias Beyer
After trying out the CLI, it seems that this is the better way to do it. Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add flag to disable highlighting for one run in CLIMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Make highlighting optionalMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add flag to show script, highlighted automaticallyMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add flag to hide logMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Use ParsedLog type for more style in log outputMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add ParsedLog type, which holds a list of parsed log linesMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add command to show a single job from the databaseMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add uuid field to "job" tableMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add flag to only show jobs of one specific submitMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add subcommand: db jobs - to list jobsMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add logging: trace!() submit object after creating submit entry in databaseMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add debug output when creating new Job in databaseMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add helper to check whether a log text reports successMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add subcommand: db submits - for listing submitsMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Remove indention in script phasesMatthias Beyer
This patch removes the indention when building the script. It also adds the phase name to the "phase-end-comment" line. Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Make JobHandle create database entry for JobMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Make LogItem::display() result able to be turned into StringMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Return Script object after job ranMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Pass Submit object around when scheduling jobsMatthias Beyer
This is needed because we need to refer to the submit when creating a Job entry in the database. Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Make model::Submit derive Clone for easy passing aroundMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Report container hash after running job on endpointMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add ContainerHash typeMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add model definition for JobMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Add model definition for EndpointMatthias Beyer
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Rename: JobHandle::get_result() -> JobHandle::run()Matthias Beyer
This renames the method, because this is way more descriptive of what the method actually does. Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2020-11-13Move log collecting to endpoint codeMatthias Beyer
This patch moves the log-setup and -collecting to the endpoint implementation, that takes care of running the job. This is required because here we aggregate the log and here we know all information that need to be written to the database, so it seems natural to move the log-collecting code (and everything else related to the database entry for the job) to this position in the code. This implies that the "job" is written to the database _after_ it was run. This is due to the fact that we cannot write the job entry before we have the logs. We _have to_ collect the logs in memory (we can also write to a log-dump-file, of course), but we cannot create the job object in the database and then append the logs to it. This might change in the future, but for now that's the way it is done. Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
generated by cgit v1.2.3 (git 2.25.1) at 2024-09-09 16:53:33 +0000