| Age | Commit message (Collapse) | Author |
|---|
|
explain why it didnt fail before though (#6007)
|
|
a check_login.sh step in prior to docker publishing, for early fail if credentials are messed up. Waiting the full build is overhead (#5928)
|
|
* netdata/packaging/ci: stronger validations during install, add some comments, change validation of CWD
* netdata/packaging/ci: Add a lifecycle test to validate update from stable to latest works smooth
-> Add updater BATS script that does the trick for installing stable then migrating to latest using current code and not nightly
-> Integrate with travis
-> Rename /code to /netdata in travis, scripts will eventually expect the TLD of repo to be actually named netdata, so test will start failing later
-> Introduce a flag on the updater, so that it doesn't download the tarball, but it rather works with a local pre-defined directory. If a user wants
to override the update process, so that it runs with a different repository he just needs to export NETDATA_LOCAL_TARBAL_OVERRIDE with the desired repo directory
* netdata/packaging/ci: Add required packages for it to run
|
|
|
|
repo slug
|
|
same job, so that they share the same context, otherwise folder is not visible, obviously
|
|
* netdata/packaging/ci: [WIP] Redesign packaging release
* netdata/packaging/ci: Add explicitly os definition, also remove sourcing of slack, its already done and rename before_install -> install
* netdata/packaging/ci: fix stray stage name
* netdata/packaging/ci: style
* netdata/packaging/ci: fix syntax for yaml
* netdata/packaging/ci: Try removing this, doesnt seem to make much sense to be there (for now)
* Revert "netdata/packaging/ci: Try removing this, doesnt seem to make much sense to be there (for now)"
This reverts commit ad5556a3ccf6c11ebc9026942af8013d4d0ea666.
wow travis - this is weird
* netdata/packaging/ci: a few more changes
* netdata/packaging/ci: This is totally experimental, will adjust after i test it
* netdata/packaging/ci: Reinstate original design - the problem i am trying to solve is more travis-related and fundamental on the way travis works
* netdata/packaging/ci: i wonder..
* Revert "netdata/packaging/ci: i wonder.."
This reverts commit a160df4fcac866375e8319bf200f47263b2bcaa3.
* netdata/packaging/ci: Fix broken items
* netdata/packaging/ci: docker images are built and published independently. As long as tagging has taken place earlier, during packaging
* netdata/packaging/ci: Following hints from https://github.com/travis-ci/travis-ci/issues/4681 and the mayhem triggered from this change
So we adding the compiler, to see if it will get travis to build the right matrix.
then linter broke on addons for some weird reason, so remove addons->apt->update too
* netdata/packaging/ci: Now do the rest of the changes, since with addition of compiler, things got worse
* netdata/packaging/ci: Ok, as expected the workaround from them did not work. Lets get creative though and assume they exclude what they added, so lets do that for os
* netdata/packaging/ci: test slack
* netdata/packaging/ci: Bring the modified slack script on this branch
* netdata/packaging/ci: Add a few more slack messages
* Fix empty stage occurrence
* netdata/packaging/ci: Another set of restructure changes, explained below
1) Publish release -> Publish for release (minor rename for clarity)
2) Add some more work on the make dist testing (temp folder for root and for now just validate the existence of the tarball)
3) Add some more slack notifications (that is still beta, will have to properly refine once we are done with changes of the flow)
4) Restructure the release process, finally
a) relase.sh becomes draft_release.sh and changelog_and_tag_release.sh, basically splitting up the steps into meaningful independent blocks
b) Adjust the pipeline, now we run 'changelog_and_tag_release.sh' to get the latest changelog and tag the release,
then we run artifacts submission and draft release generation on github
still more things to re-check on this step, it will become clearer once i get it up to speed to run on my local repos end to end
* netdata/packaging/ci: Fix wrong path usage
* netdata/packaging/ci: Update webhook URL
* netdata/packaging/ci: Update stage config as proposed by the fix from travis team
* netdata/packaging/ci: Another round of restructuring, after 1-1 comparison with old version
Briefly:
1) Publish for release stage will only run when there is a TAG defined (To be tested though, but thats the idea).
The code previously had the releaser.sh script that did exactly that, if tagger script failed, it would stop process.
Now we have distributed the processing in clear chunks and we pick up the process based on shared info that travis or the environment provides
2) Rename <<Create release (only on special commit msg)>> to a more appropriate name, we are not really creating release there its more complicated
3) Move the labeler execution to the packaging step, since publishing now not happening all the time. Previously we hacked this by exit 0 with the tagger
and allow the labeler (which was the second part of the execution) to run.
4) before_install in the publishing stage is wrong. We only need this as before_deploy before deploy execution. That is where we use it, really.
5) Rename changelog_and_tag_release.sh to generate_changelog_and_tag_release.sh, just to be clearer. No problem with slightly longer file names
6) Do not let generate_changelog_and_tag_release.sh execute if GIT_TAG is not defined, just exit quietly with a message and a success code
7) Remove the tagger from the generate_changelog_and_tag_release.sh, its confusing to shove it in there. Pull it out and let it run on a clear step from the pipeline but within the same machine to make sure it shares environment with generate_changelog_and_tag_release.sh (So it has to be within the same job)
This is it mostly, keep in mind that the nightly and release parts are not tested yet.
After this commit, there will be more code that will allow me to execute all the flows on my pipeline and fully test and adjust the workflows as needed
* netdata/packaging/ci: Add more notifications, adjust the existing ones too
* netdata/packaging/ci: Add safeguards for beta, do a rename too
1) rename generate_changelog.sh -> generate_changelog_for_nightlies.sh, just to provide clarity on the usage.
Note: we will simplify this later, this was a workaround that raised a bug on release after refactoring nightlies.
We will have to clean this up properly once we merge the new pipeline flow
2) Add checks based on TRAVIS_REPO_SLUG to identify when we running on personal repo and abort mission critical actions like deploying or generating changelogs
Note: The deployment step for now will simply fail, due to the lack of the necessary encryption keys on the environment.
At a later iteration, we should have beta buckets on GCS and also beta repositories on Docker to properly test our publishing capability on our beta stack
* netdata/packaging/ci: Refactor README.md -- still half way there, will resume when i am done with testing
* netdata/packaging/ci: At this stage make the scripts harder to run, add checks to bypass tested execution.
* netdata/packaging/ci: Seems that adding the install tag within the stage, kind of overriding the outer install
pull out install items from packaging to the main install, no point to isolate two packages, just install all required all the time
* netdata/packaging/ci: tag obviously cant work at that step, moving it down to the jobs and i need to recheck if the syntax is applicable for travis too
* netdata/packaging/ci: So the jobs in a list dont share the environment, lets see if we do it otherwise. also remove the tag check, not work as expected
* netdata/packaging/ci: Add another way to check the RC condition
* netdata/packaging/ci: experimental - check conditions on message
* netdata/packaging/ci: Introduce an alternate master branch for the workflow testing
1. I created pipeline-redesign-master that will be the master branch for my use cases
2. Modified travis to trigger all production workflows on that branch, so that i can experiment freely with test commits to trigger process and verify execution flow
* remove spaces
* netdata/packaging/ci: comment out
* netdata/packaging/ci: wording nit
* netdata/packaging/ci: adjust the condition properly
* netdata/packaging/ci: test condition for triggering on specific keywords
* netdata/packaging/ci: try different syntax
* netdata/packaging/ci: remove depth, causes problems as we dont get all required objects from repo (adds --depth 50 in clone)
* Missing equal sign in regex?
* netdata/packaging/ci: Update tagger script
1) More verbose messages
2) Update comments copyrights etc
3) Rename release_candidate to set_tag_release_candidate
4) Add curlies on the variables, also change the way we check for path
* netdata/packaging/ci: reinstate git depth
* netdata/packaging/ci: Remove git depth from packaging too
* netdata/packaging/ci: RTFM - was using wrong syntax for slack @here approach
* netdata/packaging/ci: Using install tag at the stage layer, overrides the global one for some reason. fix that
* netdata/packaging/ci: Dont use the pattern matching from travis, do it outside with a script
* netdata/packaging/ci: Do it the other way, script execution doesnt work there
* netdata/packaging/ci: Attempt to fix conditional on release, remove the faulty assumptions as provided by travis team
* netdata/packaging/ci: reintroduce the tag checking
* netdata/packaging/ci: you need to bring tag checking at the stage, as it is calculated during packaging step
* netdata/packaging/ci: you cant escape that one for the moment, merge them and we will rework it later again
* netdata/packaging/ci: dont forget git depth
* netdata/packaging/ci: [WIP] Redesign packaging release
* netdata/packaging/ci: Add explicitly os definition, also remove sourcing of slack, its already done and rename before_install -> install
* netdata/packaging/ci: fix stray stage name
* netdata/packaging/ci: Try removing this, doesnt seem to make much sense to be there (for now)
* Revert "netdata/packaging/ci: Try removing this, doesnt seem to make much sense to be there (for now)"
* netdata/packaging/ci: Reinstate original design - the problem i am trying to solve is more travis-related and fundamental on the way travis works
* netdata/packaging/ci: Fix broken items
* netdata/packaging/ci: docker images are built and published independently. As long as tagging has taken place earlier, during packaging
* netdata/packaging/ci: Following hints from https://github.com/travis-ci/travis-ci/issues/4681 and the mayhem triggered from this change
So we adding the compiler, to see if it will get travis to build the right matrix.
then linter broke on addons for some weird reason, so remove addons->apt->update too
* netdata/packaging/ci: Now do the rest of the changes, since with addition of compiler, things got worse
* netdata/packaging/ci: Ok, as expected the workaround from them did not work. Lets get creative though and assume they exclude what they added, so lets do that for os
* netdata/packaging/ci: test slack
* netdata/packaging/ci: Bring the modified slack script on this branch
* netdata/packaging/ci: Add a few more slack messages
* Fix empty stage occurrence
* netdata/packaging/ci: Another set of restructure changes, explained below
1) Publish release -> Publish for release (minor rename for clarity)
2) Add some more work on the make dist testing (temp folder for root and for now just validate the existence of the tarball)
3) Add some more slack notifications (that is still beta, will have to properly refine once we are done with changes of the flow)
4) Restructure the release process, finally
a) relase.sh becomes draft_release.sh and changelog_and_tag_release.sh, basically splitting up the steps into meaningful independent blocks
b) Adjust the pipeline, now we run 'changelog_and_tag_release.sh' to get the latest changelog and tag the release,
then we run artifacts submission and draft release generation on github
still more things to re-check on this step, it will become clearer once i get it up to speed to run on my local repos end to end
* netdata/packaging/ci: Fix wrong path usage
* netdata/packaging/ci: Update webhook URL
* netdata/packaging/ci: Update stage config as proposed by the fix from travis team
* netdata/packaging/ci: Another round of restructuring, after 1-1 comparison with old version
Briefly:
1) Publish for release stage will only run when there is a TAG defined (To be tested though, but thats the idea).
The code previously had the releaser.sh script that did exactly that, if tagger script failed, it would stop process.
Now we have distributed the processing in clear chunks and we pick up the process based on shared info that travis or the environment provides
2) Rename <<Create release (only on special commit msg)>> to a more appropriate name, we are not really creating release there its more complicated
3) Move the labeler execution to the packaging step, since publishing now not happening all the time. Previously we hacked this by exit 0 with the tagger
and allow the labeler (which was the second part of the execution) to run.
4) before_install in the publishing stage is wrong. We only need this as before_deploy before deploy execution. That is where we use it, really.
5) Rename changelog_and_tag_release.sh to generate_changelog_and_tag_release.sh, just to be clearer. No problem with slightly longer file names
6) Do not let generate_changelog_and_tag_release.sh execute if GIT_TAG is not defined, just exit quietly with a message and a success code
7) Remove the tagger from the generate_changelog_and_tag_release.sh, its confusing to shove it in there. Pull it out and let it run on a clear step from the pipeline but within the same machine to make sure it shares environment with generate_changelog_and_tag_release.sh (So it has to be within the same job)
This is it mostly, keep in mind that the nightly and release parts are not tested yet.
After this commit, there will be more code that will allow me to execute all the flows on my pipeline and fully test and adjust the workflows as needed
* netdata/packaging/ci: Add more notifications, adjust the existing ones too
* netdata/packaging/ci: Add safeguards for beta, do a rename too
1) rename generate_changelog.sh -> generate_changelog_for_nightlies.sh, just to provide clarity on the usage.
Note: we will simplify this later, this was a workaround that raised a bug on release after refactoring nightlies.
We will have to clean this up properly once we merge the new pipeline flow
2) Add checks based on TRAVIS_REPO_SLUG to identify when we running on personal repo and abort mission critical actions like deploying or generating changelogs
Note: The deployment step for now will simply fail, due to the lack of the necessary encryption keys on the environment.
At a later iteration, we should have beta buckets on GCS and also beta repositories on Docker to properly test our publishing capability on our beta stack
* netdata/packaging/ci: Refactor README.md -- still half way there, will resume when i am done with testing
* netdata/packaging/ci: At this stage make the scripts harder to run, add checks to bypass tested execution.
* netdata/packaging/ci: Seems that adding the install tag within the stage, kind of overriding the outer install
pull out install items from packaging to the main install, no point to isolate two packages, just install all required all the time
* netdata/packaging/ci: tag obviously cant work at that step, moving it down to the jobs and i need to recheck if the syntax is applicable for travis too
* netdata/packaging/ci: So the jobs in a list dont share the environment, lets see if we do it otherwise. also remove the tag check, not work as expected
* netdata/packaging/ci: Add another way to check the RC condition
* netdata/packaging/ci: experimental - check conditions on message
* netdata/packaging/ci: Introduce an alternate master branch for the workflow testing
1. I created pipeline-redesign-master that will be the master branch for my use cases
2. Modified travis to trigger all production workflows on that branch, so that i can experiment freely with test commits to trigger process and verify execution flow
* remove spaces
* netdata/packaging/ci: comment out
* netdata/packaging/ci: wording nit
* netdata/packaging/ci: adjust the condition properly
* netdata/packaging/ci: test condition for triggering on specific keywords
* netdata/packaging/ci: try different syntax
* netdata/packaging/ci: remove depth, causes problems as we dont get all required objects from repo (adds --depth 50 in clone)
* Missing equal sign in regex?
* netdata/packaging/ci: Update tagger script
1) More verbose messages
2) Update comments copyrights etc
3) Rename release_candidate to set_tag_release_candidate
4) Add curlies on the variables, also change the way we check for path
* netdata/packaging/ci: reinstate git depth
* netdata/packaging/ci: Remove git depth from packaging too
* netdata/packaging/ci: RTFM - was using wrong syntax for slack @here approach
* netdata/packaging/ci: Using install tag at the stage layer, overrides the global one for some reason. fix that
* netdata/packaging/ci: Dont use the pattern matching from travis, do it outside with a script
* netdata/packaging/ci: Do it the other way, script execution doesnt work there
* netdata/packaging/ci: Attempt to fix conditional on release, remove the faulty assumptions as provided by travis team
* netdata/packaging/ci: reintroduce the tag checking
* netdata/packaging/ci: you need to bring tag checking at the stage, as it is calculated during packaging step
* netdata/packaging/ci: you cant escape that one for the moment, merge them and we will rework it later again
* netdata/packaging/ci: dont forget git depth
* netdata/packaging/ci: bring back the dist checking
* netdata/packaging/ci: fix syntax
* netdata/packaging/ci: no depth
* netdata/packaging/ci: Introduce make dist validation tasks: Install from dist file and verify service is starting up
* netdata/packaging/ci: Add a couple more information for troubleshooting
* netdata/packaging/ci: Add extra information prior to executing crucial operations
* netdata/packaging/ci: Make rm error soft here
* netdata/packaging/ci: This is not needed here
* netdata/packaging/ci: Prepare for beta stage deployment process
- introduce a beta provider
- adjust key decryption process, different keys for beta, different for prod
- Enter conditions to deploy beta only on beta process and production only on production process
- Use travis variables, to allow multiple users have beta stages for deployment, if they want
* netdata/packaging/ci: add one more message
* netdata/packaging/ci: Bring back naming to the actual master branch
* netdata/packaging/ci: rename beta to dev
* netdata/packaging/ci: Make permanent restrictions for some of the stricts, so that we don't mess things up
* netdata/packaging/ci: Update README that was missed and also make sure git depth false is at the places needed
* netdata/packaging/ci: fix codacy complaints
|
|
not met (#5816)
I noticed the following scenario: https://api.travis-ci.com/v3/job/190900752/log.txt
Basically nightlies run without any changes being committed, which resulted in job to fail.
The GCS provider tried to push artifacts directory while the folder is not created, that should not happen.
Changes, deploy only when:
1) repo is netdata/netdata
2) branch is master
3) artifacts directory exists
|
|
* netdata/packaging/ci: fix travis syntax
* netdata/packaging/ci: fix testing for static64
|
|
* netdata/packaging/ci: Add integrity tests for kickstart scripts
* netdata/packaging/ci: Add a parallelized step to validate each build against kickstart - totally worth it
* netdata/packaging/ci: re-trigger travis with an empty commit
|
|
* netdata/packaging/ci: Integrity testing
Introduce a scriptlet that validates kickstart integrity in my-netdata.io against the ones in the repo.
Run this through the pipeline on a scheduled manner.
Will refine the conditionals later, once i verify the stage is built up as expected
* netdata/packaging/ci: remove conditionals first
* netdata/packaging/ci: Adjust the names to something more appropriate. Run it along with nightlies
* netdata/packager/ci: Cleanup checksum validation for kickstart files (continued)
1) merge validate_kickstart_integrity.sh and tests/installer/checksums.sh
2) run checksums at the new point on the pipeline
3) Change that unstable gitignore check and use a more file-agnostic check that depends only on git that we already require
4) Do not run the kickstart validation on the online website everywhere, only on the nightly runs
* netdata/packaging/ci: First rounf of PR feedback adjustments
1) changes wordings as recommended
2) pass file info in parameter and use it in the wordings
3) as shellcheck suggests, use -n instead of ! -z. Makes sense actually, ! -z is kind of reverse logic that confuses
More adjustments on a follow up commit
* netdata/packaging/ci: Enable slack integration for kickstart validation
We want to be notified in a timely manner when the kickstart on the website is outdated.
Added a wrapper for incoming webhooks from slack and instead of failing the build we notify slack
Added a debug message in the end of the script to validate the process which i will remove on a follow up commit
* netdata/packaging/ci: fixes
fix sourcing path
missed the /, this new keyboard is a pain :p
|
|
along the way (#5591)
* Refactor Travis CI docker image packaging process (preparatory step)
1) Split build and publish on two scripts
2) Update travis code to simply call both scripts wherever only build was called
Note: This is just an initial step to verify the implementation split.
There will be follow up commits to rework the implementation details
* Cleanup build process, first round
* Publish docker images: Slight refactor on input preparation and printed outputs
* Just like magic numbers are evil, manually settings the same path across the code is also dangerous - declare a WORKDIR and replace
* Use the single archs array for the loops, reference the desired ARCHs in one place
* Use MANIFEST_LIST for clarity
* Declare one docker command to use everywhere, to avoid confusion or misses
* From the way its used, seems like a temporary storage facility, make sure its cleaned up after we are done
* Add a bit more verbosity, plus a typo
* Attempt to run both scripts on the same step
* Do not call publish script within build stage, publish never ran there (to confirm with @paulfantom the details, couldnt track the workflow to justify)
* Cleanup comments as per feedback
* packaging/docker/publishing: indentation
* netdata/packaging/docker: Do not use gitignore to know whether you run on the top level directory of the repo
1) Remove check for .gitignore as its unstable
2) Use show top level and cdup to make sure you are under netdata and at the top level directory
This does not provide any functional change, it is only a more stable way of verifying we are running from TLD
* netdata/packaging/docker: Leave a comment for the next iteration
* netdata/packaging/docker: missed renamed of variable, sowwy
* netdata/packaging/docker: Use mktemp for temporary directory creation. Cleaner, safer
* netdata/packaging/travis: Make sure you get rid of the json file after completion
Just being cautious here, dont let any leftovers behind regardless the fact that this is an ephemeral host the process is running on
* netdata/packaging: make sure we dont break if the file is already removed
|
|
process change) (#5656)
* netdata/build: a small house keeping around coverity scan (no functional or process change)
1) Split coverity install on seperate scriptlet, call it through travis when needed
2) Remove shellcheck exclussions, they dont seem to be raised any more (i might be using wrong version of shellcheck, will see)
3) Fix authors,copyrights etc
4) Make sure all echo goes to stderr
5) When installing, use mktemp for the temporary directory and put everything there. Remove afterwards
* netdata/build: missed - remove temp folder once done
* Rebumping with empty commit to trigger build
* Temp commit to force execution of modified code for coverity
* [ci skip] Revert "Temp commit to force execution of modified code for coverity"
This reverts commit 70e770cbbcb2b9cb4c8040d6249688c810d823f3.
* Temp commit to force execution of modified code for coverity
* Temp commit to force execution of modified code for coverity
* packaging/netdata/coverity-scan: Missed PATH export during scan phase
* netdata/packaging/coverity-scan: the source folder not so temporary after all
Use a more permanent path for the workdir and we will deep dive on the details of this if necessary at a later stage
* netdata/packaging: This is a more general change, push REPOSITORY variable up to travis layer to help testing against personal accounts
* netdata/packaging/coverity-scan: Missed folder creation (no longer using mktemp, duh), also make sure you do a couple of validations so that you know install is failed or not, otherwise we exit with success but nothing has happened
* netdata/coverity-scan: a few more changes towards abstraction
1) move for submit email to travis environment variables
2) Use REPOSITORY in project name
3) Add some new lines on the last log message, to separate it from the curl result
* netdata/coverity: missed parameter from echo
* Reinstate the flag that controls coverity scan execution (was removed for testing purposes, to run coverity in a controlled environment)
* netdata/coverity-scan: Control coverity scan results submission and report back the potential error without blocking the pipeline by failing the step completely
|
|
|
|
|
|
* CI linting .sh.in files
* make linter passing
|
|
* self-contained uninstaller
* move lifecycle test to BATS
* remove all possible netdata service files
* add docs
|
|
* test building docker container on every PR
* fix docker building and switch lifecycle test to use ubuntu cron
* clarify stage name
|
|
|
|
|
|
* use 'git describe'
* no need to embed version string anymore
* fallback mechanism for package versioning
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* consolidate commiting in release pipeline; generate changelog before tagging
* fix how rc0 is created
|
|
* releaser needs to be on branch and not in deteached state
* dashboard.js integrity test; fix configure.ac
* fix key in travis config
* fix builder image name
* verbose diff
* revert #4707
|
|
|
|
* remove firehol release pipeline code; move tagging to separate script; add option to modify files based on tags
* fix #4572
|
|
* decouple nightly cron jobs from packaging stage
* do not fail when there is no new changelog
|
|
|
|
to be sure)
|
|
[netdata minor release] and [minor] (just to be sure)
|
|
[minor]
|
|
|
|
* fix flake8
* unify mountpoint
* remove javascript linting as we already use codacy and LGTM for that
* add python.d.plugin.in
|
|
Make python linter blocking PRs on PRs containing code not conforming to PEP8 standard.
Closes #4167
|
|
|
|
|
|
|
|
* run builds on alpine
* add some essential packages
|
|
* lint libreswan module
* disable linting obsolete shell modules
* shellcheck more shell modules
* fix review; lint helper scripts
* more shellcheck
* fix SC2235 in apache.chart.sh
* shellcheck last portion of charts.d scripts
* final shellcheck; make travis blocking
* fix cppcheck
* fix indents; fix expansion
* fix local var assignments
|
