summaryrefslogtreecommitdiffstats
path: root/.travis/package_management
AgeCommit message (Collapse)Author
2020-08-20Don't run packaging checks on Travis builds that build packages. (#9795)Austin S. Hemmelgarn
* Revert "Fix Travis CI builds and skip Fedora 31 i386 build/test cycles (#9781)" This reverts commit 07e67c4b62b100ab048070f1a0291fe904ff7513. This will be fixed differently. * Skip package build checks on builds that build packages. We don't need to check that package builds work if we're actually building packages.
2020-08-20Fix Travis CI builds and skip Fedora 31 i386 build/test cycles (#9781)James Mills
* Fix Travis CI builds and skip Fedora 31 i386 build/test cycles * Fix lint and address @Ferroin's feedback
2020-08-19Debian packaging (#8982)Sander Klein
* Add protobuf-compiler This adds the package protobuf-compiler to the dependencies. Without it, prometheus support will not be included. * Remove libraries from dependencies There is no need to include the libraries since they will automaticaly be added by ${shlibs:Depends} * Remove various control files - control.buster - control.eoan - control.xenial Default control file can be used - control.jessie About to be unsupported - control.wheezy unsupported - control.trusty Since Netdata need libuv1-dev to build, it cannot be build on Trusty anymore because it doesn't have that library. * Fix sysv init scripts Set the location of the pidfile using $PIDFILE in the init script and in the defaults file. This way no changes are needed when compiling for systemd or sysv. * Update documentation to reflect the changes * Add note about updating the changelog * Update install instructions * Remove travis control file adjustments Since these control files have been removed, this code is now obsolete. * Remove unused control.focal file
2020-08-10Fix print message when building for Ubuntu Focal (#9694)Devin Smith
2020-08-07Add support for DEB packages for Ubuntu 20.04 (focal) (#9592)James Mills
* Add support for Ubuntu 20.04 DEB packages * Add Travis config to also build the DEB packages for Ubuntu 20.04 (focal)
2020-07-30Added eBPF collector support to DEB and RPM packages. (#9628)Austin S. Hemmelgarn
* Add DEB/RPM package build tests to Travis. * Add working support for bundling eBPF in binary packages. * Show stdout and stderr from commands run in LXC. * Add proper bundling code for libbpf. * Use AC_CHECK_FILE for libbpf.a external dep. This way it gets properly logged in both configure output and the configure log.
2020-04-03Correctly fixed RPM package builds on Fedora. (#8595)Austin S. Hemmelgarn
* Revert "Fix broken Fedora 30/31 RPM builds (#8572)" This reverts commit 71290d9cb904e8ea08456e5c4ab24bc274e593bf. It didn't actually fix things and caused a different set of issues. * Build LWS and mosquitto with -fPIC in RPM builds. This fixeslinking errors that are currently causing RPM package build failures for Fedora 30/31.
2020-04-02Fix broken Fedora 30/31 RPM builds (#8572)James Mills
2020-04-01Add missing override for Ubuntu eoan (#8547)James Mills
2020-03-20Fix broken dependencies for Ubuntu 19.10 (eoan) (#8397)James Mills
* Fix broken dependencies for Ubuntu 19.10 (eoan) * Update Python-based configure LXC env to also handle Ubuntu 19.10 (eoan)
2020-01-16Move the script for installing required packages into the main repo. (#7563)Austin S. Hemmelgarn
* Move the script for installing required packages into the main repo. Based on discussion with Costa. Most of what this script is actually used for directly is in the main repo anyway, so it makes more sense to have it there so that any changes get reviewed properly. * Fix typo in RPM package build setup.
2019-12-13Inject archived backports repository on Debian Jessie for CI package builds. ↵Austin S. Hemmelgarn
(#7495) * Inject archived backports repository on Debian Jessie. Debian Jessie only ever had libuv1 as a backport, not part of the base release. This means that we need the backports repository on Jessie for the build to work, which has been archived. This updates the environment prep script for building Debian packages for Jessie to inject a file into `/etc/apt/sources.list.d` to add in the archived repository so that everything works correctly. * Change how the archived backports are injected for jessie. This shifts to just appending to the main sources.list file, which should be more reliable. It also explicitly updates caches after changing the sources. * Switch back to using a separate sources file. * Moved Jessie archived backports handling to repo setup function. This makes the code a bit cleaner overall. Also, we need to disable validity checking for the archives during the call to `apt-get update` otherwise things will just refuse to work. * Fix syntax error.
2019-12-10ci: remove ubuntu trusty 14.04 from build (#7481)Mansour
* ci: remove ubuntu trusty 14.04 from build * Update packaging matrix with ubuntu 14.04 changes
2019-09-16netdata/packaging: Introduce separate CUPS package for debian distributions ↵Paul Emm. Katsoulakis
(#6724) * netdata/packaging: baby steps, create the new package and introduce the install file * netdata/packaging: [ci skip] change branch to be able to test * netdata/packaging: we do define usrlib path to be /usr/lib * netdata/packaging: silence sign errors for now * netdata/packaging: Add rules for plugin-cups sub package * netdata/packaging: [ci skip] That is libexec, not lib, duh * netdata/packaging: [ci skip] explicitly arrange the other plugins also * netdata/packaging: [ci skip] fix bug * netdata/packaging: copy, dont move * netdata/packaging: move all plugins around, also remove a slash * netdata/packaging: [ci skip] align travis.yml changes with the ones we used on rpm cups pkg split * netdata/packaging:[ci skip] add netdata dependency * netdata/packaging:[ci skip] Update control files * netdata/packaging: Add separate control file for buster, dependencies deviate (libprotoc17 instead of libprotoc10....) * netdata/packaging: [ci skip] whoops * netdata/packaging: [ci skip] Copy over files from temp dir, if destination non existent * netdata/packaging:[ci skip] -d, not -f * netdata/packaging: [ci skip] long shot * netdata/packaging: [ci skip] revert this - need to retest * netdata/packaging: [ci skip] remove the .install, as per an article input (https://askubuntu.com/questions/660373/dh-install-fails-with-missing-files) these files are for handling items not managed by the upstream build system, so bye bye * netdata/packaging: Ok, lets ditch dh_auto_configure * Revert "netdata/packaging: Ok, lets ditch dh_auto_configure" This reverts commit 3b497cf5f6e157ce5429ffce38452515670a934a. Well, that didn't affect anything * netdata/packaging: [ci skip] here's a crazy thought Okay, assuming that packager decided to use debian/tmp because we introduced multiple packages. If that is the case, then move the custom stuff out first and then just move all debian/tmp to debian/netdata. In theory, that should get me back on track for the rest of the package processing. Lets see if i was right * netdata/packaging: [ci skip] mv complains, forgot about that, fix forgot about that, just copy for now to verify the use case and we will consider rsync approach for faster processing afterwards * netdata/packaging: nope, that didn't work. rollback and remove also a suspicious backslash * netdata/packaging: ok, trying this * netdata/packaging: really? * netdata/packaging: [ci skip] reinstate original setup, the bug probably was too obvious to spot * netdata/packaging:[ci skip] now that you finally noted the www bug, try to copy over the rest of the stuff in * netdata/packaging: [ci skip] long shot, etc is not properly created, lets see if it was already there and we messed it up * netdata/packaging: [ci skip] fix perms, add placeholder folders in etc * netdata/packaging: [ci skip] preserve ownership details when copying over * netdata/packaging:[ci skip] reinstate branch condition * netdata/packaging: redudant * netdata/packaging: xenial - add cups subpackage * netdata/packaging: remove cups dependency from those distros, missed to update the control file * netdata/packaging: dont use personal account info
2019-09-15netdata/packaging: Bring on board two scripts that build libuv and judy from ↵Paul Emm. Katsoulakis
source (#6850) * netdata/packaging: Script to build judy from source * netdata/packaging: Add script to build libuv from source
2019-09-13netdata/packaging: nits and fixes for packaging (#6842)Paul Emm. Katsoulakis
* netdata/packaging:[ci skip] Bring over netdata.spec before removing the temp directory. We need it to resume RPM workflow * [Package amd64 DEB] Package build stable test * netdata/packaging:[ci skip] add branch for testing * [Package amd64 DEB] Package build stable test * [Package i386 DEB] Package build stable test * [Package i386 DEB] Package build stable test * [Package amd64 DEB] Package build stable test * [Package i386 DEB] Package build stable test * [Package amd64 RPM] Package build stable test * [Package i386 RPM] Package build stable test * netdata/doc: update distro support, due to lxc limitations we dont have those yet * [Package i386 RPM] Package build stable test * [Package i386 RPM Enterprise Linux] Package build stable test * netdata/ci:[ci skip] yes, we must test rpm too * [Package i386 RPM] Package build stable test * [Package amd64 RPM] Package build stable test * netdata/packaging:[ci skip] Add i386 RPM and DEB distro triggers * netdata/packaging: [ci skip] remove the test branch info * netdata/doc: [ci skip] fix doc * netdata/doc: [ci skip] update doc more
2019-09-13netdata/packaging: nit - missed trailing slash (#6840)Paul Emm. Katsoulakis
2019-09-13netdata/packaging: Ensure that we do not mess with CI tooling, when building ↵Paul Emm. Katsoulakis
stable (#6838) * netdata/packaging: Dont mess with the CI code when playing with the repo Sadly, we need to run consistently the same code for the CI side. When doing checkouts to build stable code, we move the whole repo thus resulting on changing the ci code too. This is usually fine when building nightlies, on stable though is a completely different story. To mitigate this, copy the repo into a temporary folder. Then do whatever you need to prepare and copy the tarball. Once you are done, just go back to the original repo folder and remove the temp. That should work fine for both nightlies and stable, with a little overhead of copying over the repository, but we have to. We want to be able to run with latest CI code, when trying to build a stable release. * [Package amd64 DEB] Package build stable test * netdata/packaging: allow me to test on my repo the build process (dont care for deploy) * [Package amd64 DEB] Package build stable test * netdata/packaging: [ci skip] copy under tmp * [Package amd64 DEB] Package build stable test * netdata/packaging:[ci skip] play with CWD of the command, thats cleaner and well, the right thing * [Package amd64 DEB] Package build stable test * netdata/packaging: [ci skip] cd did not fall for cwd, provide full path and move on * netdata/packaging: a few nits like dont use cwd for some commands, import sys, simplify some variable usage * netdata/packaging: [ci skip] focus, missed a reference to the tarballen * netdata/packaging: [ci skip] remove test branch * netdata/packaging: Thanks for taking note Codacy, here's a better temp file for the job
2019-09-12netdata/packaging: we didnt fix changelog handling, fixes and nits now (#6837)Paul Emm. Katsoulakis
1) We missed a dot when reconstructing since_version 2) When we are on a patch release, just step back to fetch history from the previous patch release (or last minor release, respectively)
2019-09-11netdata/packaging: fix ubuntu/xenial runtime dependencies (#6825)Paul Emm. Katsoulakis
2019-09-05netdata/packaging: fix changelog generation failing the build (#6778)Paul Emm. Katsoulakis
We bumped into an error that caused debian/jessie package generation to fail. When building stable release, we don't properly detect the change from previous stable to this one. So the changelog generation fails and only on some distros causes the process to fail. With this change we attempt to detect this peculiarity and just try fo decrease latest release version, to properly fetch the full changes
2019-09-05netdata/packaging: override control file for debian/buster, different ↵Paul Emm. Katsoulakis
library available for protoc runtime (#6777)
2019-08-29netdata/packaging: Add purging logic for package cloud repositories (#6732)Paul Emm. Katsoulakis
* netdata/packaging: Add purging logic for package cloud repositories 1) Introduce scriptlet that is executing the cleanup on a designated repository, with a designated retention period 2) Add .travis.yml configuration to execute purging logic with the right config on the nightly operations * netdata/packaging: uncomment the temp dir clean up * netdata/packaging: fix codacy * netdata/packaging: [ci skip] Purge only -edge and -devel packages
2019-08-19netdata/packaging: fix script naming, rpm was not accurate on that contexTPavlos Emm. Katsoulakis
2019-08-16netdata/packaging: Align libdir in all configure commands (#6682)Paul Emm. Katsoulakis
* netdata/packaging: Align libdir with the one we expect our installation to use, following the rest of the directory correllations * netdata/packaging:[ci skip] One more
2019-08-16netdata/packaging: in the peculiar case where a patch release has been ↵Pavlos Emm. Katsoulakis
applied, versioning will be messed up for nightlies. To mitigate this, just prepend patch release number to the build number and this should provide with a proper higher number for nightlies every time
2019-08-16netdata/packaging: [ci skip] Correlate configure command (#6678)Paul Emm. Katsoulakis
Overriding as i will be closely monitoring the results
2019-08-09netdata/packaging: Documentation on distribution support matrix and ↵Paul Emm. Katsoulakis
functionality availability (#6552) * netdata/packaging: [ci skip] Add trusty and xenial * netdata/packaging: [ci skip] check directly for the flag existence * netdata/packaging: Append table for functionality-related support matrix * netdata/packaging: add line ending * [ci skip] revert this - will get help from joel * netdata/packaging/doc: [ci skip] experiment * netdata/packaging: [ci skip] fix * netdata/packaging: [ci skip] Add more info needed for the rest to check the style * netdata/packaging: [ci skip] different icon * netdata/packaging: [ci skip] split per arch * netdata/packaging:[ci skip] More adjustments * netdata/packaging:[ci skip] more adjustments * netdata/packaging:[ci skip] style * netdata/packaging: update status * netdata/packaging: [ci skip] update rpm/deb state * netdata/packaging:[ci skip] different icon for unchecked * Revert "netdata/packaging:[ci skip] different icon for unchecked" This reverts commit 6ed2a46859f0bb1dd3c1a266280074b023028382. oops, the other one, not this * netdata/packaging: [ci skip] change symbol for unchecked * netdata/packaging: we dont know source state for all other linux, so make it questionmark * netdata/packaging:[ci skip] fix suse leap state * netdata/packaging: [ci skip] fix deb state * Update DISTRIBUTIONS.md * Update DISTRIBUTIONS.md * Update DISTRIBUTIONS.md * netdata/pkg: [ci skip]Update DISTRIBUTIONS.md Sync x86 table with amd64 table. They are not 1 on 1 but close enough, so syncing and then will adjust to the differences at another pass * netdata/packaging: [ci skip] code health doesnt make sense, rename to ci smoke testing which makes more sense * Update DISTRIBUTIONS.md * Update DISTRIBUTIONS.md * Update DISTRIBUTIONS.md * Update DISTRIBUTIONS.md * Update DISTRIBUTIONS.md * netdata/packaging:[ci skip] Stricter check on protobuf, plus separate config for trusty because no libuv1 available there * netdata/packaging: add functionality-related details * netdata/packaging/docs: add skeleton sections * netdata/packaging/docs: remove that * netdata/packaging: fix header * Update DISTRIBUTIONS.md * netdata/packaging/docs:[ci skip] fix indent * netdata/packaging/docs: [ci skip] change wording * netdata/packaging/docs: kickstart-static64 everywhere * netdata/packaging/docs: [ci skip] Add legend and update some fields * netdata/packaging/docs: add disable flags * Update DISTRIBUTIONS.md * Update DISTRIBUTIONS.md * Update DISTRIBUTIONS.md * Update packaging/DISTRIBUTIONS.md Co-Authored-By: Joel Hans <joel.g.hans@gmail.com> * Update packaging/DISTRIBUTIONS.md Co-Authored-By: Joel Hans <joel.g.hans@gmail.com> * Update packaging/DISTRIBUTIONS.md Co-Authored-By: Joel Hans <joel.g.hans@gmail.com> * netdata/packaging/docs: feedback * Update packaging/DISTRIBUTIONS.md Co-Authored-By: Joel Hans <joel.g.hans@gmail.com> * Update packaging/DISTRIBUTIONS.md Co-Authored-By: Joel Hans <joel.g.hans@gmail.com> * Update packaging/DISTRIBUTIONS.md Co-Authored-By: Joel Hans <joel.g.hans@gmail.com> * Update packaging/DISTRIBUTIONS.md Co-Authored-By: Joel Hans <joel.g.hans@gmail.com> * Update packaging/DISTRIBUTIONS.md Co-Authored-By: Joel Hans <joel.g.hans@gmail.com> * netdata/packaging: more feedback
2019-07-17netdata/packaging: Enable built-in support for prometheus remote write in ↵Paul Emm. Katsoulakis
packaging (#6480) * netdata/packaging: [ci skip] Add prometheus remote write plugin deps * netdata/packaging: [ci skip] adjust for the tests * netdata/packaging: [ci skip] Apparently we need proto-c too * netdata/packaging: [ci skip] Add dependencies also * netdata/packaging: [ci skip] fix suse dependency * netdata/packaging: [ci skip] epel-release should always go first dumdum * netdata/packaging: [ci skip] g++ needed for prometheus * netdata/packaging: First prepare the repo, then start throwing in packages * netdata/packaging: [ci skip] fix attempt for missing snappy on some debian-based packages * netdata/packaging: [ci skip] protobuf compiler dep * netdata/packaging: [ci skip] Add the generic maintainer info and the new website. Also stop pushing snappy to Debian/Stretch, doesnt have it * netdata/packaging: [ci skip] Remove branch dev setup * Empty commit to re-trigger Travis
2019-07-16netdata/packaging: [ci skip] well, when no match in place, its equal to ↵Pavlos Emm. Katsoulakis
zero. Fix it.
2019-07-16netdata/packaging: [ci skip] Make an update early in the repoPavlos Emm. Katsoulakis
2019-07-15netdata/packaging: Binary distributions - clean up .DEB package generation ↵Paul Emm. Katsoulakis
process (#6465) * netdata/packaging: [ci skip] reinstate dev branch to help testing * netdata/packaging: [ci skip] Yes, make the right package choice on json-c for debian based distros.. * netdata/packaging: [ci skip] two more packages that differentiate on debian/ubuntu............... * netdata/packaging: [ci skip] Add more package dependencies around building * netdata/packaging: [ci skip] Introduce changelog generation technique for .DEB 1) Remove contrib/debian/changelog from gitignore, we want it in 2) Add variables to monitor latest release version and latest release date from git 3) Add a templated contrib/debian/changelog, that will be the starting point for our changelog generation. Either on stable release or nightly releases, we will be generating the logs since the previous release, to manage the content. Note: We might eventually take the effort to produce one large changelog and append each time the extra stuff 4) Add the steps to prepare the source and the changelog for building and execute the build At this stage things should still be broken, but we should have come closer to a ready-to-build-the-package environment per distro * netdata/packaging: [ci skip] Extract to specified directory, otherwise we dump it on a privileged folder using an unprivileged user (builder) Also, a fix on sed command, missed the s keyword on the string replacement * netdata/packaging: [ci skip] Should be in-place replacement, use -i instead of -e * netdata/packaging: [ci skip] wrong parameter usage * netdata/packaging: [ci skip] factor out the two build commands to run as a single bundle. refactor trigger deb build script respectively * netdata/packaging: [ci skip] Be sure to copy over the build script to the build path * netdata/packaging: [ci skip] OCD - unnecessary s added * netdata/packaging: [ci skip] Changelog generation should take place on the repo side, then copy it over to the destination (If that doesnt work well, will just provide a static link for changelog * netdata/packaging: [ci skip] missed the ignore branch option Also, a nit on the gbp command * netdata/packaging: [ci skip] link to the original path, then create symlink on build folder Also, fix --since option * netdata/packaging: [ci skip] Run replacement of changelog fields on the host, not on the container * netdata/packaging: [ci skip] fix contrib path, you are no longer reading from container path * netdata/packaging: [ci skip] fix version * netdata/packaging: [ci skip] add dep * netdata/packaging: [ci skip] fix changelog template * netdata/packaging: [ci skip] Fix debian package build, we were looping through forever. Exclude contrib all together * netdata/packaging: [ci skip] Add logic to gather all package related content from the container folder and push it for publishing * netdata/packaging: [ci skip] fixes and nits 1) use the branch for the beta deployment 2) change the cp pattern * netdata/packaging:[ci skip] add more debug info * netdata/packaging: [ci skip] Adjust LXC directory permissions, so that the rest of the workflow can pull through * netdata/packaging: [ci skip] Using expressions within double quotes doesnt work, use quotes only on the variable to avoid syntax warnings from shellchecking * netdata/packaging: [ci skip] Update build deps * netdata/packaging: Add runtime deps (hopefully) * netdata/packaging: [ci skip] remove old distro * netdata/packaging: [ci skip] dont let that space there, breaks the syntax * netdata/packaging: [ci skip] missed one * netdata/packaging: [ci skip] yank yank yank * netdata/packaging: [ci skip] You missed www over there * netdata/packaging: [ci skip] Remove broken distros (no container images). We will revisit the way to support multiple distros in a different way, lxc failed us * netdata/packaging: Pull in go.d plugin, also fix chmod command to just pass the folder since with the -R doesnt require anything else * netdata/packaging: [ci skip] Get cups, freeipmi and nfacct on * netdata/packaging: [ci skip] pull different package in * netdata/packaging: [ci skip] Install dependencies on the build machine * netdata/packaging: [ci skip] dont do post-clean, not on all versions * netdata/packaging: [ci skip] move go install seperately * netdata/packaging: [ci skip] Towards a more appropriate go.d plugin install * netdata/packaging: [ci skip] fix permissions setting * netdata/packaging: [ci skip] fix freeipmi package install * netdata/packaging: [ci skip] Fix build arch setting on the build command, also remove pre-clean, it breaks on debian/jessie * netdata/packaging: [ci skip] fix postinst actions, we use libexec * netdata/packaging: adjust www and go.d permissions * netdata/packaging: [ci skip] Handle the different build tool version cases, also take care some dependency issues on older version jessie * netdata/packaging: [ci skip] adjust the control file based on the distro * netdata/packaging: [ci skip] Move control file preparation prior to packaging the source * netdata/packaging: [ci skip] Adjust versioning scheme for the nightlies * netdata/packaging: [ci skip] Let RPM be built too for testing * netdata/packaging: [ci skip] Factor out version calculation * netdata/packaging: [ci skip] Attempt to modify version properly * netdata/packaging: [ci skip] Update documentationg regarding .DEB files * netdata/packaging: [ci skip] autogen not available everywhere, also fix json-c deps * netdata/packaging: Remove development settings, also remove beta deployment as its not needed
2019-07-10netdata/packaging: [ci skip] replace all necessary items with amd64Pavlos Emm. Katsoulakis
2019-07-10netdata/packaging: [ci skip] Wrong positioning of argumentsPavlos Emm. Katsoulakis
2019-07-10netdata/packaging: [ci skip] Nit - pass container parameterPavlos Emm. Katsoulakis
2019-07-10netdata/packaging: Fix RPM packaging workflow issues, plus draft changes for ↵Paul Emm. Katsoulakis
.DEB packaging (#6415) * netdata/packaging: enable builds on our dev repo to initiate working * netdata/packaging: first round of fixes and changes 1) Factor out source preparation for latest code 2) Prepare stable/latest build branching on deb code 3) In Travis, make branch code selection variables to make changes easier (Found a missed change when merging to prod, so the variables will make switching cleaner and easier) 4) Clean v letter from stable version for RPM (side fix) More to come, at this point nothing should be yet working * netdata/packaging: [ci skip] dont run epel install for centos 7/i386 * netdata/packaging: [ci skip] add all of them * netdata/packaging: [ci skip] redesign stable build. Just checkout the tag, dont rely on the published source because we want the binaries out before the release * netdata/packaging: oh lord.. the versioning.. * netdata/packaging: [ci skip] add git depth false * netdata/packaging: [ci skip] Do a git fetch first, so that checkout can actually work.. (mercy) * netdata/packaging: [ci skip] Do it all * netdata/packaging: [ci skip] Some more debug information and make sure you checkout master before any other action * netdata/packaging: [ci skip] didnt fix the problem, remove * netdata/packaging:[ci skip] Attempt to bring the right branch * netdata/packaging: [ci skip] Extreme, use a large depth and see what it will bring onboard. remove useless commands * netdata/packaging: [ci skip] found the culprit, was on the forked repo - bring back the expected commands * netdata/packaging: [ci skip] Well, do it a bit different otherwise we cant cheat the versioning problem. Check out the tag if necessary and always do anotated tagging with the right version * netdata/packaging: [ci skip] a couple of debug message nits and styles, plus one more further alignment of packaging process for .DEB 1) Clean style a bit 2) Clean some messages and comments 3) Bring the RPM processs to DEB packaging, for the tarball retrieval and placement * netdata/packaging: [ci skip] factor out common dependencies, to reuse on both cases * netdata/packaging: [ci skip] Reinstate the production branch assignments we added for testing, also fix call to method * netdata/packaging: do NOT trigger on pull requests, of course
2019-07-03netdata/packaging: Netdata binary packages generation - spec file ↵Paul Emm. Katsoulakis
refinement, support for nightlies (RPM) (#6369) Detailed list of changes: * netdata/packaging: First round of netdata.spec refactoring - package dependencies - Remove freeipmi/nfacct plugin flags. We auto-detect all plugins by decision - Start refactor of package dependencies - Add missing dependencies, with respect to distro peculiarities - Adjust existing dependencies, so that distro-specific package names is applied * netdata/packaging: add packaging trigger mechanism on binary-packages-specfile-cleanup to help testing * netdata/packaging: WIP - Allow building nightly versions for RPM (Still investigating the right version scheme) * netdata/packaging: Attempt to fix syntax errors - wow travis, mercy * netdata/packaging: tiny nits and fixes 1) there was a syntax error in version definition 2) bring default deploy repo name at the top, then overwrite when latest is published 3) Leave a comment * netdata/packaging: Move these core package requirements to the right section * netdata/packaging: libuv requires >= 1, missed to add that on devel libs * netdata/packaging: some comments, also when you put logrotate in our conf directory, link it to actual log rotate * netdata/packaging: Adjust configure options on spec file (align with netdata-installer.sh) * netdata/packaging: In order to properly support packaging both latest and stable releases, you need to star that extraction part, folder name contains the version with the hash that we cant tell * netdata/packaging: until we fix versioning, bring the original unprocessed spec file and pass the version we like in * netdata/packaging: fix version string in the spec file * netdata/packaging: Replace source path, we need to work with whatever we downloaded (to be refined) * netdata/packaging: [ci skip] you mixed the paths, reverse them * netdata/packaging: [ci skip] attempt to fix source path when needed * netdata/packaging: fixes and nits 1) generalize the spec file refinement method 2) Do both replacement that way 3) Adjust file names and paths * netdata/packaging: Replace Source0 explicitly and also make sure you remove extra new lines that may confuse the replacement later * netdata/packaging: [ci skip] move env outside, add some more log * netdata/packaging: [ci skip] adjust logic of replacing tag lines in spec file * netdata/packaging: [ci skip] Alas, fetch the archive, rename the structure to the new version, repack and proceed * netdata/packaging: [ci skip] Extract to specific path * netdata/packaging: [ci skip] replace problematic tar call with the respective pythonized one * netdata/packaging: [ci skip] Add new friendly octal format for chmod * netdata/packaging: [ci skip] more checks * netdata/packaging: [ci skip] dont chmod, chown the whole folder after extraction * netdata/packaging: nit * netdata/packaging: [ci skip] attempt to fix permissions differently * netdata/packaging: [ci skip] Another hack, attempt to fix permissions on the host directly * netdata/packaging: [ci skip] fine, do the move also on the host, pointless to run all through the container * netdata/packaging: [ci skip] Do the move through the container. Also, move a more specific folder * netdata/packaging: [ci skip] move as w00t * netdata/packaging: [ci skip] fix * netdata/packaging: sync permissions handling, round two * netdata/packaging: scratch those hacks, we should do latest differently * netdata/packaging: [ci skip] Set up rpm version string depending on whether we are on latest or not * netdata/packaging: [ci skip] Branch the code here, latest build up should be on the commit rather the tar files * netdata/packaging: Attempt to handle latest version build up * netdata/packaging: [ci skip] Dont continue processing if the file was not generated. Also do the tagging before autoreconf * netdata/packaging: [ci skip] fixes and nits * netdata/packaging: [ci skip] When doing annotated tags, you need to provide a message otherwise an editor pops up and will block the build * netdata/packaging: on the latest code case go all the way host level operations * netdata/packaging: [ci skip] Do not modify version, no longer need it because we properly generated the package with the right version * netdata/packaging: [ci skip] fix sensors line, you missed the Recommends keyword * netdata/packaging: Some code style (remove tabification from some tags) and also remove Recommends, we start with required packages for build and runtime and then we document further needs * netdata/packaging: [ci skip] Make sure you get epel in there, its needed * netdata/packaging: [ci skip] Fix bogus date * netdata/packaging: [ci skip] add these packages to include functionality * netdata/packaging: [ci skip] more fine grained dependency handling * netdata/packaging: [ci skip] Dependency hell - do not depend on libnetacctfilter if on el/7 * netdata/packaging: [ci skip] Be serious, that is not what you do when building the package silly * netdata/packaging: [ci skip] define directory attributes too * netdata/packaging: [ci skip] mystery unfold - comment out as we need to figure out how to handle missing directories that our installer creates if necessary * netdata/packaging: [ci skip] Make sure you properly set deployment repo based on commit message * netdata/packaging: [ci skip] Install these packages for specific versions, remove optimisations for now * netdata/packaging: [ci skip] Make package yanking flexible, so that we can manage both netdata and netdata-edge cleanup properly * netdata/packaging: [ci skip] Actually, set user at travis layer and let the script consume it only. We need to be sure the same user is used on yanking and the deployment process * netdata/packaging: [ci skip] wow pkatsoul * netdata/packaging: Dont forget to install some of the enabled plugins * netdata/packaging: nfacct.plugin not really working on rhel, disable temporarily * netdata/packaging: include installed directories to package * netdata/packaging: [ci skip] nits * netdata/packaging: fixes * netdata/packaging: [ci skip] dont add ghost files * netdata/packaging: [ci skip] Dont do that, make install already places the files in the right location. Also, seems that libexec is not really getting overriden there * netdata/packaging: [ci skip] Extreme - override _libexecdir macro * Revert "netdata/packaging: [ci skip] Dont do that, make install already places the files in the right location. Also, seems that libexec is not really getting overriden there" This reverts commit 320102c96284e4622266f6f089d6ce87319976f3. That wasnt really the problem here * netdata/packaging: [ci skip] Include contents of dir here * netdata/packaging: fixes * netdata/packaging: [ci skip] Fixes and additions - Tighter permissions for some files - Introduce go.d plugin install (likely broken like this, dont recall if bash style code is fine here) - Raise the path overrides to the spec level, otherwise we build with the overrides and package with the defaults * netdata/packaging: [ci skip] Missed an invalid variable. These files are under our current path, no need for complexity * netdata/packaging: [ci skip] bring sha method onboard * netdata/packaging: [ci skip] Wrong path provided, should be under build root * netdata/packaging: [ci skip] Add registry directory * netdata/packaging: [ci skip] add missing config directory definitions * netdata/packaging: [ci skip] Fix missing dirs * netdata/packaging:[ci fix] hos * netdata/packaging: [ci skip] Fixes and additions - Change versioning to v1.15.0.114 from v1.15.0114 - Add CUPS, Xenstat, json-c * netdata/packaging: [ci skip] fix package dependencies on the builder setup, otherwise spec will fail * netdata/packaging: [ci skip] limit xenstat to suse and fedora for now * netdata/packaging: [ci skip] separate fedora and suse cases, unfortunately suse has different package names.... * netdata/packaging: [ci skip] yajl is not the same package everywhere, of course * netdata/packagin: [ci skip] skip xenstat for now * netdata/packaging: [ci skip] remove the vvvvvvvvv * netdata/packaging: [ci skip] one more fix attempt * netdata/packaging: [ci skip] Spec file extraction is different when building from stable. We *must* fetch it from the tarball rather than the local one * netdata/packaging: [ci skip] spec_file defined on all cases, latest or stable. Also move latest spec file copy after make dist operation, that is where is suppose to be * netdata/packaging: [ci skip] Tag source with the local version, as we build upon the very latest * netdata/packaging: oopsie -- remove json-c from that place, it was forgotten * netdata/packaging: fix packaging, even though devel is consistent between fedora and suse, the binary one is not .... (also fix default perms to be root:netdata rather root:root) * netdata/packaging: Suse love - again, libuuid-devel same, but libuuid is named libuuid1 on suse...... * netdata/packaging: [ci skip] libcap same.... * netdata/packaging: [ci skip] use the noreplace directive for the configuration files * netdata/packaging: [ci skip] Bring back master as the triggering repository * netdata/packaging: [ci skip] attempt to enable/disable service where necessary. Default preset action requires further setup, so we override that * Revert "netdata/packaging: [ci skip] Bring back master as the triggering repository" This reverts commit 9b25a622dc5f12b1539f6863bab52d7bb225fd8d. ** Reverting as we need to test further the service enable action ** * netdata/packaging: [ci skip] revert this change - not the right way, we have empty files * netdata/packaging: [ci skip] Add daemon-reload option, to make sure systemd learns about the newly installed service * netdata/packaging: [ci skip] Attempt to silence codacy by factoring out the few small methods for common usage, then kill directories to reduce complexity adn thats it * netdata/packaging: [ci skip] Just making sure codacy reprocess everything * netdata/packaging: [ci skip] missed one adjustment * netdata/packaging: python style - print shouldnt have space before open parenthesis. Also, add restart on all cases on the spec file * netdata/packaging: [ci skip] fix syntax * netdata/packaging: [ci skip] syntax fixes 2 * netdata/packaging: [ci skip] another miss * netdata/packaging: [ci skip] You need to explicitly trigger a restart, apparently * netdata/packaging: [ci skip] override packaging user definition temporarily, to get things going on testing
2019-06-09netdata/packaging: Introducing automatic binary packages generation and ↵Paul Emm. Katsoulakis
delivery for RPM types (Phase 1) (#6223) * netdata/packaging: Introducing automatic binary packages generation and delivery for RPM types (Phase 1) With this commit we introduce our packaging toolkits and workflows to automate the delivery for RPM packages to packagecloud.io, our packaging content delivery infrastructure. At this stage we have only prepared the required flows to publish our latest stable release. Missing items for now: 1) Packaging versioning - we won't be providing increasing version numbers for the same upstream netdata version 2) Nightlies - our nightlies won't be provided by this channel, yet With this changeset, we also introduce the basic artifacts for DEB packaging generation, with the same misses mentioned above. Debian is likely not to be functional within this sprint run though. Here's a more detailed list of the changes, as they were generated from the original branch: 89baa00b netdata/packging: Prior to the PR, place the master bracnh as the place to execute packaging acfd2804 export variables at before_script b95b112b remove that README.md, we will cover details from the distributions document in packaging folder e52ab82e Update README.md 8df1f8e9 [ci skip] There is no need for package all option. There is absolutely no uniformity on available images or distro per arch to do our job for all at the same time, so remove it for now 6fc1600c remove a few of the unavailable architectures from distros 7e2741de missed arch, we actually have i686 64c72b4e Add arm64 architecture as a build option (to be refined on follow up commit for the distros not having arm64 or amd64 not to trigger build) ead672b4 silence shellchecking, also fix a shameful recommendation from shellcheck so that array parsing can work 986d0e00 you disappoint me shellcheck, shame on you a5c11a75 remove debug ls command 4b5fade6 nit - too obvious to spot 7edb9215 nits and fixes 06cf10b9 fix the obvious miss - you need to the full path appended before the container name, otherwise no folder.. d777f8ae Give me a break travis, whats wrong with you -- define packages directory at the yaml config level f9756642 nits ee253493 missed some old parts 1506105c we dont need container root variable here, we just need to go through the base LXC directory 03dc231c restructure packaging preparation process 529cdedb simplify, unify logic around build arch handling between creation and building 17624c14 pull packages directory variable in the code. We calculate base on arch c47ea326 remove redudant package type var, also hard exit when not found the right stuff 8890041d Pull more arch dependent variables out of yaml config and into the code 832fbf71 remove unused variable b54d6ccb Move check lower 0b1a3a7e handle build arch requirement, plus some shellcheck fixes 07bb01fe missed this, should pass container name instead 1ab8a3a4 we dont use build arch within this scope, ditch the check 6d3df1b3 Add i386 packaging keywords 1b403680 Remove arch from the descriptions and the environment variables We wont be building sections per arch in travis, to minimize config file size. for each distro we will always build both ARCHS unless the commit message instructs us to build specific architecture 8f38caa5 Drop support of 42.3 8e394906 its word counting here, focus d3f6009c fix yanking command 5b4d7b9a dont call gpg flag on all distros, focus 7e0f4d64 better listing when yanking packages bdb2d3aa skipping gpg checks, for now 75841a67 use newer distro instead a439a9f9 downgrade base dist eba0fba4 Merge branch 'master' into package-cloud-deployer 241c0e10 nits and fixeS aad8d8bf fix naming, add update process in debian preparation 872b15af Separately list rpm and srpm contents for yanking b3ef3a6f el/8 and fedora/31 not ready yet d468cad7 Null commit 22135a2f fix stages for packaging 1464cd6b Null commit 5d44d4f9 implement yanking - when same version is uploaded, remove to re-add 0a4e2e46 sync the repo first of all 9f71583c Generalize repo tool definition on yaml config, so that we adjust per distro as needed 3988eb74 adjustments on the distro list, also remove coreutils -- not important 51571968 check if its just permission issue 38fa8b2b adjust syntax e08ebd2b Due to the peculiarity of the way we do things, script out the packages preparation so that we can sudo them 0a3ce2c1 Now that we solved all other issues, make sure you add rpm building dependencies to LXC aa0b370e a couple of nits and fixes 1) we need sudo within the container 2) just use wget everywhere, dont mix up too many packages for the same thing 3) do not try to add packages again, our dependency script should take care that. Single point of responsibility a3f42e8d You need to request home directory to be created (need to review under the different distros for the right syntax though) d7a3d0d0 dont go and add manually the required packages, call our dependency scriptlet 5c0888b7 skip GPG checking for now 9aa64bbe dont create from the python lib 19f8cc6b Add more keywords for better grouping 0d5b89d7 nits e260a57d python libs not in good shape, try running create from command line instead 7fa58921 netdata/packaging/ci: Bingo - add container name on the RPM jobs b99ff89e some nits and fixes, lets see 636057b2 bring back sudo, move to /var/lib/lxc, pass -E to sudo to preserve vars 1e487517 i wonder.. 97583506 Update SuSE list 2e92649b fix doc - wrong distro details 6f53a4c2 Initial distro document -- WIP d2ef5e4f Add some README stuff, also introduce yanking step in pre-deploy (empty script at this point) 465f293f Change the token 36431ab5 Add a package cloud wrapper (required for yanking RPMs) 2142a024 Auto-detect username for the beta deployment, so that others can use it. Secret key will remain as is for now, will do revisit this also, so that others can beta test the implementation too. Also include provide for the production deployment, when running on master ab5a85ad Add more special conditions, also fix spacing and style a little bit more: ce893eae build only on special conditions (branch name for now) 82340536 Attempt to find how to ref template in stages 5375cd93 one more thing - use ubuntu instead of download template, lets see 4ff26df4 Attempt to fetch lxc-templates (not installed by default 52715ba2 Bring back original implementation - tests gave some light to the problem 4607e7b8 lets see, try different template 4a453425 add some more debug stuff - will remove them once i figure it out 5ee2e03d Try to use the binaries for the creation, see what changes 95a22ef7 another re-arrangement 82855f91 Attempt to install a different version of libraries, as per another issue fbda0d3a Attempt to debug weird container failure d828de8c debugging 07bad55d reduce the noise, when not possible to create/modify the required directories dont do anything 6a2bd46b rename bba22329 netdata/health: shellchecking - SC2236 103a7df9 Adjust naming ea70d482 Run on Xenial c2b3ad77 You should sudo create the folder a2f00c26 Do not stop the build if one of the RPM fails, we want all of them to attempt to build c367ab6c Cleanup, bring also RPM template in and lets see what happens ba7b441b more nits -- add the right replacements 2dfbb3f5 move templated parts outside 3211bc11 another approach, missed something 2a47dd17 A more educated attempt based on other resources 670c35b7 first time in anchors, most likely is wrong but got to see what travis will do 6c00628e Binary release flows and other side fixes 1d89e519 Just in case,