netdata/packaging: Netdata binary packages generation - spec file refinement, support for nightlies (RPM) (#6369)

Detailed list of changes:

* netdata/packaging: First round of netdata.spec refactoring - package dependencies

- Remove freeipmi/nfacct plugin flags. We auto-detect all plugins by decision
- Start refactor of package dependencies
- Add missing dependencies, with respect to distro peculiarities
- Adjust existing dependencies, so that distro-specific package names is applied

* netdata/packaging: add packaging trigger mechanism on binary-packages-specfile-cleanup to help testing

* netdata/packaging: WIP - Allow building nightly versions for RPM (Still investigating the right version scheme)

* netdata/packaging: Attempt to fix syntax errors - wow travis, mercy

* netdata/packaging: tiny nits and fixes

1) there was a syntax error in version definition
2) bring default deploy repo name at the top, then overwrite when latest is published
3) Leave a comment

* netdata/packaging: Move these core package requirements to the right section

* netdata/packaging: libuv requires >= 1, missed to add that on devel libs

* netdata/packaging: some comments, also when you put logrotate in our conf directory, link it to actual log rotate

* netdata/packaging: Adjust configure options on spec file (align with netdata-installer.sh)

* netdata/packaging: In order to properly support packaging both latest and stable releases, you need to star that extraction part, folder name contains the version with the hash that we cant tell

* netdata/packaging: until we fix versioning, bring the original unprocessed spec file and pass the version we like in

* netdata/packaging: fix version string in the spec file

* netdata/packaging: Replace source path, we need to work with whatever we downloaded (to be refined)

* netdata/packaging: [ci skip] you mixed the paths, reverse them

* netdata/packaging: [ci skip] attempt to fix source path when needed

* netdata/packaging: fixes and nits

1) generalize the spec file refinement method
2) Do both replacement that way
3) Adjust file names and paths

* netdata/packaging: Replace Source0 explicitly and also make sure you remove extra new lines that may confuse the replacement later

* netdata/packaging: [ci skip] move env outside, add some more log

* netdata/packaging: [ci skip] adjust logic of replacing tag lines in spec file

* netdata/packaging: [ci skip] Alas, fetch the archive, rename the structure to the new version, repack and proceed

* netdata/packaging: [ci skip] Extract to specific path

* netdata/packaging: [ci skip] replace problematic tar call with the respective pythonized one

* netdata/packaging: [ci skip] Add new friendly octal format for chmod

* netdata/packaging: [ci skip] more checks

* netdata/packaging: [ci skip] dont chmod, chown the whole folder after extraction

* netdata/packaging: nit

* netdata/packaging: [ci skip] attempt to fix permissions differently

* netdata/packaging: [ci skip] Another hack, attempt to fix permissions on the host directly

* netdata/packaging: [ci skip] fine, do the move also on the host, pointless to run all through the container

* netdata/packaging: [ci skip] Do the move through the container. Also, move a more specific folder

* netdata/packaging: [ci skip] move as w00t

* netdata/packaging: [ci skip] fix

* netdata/packaging: sync permissions handling, round two

* netdata/packaging: scratch those hacks, we should do latest differently

* netdata/packaging: [ci skip] Set up rpm version string depending on whether we are on latest or not

* netdata/packaging: [ci skip] Branch the code here, latest build up should be on the commit rather the tar files

* netdata/packaging: Attempt to handle latest version build up

* netdata/packaging: [ci skip] Dont continue processing if the file was not generated. Also do the tagging before autoreconf

* netdata/packaging: [ci skip] fixes and nits

* netdata/packaging: [ci skip] When doing annotated tags, you need to provide a message otherwise an editor pops up and will block the build

* netdata/packaging: on the latest code case go all the way host level operations

* netdata/packaging: [ci skip] Do not modify version, no longer need it because we properly generated the package with the right version

* netdata/packaging: [ci skip] fix sensors line, you missed the Recommends keyword

* netdata/packaging: Some code style (remove tabification from some tags) and also remove Recommends, we start with required packages for build and runtime and then we document further needs

* netdata/packaging: [ci skip] Make sure you get epel in there, its needed

* netdata/packaging: [ci skip] Fix bogus date

* netdata/packaging: [ci skip] add these packages to include functionality

* netdata/packaging: [ci skip] more fine grained dependency handling

* netdata/packaging: [ci skip] Dependency hell - do not depend on libnetacctfilter if on el/7

* netdata/packaging: [ci skip] Be serious, that is not what you do when building the package silly

* netdata/packaging: [ci skip] define directory attributes too

* netdata/packaging: [ci skip] mystery unfold - comment out as we need to figure out how to handle missing directories that our installer creates if necessary

* netdata/packaging: [ci skip] Make sure you properly set deployment repo based on commit message

* netdata/packaging: [ci skip] Install these packages for specific versions, remove optimisations for now

* netdata/packaging: [ci skip] Make package yanking flexible, so that we can manage both netdata and netdata-edge cleanup properly

* netdata/packaging: [ci skip] Actually, set user at travis layer and let the script consume it only. We need to be sure the same user is used on yanking and the deployment process

* netdata/packaging: [ci skip] wow pkatsoul

* netdata/packaging: Dont forget to install some of the enabled plugins

* netdata/packaging: nfacct.plugin not really working on rhel, disable temporarily

* netdata/packaging: include installed directories to package

* netdata/packaging: [ci skip] nits

* netdata/packaging: fixes

* netdata/packaging: [ci skip] dont add ghost files

* netdata/packaging: [ci skip] Dont do that, make install already places the files in the right location. Also, seems that libexec is not really getting overriden there

* netdata/packaging: [ci skip] Extreme - override _libexecdir macro

* Revert "netdata/packaging: [ci skip] Dont do that, make install already places the files in the right location. Also, seems that libexec is not really getting overriden there"

This reverts commit 320102c96284e4622266f6f089d6ce87319976f3.
That wasnt really the problem here

* netdata/packaging: [ci skip] Include contents of dir here

* netdata/packaging: fixes

* netdata/packaging: [ci skip] Fixes and additions

- Tighter permissions for some files
- Introduce go.d plugin install (likely broken like this, dont recall if bash style code is fine here)
- Raise the path overrides to the spec level, otherwise we build with the overrides and package with the defaults

* netdata/packaging: [ci skip] Missed an invalid variable. These files are under our current path, no need for complexity

* netdata/packaging: [ci skip] bring sha method onboard

* netdata/packaging: [ci skip] Wrong path provided, should be under build root

* netdata/packaging: [ci skip] Add registry directory

* netdata/packaging: [ci skip] add missing config directory definitions

* netdata/packaging: [ci skip] Fix missing dirs

* netdata/packaging:[ci fix] hos

* netdata/packaging: [ci skip] Fixes and additions
- Change versioning to v1.15.0.114 from v1.15.0114
- Add CUPS, Xenstat, json-c

* netdata/packaging: [ci skip] fix package dependencies on the builder setup, otherwise spec will fail

* netdata/packaging: [ci skip] limit xenstat to suse and fedora for now

* netdata/packaging: [ci skip] separate fedora and suse cases, unfortunately suse has different package names....

* netdata/packaging: [ci skip] yajl is not the same package everywhere, of course

* netdata/packagin: [ci skip] skip xenstat for now

* netdata/packaging: [ci skip] remove the vvvvvvvvv

* netdata/packaging: [ci skip] one more fix attempt

* netdata/packaging: [ci skip] Spec file extraction is different when
building from stable. We *must* fetch it from the tarball rather than
the local one

* netdata/packaging: [ci skip] spec_file defined on all cases, latest or stable. Also move latest spec file copy after make dist operation, that is where is suppose to be

* netdata/packaging: [ci skip] Tag source with the local version, as we build upon the very latest

* netdata/packaging: oopsie -- remove json-c from that place, it was forgotten

* netdata/packaging: fix packaging, even though devel is consistent between fedora and suse, the binary one is not .... (also fix default perms to be root:netdata rather root:root)

* netdata/packaging: Suse love - again, libuuid-devel same, but libuuid is named libuuid1 on suse......

* netdata/packaging: [ci skip] libcap same....

* netdata/packaging: [ci skip] use the noreplace directive for the configuration files

* netdata/packaging: [ci skip] Bring back master as the triggering repository

* netdata/packaging: [ci skip] attempt to enable/disable service where necessary. Default preset action requires further setup, so we override that

* Revert "netdata/packaging: [ci skip] Bring back master as the triggering repository"

This reverts commit 9b25a622dc5f12b1539f6863bab52d7bb225fd8d.

** Reverting as we need to test further the service enable action **

* netdata/packaging: [ci skip] revert this change - not the right way, we have empty files

* netdata/packaging: [ci skip] Add daemon-reload option, to make sure systemd learns about the newly installed service

* netdata/packaging: [ci skip] Attempt to silence codacy by factoring out the few small methods for common usage, then kill directories to reduce complexity adn thats it

* netdata/packaging: [ci skip] Just making sure codacy reprocess everything

* netdata/packaging: [ci skip] missed one adjustment

* netdata/packaging: python style - print shouldnt have space before open parenthesis. Also, add restart on all cases on the spec file

* netdata/packaging: [ci skip] fix syntax

* netdata/packaging: [ci skip] syntax fixes 2

* netdata/packaging: [ci skip] another miss

* netdata/packaging: [ci skip] You need to explicitly trigger a restart, apparently

* netdata/packaging: [ci skip] override packaging user definition temporarily, to get things going on testing

13 files changed, 661 insertions, 332 deletions

diff --git a/.travis.yml b/.travis.yml
index ccd2f78d6d..3d3d6ffcc4 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -23,6 +23,13 @@ install:
 - source tests/installer/slack.sh
 - export NOTIF_CHANNEL="automation-beta"
 - if [ "${TRAVIS_REPO_SLUG}" = "netdata/netdata" ]; then export NOTIF_CHANNEL="automation"; fi;
+- export BUILD_VERSION="$(cat packaging/version | cut -d'-' -f1)"
+- if [[ "${TRAVIS_COMMIT_MESSAGE}" = *"[Build latest]"* ]]; then export BUILD_VERSION="$(cat packaging/version | cut -d'-' -f1,2 | sed -e 's/-/./g').latest"; fi;
+- export DEPLOY_REPO="netdata" # Default production packaging repository
+- if [[ "${TRAVIS_COMMIT_MESSAGE}" = *"[Build latest]"* ]]; then export DEPLOY_REPO="netdata-edge"; fi;
+#- export PACKAGING_USER="$(echo ${TRAVIS_REPO_SLUG} | cut -d'/' -f1)"
+#Override to help push packages towards netdata/netdata-edge
+- export PACKAGING_USER="netdata"
 
 
 
@@ -80,23 +87,23 @@ stages:
   # Build RPM packages under special conditions
   # Enterprise linux (Covers CentOS, Redhat, Amazon linux)
 - name: "Package Enterprise Linux 7"
-  if: type != cron AND branch = master AND commit_message =~ /(\[Package arm64 RPM Enterprise Linux\]|\[Package arm64 RPM\]|\[Package i386 RPM Enterprise Linux\]|\[Package i386 RPM\]|\[Package AMD64 RPM Enterprise Linux\]|\[Package AMD64 RPM\])/
+  if: type != cron AND branch = binary-packages-specfile-cleanup AND commit_message =~ /(\[Package arm64 RPM Enterprise Linux\]|\[Package arm64 RPM\]|\[Package i386 RPM Enterprise Linux\]|\[Package i386 RPM\]|\[Package AMD64 RPM Enterprise Linux\]|\[Package AMD64 RPM\])/
 - name: "Package Enterprise linux 6"
-  if: type != cron AND branch = master AND commit_message =~ /(\[Package i386 RPM Enterprise Linux\]|\[Package i386 RPM\]|\[Package AMD64 RPM Enterprise Linux\]|\[Package AMD64 RPM\])/
+  if: type != cron AND branch = binary-packages-specfile-cleanup AND commit_message =~ /(\[Package i386 RPM Enterprise Linux\]|\[Package i386 RPM\]|\[Package AMD64 RPM Enterprise Linux\]|\[Package AMD64 RPM\])/
 
   # Fedora
 - name: "Package Fedora 30"
-  if: type != cron AND branch = master AND commit_message =~ /(\[Package arm64 RPM Fedora\]|\[Package arm64 RPM\]|\[Package AMD64 RPM Fedora\]|\[Package AMD64 RPM\])/
+  if: type != cron AND branch = binary-packages-specfile-cleanup AND commit_message =~ /(\[Package arm64 RPM Fedora\]|\[Package arm64 RPM\]|\[Package AMD64 RPM Fedora\]|\[Package AMD64 RPM\])/
 - name: "Package Fedora 29"
-  if: type != cron AND branch = master AND commit_message =~ /(\[Package arm64 RPM Fedora\]|\[Package arm64 RPM\]|\[Package AMD64 RPM Fedora\]|\[Package AMD64 RPM\])/
+  if: type != cron AND branch = binary-packages-specfile-cleanup AND commit_message =~ /(\[Package arm64 RPM Fedora\]|\[Package arm64 RPM\]|\[Package AMD64 RPM Fedora\]|\[Package AMD64 RPM\])/
 - name: "Package Fedora 28"
-  if: type != cron AND branch = master AND commit_message =~ /(\[Package arm64 RPM Fedora\]|\[Package arm64 RPM\]|\[Package AMD64 RPM Fedora\]|\[Package AMD64 RPM\])/
+  if: type != cron AND branch = binary-packages-specfile-cleanup AND commit_message =~ /(\[Package arm64 RPM Fedora\]|\[Package arm64 RPM\]|\[Package AMD64 RPM Fedora\]|\[Package AMD64 RPM\])/
 
   # OpenSuSE
 - name: "Package OpenSuSE 15.1"
-  if: type != cron AND branch = master AND commit_message =~ /(\[Package arm64 RPM openSuSE\]|\[Package arm64 RPM\]|\[Package AMD64 RPM openSuSE\]|\[Package AMD64 RPM\])/
+  if: type != cron AND branch = binary-packages-specfile-cleanup AND commit_message =~ /(\[Package arm64 RPM openSuSE\]|\[Package arm64 RPM\]|\[Package AMD64 RPM openSuSE\]|\[Package AMD64 RPM\])/
 - name: "Package OpenSuSE 15.0"
-  if: type != cron AND branch = master AND commit_message =~ /(\[Package arm64 RPM openSuSE\]|\[Package arm64 RPM\]|\[Package AMD64 RPM openSuSE\]|\[Package AMD64 RPM\])/
+  if: type != cron AND branch = binary-packages-specfile-cleanup AND commit_message =~ /(\[Package arm64 RPM openSuSE\]|\[Package arm64 RPM\]|\[Package AMD64 RPM openSuSE\]|\[Package AMD64 RPM\])/
 
 
 
@@ -109,20 +116,20 @@ stages:
     before_script:
     - post_message "TRAVIS_MESSAGE" "Starting package preparation and publishing for ${BUILD_STRING}.${BUILD_ARCH}" "${NOTIF_CHANNEL}"
     - export PACKAGES_DIRECTORY="$(mktemp -d -t netdata-packaging-contents-dir-XXXXXX)" && echo "Created packaging directory ${PACKAGES_DIRECTORY}"
-    - export BUILD_VERSION=$(cat packaging/version | cut -d'-' -f1) # We always build on the latest release
     script:
     - echo "Creating LXC environment for the build" && sudo -E .travis/package_management/create_lxc_for_build.sh
     - echo "Building package in container" && sudo -E .travis/package_management/build_package_in_container.sh
     - sudo chmod -R 755 "/var/lib/lxc"
     - echo "Preparing RPM packaging contents for upload" && sudo -E .travis/package_management/prepare_packages.sh
     after_failure: post_message "TRAVIS_MESSAGE" "Failed to build RPM for ${BUILD_STRING}.${BUILD_ARCH}"
+
     before_deploy:
     - .travis/package_management/yank_stale_rpm.sh "${PACKAGES_DIRECTORY}" "${BUILD_STRING}" || echo "No stale RPM found"
     deploy:
       # Beta packages deployment
       - provider: packagecloud
-        repository: "netdata"
-        username: "$(echo ${TRAVIS_REPO_SLUG} | cut -d'/' -f1)"
+        repository: "${DEPLOY_REPO}"
+        username: "${PACKAGING_USER}"
         token: "${PKG_CLOUD_TOKEN}"
         dist: "${BUILD_STRING}"
         local_dir: "${PACKAGES_DIRECTORY}"
@@ -130,7 +137,7 @@ stages:
         on:
           # Only deploy on ${USER}/netdata, master branch, when packages directory is created
           repo: ${TRAVIS_REPO_SLUG}
-          branch: master
+          branch: binary-packages-specfile-cleanup
           condition: -d "${PACKAGES_DIRECTORY}"
       # Production release packages deployment
       - provider: packagecloud
@@ -143,14 +150,14 @@ stages:
         on:
           # Only deploy on ${USER}/netdata, master branch, when packages directory is created
           repo: "${TRAVIS_REPO_SLUG}"
-          branch: master
+          branch: binary-packages-specfile-cleanup
           condition: -d "${PACKAGES_DIRECTORY}"
     after_deploy:
       - if [ -n "${BUILDER_NAME}" ]; then rm -rf /home/${BUILDER_NAME}/* && echo "Cleared /home/${BUILDER_NAME} directory" || echo "Failed to clean /home/${BUILDER_NAME} directory"; fi;
       - if [ -d "${PACKAGES_DIRECTORY}" ]; then rm -rf "${PACKAGES_DIRECTORY}"; fi;
 
 
-
+  # TODO: This section is stale, will be aligned with the RPM implementation when we get to DEB packaging
 - stage: &_DEB_TEMPLATE
     name: "Build & Publish DEB package"
     before_install:
@@ -159,7 +166,6 @@ stages:
     before_script:
     - post_message "TRAVIS_MESSAGE" "Starting package preparation and publishing for ${BUILD_STRING}.${BUILD_ARCH}" "${NOTIF_CHANNEL}"
     - export PACKAGES_DIRECTORY="$(mktemp -d -t netdata-packaging-contents-dir-XXXXXX)" && echo "Created packaging directory ${PACKAGES_DIRECTORY}"
-    - export BUILD_VERSION=$(cat packaging/version | cut -d'-' -f1) # We always build on the latest release
     script:
     - echo "Creating LXC environment for the build" && sudo -E .travis/package_management/create_lxc_for_build.sh
     - echo "Building package in container" && sudo -E .travis/package_management/build_package_in_container.sh
@@ -171,8 +177,8 @@ stages:
     deploy:
       # Beta packages deployment
       - provider: packagecloud
-        repository: "netdata"
-        username: "$(echo ${TRAVIS_REPO_SLUG} | cut -d'/' -f1)"
+        repository: "${DEPLOY_REPO}"
+        username: "${PACKAGING_USER}"
         token: "${PKG_CLOUD_TOKEN}"
         dist: "${BUILD_STRING}"
         local_dir: "${PACKAGES_DIRECTORY}"
diff --git a/.travis/package_management/build_package_in_container.sh b/.travis/package_management/build_package_in_container.sh
index 2719e7b6b3..95a68e7a83 100755
--- a/.travis/package_management/build_package_in_container.sh
+++ b/.travis/package_management/build_package_in_container.sh
@@ -54,24 +54,24 @@ case "${BUILD_ARCH}" in
 	echo "Building for amd64.."
 	export CONTAINER_NAME="${BUILDER_NAME}-${BUILD_DISTRO}${BUILD_RELEASE}-amd64"
 	export LXC_CONTAINER_ROOT="/var/lib/lxc/${CONTAINER_NAME}/rootfs"
-	.travis/package_management/"${PACKAGE_TYPE}"/trigger_lxc_build.py "${CONTAINER_NAME}"
+	.travis/package_management/trigger_${PACKAGE_TYPE}_lxc_build.py "${CONTAINER_NAME}"
 
 	echo "Building for arm64.."
 	export CONTAINER_NAME="${BUILDER_NAME}-${BUILD_DISTRO}${BUILD_RELEASE}-arm64"
 	export LXC_CONTAINER_ROOT="/var/lib/lxc/${CONTAINER_NAME}/rootfs"
-	.travis/package_management/"${PACKAGE_TYPE}"/trigger_lxc_build.py "${CONTAINER_NAME}"
+	.travis/package_management/trigger_${PACKAGE_TYPE}_lxc_build.py "${CONTAINER_NAME}"
 
 	echo "Building for i386.."
 	export CONTAINER_NAME="${BUILDER_NAME}-${BUILD_DISTRO}${BUILD_RELEASE}-i386"
 	export LXC_CONTAINER_ROOT="/var/lib/lxc/${CONTAINER_NAME}/rootfs"
-	.travis/package_management/"${PACKAGE_TYPE}"/trigger_lxc_build.py "${CONTAINER_NAME}"
+	.travis/package_management/trigger_${PACKAGE_TYPE}_lxc_build.py "${CONTAINER_NAME}"
 
 	;;
 "amd64"|"arm64"|"i386")
 	echo "Building for ${BUILD_ARCH}.."
 	export CONTAINER_NAME="${BUILDER_NAME}-${BUILD_DISTRO}${BUILD_RELEASE}-${BUILD_ARCH}"
 	export LXC_CONTAINER_ROOT="/var/lib/lxc/${CONTAINER_NAME}/rootfs"
-	.travis/package_management/"${PACKAGE_TYPE}"/trigger_lxc_build.py "${CONTAINER_NAME}"
+	.travis/package_management/trigger_${PACKAGE_TYPE}_lxc_build.py "${CONTAINER_NAME}"
 	;;
 *)
 	echo "Unknown build architecture '${BUILD_ARCH}', nothing to do for build"
diff --git a/.travis/package_management/common.py b/.travis/package_management/common.py
new file mode 100755
index 0000000000..6cf59293d9
--- /dev/null
+++ b/.travis/package_management/common.py
@@ -0,0 +1,46 @@
+#
+#
+# Python library with commonly used functions within the package management scope
+#
+# Author   : Pavlos Emm. Katsoulakis <paul@netdata.cloud>
+
+import lxc
+import subprocess
+
+def replace_tag(tag_name, spec, new_tag_content):
+    print("Fixing tag %s in %s" % (tag_name, spec))
+
+    ifp = open(spec, "r")
+    config = ifp.readlines()
+    ifp.close()
+
+    source_line = -1
+    for line in config:
+        if str(line).count(tag_name + ":") > 0:
+            source_line = config.index(line)
+            print("Found line: %s in item %d" % (line, source_line))
+            break
+
+    if source_line >= 0:
+        print("Replacing line %s with %s in spec file" %(config[source_line], new_tag_content))
+        config[source_line] = "%s: %s\n" % (tag_name, new_tag_content)
+        config_str = ''.join(config)
+        ofp = open(spec, 'w')
+        ofp.write(config_str)
+        ofp.close()
+
+def run_command(container, command):
+    print("Running command: %s" % command)
+    command_result = container.attach_wait(lxc.attach_run_command, command)
+
+    if command_result != 0:
+        raise Exception("Command failed with exit code %d" % command_result)
+
+def run_command_in_host(cmd):
+    print("Issue command in host: %s" % str(cmd))
+
+    proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    o, e = proc.communicate()
+    print('Output: ' + o.decode('ascii'))
+    print('Error: '  + e.decode('ascii'))
+    print('code: ' + str(proc.returncode))
diff --git a/.travis/package_management/configure_deb_lxc_environment.py b/.travis/package_management/configure_deb_lxc_environment.py
new file mode 100755
index 0000000000..58999ad38b
--- /dev/null
+++ b/.travis/package_management/configure_deb_lxc_environment.py
@@ -0,0 +1,65 @@
+#!/usr/bin/env python3
+#
+# Prepare the build environment within the container
+# The script attaches to the running container and does the following:
+# 1) Create the container
+# 2) Start the container up
+# 3) Create the builder user
+# 4) Prepare the environment for DEB build
+#
+# Copyright: SPDX-License-Identifier: GPL-3.0-or-later
+#
+# Author   : Pavlos Emm. Katsoulakis <paul@netdata.cloud>
+
+import common
+import os
+import sys
+import lxc
+
+if len(sys.argv) != 2:
+    print('You need to provide a container name to get things started')
+    sys.exit(1)
+container_name=sys.argv[1]
+
+# Setup the container object
+print("Defining container %s" % container_name)
+container = lxc.Container(container_name)
+if not container.defined:
+    raise Exception("Container %s not defined!" % container_name)
+
+# Start the container
+if not container.start():
+    raise Exception("Failed to start the container")
+
+if not container.running or not container.state == "RUNNING":
+    raise Exception('Container %s is not running, configuration process aborted ' % container_name)
+
+# Wait for connectivity
+print("Waiting for container connectivity to start configuration sequence")
+if not container.get_ips(timeout=30):
+    raise Exception("Timeout while waiting for container")
+
+# Run the required activities now
+# 1. Create the builder user
+print("1. Adding user %s" % os.environ['BUILDER_NAME'])
+common.run_command(container, ["useradd", "-m", os.environ['BUILDER_NAME']])
+
+# Fetch package dependencies for the build
+print("2. Installing package dependencies within LXC container")
+common.run_command(container, ["apt-get", "update", "-y"])
+common.run_command(container, ["apt-get", "install", "-y", "sudo"])
+common.run_command(container, ["apt-get", "install", "-y", "wget"])
+common.run_command(container, ["apt-get", "install", "-y", "bash"])
+common.run_command(container, ["wget", "-T", "15", "-O", "~/.install-required-packages.sh", "https://raw.githubusercontent.com/netdata/netdata-demo-site/master/install-required-packages.sh"])
+common.run_command(container, ["bash", "~/.install-required-packages.sh", "netdata", "--dont-wait", "--non-interactive"])
+
+# Download the source
+dest_archive="/home/%s/netdata-%s.tar.gz" % (os.environ['BUILDER_NAME'],os.environ['BUILD_VERSION'])
+release_url="https://github.com/netdata/netdata/releases/download/%s/netdata-%s.tar.gz" % (os.environ['BUILD_VERSION'], os.environ['BUILD_VERSION'])
+print("3. Fetch netdata source (%s -> %s)" % (release_url, dest_archive))
+common.run_command(container, ["sudo", "-u", os.environ['BUILDER_NAME'], "wget", "-T", "15", "--output-document=" + dest_archive, release_url])
+
+print("4. Extracting directory contents to /home " + os.environ['BUILDER_NAME'])
+common.run_command(container, ["sudo", "-u", os.environ['BUILDER_NAME'], "tar", "xf", dest_archive, "-C", "/home/" + os.environ['BUILDER_NAME']])
+
+print("Done!")
diff --git a/.travis/package_management/configure_rpm_lxc_environment.py b/.travis/package_management/configure_rpm_lxc_environment.py
new file mode 100755
index 0000000000..644e027b7b
--- /dev/null
+++ b/.travis/package_management/configure_rpm_lxc_environment.py
@@ -0,0 +1,159 @@
+#!/usr/bin/env python3
+#
+#
+# Prepare the build environment within the container
+# The script attaches to the running container and does the following:
+# 1) Create the container
+# 2) Start the container up
+# 3) Create the builder user
+# 4) Prepare the environment for RPM build
+#
+# Copyright: SPDX-License-Identifier: GPL-3.0-or-later
+#
+# Author   : Pavlos Emm. Katsoulakis <paul@netdata.cloud>
+
+import common
+import os
+import sys
+import lxc
+
+if len(sys.argv) != 2:
+    print('You need to provide a container name to get things started')
+    sys.exit(1)
+container_name=sys.argv[1]
+
+# Setup the container object
+print("Defining container %s" % container_name)
+container = lxc.Container(container_name)
+if not container.defined:
+    raise Exception("Container %s not defined!" % container_name)
+
+# Start the container
+if not container.start():
+    raise Exception("Failed to start the container")
+
+if not container.running or not container.state == "RUNNING":
+    raise Exception('Container %s is not running, configuration process aborted ' % container_name)
+
+# Wait for connectivity
+print("Waiting for container connectivity to start configuration sequence")
+if not container.get_ips(timeout=30):
+    raise Exception("Timeout while waiting for container")
+
+# Run the required activities now
+# Create the builder user
+print("1. Adding user %s" % os.environ['BUILDER_NAME'])
+common.run_command(container, ["useradd", "-m", os.environ['BUILDER_NAME']])
+
+# Fetch package dependencies for the build
+print("2. Installing package dependencies within LXC container")
+if str(os.environ["REPO_TOOL"]).count("zypper") == 1:
+    common.run_command(container, [os.environ["REPO_TOOL"], "clean", "-a"])
+    common.run_command(container, [os.environ["REPO_TOOL"], "--no-gpg-checks", "update", "-y"])
+    common.run_command(container, [os.environ["REPO_TOOL"], "install", "-y", "json-glib-devel"])
+
+elif str(os.environ["REPO_TOOL"]).count("yum") == 1:
+    common.run_command(container, [os.environ["REPO_TOOL"], "install", "-y", "json-c-devel"])
+    common.run_command(container, [os.environ["REPO_TOOL"], "clean", "all"])
+    common.run_command(container, [os.environ["REPO_TOOL"], "update", "-y"])
+    common.run_command(container, [os.environ["REPO_TOOL"], "install", "-y", "epel-release"])
+else:
+    common.run_command(container, [os.environ["REPO_TOOL"], "install", "-y", "json-c-devel"])
+    common.run_command(container, [os.environ["REPO_TOOL"], "update", "-y"])
+
+common.run_command(container, [os.environ["REPO_TOOL"], "install", "-y", "sudo"])
+common.run_command(container, [os.environ["REPO_TOOL"], "install", "-y", "wget"])
+common.run_command(container, [os.environ["REPO_TOOL"], "install", "-y", "bash"])
+common.run_command(container, [os.environ["REPO_TOOL"], "install", "-y", "freeipmi-devel"])
+common.run_command(container, [os.environ["REPO_TOOL"], "install", "-y", "cups-devel"])
+
+# Exceptional cases, not available everywhere
+#
+
+# Not on Centos-7
+if os.environ["BUILD_STRING"].count("el/7") <= 0:
+    common.run_command(container, [os.environ["REPO_TOOL"], "install", "-y", "libnetfilter_acct-devel"])
+
+# Not on Centos-6
+if os.environ["BUILD_STRING"].count("el/6") <= 0:
+    common.run_command(container, [os.environ["REPO_TOOL"], "install", "-y", "autoconf-archive"])
+
+common.run_command(container, ["wget", "-T", "15", "-O", "/home/%s/.install-required-packages.sh" % (os.environ['BUILDER_NAME']), "https://raw.githubusercontent.com/netdata/netdata-demo-site/master/install-required-packages.sh"])
+common.run_command(container, ["bash", "/home/%s/.install-required-packages.sh" % (os.environ['BUILDER_NAME']), "netdata", "--dont-wait", "--non-interactive"])
+
+print("3. Setting up macros")
+common.run_command(container, ["sudo", "-u", os.environ['BUILDER_NAME'], "/bin/echo", "'%_topdir %(echo /home/" + os.environ['BUILDER_NAME'] + ")/rpmbuild' > /home/" + os.environ['BUILDER_NAME'] + "/.rpmmacros"])
+
+print("4. Create rpmbuild directory")
+common.run_command(container, ["sudo", "-u", os.environ['BUILDER_NAME'], "mkdir", "-p", "/home/" + os.environ['BUILDER_NAME'] + "/rpmbuild/BUILD"])
+common.run_command(container, ["sudo", "-u", os.environ['BUILDER_NAME'], "mkdir", "-p", "/home/" + os.environ['BUILDER_NAME'] + "/rpmbuild/RPMS"])
+common.run_command(container, ["sudo", "-u", os.environ['BUILDER_NAME'], "mkdir", "-p", "/home/" + os.environ['BUILDER_NAME'] + "/rpmbuild/SOURCES"])
+common.run_command(container, ["sudo", "-u", os.environ['BUILDER_NAME'], "mkdir", "-p", "/home/" + os.environ['BUILDER_NAME'] + "/rpmbuild/SPECS"])
+common.run_command(container, ["sudo", "-u", os.environ['BUILDER_NAME'], "mkdir", "-p", "/home/" + os.environ['BUILDER_NAME'] + "/rpmbuild/SRPMS"])
+common.run_command(container, ["sudo", "-u", os.environ['BUILDER_NAME'], "ls", "-ltrR", "/home/" + os.environ['BUILDER_NAME'] + "/rpmbuild"])
+
+# Download the source
+rpm_friendly_version=""
+dest_archive=""
+download_url=""
+spec_file="/home/%s/rpmbuild/SPECS/netdata.spec" % os.environ['BUILDER_NAME']
+
+# TODO: Checksum validations
+if str(os.environ['BUILD_VERSION']).count(".latest") == 1:
+    version_list=str(os.environ['BUILD_VERSION']).replace('v', '').split('.')
+    rpm_friendly_version='.'.join(version_list[0:3]) + "." + version_list[3]
+
+    print("Building latest nightly version of netdata..(%s)" % os.environ['BUILD_VERSION'])
+    dest_archive="/home/%s/rpmbuild/SOURCES/netdata-%s.tar.gz" % (os.environ['BUILDER_NAME'], rpm_friendly_version)
+
+    print("5. Preparing local latest implementation tarball for version %s" % rpm_friendly_version)
+    tar_file = os.environ['LXC_CONTAINER_ROOT'] + dest_archive
+
+    print("5.1 Tagging the code with latest version: %s" % rpm_friendly_version)
+    common.run_command_in_host(['git', 'tag', '-a', rpm_friendly_version, '-m', 'Tagging while packaging on %s' % os.environ["CONTAINER_NAME"]])
+
+    print("5.2 Run autoreconf -ivf")
+    common.run_command_in_host(['autoreconf', '-ivf'])
+
+    print("5.3 Run configure")
+    common.run_command_in_host(['./configure', '--with-math', '--with-zlib', '--with-user=netdata'])
+
+    print("5.4 Run make dist")
+    common.run_command_in_host(['make', 'dist'])
+
+    print("5.5 Copy generated tarbal to desired path")
+    if os.path.exists('netdata-%s.tar.gz' % rpm_friendly_version):
+        common.run_command_in_host(['sudo', 'cp', 'netdata-%s.tar.gz' % rpm_friendly_version, tar_file])
+
+        print("5.6 Fixing permissions on tarball")
+        common.run_command_in_host(['sudo', 'chmod', '777', tar_file])
+    else:
+        print("I could not find (%s) on the disk, stopping the build. Kindly check the logs and try again" % 'netdata-%s.tar.gz' % rpm_friendly_version)
+        sys.exit(1)
+
+    # Extract the spec file in place
+    print("6. Extract spec file from the source")
+    common.run_command_in_host(['sudo', 'cp', 'netdata.spec', os.environ['LXC_CONTAINER_ROOT'] + spec_file])
+    common.run_command_in_host(['sudo', 'chmod', '777', os.environ['LXC_CONTAINER_ROOT'] + spec_file])
+
+    print("7. Temporary hack: Change Source0 to %s on spec file %s" % (dest_archive, spec_file))
+    common.replace_tag("Source0", os.environ['LXC_CONTAINER_ROOT'] + spec_file, tar_file)
+else:
+    rpm_friendly_version = os.environ['BUILD_VERSION']
+
+    print("Building latest stable version of netdata.. (%s)" % os.environ['BUILD_VERSION'])
+    dest_archive="/home/%s/rpmbuild/SOURCES/netdata-%s.tar.gz" % (os.environ['BUILDER_NAME'],os.environ['BUILD_VERSION'])
+    download_url="https://github.com/netdata/netdata/releases/download/%s/netdata-%s.tar.gz" % (os.environ['BUILD_VERSION'], os.environ['BUILD_VERSION'])
+
+    print("5. Fetch netdata source into the repo structure(%s -> %s)" % (download_url, dest_archive))
+    tar_file="%s/netdata-%s.tar.gz" % (os.path.dirname(dest_archive), rpm_friendly_version)
+    common.run_command(container, ["sudo", "-u", os.environ['BUILDER_NAME'], "wget", "-T", "15", "--output-document=" + dest_archive, download_url])
+
+    print("6.Extract spec file from the source")
+    common.run_command(container, ["sudo", "-u", os.environ['BUILDER_NAME'], "tar", "--to-command=cat > %s" % spec_file, "-xvf", dest_archive, "netdata-%s/netdata.spec" % os.environ['BUILD_VERSION']])
+
+    print("7. Temporary hack: Adjust version string on the spec file (%s) to %s and Source0 to %s" % (os.environ['LXC_CONTAINER_ROOT'] + spec_file, rpm_friendly_version, download_url))
+    common.replace_tag("Version", os.environ['LXC_CONTAINER_ROOT'] + spec_file, rpm_friendly_version)
+    common.replace_tag("Source0", os.environ['LXC_CONTAINER_ROOT'] + spec_file, tar_file)
+
+print('Done!')
diff --git a/.travis/package_management/create_lxc_for_build.sh b/.travis/package_management/create_lxc_for_build.sh
index 83ef9d1fc1..ae855a7420 100755
--- a/.travis/package_management/create_lxc_for_build.sh
+++ b/.travis/package_management/create_lxc_for_build.sh
@@ -62,7 +62,7 @@ case "${BUILD_ARCH}" in
 	lxc-create -n "${CONTAINER_NAME}" -t "download" -- --dist "${BUILD_DISTRO}" --release "${BUILD_RELEASE}" --arch "i386" --no-validate
 
 	echo "Container(s) ready. Configuring container(s).."
-	.travis/package_management/"${PACKAGE_TYPE}"/configure_lxc_environment.py "${CONTAINER_NAME}"
+	.travis/package_management/configure_${PACKAGE_TYPE}_lxc_environment.py "${CONTAINER_NAME}"
 
 	# amd64
 	echo "Creating LXC Container for amd64.."
@@ -71,7 +71,7 @@ case "${BUILD_ARCH}" in
 	lxc-create -n "${CONTAINER_NAME}" -t "download" -- --dist "${BUILD_DISTRO}" --release "${BUILD_RELEASE}" --arch "amd64" --no-validate
 
 	echo "Container(s) ready. Configuring container(s).."
-	.travis/package_management/"${PACKAGE_TYPE}"/configure_lxc_environment.py "${CONTAINER_NAME}"
+	.travis/package_management/configure_${PACKAGE_TYPE}_lxc_environment.py "${CONTAINER_NAME}"
 
 	# arm64
 	echo "Creating LXC Container for arm64.."
@@ -80,7 +80,7 @@ case "${BUILD_ARCH}" in
 	lxc-create -n "${CONTAINER_NAME}" -t "download" -- --dist "${BUILD_DISTRO}" --release "${BUILD_RELEASE}" --arch "arm64" --no-validate
 
 	echo "Container(s) ready. Configuring container(s).."
-	.travis/package_management/"${PACKAGE_TYPE}"/configure_lxc_environment.py "${CONTAINER_NAME}"
+	.travis/package_management/configure_${PACKAGE_TYPE}_lxc_environment.py "${CONTAINER_NAME}"
 	;;
 "i386"|"amd64"|"arm64")
 	# AMD64 or i386
@@ -90,7 +90,7 @@ case "${BUILD_ARCH}" in
 	lxc-create -n "${CONTAINER_NAME}" -t "download" -- --dist "${BUILD_DISTRO}" --release "${BUILD_RELEASE}" --arch "${BUILD_ARCH}" --no-validate
 
 	echo "Container(s) ready. Configuring container(s).."
-	.travis/package_management/"${PACKAGE_TYPE}"/configure_lxc_environment.py "${CONTAINER_NAME}"
+	.travis/package_management/configure_${PACKAGE_TYPE}_lxc_environment.py "${CONTAINER_NAME}"
 	;;
 *)
 	echo "Unknown BUILD_ARCH value '${BUILD_ARCH}' given, process failed"
diff --git a/.travis/package_management/deb/configure_lxc_environment.py b/.travis/package_management/deb/configure_lxc_environment.py
deleted file mode 100755
index 496cdf5aa5..0000000000
--- a/.travis/package_management/deb/configure_lxc_environment.py
+++ /dev/null
@@ -1,71 +0,0 @@
-#!/usr/bin/env python3
-#
-# Prepare the build environment within the container
-# The script attaches to the running container and does the following:
-# 1) Create the container
-# 2) Start the container up
-# 3) Create the builder user
-# 4) Prepare the environment for DEB build
-#
-# Copyright: SPDX-License-Identifier: GPL-3.0-or-later
-#
-# Author   : Pavlos Emm. Katsoulakis <paul@netdata.cloud>
-
-import os
-import sys
-import lxc
-
-def run_command(command):
-    print ("Running command: %s" % command)
-    command_result = container.attach_wait(lxc.attach_run_command, command)
-
-    if command_result != 0:
-        raise Exception("Command failed with exit code %d" % command_result)
-
-if len(sys.argv) != 2:
-    print ('You need to provide a container name to get things started')
-    sys.exit(1)
-container_name=sys.argv[1]
-
-# Setup the container object
-print ("Defining container %s" % container_name)
-container = lxc.Container(container_name)
-if not container.defined:
-    raise Exception("Container %s not defined!" % container_name)
-
-# Start the container
-if not container.start():
-    raise Exception("Failed to start the container")
-
-if not container.running or not container.state == "RUNNING":
-    raise Exception('Container %s is not running, configuration process aborted ' % container_name)
-
-# Wait for connectivity
-print ("Waiting for container connectivity to start configuration sequence")
-if not container.get_ips(timeout=30):
-    raise Exception("Timeout while waiting for container")
-