Add initial tooling for generating integrations.js file. (#15406)

* Fix link tags in deploy.

* Add initial tooling for generating integrations.js file.

* Skip integrations directory for eslint.

* Add README to explain how to generate integrations.js locally.

* Fix ID/name for top-level categories.

* Deduplicate categories entries.

* Properly render related resources information.

* Warn on and skip bad references for related resources.

* Add CI workflow to rebuild integrations as-needed.

* Add integrations.js to build artifacts.

* Fix actionlint complaints.

* Assorted template fixes.

* Add script to check collector metadata.

* Add default categories for collectors when they have no categories.

* Fix template formatting issues.

* Link related resources properly.

* Skip more sections in rendered output if they are not present in source data.

* Temporarily skip config syntax section.

It needs further work and is not critical at the moment.

* Fix metrics table rendering.

* Hide most overview content if method_description is empty.

* Fix metrics table rendering (again).

* Add detailed description to setup options section.

* Fix detailed description handling for config options.

* Fix config example folding logic.

* Fix multi-instance selection.

* Properly fix multi-instance selection.

* Add titles for labels and metrics charts.

* Include monitored instance name in integration ID.

This is required to disambiguate some ‘virtual’ integrations.

* Indicate if there are no alerts defined for an integration.

* Fix multi-instance in template.

* Improve warning handling in script and fix category handling.

* Hide debug messages by default.

* Fix invalid category name in cgroups plugin.

* Completely fix invalid categories in cgroups plugin.

* Warn about and ignore duplicate integration ids.

* Flag integration type in integrations list.

* Add configuration syntax samples.

* Fix issues in gen_integrations.py

* Validate categories.yaml on load.

* Add support for handling deployment information.

* Fix bugs in gen_integrations.py

* Add code to handle exporters.

* Add link to integrations pointing to their source files.

* Fix table justification.

* Add notification handling to script.

Also tidy up a few other things.

* Fix numerous bugs in gen_integrations.py

* remove trailing space from deploy.yaml command

* make availability one column

* Switch back to multiple columns for availability.

And also switch form +/- to a dot for positive and empty cell for
negative.

* Render setup description.

* Fix platform info rendering in deploy integrations.

* Fix sourcing of cloud-notifications metadata.

* Fix rendering of empty metrics.

* Fix alerts template.

* Add per-instance templating for templated keys.

* Fix go plugin links.

* Fix overview template.

* Fix handling of exporters.

* Fix loading of cloud notification integrations.

* Always show full collector overview.

* Add static troubleshooting content when appropriate.

* Assorted deploy integration updates.

* Add initial copy of integrations.js.

---------

Co-authored-by: Fotis Voutsas <fotis@netdata.cloud>

1 files changed, 26 insertions, 0 deletions

diff --git a/integrations/README.md b/integrations/README.md
new file mode 100644
index 0000000000..377c1a3061
--- /dev/null
+++ b/integrations/README.md
@@ -0,0 +1,26 @@
+To generate a copy of `integrations.js` locally, you will need:
+
+- Python 3.6 or newer (only tested on Python 3.10 currently, should work
+  on any version of Python newer than 3.6).
+- The following third-party Python modules:
+    - `jsonschema`
+    - `referencing`
+    - `jinja2`
+    - `ruamel.yaml`
+- A local checkout of https://github.com/netdata/netdata
+- A local checkout of https://github.com/netdata/go.d.plugin. The script
+  expects this to be checked out in a directory called `go.d.plugin`
+  in the root directory of the agent repo, though a symlink with that
+  name pointing at the actual location of the repo will work as well.
+
+The first two parts can be easily covered in a Linux environment, such
+as a VM or Docker container:
+
+- On Debian or Ubuntu: `apt-get install python3-jsonschema python3-referencing python3-jinja2 python3-ruamel.yaml`
+- On Alpine: `apk add py3-jsonschema py3-referencing py3-jinja2 py3-ruamel.yaml`
+- On Fedora or RHEL (EPEL is required on RHEL systems): `dnf install python3-jsonschema python3-referencing python3-jinja2 python3-ruamel-yaml`
+
+Once the environment is set up, simply run
+`integrations/gen_integrations.py` from the agent repo. Note that the
+script must be run _from this specific location_, as it uses it’s own
+path to figure out where all the files it needs are.