Adding upstream version 1.39.0.upstream/1.39.0

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 92 insertions, 117 deletions

diff --git a/collectors/REFERENCE.md b/collectors/REFERENCE.md
index 270dded29..f19533f21 100644
--- a/collectors/REFERENCE.md
+++ b/collectors/REFERENCE.md
@@ -4,81 +4,29 @@ custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/REFE
 sidebar_label: "Collectors configuration"
 learn_status: "Published"
 learn_topic_type: "Tasks"
-learn_rel_path: "Setup"
+learn_rel_path: "Configuration"
 -->
 
 # Collectors configuration reference
 
-Welcome to the collector configuration reference guide.
+The list of supported collectors can be found in [the documentation](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md), 
+and on [our website](https://www.netdata.cloud/integrations). The documentation of each collector provides all the 
+necessary configuration options and prerequisites for that collector. In most cases, either the charts are automatically generated 
+without any configuration, or you just fulfil those prerequisites and [configure the collector](#configure-a-collector).
 
-This guide contains detailed information about enabling/disabling plugins or modules, in addition a quick reference to
-the internal plugins API.
+If the application you are interested in monitoring is not listed in our integrations, the collectors list includes 
+the available options to 
+[add your application to Netdata](https://github.com/netdata/netdata/edit/master/collectors/COLLECTORS.md#add-your-application-to-netdata).
 
-## Netdata's collector architecture
+If we do support your collector but the charts described in the documentation don't appear on your dashboard, the reason will 
+be one of the following:
 
-Netdata has an intricate system for organizing and managing its collectors. **Collectors** are the processes/programs
-that actually gather metrics from various sources. Collectors are organized by **plugins**, which help manage all the
-independent processes in a variety of programming languages based on their purpose and performance requirements.
-**Modules** are a type of collector, used primarily to connect to external applications, such as an Nginx web server or
-MySQL database, among many others.
+-   The entire data collection plugin is disabled by default. Read how to [enable and disable plugins](#enable-and-disable-plugins)
 
-For most users, enabling individual collectors for the application/service you're interested in is far more important
-than knowing which plugin it uses. See our [collectors list](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md) to see whether your favorite app/service has
-a collector, and then read the documentation for that specific collector to figure out how to enable it.
+-   The data collection plugin is enabled, but a specific data collection module is disabled. Read how to
+    [enable and disable a specific collection module](#enable-and-disable-a-specific-collection-module). 
 
-There are three types of plugins:
-
--   **Internal** plugins organize collectors that gather metrics from `/proc`, `/sys` and other Linux kernel sources.
-    They are written in `C`, and run as threads within the Netdata daemon.
--   **External** plugins organize collectors that gather metrics from external processes, such as a MySQL database or
-    Nginx web server. They can be written in any language, and the `netdata` daemon spawns them as long-running
-    independent processes. They communicate with the daemon via pipes.
--   **Plugin orchestrators**, which are external plugins that instead support a number of **modules**. Modules are a
-    type of collector. We have a few plugin orchestrators available for those who want to develop their own collectors,
-    but focus most of our efforts on the [Go plugin](https://github.com/netdata/go.d.plugin/blob/master/README.md).
-
-## Enable, configure, and disable modules
-
-Most collector modules come with **auto-detection**, configured to work out-of-the-box on popular operating systems with
-the default settings.
-
-However, there are cases that auto-detection fails. Usually, the reason is that the applications to be monitored do not
-allow Netdata to connect. In most of the cases, allowing the user `netdata` from `localhost` to connect and collect
-metrics, will automatically enable data collection for the application in question (it will require a Netdata restart).
-
-
-## Troubleshoot a collector
-
-First, navigate to your plugins directory, which is usually at `/usr/libexec/netdata/plugins.d/`. If that's not the case
-on your system, open `netdata.conf` and look for the setting `plugins directory`. Once you're in the plugins directory,
-switch to the `netdata` user.
-
-```bash
-cd /usr/libexec/netdata/plugins.d/
-sudo su -s /bin/bash netdata
-```
-
-The next step is based on the collector's orchestrator. You can figure out which orchestrator the collector uses by 
-
-uses either
-by viewing the [collectors list](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md) and referencing the _configuration file_ field. For example, if that
-field contains `go.d`, that collector uses the Go orchestrator.
-
-```bash
-# Go orchestrator (go.d.plugin)
-./go.d.plugin -d -m <MODULE_NAME>
-
-# Python orchestrator (python.d.plugin)
-./python.d.plugin <MODULE_NAME> debug trace
-
-# Bash orchestrator (bash.d.plugin)
-./charts.d.plugin debug 1 <MODULE_NAME>
-```
-
-The output from the relevant command will provide valuable troubleshooting information. If you can't figure out how to
-enable the collector using the details from this output, feel free to [create an issue on our
-GitHub](https://github.com/netdata/netdata/issues/new?assignees=&labels=bug%2Cneeds+triage&template=BUG_REPORT.yml) to get some
-help from our collectors experts.
+-   Autodetection failed. Read how to [configure](#configure-a-collector) and [troubleshoot](#troubleshoot-a-collector) a collector.
 
 ## Enable and disable plugins
 
@@ -88,87 +36,114 @@ This section features a list of Netdata's plugins, with a boolean setting to ena
 
 ```conf
 [plugins]
-	# proc = yes
-	# diskspace = yes
 	# timex = yes
-	# cgroups = yes
-	# tc = yes
 	# idlejitter = yes
+	# netdata monitoring = yes
+	# tc = yes
+	# diskspace = yes
+	# proc = yes
+	# cgroups = yes
 	# enable running new plugins = yes
 	# check for new plugins every = 60
 	# slabinfo = no
-	# ioping = yes
 	# python.d = yes
+	# perf = yes
+	# ioping = yes
+	# fping = yes
+	# nfacct = yes
 	# go.d = yes
 	# apps = yes
-	# perf = yes
+	# ebpf = yes
 	# charts.d = yes
+	# statsd = yes
 ```
 
 By default, most plugins are enabled, so you don't need to enable them explicitly to use their collectors. To enable or
 disable any specific plugin, remove the comment (`#`) and change the boolean setting to `yes` or `no`.
 
-All **external plugins** are managed by [plugins.d](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md), which provides additional management options.
+## Enable and disable a specific collection module
 
-## Internal plugins
+You can enable/disable of the collection modules supported by `go.d`, `python.d` or `charts.d` individually, using the 
+configuration file of that orchestrator. For example, you can change the behavior of the Go orchestrator, or any of its 
+collectors, by editing `go.d.conf`.
 
-Each of the internal plugins runs as a thread inside the `netdata` daemon. Once this thread has started, the plugin may
-spawn additional threads according to its design.
+Use `edit-config` from your [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory) 
+to open the orchestrator primary configuration file:
 
-### Internal plugins API
+```bash
+cd /etc/netdata
+sudo ./edit-config go.d.conf
+```
 
-The internal data collection API consists of the following calls:
+Within this file, you can either disable the orchestrator entirely (`enabled: yes`), or find a specific collector and
+enable/disable it with `yes` and `no` settings. Uncomment any line you change to ensure the Netdata daemon reads it on
+start.
 
-```c
-collect_data() {
-    // collect data here (one iteration)
+After you make your changes, restart the Agent with `sudo systemctl restart netdata`, or the [appropriate
+method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system.
 
-    collected_number collected_value = collect_a_value();
+## Configure a collector
 
-    // give the metrics to Netdata
+Most collector modules come with **auto-detection**, configured to work out-of-the-box on popular operating systems with
+the default settings. 
 
-    static RRDSET *st = NULL; // the chart
-    static RRDDIM *rd = NULL; // a dimension attached to this chart
+However, there are cases that auto-detection fails. Usually, the reason is that the applications to be monitored do not
+allow Netdata to connect. In most of the cases, allowing the user `netdata` from `localhost` to connect and collect
+metrics, will automatically enable data collection for the application in question (it will require a Netdata restart).
 
-    if(unlikely(!st)) {
-        // we haven't created this chart before
-        // create it now
-        st = rrdset_create_localhost(
-                "type"
-                , "id"
-                , "name"
-                , "family"
-                , "context"
-                , "Chart Title"
-                , "units"
-                , "plugin-name"
-                , "module-name"
-                , priority
-                , update_every
-                , chart_type
-        );
+When Netdata starts up, each collector searches for exposed metrics on the default endpoint established by that service
+or application's standard installation procedure. For example, 
+the [Nginx collector](https://github.com/netdata/go.d.plugin/blob/master/modules/nginx/README.md) searches at
+`http://127.0.0.1/stub_status` for exposed metrics in the correct format. If an Nginx web server is running and exposes
+metrics on that endpoint, the collector begins gathering them.
 
-        // attach a metric to it
-        rd = rrddim_add(st, "id", "name", multiplier, divider, algorithm);
-    }
+However, not every node or infrastructure uses standard ports, paths, files, or naming conventions. You may need to
+enable or configure a collector to gather all available metrics from your systems, containers, or applications.
 
-    // give the collected value(s) to the chart
-    rrddim_set_by_pointer(st, rd, collected_value);
+First, [find the collector](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md) you want to edit 
+and open its documentation. Some software has collectors written in multiple languages. In these cases, you should always 
+pick the collector written in Go.
 
-    // signal Netdata we are done with this iteration
-    rrdset_done(st);
-}
+Use `edit-config` from your 
+[Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory) 
+to open a collector's configuration file. For example, edit the Nginx collector with the following:
+
+```bash
+./edit-config go.d/nginx.conf
 ```
 
-Of course, Netdata has a lot of libraries to help you also in collecting the metrics. The best way to find your way
-through this, is to examine what other similar plugins do.
+Each configuration file describes every available option and offers examples to help you tweak Netdata's settings
+according to your needs. In addition, every collector's documentation shows the exact command you need to run to
+configure that collector. Uncomment any line you change to ensure the collector's orchestrator or the Netdata daemon
+read it on start.
+
+After you make your changes, restart the Agent with `sudo systemctl restart netdata`, or the [appropriate
+method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system.
+
+## Troubleshoot a collector
 
-## External Plugins
+First, navigate to your plugins directory, which is usually at `/usr/libexec/netdata/plugins.d/`. If that's not the case
+on your system, open `netdata.conf` and look for the setting `plugins directory`. Once you're in the plugins directory,
+switch to the `netdata` user.
 
-**External plugins** use the API and are managed 
-by [plugins.d](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md).
+```bash
+cd /usr/libexec/netdata/plugins.d/
+sudo su -s /bin/bash netdata
+```
 
-## Write a custom collector
+The next step is based on the collector's orchestrator. 
 
-You can add custom collectors by following the [external plugins documentation](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md).
+```bash
+# Go orchestrator (go.d.plugin)
+./go.d.plugin -d -m <MODULE_NAME>
 
+# Python orchestrator (python.d.plugin)
+./python.d.plugin <MODULE_NAME> debug trace
+
+# Bash orchestrator (bash.d.plugin)
+./charts.d.plugin debug 1 <MODULE_NAME>
+```
+
+The output from the relevant command will provide valuable troubleshooting information. If you can't figure out how to
+enable the collector using the details from this output, feel free to [join our Discord server](https://discord.com/invite/mPZ6WZKKG2), 
+to get help from our experts.