summaryrefslogtreecommitdiffstats
path: root/collectors/REFERENCE.md
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2023-05-08 16:27:08 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2023-05-08 16:27:08 +0000
commit81581f9719bc56f01d5aa08952671d65fda9867a (patch)
tree0f5c6b6138bf169c23c9d24b1fc0a3521385cb18 /collectors/REFERENCE.md
parentReleasing debian version 1.38.1-1. (diff)
downloadnetdata-81581f9719bc56f01d5aa08952671d65fda9867a.tar.xz
netdata-81581f9719bc56f01d5aa08952671d65fda9867a.zip
Merging upstream version 1.39.0.
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'collectors/REFERENCE.md')
-rw-r--r--collectors/REFERENCE.md209
1 files changed, 92 insertions, 117 deletions
diff --git a/collectors/REFERENCE.md b/collectors/REFERENCE.md
index 270dded29..f19533f21 100644
--- a/collectors/REFERENCE.md
+++ b/collectors/REFERENCE.md
@@ -4,81 +4,29 @@ custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/REFE
sidebar_label: "Collectors configuration"
learn_status: "Published"
learn_topic_type: "Tasks"
-learn_rel_path: "Setup"
+learn_rel_path: "Configuration"
-->
# Collectors configuration reference
-Welcome to the collector configuration reference guide.
+The list of supported collectors can be found in [the documentation](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md),
+and on [our website](https://www.netdata.cloud/integrations). The documentation of each collector provides all the
+necessary configuration options and prerequisites for that collector. In most cases, either the charts are automatically generated
+without any configuration, or you just fulfil those prerequisites and [configure the collector](#configure-a-collector).
-This guide contains detailed information about enabling/disabling plugins or modules, in addition a quick reference to
-the internal plugins API.
+If the application you are interested in monitoring is not listed in our integrations, the collectors list includes
+the available options to
+[add your application to Netdata](https://github.com/netdata/netdata/edit/master/collectors/COLLECTORS.md#add-your-application-to-netdata).
-## Netdata's collector architecture
+If we do support your collector but the charts described in the documentation don't appear on your dashboard, the reason will
+be one of the following:
-Netdata has an intricate system for organizing and managing its collectors. **Collectors** are the processes/programs
-that actually gather metrics from various sources. Collectors are organized by **plugins**, which help manage all the
-independent processes in a variety of programming languages based on their purpose and performance requirements.
-**Modules** are a type of collector, used primarily to connect to external applications, such as an Nginx web server or
-MySQL database, among many others.
+- The entire data collection plugin is disabled by default. Read how to [enable and disable plugins](#enable-and-disable-plugins)
-For most users, enabling individual collectors for the application/service you're interested in is far more important
-than knowing which plugin it uses. See our [collectors list](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md) to see whether your favorite app/service has
-a collector, and then read the documentation for that specific collector to figure out how to enable it.
+- The data collection plugin is enabled, but a specific data collection module is disabled. Read how to
+ [enable and disable a specific collection module](#enable-and-disable-a-specific-collection-module).
-There are three types of plugins:
-
-- **Internal** plugins organize collectors that gather metrics from `/proc`, `/sys` and other Linux kernel sources.
- They are written in `C`, and run as threads within the Netdata daemon.
-- **External** plugins organize collectors that gather metrics from external processes, such as a MySQL database or
- Nginx web server. They can be written in any language, and the `netdata` daemon spawns them as long-running
- independent processes. They communicate with the daemon via pipes.
-- **Plugin orchestrators**, which are external plugins that instead support a number of **modules**. Modules are a
- type of collector. We have a few plugin orchestrators available for those who want to develop their own collectors,
- but focus most of our efforts on the [Go plugin](https://github.com/netdata/go.d.plugin/blob/master/README.md).
-
-## Enable, configure, and disable modules
-
-Most collector modules come with **auto-detection**, configured to work out-of-the-box on popular operating systems with
-the default settings.
-
-However, there are cases that auto-detection fails. Usually, the reason is that the applications to be monitored do not
-allow Netdata to connect. In most of the cases, allowing the user `netdata` from `localhost` to connect and collect
-metrics, will automatically enable data collection for the application in question (it will require a Netdata restart).
-
-
-## Troubleshoot a collector
-
-First, navigate to your plugins directory, which is usually at `/usr/libexec/netdata/plugins.d/`. If that's not the case
-on your system, open `netdata.conf` and look for the setting `plugins directory`. Once you're in the plugins directory,
-switch to the `netdata` user.
-
-```bash
-cd /usr/libexec/netdata/plugins.d/
-sudo su -s /bin/bash netdata
-```
-
-The next step is based on the collector's orchestrator. You can figure out which orchestrator the collector uses by
-
-uses either
-by viewing the [collectors list](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md) and referencing the _configuration file_ field. For example, if that
-field contains `go.d`, that collector uses the Go orchestrator.
-
-```bash
-# Go orchestrator (go.d.plugin)
-./go.d.plugin -d -m <MODULE_NAME>
-
-# Python orchestrator (python.d.plugin)
-./python.d.plugin <MODULE_NAME> debug trace
-
-# Bash orchestrator (bash.d.plugin)
-./charts.d.plugin debug 1 <MODULE_NAME>
-```
-
-The output from the relevant command will provide valuable troubleshooting information. If you can't figure out how to
-enable the collector using the details from this output, feel free to [create an issue on our
-GitHub](https://github.com/netdata/netdata/issues/new?assignees=&labels=bug%2Cneeds+triage&template=BUG_REPORT.yml) to get some
-help from our collectors experts.
+- Autodetection failed. Read how to [configure](#configure-a-collector) and [troubleshoot](#troubleshoot-a-collector) a collector.
## Enable and disable plugins
@@ -88,87 +36,114 @@ This section features a list of Netdata's plugins, with a boolean setting to ena
```conf
[plugins]
- # proc = yes
- # diskspace = yes
# timex = yes
- # cgroups = yes
- # tc = yes
# idlejitter = yes
+ # netdata monitoring = yes
+ # tc = yes
+ # diskspace = yes
+ # proc = yes
+ # cgroups = yes
# enable running new plugins = yes
# check for new plugins every = 60
# slabinfo = no
- # ioping = yes
# python.d = yes
+ # perf = yes
+ # ioping = yes
+ # fping = yes
+ # nfacct = yes
# go.d = yes
# apps = yes
- # perf = yes
+ # ebpf = yes
# charts.d = yes
+ # statsd = yes
```
By default, most plugins are enabled, so you don't need to enable them explicitly to use their collectors. To enable or
disable any specific plugin, remove the comment (`#`) and change the boolean setting to `yes` or `no`.
-All **external plugins** are managed by [plugins.d](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md), which provides additional management options.
+## Enable and disable a specific collection module
-## Internal plugins
+You can enable/disable of the collection modules supported by `go.d`, `python.d` or `charts.d` individually, using the
+configuration file of that orchestrator. For example, you can change the behavior of the Go orchestrator, or any of its
+collectors, by editing `go.d.conf`.
-Each of the internal plugins runs as a thread inside the `netdata` daemon. Once this thread has started, the plugin may
-spawn additional threads according to its design.
+Use `edit-config` from your [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory)
+to open the orchestrator primary configuration file:
-### Internal plugins API
+```bash
+cd /etc/netdata
+sudo ./edit-config go.d.conf
+```
-The internal data collection API consists of the following calls:
+Within this file, you can either disable the orchestrator entirely (`enabled: yes`), or find a specific collector and
+enable/disable it with `yes` and `no` settings. Uncomment any line you change to ensure the Netdata daemon reads it on
+start.
-```c
-collect_data() {
- // collect data here (one iteration)
+After you make your changes, restart the Agent with `sudo systemctl restart netdata`, or the [appropriate
+method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system.
- collected_number collected_value = collect_a_value();
+## Configure a collector
- // give the metrics to Netdata
+Most collector modules come with **auto-detection**, configured to work out-of-the-box on popular operating systems with
+the default settings.
- static RRDSET *st = NULL; // the chart
- static RRDDIM *rd = NULL; // a dimension attached to this chart
+However, there are cases that auto-detection fails. Usually, the reason is that the applications to be monitored do not
+allow Netdata to connect. In most of the cases, allowing the user `netdata` from `localhost` to connect and collect
+metrics, will automatically enable data collection for the application in question (it will require a Netdata restart).
- if(unlikely(!st)) {
- // we haven't created this chart before
- // create it now
- st = rrdset_create_localhost(
- "type"
- , "id"
- , "name"
- , "family"
- , "context"
- , "Chart Title"
- , "units"
- , "plugin-name"
- , "module-name"
- , priority
- , update_every
- , chart_type
- );
+When Netdata starts up, each collector searches for exposed metrics on the default endpoint established by that service
+or application's standard installation procedure. For example,
+the [Nginx collector](https://github.com/netdata/go.d.plugin/blob/master/modules/nginx/README.md) searches at
+`http://127.0.0.1/stub_status` for exposed metrics in the correct format. If an Nginx web server is running and exposes
+metrics on that endpoint, the collector begins gathering them.
- // attach a metric to it
- rd = rrddim_add(st, "id", "name", multiplier, divider, algorithm);
- }
+However, not every node or infrastructure uses standard ports, paths, files, or naming conventions. You may need to
+enable or configure a collector to gather all available metrics from your systems, containers, or applications.
- // give the collected value(s) to the chart
- rrddim_set_by_pointer(st, rd, collected_value);
+First, [find the collector](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md) you want to edit
+and open its documentation. Some software has collectors written in multiple languages. In these cases, you should always
+pick the collector written in Go.
- // signal Netdata we are done with this iteration
- rrdset_done(st);
-}
+Use `edit-config` from your
+[Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory)
+to open a collector's configuration file. For example, edit the Nginx collector with the following:
+
+```bash
+./edit-config go.d/nginx.conf
```
-Of course, Netdata has a lot of libraries to help you also in collecting the metrics. The best way to find your way
-through this, is to examine what other similar plugins do.
+Each configuration file describes every available option and offers examples to help you tweak Netdata's settings
+according to your needs. In addition, every collector's documentation shows the exact command you need to run to
+configure that collector. Uncomment any line you change to ensure the collector's orchestrator or the Netdata daemon
+read it on start.
+
+After you make your changes, restart the Agent with `sudo systemctl restart netdata`, or the [appropriate
+method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system.
+
+## Troubleshoot a collector
-## External Plugins
+First, navigate to your plugins directory, which is usually at `/usr/libexec/netdata/plugins.d/`. If that's not the case
+on your system, open `netdata.conf` and look for the setting `plugins directory`. Once you're in the plugins directory,
+switch to the `netdata` user.
-**External plugins** use the API and are managed
-by [plugins.d](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md).
+```bash
+cd /usr/libexec/netdata/plugins.d/
+sudo su -s /bin/bash netdata
+```
-## Write a custom collector
+The next step is based on the collector's orchestrator.
-You can add custom collectors by following the [external plugins documentation](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md).
+```bash
+# Go orchestrator (go.d.plugin)
+./go.d.plugin -d -m <MODULE_NAME>
+# Python orchestrator (python.d.plugin)
+./python.d.plugin <MODULE_NAME> debug trace
+
+# Bash orchestrator (bash.d.plugin)
+./charts.d.plugin debug 1 <MODULE_NAME>
+```
+
+The output from the relevant command will provide valuable troubleshooting information. If you can't figure out how to
+enable the collector using the details from this output, feel free to [join our Discord server](https://discord.com/invite/mPZ6WZKKG2),
+to get help from our experts.