From 386ccdd61e8256c8b21ee27ee2fc12438fc5ca98 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Tue, 17 Oct 2023 11:30:20 +0200 Subject: Adding upstream version 1.43.0. Signed-off-by: Daniel Baumann --- exporting/prometheus/README.md | 2 +- exporting/prometheus/integrations/appoptics.md | 158 +++++++++++++++++++++ .../prometheus/integrations/azure_data_explorer.md | 158 +++++++++++++++++++++ .../prometheus/integrations/azure_event_hub.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/chronix.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/cortex.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/cratedb.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/elasticsearch.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/gnocchi.md | 158 +++++++++++++++++++++ .../prometheus/integrations/google_bigquery.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/irondb.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/kafka.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/m3db.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/metricfire.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/new_relic.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/postgresql.md | 158 +++++++++++++++++++++ .../integrations/prometheus_remote_write.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/quasardb.md | 158 +++++++++++++++++++++ .../prometheus/integrations/splunk_signalfx.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/thanos.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/tikv.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/timescaledb.md | 158 +++++++++++++++++++++ .../prometheus/integrations/victoriametrics.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/vmware_aria.md | 158 +++++++++++++++++++++ exporting/prometheus/integrations/wavefront.md | 158 +++++++++++++++++++++ exporting/prometheus/prometheus.c | 10 +- exporting/prometheus/remote_write/README.md | 61 +------- exporting/prometheus/remote_write/remote_write.c | 4 +- 28 files changed, 3800 insertions(+), 69 deletions(-) create mode 100644 exporting/prometheus/integrations/appoptics.md create mode 100644 exporting/prometheus/integrations/azure_data_explorer.md create mode 100644 exporting/prometheus/integrations/azure_event_hub.md create mode 100644 exporting/prometheus/integrations/chronix.md create mode 100644 exporting/prometheus/integrations/cortex.md create mode 100644 exporting/prometheus/integrations/cratedb.md create mode 100644 exporting/prometheus/integrations/elasticsearch.md create mode 100644 exporting/prometheus/integrations/gnocchi.md create mode 100644 exporting/prometheus/integrations/google_bigquery.md create mode 100644 exporting/prometheus/integrations/irondb.md create mode 100644 exporting/prometheus/integrations/kafka.md create mode 100644 exporting/prometheus/integrations/m3db.md create mode 100644 exporting/prometheus/integrations/metricfire.md create mode 100644 exporting/prometheus/integrations/new_relic.md create mode 100644 exporting/prometheus/integrations/postgresql.md create mode 100644 exporting/prometheus/integrations/prometheus_remote_write.md create mode 100644 exporting/prometheus/integrations/quasardb.md create mode 100644 exporting/prometheus/integrations/splunk_signalfx.md create mode 100644 exporting/prometheus/integrations/thanos.md create mode 100644 exporting/prometheus/integrations/tikv.md create mode 100644 exporting/prometheus/integrations/timescaledb.md create mode 100644 exporting/prometheus/integrations/victoriametrics.md create mode 100644 exporting/prometheus/integrations/vmware_aria.md create mode 100644 exporting/prometheus/integrations/wavefront.md mode change 100644 => 120000 exporting/prometheus/remote_write/README.md (limited to 'exporting/prometheus') diff --git a/exporting/prometheus/README.md b/exporting/prometheus/README.md index d3b37f126..abd81554d 100644 --- a/exporting/prometheus/README.md +++ b/exporting/prometheus/README.md @@ -24,7 +24,7 @@ Each chart in Netdata has several properties (common to all its metrics): - `chart_name` - a more human friendly name for `chart_id`, also unique. - `context` - this is the template of the chart. All disk I/O charts have the same context, all mysql requests charts - have the same context, etc. This is used for alarm templates to match all the charts they should be attached to. + have the same context, etc. This is used for alert templates to match all the charts they should be attached to. - `family` groups a set of charts together. It is used as the submenu of the dashboard. diff --git a/exporting/prometheus/integrations/appoptics.md b/exporting/prometheus/integrations/appoptics.md new file mode 100644 index 000000000..29954b654 --- /dev/null +++ b/exporting/prometheus/integrations/appoptics.md @@ -0,0 +1,158 @@ + + +# AppOptics + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/azure_data_explorer.md b/exporting/prometheus/integrations/azure_data_explorer.md new file mode 100644 index 000000000..c2ff6f21e --- /dev/null +++ b/exporting/prometheus/integrations/azure_data_explorer.md @@ -0,0 +1,158 @@ + + +# Azure Data Explorer + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/azure_event_hub.md b/exporting/prometheus/integrations/azure_event_hub.md new file mode 100644 index 000000000..0d6f97d81 --- /dev/null +++ b/exporting/prometheus/integrations/azure_event_hub.md @@ -0,0 +1,158 @@ + + +# Azure Event Hub + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/chronix.md b/exporting/prometheus/integrations/chronix.md new file mode 100644 index 000000000..5f00e6d1a --- /dev/null +++ b/exporting/prometheus/integrations/chronix.md @@ -0,0 +1,158 @@ + + +# Chronix + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/cortex.md b/exporting/prometheus/integrations/cortex.md new file mode 100644 index 000000000..64e7aed1c --- /dev/null +++ b/exporting/prometheus/integrations/cortex.md @@ -0,0 +1,158 @@ + + +# Cortex + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/cratedb.md b/exporting/prometheus/integrations/cratedb.md new file mode 100644 index 000000000..7e2ca3ffa --- /dev/null +++ b/exporting/prometheus/integrations/cratedb.md @@ -0,0 +1,158 @@ + + +# CrateDB + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/elasticsearch.md b/exporting/prometheus/integrations/elasticsearch.md new file mode 100644 index 000000000..67bc9d0ee --- /dev/null +++ b/exporting/prometheus/integrations/elasticsearch.md @@ -0,0 +1,158 @@ + + +# ElasticSearch + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/gnocchi.md b/exporting/prometheus/integrations/gnocchi.md new file mode 100644 index 000000000..c3b11c247 --- /dev/null +++ b/exporting/prometheus/integrations/gnocchi.md @@ -0,0 +1,158 @@ + + +# Gnocchi + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/google_bigquery.md b/exporting/prometheus/integrations/google_bigquery.md new file mode 100644 index 000000000..3639fd48f --- /dev/null +++ b/exporting/prometheus/integrations/google_bigquery.md @@ -0,0 +1,158 @@ + + +# Google BigQuery + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/irondb.md b/exporting/prometheus/integrations/irondb.md new file mode 100644 index 000000000..c2525848a --- /dev/null +++ b/exporting/prometheus/integrations/irondb.md @@ -0,0 +1,158 @@ + + +# IRONdb + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/kafka.md b/exporting/prometheus/integrations/kafka.md new file mode 100644 index 000000000..de98992b1 --- /dev/null +++ b/exporting/prometheus/integrations/kafka.md @@ -0,0 +1,158 @@ + + +# Kafka + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/m3db.md b/exporting/prometheus/integrations/m3db.md new file mode 100644 index 000000000..38be54a65 --- /dev/null +++ b/exporting/prometheus/integrations/m3db.md @@ -0,0 +1,158 @@ + + +# M3DB + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/metricfire.md b/exporting/prometheus/integrations/metricfire.md new file mode 100644 index 000000000..e9c4f7eab --- /dev/null +++ b/exporting/prometheus/integrations/metricfire.md @@ -0,0 +1,158 @@ + + +# MetricFire + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/new_relic.md b/exporting/prometheus/integrations/new_relic.md new file mode 100644 index 000000000..6d5417409 --- /dev/null +++ b/exporting/prometheus/integrations/new_relic.md @@ -0,0 +1,158 @@ + + +# New Relic + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/postgresql.md b/exporting/prometheus/integrations/postgresql.md new file mode 100644 index 000000000..998659880 --- /dev/null +++ b/exporting/prometheus/integrations/postgresql.md @@ -0,0 +1,158 @@ + + +# PostgreSQL + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/prometheus_remote_write.md b/exporting/prometheus/integrations/prometheus_remote_write.md new file mode 100644 index 000000000..213414d63 --- /dev/null +++ b/exporting/prometheus/integrations/prometheus_remote_write.md @@ -0,0 +1,158 @@ + + +# Prometheus Remote Write + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/quasardb.md b/exporting/prometheus/integrations/quasardb.md new file mode 100644 index 000000000..66d65766c --- /dev/null +++ b/exporting/prometheus/integrations/quasardb.md @@ -0,0 +1,158 @@ + + +# QuasarDB + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/splunk_signalfx.md b/exporting/prometheus/integrations/splunk_signalfx.md new file mode 100644 index 000000000..eba1cec53 --- /dev/null +++ b/exporting/prometheus/integrations/splunk_signalfx.md @@ -0,0 +1,158 @@ + + +# Splunk SignalFx + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/thanos.md b/exporting/prometheus/integrations/thanos.md new file mode 100644 index 000000000..09fa6d8a9 --- /dev/null +++ b/exporting/prometheus/integrations/thanos.md @@ -0,0 +1,158 @@ + + +# Thanos + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/tikv.md b/exporting/prometheus/integrations/tikv.md new file mode 100644 index 000000000..3735e52c1 --- /dev/null +++ b/exporting/prometheus/integrations/tikv.md @@ -0,0 +1,158 @@ + + +# TiKV + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/timescaledb.md b/exporting/prometheus/integrations/timescaledb.md new file mode 100644 index 000000000..41cfc1931 --- /dev/null +++ b/exporting/prometheus/integrations/timescaledb.md @@ -0,0 +1,158 @@ + + +# TimescaleDB + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/victoriametrics.md b/exporting/prometheus/integrations/victoriametrics.md new file mode 100644 index 000000000..d51dd82fb --- /dev/null +++ b/exporting/prometheus/integrations/victoriametrics.md @@ -0,0 +1,158 @@ + + +# VictoriaMetrics + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/vmware_aria.md b/exporting/prometheus/integrations/vmware_aria.md new file mode 100644 index 000000000..9311f1489 --- /dev/null +++ b/exporting/prometheus/integrations/vmware_aria.md @@ -0,0 +1,158 @@ + + +# VMware Aria + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/integrations/wavefront.md b/exporting/prometheus/integrations/wavefront.md new file mode 100644 index 000000000..fd199dab9 --- /dev/null +++ b/exporting/prometheus/integrations/wavefront.md @@ -0,0 +1,158 @@ + + +# Wavefront + + + + + +Use the Prometheus remote write exporting connector to archive your Netdata metrics to the external storage provider of your choice for long-term storage and further analysis. + + + + +## Limitations + +The remote write exporting connector does not support buffer on failures. + + +## Setup + +### Prerequisites + +#### + +- Netdata and the external storage provider of your choice, installed, configured and operational. +- `protobuf` and `snappy` libraries installed. +- Netdata reinstalled after the libraries. + + + +### Configuration + +#### File + +The configuration file name for this integration is `exporting.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config exporting.conf +``` +#### Options + +The following options can be defined for this exporter. + +
Config options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| enabled | Enables or disables an exporting connector instance (yes/no). | no | True | +| destination | Accepts a space separated list of hostnames, IPs (IPv4 and IPv6) and ports to connect to. Netdata will use the first available to send the metrics. | no | True | +| username | Username for HTTP authentication | my_username | False | +| password | Password for HTTP authentication | my_password | False | +| data source | Selects the kind of data that will be sent to the external database. (as collected/average/sum) | | False | +| hostname | The hostname to be used for sending data to the external database server. | [global].hostname | False | +| prefix | The prefix to add to all metrics. | netdata | False | +| update every | Frequency of sending sending data to the external database, in seconds. | 10 | False | +| buffer on failures | The number of iterations (`update every` seconds) to buffer data, when the external database server is not available. | 10 | False | +| timeout ms | The timeout in milliseconds to wait for the external database server to process the data. | 20000 | False | +| send hosts matching | Hosts filter. Determines which hosts will be sent to the external database. The syntax is [simple patterns](https://github.com/netdata/netdata/tree/master/libnetdata/simple_pattern#simple-patterns). | localhost * | False | +| send charts matching | One or more space separated patterns (use * as wildcard) checked against both chart id and chart name. | * | False | +| send names instead of ids | Controls the metric names Netdata should send to the external database (yes/no). | | False | +| send configured labels | Controls if host labels defined in the `[host labels]` section in `netdata.conf` should be sent to the external database (yes/no). | | False | +| send automatic labels | Controls if automatically created labels, like `_os_name` or `_architecture` should be sent to the external database (yes/no). | | False | + +##### destination + +The format of each item in this list, is: [PROTOCOL:]IP[:PORT]. +- PROTOCOL can be udp or tcp. tcp is the default and only supported by the current exporting engine. +- IP can be XX.XX.XX.XX (IPv4), or [XX:XX...XX:XX] (IPv6). For IPv6 you can to enclose the IP in [] to separate it from the port. +- PORT can be a number of a service name. If omitted, the default port for the exporting connector will be used. + +Example IPv4: + ```yaml + destination = 10.11.14.2:2003 10.11.14.3:4242 10.11.14.4:2003 + ``` +Example IPv6 and IPv4 together: +```yaml +destination = [ffff:...:0001]:2003 10.11.12.1:2003 +``` +When multiple servers are defined, Netdata will try the next one when the previous one fails. + + +##### update every + +Netdata will add some randomness to this number, to prevent stressing the external server when many Netdata servers +send data to the same database. This randomness does not affect the quality of the data, only the time they are sent. + + +##### buffer on failures + +If the server fails to receive the data after that many failures, data loss on the connector instance is expected (Netdata will also log it). + + +##### send hosts matching + +Includes one or more space separated patterns, using * as wildcard (any number of times within each pattern). +The patterns are checked against the hostname (the localhost is always checked as localhost), allowing us to +filter which hosts will be sent to the external database when this Netdata is a central Netdata aggregating multiple hosts. + +A pattern starting with `!` gives a negative match. So to match all hosts named `*db*` except hosts containing `*child*`, +use `!*child* *db*` (so, the order is important: the first pattern matching the hostname will be used - positive or negative). + + +##### send charts matching + +A pattern starting with ! gives a negative match. So to match all charts named apps.* except charts ending in *reads, +use !*reads apps.* (so, the order is important: the first pattern matching the chart id or the chart name will be used, +positive or negative). There is also a URL parameter filter that can be used while querying allmetrics. The URL parameter +has a higher priority than the configuration option. + + +##### send names instead of ids + +Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system and names +are human friendly labels (also unique). Most charts and metrics have the same ID and name, but in several cases they are +different : disks with device-mapper, interrupts, QoS classes, statsd synthetic charts, etc. + + +
+ +#### Examples + +##### Example configuration + +Basic example configuration for Prometheus remote write. + +```yaml +[prometheus_remote_write:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + +``` +##### Example configuration with HTTPS and HTTP authentication + +Add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: `remote_write:https:my_instance`. + +```yaml +[prometheus_remote_write:https:my_instance] + enabled = yes + destination = 10.11.14.2:2003 + remote write URL path = /receive + username = my_username + password = my_password + +``` + diff --git a/exporting/prometheus/prometheus.c b/exporting/prometheus/prometheus.c index 2d0611fdf..9e3f4bbf1 100644 --- a/exporting/prometheus/prometheus.c +++ b/exporting/prometheus/prometheus.c @@ -292,7 +292,7 @@ struct format_prometheus_label_callback { size_t count; }; -static int format_prometheus_label_callback(const char *name, const char *value, RRDLABEL_SRC ls, void *data) { +static int format_prometheus_label_callback(const char *name, const char *value, RRDLABEL_SRC ls __maybe_unused, void *data) { struct format_prometheus_label_callback *d = (struct format_prometheus_label_callback *)data; if (!should_send_label(d->instance, ls)) return 0; @@ -333,11 +333,9 @@ void format_host_labels_prometheus(struct instance *instance, RRDHOST *host) * @param data is the buffer used to add labels. */ -static int format_prometheus_chart_label_callback(const char *name, const char *value, RRDLABEL_SRC ls, void *data) { +static int format_prometheus_chart_label_callback(const char *name, const char *value, RRDLABEL_SRC ls __maybe_unused, void *data) { BUFFER *wb = data; - (void)ls; - if (name[0] == '_' ) return 1; @@ -496,7 +494,7 @@ static void generate_as_collected_prom_metric(BUFFER *wb, struct gen_parameters *p, int homogeneous, int prometheus_collector, - DICTIONARY *chart_labels) + RRDLABELS *chart_labels) { buffer_sprintf(wb, "%s_%s", p->prefix, p->context); @@ -524,7 +522,7 @@ static void generate_as_collected_prom_metric(BUFFER *wb, buffer_sprintf(wb, COLLECTED_NUMBER_FORMAT, p->rd->collector.last_collected_value); if (p->output_options & PROMETHEUS_OUTPUT_TIMESTAMPS) - buffer_sprintf(wb, " %llu\n", timeval_msec(&p->rd->collector.last_collected_time)); + buffer_sprintf(wb, " %"PRIu64"\n", timeval_msec(&p->rd->collector.last_collected_time)); else buffer_sprintf(wb, "\n"); } diff --git a/exporting/prometheus/remote_write/README.md b/exporting/prometheus/remote_write/README.md deleted file mode 100644 index c2ad22a6c..000000000 --- a/exporting/prometheus/remote_write/README.md +++ /dev/null @@ -1,60 +0,0 @@ - - -# Export metrics to Prometheus remote write providers - -The Prometheus remote write exporting connector uses the exporting engine to send Netdata metrics to your choice of more -than 20 external storage providers for long-term archiving and further analysis. - -## Prerequisites - -To use the Prometheus remote write API with [storage -providers](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage), install -[protobuf](https://developers.google.com/protocol-buffers/) and [snappy](https://github.com/google/snappy) libraries. -Next, [reinstall Netdata](https://github.com/netdata/netdata/blob/master/packaging/installer/REINSTALL.md), which detects that the required libraries and utilities -are now available. - -## Configuration - -To enable data exporting to a storage provider using the Prometheus remote write API, run `./edit-config exporting.conf` -in the Netdata configuration directory and set the following options: - -```conf -[prometheus_remote_write:my_instance] - enabled = yes - destination = example.domain:example_port - remote write URL path = /receive -``` - -You can also add `:https` modifier to the connector type if you need to use the TLS/SSL protocol. For example: -`remote_write:https:my_instance`. - -`remote write URL path` is used to set an endpoint path for the remote write protocol. The default value is `/receive`. -For example, if your endpoint is `http://example.domain:example_port/storage/read`: - -```conf - destination = example.domain:example_port - remote write URL path = /storage/read -``` - -You can set basic HTTP authentication credentials using - -```conf - username = my_username - password = my_password -``` - -`buffered` and `lost` dimensions in the Netdata Exporting Connector Data Size operation monitoring chart estimate uncompressed -buffer size on failures. - -## Notes - -The remote write exporting connector does not support `buffer on failures` - - diff --git a/exporting/prometheus/remote_write/README.md b/exporting/prometheus/remote_write/README.md new file mode 120000 index 000000000..8ca4673a6 --- /dev/null +++ b/exporting/prometheus/remote_write/README.md @@ -0,0 +1 @@ +../integrations/prometheus_remote_write.md \ No newline at end of file diff --git a/exporting/prometheus/remote_write/remote_write.c b/exporting/prometheus/remote_write/remote_write.c index 2b53b1c20..ed431c9dc 100644 --- a/exporting/prometheus/remote_write/remote_write.c +++ b/exporting/prometheus/remote_write/remote_write.c @@ -139,11 +139,11 @@ struct format_remote_write_label_callback { void *write_request; }; -static int format_remote_write_label_callback(const char *name, const char *value, RRDLABEL_SRC ls, void *data) { +static int format_remote_write_label_callback(const char *name, const char *value, RRDLABEL_SRC ls __maybe_unused, void *data) +{ struct format_remote_write_label_callback *d = (struct format_remote_write_label_callback *)data; if (!should_send_label(d->instance, ls)) return 0; - char k[PROMETHEUS_ELEMENT_MAX + 1]; char v[PROMETHEUS_ELEMENT_MAX + 1]; -- cgit v1.2.3