diff options
| author | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-11-25 17:33:56 +0000 |
|---|---|---|
| committer | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-11-25 17:34:10 +0000 |
| commit | 83ba6762cc43d9db581b979bb5e3445669e46cc2 (patch) | |
| tree | 2e69833b43f791ed253a7a20318b767ebe56cdb8 /docs/netdata-agent/configuration | |
| parent | Releasing debian version 1.47.5-1. (diff) | |
| download | netdata-83ba6762cc43d9db581b979bb5e3445669e46cc2.tar.xz netdata-83ba6762cc43d9db581b979bb5e3445669e46cc2.zip | |
Merging upstream version 2.0.3+dfsg (Closes: #923993, #1042533, #1045145).
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'docs/netdata-agent/configuration')
16 files changed, 387 insertions, 549 deletions
diff --git a/docs/netdata-agent/configuration/README.md b/docs/netdata-agent/configuration/README.md index 097fb9310..abe511313 100644 --- a/docs/netdata-agent/configuration/README.md +++ b/docs/netdata-agent/configuration/README.md @@ -1,21 +1,28 @@ # Netdata Agent Configuration -The main Netdata agent configuration is `netdata.conf`. +> **Info** +> +> Netdata Cloud lets you configure Agents on the fly. Check the [Dynamic Configuration Manager](/docs/netdata-agent/configuration/dynamic-configuration.md) documentation for details. + +The main Netdata Agent configuration is `netdata.conf`. ## The Netdata config directory -On most Linux systems, by using our [recommended one-line installation](/packaging/installer/README.md#install-on-linux-with-one-line-installer), the **Netdata config +On most Linux systems, the **Netdata config directory** will be `/etc/netdata/`. The config directory contains several configuration files with the `.conf` extension, a few directories, and a shell script named `edit-config`. > Some operating systems will use `/opt/netdata/etc/netdata/` as the config directory. If you're not sure where yours > is, navigate to `http://NODE:19999/netdata.conf` in your browser, replacing `NODE` with the IP address or hostname of -> your node, and find the `# config directory = ` setting. The value listed is the config directory for your system. +> your node, and find the `# config directory =` setting. The value listed is the config directory for your system. All of Netdata's documentation assumes that your config directory is at `/etc/netdata`, and that you're running any scripts from inside that directory. +## Edit a configuration file using `edit-config` + +We recommend the use of the `edit-config` script for configuration changes. -## edit `netdata.conf` +It exists inside your config directory (read above) and helps manage and safely edit configuration files. To edit `netdata.conf`, run this on your terminal: @@ -28,9 +35,9 @@ Your editor will open. ## downloading `netdata.conf` -The running version of `netdata.conf` can be downloaded from a running Netdata agent, at this URL: +The running version of `netdata.conf` can be downloaded from a running Netdata Agent, at this URL: -``` +```url http://agent-ip:19999/netdata.conf ``` @@ -40,4 +47,3 @@ You can save and use this version, using these commands: cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata curl -ksSLo /tmp/netdata.conf.new http://localhost:19999/netdata.conf && sudo mv -i /tmp/netdata.conf.new netdata.conf ``` - diff --git a/docs/netdata-agent/configuration/anonymous-telemetry-events.md b/docs/netdata-agent/configuration/anonymous-telemetry-events.md index b943ea9a3..4d48de4a2 100644 --- a/docs/netdata-agent/configuration/anonymous-telemetry-events.md +++ b/docs/netdata-agent/configuration/anonymous-telemetry-events.md @@ -1,30 +1,22 @@ -<!-- -title: "Anonymous telemetry events" -custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/netdata-agent/configuration/anonymous-telemetry-events.md -sidebar_label: "Anonymous telemetry events" -learn_status: "Published" -learn_rel_path: "Configuration" ---> - # Anonymous telemetry events -By default, Netdata collects anonymous usage information from the open-source monitoring agent. For agent events like start,stop,crash etc we use our own cloud function in GCP. For frontend telemetry (pageviews etc.) on the agent dashboard itself we use the open-source +By default, Netdata collects anonymous usage information from the open-source monitoring agent. For agent events like start, stop, crash, etc. we use our own cloud function in GCP. For frontend telemetry (page views etc.) on the agent dashboard itself, we use the open-source product analytics platform [PostHog](https://github.com/PostHog/posthog). We are strongly committed to your [data privacy](https://netdata.cloud/privacy/). We use the statistics gathered from this information for two purposes: -1. **Quality assurance**, to help us understand if Netdata behaves as expected, and to help us classify repeated - issues with certain distributions or environments. +1. **Quality assurance**, to help us understand if Netdata behaves as expected, and to help us classify repeated + issues with certain distributions or environments. -2. **Usage statistics**, to help us interpret how people use the Netdata agent in real-world environments, and to help - us identify how our development/design decisions influence the community. +2. **Usage statistics**, to help us interpret how people use the Netdata agent in real-world environments, and to help + us identify how our development/design decisions influence the community. Netdata collects usage information via two different channels: -- **Agent dashboard**: We use the [PostHog JavaScript integration](https://posthog.com/docs/integrations/js-integration) (with sensitive event attributes overwritten to be anonymized) to send product usage events when you access an [Agent's dashboard](/docs/dashboards-and-charts/README.md). -- **Agent backend**: The `netdata` daemon executes the [`anonymous-statistics.sh`](https://github.com/netdata/netdata/blob/6469cf92724644f5facf343e4bdd76ac0551a418/daemon/anonymous-statistics.sh.in) script when Netdata starts, stops cleanly, or fails. +- **Agent dashboard**: We use the [PostHog JavaScript integration](https://posthog.com/docs/integrations/js-integration) (with sensitive event attributes overwritten to be anonymized) to send product usage events when you access an [Agent's dashboard](/docs/dashboards-and-charts/README.md). +- **Agent backend**: The `netdata` daemon executes the [`anonymous-statistics.sh`](https://github.com/netdata/netdata/blob/6469cf92724644f5facf343e4bdd76ac0551a418/daemon/anonymous-statistics.sh.in) script when Netdata starts, stops cleanly, or fails. You can opt-out from sending anonymous statistics to Netdata through three different [opt-out mechanisms](#opt-out). @@ -32,7 +24,7 @@ You can opt-out from sending anonymous statistics to Netdata through three diffe When you kick off an Agent dashboard session by visiting `http://NODE:19999`, Netdata initializes a PostHog session and masks various event attributes. -_Note_: You can see the relevant code in the [dashboard repository](https://github.com/netdata/dashboard/blob/master/src/domains/global/sagas.ts#L107) where the `window.posthog.register()` call is made. +_Note_: You can see the relevant code in the [dashboard repository](https://github.com/netdata/dashboard/blob/master/src/domains/global/sagas.ts#L107) where the `window.posthog.register()` call is made. ```JavaScript window.posthog.register({ @@ -52,28 +44,28 @@ variable is controlled via the [opt-out mechanism](#opt-out). ## Agent Backend - Anonymous Statistics Script Every time the daemon is started or stopped and every time a fatal condition is encountered, Netdata uses the anonymous -statistics script to collect system information and send it to the Netdata telemetry cloud function via an http call. The information collected for all +statistics script to collect system information and send it to the Netdata telemetry cloud function via a http call. The information collected for all events is: -- Netdata version -- OS name, version, id, id_like -- Kernel name, version, architecture -- Virtualization technology -- Containerization technology +- Netdata version +- OS name, version, id, id_like +- Kernel name, version, architecture +- Virtualization technology +- Containerization technology -Furthermore, the FATAL event sends the Netdata process & thread name, along with the source code function, source code +Furthermore, the FATAL event sends the Netdata process and thread name, along with the source code function, source code filename and source code line number of the fatal error. Starting with v1.21, we additionally collect information about: -- Failures to build the dependencies required to use Cloud features. -- Unavailability of Cloud features in an agent. -- Failures to connect to the Cloud in case the [connection process](/src/claim/README.md) has been completed. This includes error codes - to inform the Netdata team about the reason why the connection failed. +- Failures to build the dependencies required to use Cloud features. +- Unavailability of Cloud features in an agent. +- Failures to connect to the Cloud in case the [connection process](/src/claim/README.md) has been completed. This includes error codes + to inform the Netdata team about the reason why the connection failed. To see exactly what and how is collected, you can review the script template `daemon/anonymous-statistics.sh.in`. The template is converted to a bash script called `anonymous-statistics.sh`, installed under the Netdata `plugins -directory`, which is usually `/usr/libexec/netdata/plugins.d`. +directory`, which is usually `/usr/libexec/netdata/plugins.d`. ## Opt-out @@ -87,17 +79,15 @@ installation, including manual, offline, and macOS installations. Create the fil **Pass the option `--disable-telemetry` to any of the installer scripts in the [installation docs](/packaging/installer/README.md).** You can append this option during the initial installation or a manual update. You can also export the environment variable `DISABLE_TELEMETRY` with a non-zero or non-empty value -(e.g: `export DISABLE_TELEMETRY=1`). +(e.g.,: `export DISABLE_TELEMETRY=1`). When using Docker, **set your `DISABLE_TELEMETRY` environment variable to `1`.** You can set this variable with the following command: `export DISABLE_TELEMETRY=1`. When creating a container using Netdata's [Docker image](/packaging/docker/README.md#create-a-new-netdata-agent-container) for the first time, this variable will disable -the anonymous statistics script inside of the container. +the anonymous statistics script inside the container. Each of these opt-out processes does the following: -- Prevents the daemon from executing the anonymous statistics script. -- Forces the anonymous statistics script to exit immediately. -- Stops the PostHog JavaScript snippet, which remains on the dashboard, from firing and sending any data to the Netdata PostHog. - - +- Prevents the daemon from executing the anonymous statistics script. +- Forces the anonymous statistics script to exit immediately. +- Stops the PostHog JavaScript snippet, which remains on the dashboard, from firing and sending any data to the Netdata PostHog. diff --git a/docs/netdata-agent/configuration/cheatsheet.md b/docs/netdata-agent/configuration/cheatsheet.md index 3e1428694..ecd8e8a84 100644 --- a/docs/netdata-agent/configuration/cheatsheet.md +++ b/docs/netdata-agent/configuration/cheatsheet.md @@ -1,8 +1,8 @@ # Useful management and configuration actions -Below you will find some of the most common actions that one can take while using Netdata. You can use this page as a quick reference for installing Netdata, connecting a node to the Cloud, properly editing the configuration, accessing Netdata's API, and more! +Below are some of the most common actions one can take while using Netdata. You can use this page as a quick reference for installing Netdata, connecting a node to the Cloud, properly editing the configuration, accessing Netdata's API, and more! -### Install Netdata +## Install Netdata ```bash wget -O /tmp/netdata-kickstart.sh https://get.netdata.cloud/kickstart.sh && sh /tmp/netdata-kickstart.sh @@ -11,12 +11,12 @@ wget -O /tmp/netdata-kickstart.sh https://get.netdata.cloud/kickstart.sh && sh / curl https://get.netdata.cloud/kickstart.sh > /tmp/netdata-kickstart.sh && sh /tmp/netdata-kickstart.sh ``` -#### Connect a node to Netdata Cloud +### Connect a node to Netdata Cloud To do so, sign in to Netdata Cloud, on your Space under the Nodes tab, click `Add Nodes` and paste the provided command into your node’s terminal and run it. You can also copy the Claim token and pass it to the installation script with `--claim-token` and re-run it. -### Configuration +## Configuration **Netdata's config directory** is `/etc/netdata/` but in some operating systems it might be `/opt/netdata/etc/netdata/`. Look for the `# config directory =` line over at `http://NODE_IP:19999/netdata.conf` to find your config directory. @@ -25,63 +25,19 @@ From within that directory you can run `sudo ./edit-config netdata.conf` **to ed You can edit other config files too, by specifying their filename after `./edit-config`. You are expected to use this method in all following configuration changes. -<!-- #### Edit Netdata's other config files (examples): - -- `$ sudo ./edit-config apps_groups.conf` -- `$ sudo ./edit-config ebpf.conf` -- `$ sudo ./edit-config health.d/load.conf` -- `$ sudo ./edit-config go.d/prometheus.conf` - -#### View the running Netdata configuration: `http://NODE:19999/netdata.conf` - -> Replace `NODE` with the IP address or hostname of your node. Often `localhost`. - -## Metrics collection & retention - -You can tweak your settings in the netdata.conf file. -📄 [Find your netdata.conf file](/src/daemon/config/README.md) - -Open a new terminal and navigate to the netdata.conf file. Use the edit-config script to make changes: `sudo ./edit-config netdata.conf` - -The most popular settings to change are: - -#### Increase metrics retention (4GiB) - -``` -sudo ./edit-config netdata.conf -``` - -``` -[global] - dbengine multihost disk space = 4096 -``` - -#### Reduce the collection frequency (every 5 seconds) - -``` -sudo ./edit-config netdata.conf -``` - -``` -[global] - update every = 5 -``` --> - ---- - -#### Enable/disable plugins (groups of collectors) +### Enable/disable plugins (groups of collectors) ```bash sudo ./edit-config netdata.conf ``` -```conf +```text [plugins] go.d = yes # enabled node.d = no # disabled ``` -#### Enable/disable specific collectors +### Enable/disable specific collectors ```bash sudo ./edit-config go.d.conf # edit a plugin's config @@ -89,24 +45,18 @@ sudo ./edit-config go.d.conf # edit a plugin's config ```yaml modules: - activemq: no # disabled - cockroachdb: yes # enabled + activemq: no # disabled + cockroachdb: yes # enabled ``` -#### Edit a collector's config +### Edit a collector's config ```bash sudo ./edit-config go.d/mysql.conf ``` -### Alerts & notifications - -<!-- #### Add a new alert +## Alerts & notifications -``` -sudo touch health.d/example-alert.conf -sudo ./edit-config health.d/example-alert.conf -``` --> After any change, reload the Netdata health configuration: ```bash @@ -115,32 +65,23 @@ netdatacli reload-health killall -USR2 netdata ``` -#### Configure a specific alert +### Configure a specific alert ```bash sudo ./edit-config health.d/example-alert.conf ``` -#### Silence a specific alert +### Silence a specific alert ```bash sudo ./edit-config health.d/example-alert.conf ``` -``` +```text to: silent ``` -<!-- #### Disable alerts and notifications - -```conf -[health] - enabled = no -``` --> - ---- - -### Manage the daemon +## Manage the daemon | Intent | Action | |:----------------------------|------------------------------------------------------------:| @@ -151,65 +92,22 @@ sudo ./edit-config health.d/example-alert.conf | View error logs | `less /var/log/netdata/error.log` | | View collectors logs | `less /var/log/netdata/collector.log` | -#### Change the port Netdata listens to (example, set it to port 39999) +### Change the port Netdata listens to (example, set it to port 39999) -```conf +```text [web] default port = 39999 ``` -### See metrics and dashboards +## See metrics and dashboards -#### Netdata Cloud: `https://app.netdata.cloud` +### Netdata Cloud: `https://app.netdata.cloud` -#### Local dashboard: `https://NODE:19999` +### Local dashboard: `https://NODE:19999` > Replace `NODE` with the IP address or hostname of your node. Often `localhost`. -### Access the Netdata API +## Access the Netdata API You can access the API like this: `http://NODE:19999/api/VERSION/REQUEST`. If you want to take a look at all the API requests, check our API page at <https://learn.netdata.cloud/api> -<!-- -## Interact with charts - -| Intent | Action | -| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | -| Stop a chart from updating | `click` | -| Zoom | **Cloud** <br/> use the `zoom in` and `zoom out` buttons on any chart (upper right corner) <br/><br/> **Agent**<br/>`SHIFT` or `ALT` + `mouse scrollwheel` <br/> `SHIFT` or `ALT` + `two-finger pinch` (touchscreen) <br/> `SHIFT` or `ALT` + `two-finger scroll` (touchscreen) | -| Zoom to a specific timeframe | **Cloud**<br/>use the `select and zoom` button on any chart and then do a `mouse selection` <br/><br/> **Agent**<br/>`SHIFT` + `mouse selection` | -| Pan forward or back in time | `click` & `drag` <br/> `touch` & `drag` (touchpad/touchscreen) | -| Select a certain timeframe | `ALT` + `mouse selection` <br/> WIP need to evaluate this `command?` + `mouse selection` (macOS) | -| Reset to default auto refreshing state | `double click` | --> - -<!-- ## Dashboards - -#### Disable the local dashboard - -Use the `edit-config` script to edit the `netdata.conf` file. - -``` -[web] -mode = none -``` --> - -<!-- #### Opt out from anonymous statistics - -``` -sudo touch .opt-out-from-anonymous-statistics -``` --> - -<!-- ## Understanding the dashboard - -**Charts**: A visualization displaying one or more collected/calculated metrics in a time series. Charts are generated -by collectors. - -**Dimensions**: Any value shown on a chart, which can be raw or calculated values, such as percentages, averages, -minimums, maximums, and more. - -**Families**: One instance of a monitored hardware or software resource that needs to be monitored and displayed -separately from similar instances. Example, disks named -**sda**, **sdb**, **sdc**, and so on. - -**Contexts**: A grouping of charts based on the types of metrics collected and visualized. -**disk.io**, **disk.ops**, and **disk.backlog** are all contexts. --> diff --git a/docs/netdata-agent/configuration/common-configuration-changes.md b/docs/netdata-agent/configuration/common-configuration-changes.md index e9d8abadc..0eda7dd86 100644 --- a/docs/netdata-agent/configuration/common-configuration-changes.md +++ b/docs/netdata-agent/configuration/common-configuration-changes.md @@ -19,11 +19,7 @@ changes reflected in those visualizations due to the way Netdata Cloud proxies m ### Increase the long-term metrics retention period -Read our doc -on [increasing long-term metrics storage](/docs/netdata-agent/configuration/optimizing-metrics-database/change-metrics-storage.md) -for details, including a -[calculator](/docs/netdata-agent/configuration/optimizing-metrics-database/change-metrics-storage.md#calculate-the-system-resources-ram-disk-space-needed-to-store-metrics) -to help you determine the exact settings for your desired retention period. +Read our doc on [increasing long-term metrics storage](/docs/netdata-agent/configuration/optimizing-metrics-database/change-metrics-storage.md) for details. ### Reduce the data collection frequency @@ -33,7 +29,7 @@ of `netdata.conf` so that it is greater than `1`. An `update every` of `5` means the Netdata Agent enforces a _minimum_ collection frequency of 5 seconds. -```conf +```text [global] update every = 5 ``` @@ -56,7 +52,7 @@ for that specific module. Uncomment the line and change its value to `no`. ## Modify alerts and notifications -Netdata's health monitoring watchdog uses hundreds of preconfigured health entities, with intelligent thresholds, to +Netdata's health monitoring watchdog uses hundreds of pre-configured health entities, with intelligent thresholds, to generate warning and critical alerts for most production systems and their applications without configuration. However, each alert and notification method is completely customizable. @@ -94,7 +90,7 @@ Because the source path contains `health.d/cpu.conf`, run `sudo edit-config heal Open the configuration file for that alert and set the `to` line to `silent`. -```conf +```text template: disk_fill_rate on: disk.space lookup: max -1s at -30m unaligned of avail @@ -111,7 +107,7 @@ section of `netdata.conf`. ### Enable alert notifications -Open `health_alarm_notify.conf` for editing. First, read the [enabling notifications](/docs/alerts-and-notifications/notifications/README.md#netdata-agent) doc +Open `health_alarm_notify.conf` for editing. First, read the [enabling notifications](/src/health/notifications/README.md) doc for an example of the process using Slack, then click on the link to your preferred notification method to find documentation for that specific endpoint. @@ -143,6 +139,6 @@ The following restrictions apply to host label names: - Names cannot start with `_`, but it can be present in other parts of the name. - Names only accept alphabet letters, numbers, dots, and dashes. -The policy for values is more flexible, but you can not use exclamation marks (`!`), whitespaces (` `), single quotes +The policy for values is more flexible, but you cannot use exclamation marks (`!`), whitespaces (` `), single quotes (`'`), double quotes (`"`), or asterisks (`*`), because they are used to compare label values in health alerts and templates. diff --git a/docs/netdata-agent/configuration/dynamic-configuration.md b/docs/netdata-agent/configuration/dynamic-configuration.md index 7064abf9a..c419a82d9 100644 --- a/docs/netdata-agent/configuration/dynamic-configuration.md +++ b/docs/netdata-agent/configuration/dynamic-configuration.md @@ -1,6 +1,8 @@ # Dynamic Configuration Manager -**Netdata Cloud paid subscription required.** +> **Info** +> +> Netdata Cloud paid subscription is required. The Dynamic Configuration Manager allows direct configuration of collectors and alerts through the Netdata UI. This feature allows users to: @@ -11,7 +13,7 @@ The Dynamic Configuration Manager allows direct configuration of collectors and > **Info** > -> To understand what actions users can perform based on their role, refer to the [Role Based Access documentation](/docs/netdata-cloud/authentication-and-authorization/role-based-access-model.md#dynamic-configuration-manager). +> To understand what actions users can perform based on their role, refer to the [Role-Based Access documentation](/docs/netdata-cloud/authentication-and-authorization/role-based-access-model.md#dynamic-configuration-manager). ## Collectors @@ -35,9 +37,9 @@ A job represents a running instance of a module with a specific configuration. T Every job has a designated "source type" indicating its origin: - **Stock**: Pre-installed with Netdata and provides basic data collection for common services. -- **User**: Originates from user-created files on the node. +- **User**: Created from user-defined configuration files on the node. - **Discovered**: Automatically generated by Netdata upon discovering a service running on the node. -- **Dynamic Configuration**: Created and managed using the Dynamic Configuration Manager. +- **Dynamic Configuration**: Managed and created through the Dynamic Configuration Manager. You can manage individual jobs using the following actions: @@ -51,7 +53,7 @@ You can manage individual jobs using the following actions: ## Health -Each entry in the Health tab contains an Alert template, that then is used to create Alerts. +Each entry in the Health tab contains an Alert template that then is used to create Alerts. The functionality in the main view is the same as with the [Collectors tab](#collectors). diff --git a/docs/netdata-agent/configuration/optimize-the-netdata-agents-performance.md b/docs/netdata-agent/configuration/optimize-the-netdata-agents-performance.md index 6acbd4977..ff51fbf78 100644 --- a/docs/netdata-agent/configuration/optimize-the-netdata-agents-performance.md +++ b/docs/netdata-agent/configuration/optimize-the-netdata-agents-performance.md @@ -1,9 +1,9 @@ # How to optimize the Netdata Agent's performance We designed the Netdata Agent to be incredibly lightweight, even when it's collecting a few thousand dimensions every -second and visualizing that data into hundreds of charts. However, the default settings of the Netdata Agent are not -optimized for performance, but for a simple, standalone setup. We want the first install to give you something you can -run without any configuration. Most of the settings and options are enabled, since we want you to experience the full +second and visualizing that data into hundreds of charts. However, the default settings of the Netdata Agent aren’t +optimized for performance, but for a simple, standalone setup. We want the first installation to give you something you can +run without any configuration. Most of the settings and options are enabled since we want you to experience the full thing. By default, Netdata will automatically detect applications running on the node it is installed to start collecting @@ -17,16 +17,16 @@ Netdata for production use. The following table summarizes the effect of each optimization on the CPU, RAM and Disk IO utilization in production. -| Optimization | CPU | RAM | Disk IO | -|-------------------------------------------------------------------------------------------------------------------------------|--------------------|--------------------|--------------------| -| [Use streaming and replication](#use-streaming-and-replication) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | -| [Disable unneeded plugins or collectors](#disable-unneeded-plugins-or-collectors) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | -| [Reduce data collection frequency](#reduce-collection-frequency) | :heavy_check_mark: | | :heavy_check_mark: | +| Optimization | CPU | RAM | Disk IO | +|-----------------------------------------------------------------------------------------------------------------------------------|--------------------|--------------------|--------------------| +| [Use streaming and replication](#use-streaming-and-replication) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | +| [Disable unneeded plugins or collectors](#disable-unneeded-plugins-or-collectors) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | +| [Reduce data collection frequency](#reduce-collection-frequency) | :heavy_check_mark: | | :heavy_check_mark: | | [Change how long Netdata stores metrics](/docs/netdata-agent/configuration/optimizing-metrics-database/change-metrics-storage.md) | | :heavy_check_mark: | :heavy_check_mark: | -| [Use a different metric storage database](/src/database/README.md) | | :heavy_check_mark: | :heavy_check_mark: | -| [Disable machine learning](#disable-machine-learning) | :heavy_check_mark: | | | -| [Use a reverse proxy](#run-netdata-behind-a-proxy) | :heavy_check_mark: | | | -| [Disable/lower gzip compression for the agent dashboard](#disablelower-gzip-compression-for-the-dashboard) | :heavy_check_mark: | | | +| [Use a different metric storage database](/src/database/README.md) | | :heavy_check_mark: | :heavy_check_mark: | +| [Disable machine learning](#disable-machine-learning) | :heavy_check_mark: | | | +| [Use a reverse proxy](#run-netdata-behind-a-proxy) | :heavy_check_mark: | | | +| [Disable/lower gzip compression for the agent dashboard](#disablelower-gzip-compression-for-the-dashboard) | :heavy_check_mark: | | | ## Resources required by a default Netdata installation @@ -39,15 +39,15 @@ You can configure almost all aspects of data collection/retention, and certain a Expect about: - 1-3% of a single core for the netdata core -- 1-3% of a single core for the various collectors (e.g. go.d.plugin, apps.plugin) +- 1-3% of a single core for the various collectors (e.g., go.d.plugin, apps.plugin) - 5-10% of a single core, when ML training runs Your experience may vary depending on the number of metrics collected, the collectors enabled and the specific -environment they run on, i.e. the work they have to do to collect these metrics. +environment they run on, i.e., the work they have to do to collect these metrics. As a general rule, for modern hardware and VMs, the total CPU consumption of a standalone Netdata installation, including all its components, should be below 5 - 15% of a single core. For example, on 8 core server it will use only -0.6% - 1.8% of a total CPU capacity, depending on the CPU characteristics. +0.6% - 1.8% of the total CPU capacity, depending on the CPU characteristics. The Netdata Agent runs with the lowest possible [process scheduling policy](/src/daemon/README.md#netdata-process-scheduling-policy), @@ -55,7 +55,7 @@ which is `nice 19`, and uses the `idle` process scheduler. Together, these setti resources when the node has CPU resources to space. If the node reaches 100% CPU utilization, the Agent is stopped first to ensure your applications get any available resources. -To reduce CPU usage you can (either one or a combination of the following actions): +To reduce CPU usage, you can (either one or a combination of the following actions): 1. [Disable machine learning](#disable-machine-learning), 2. [Use streaming and replication](#use-streaming-and-replication), @@ -77,19 +77,18 @@ To estimate and control memory consumption, you can (either one or a combination ### Disk footprint and I/O -By default, Netdata should not use more than 1GB of disk space, most of which is dedicated for storing metric data and -metadata. For typical installations collecting 2000 - 3000 metrics, this storage should provide a few days of +By default, Netdata shouldn’t use more than 1GB of disk space, most of which is dedicated to storing metric data and +metadata. For typical installations collecting 2000–3000 metrics, this storage should provide a few days of high-resolution retention (per second), about a month of mid-resolution retention (per minute) and more than a year of low-resolution retention (per hour). -Netdata spreads I/O operations across time. For typical standalone installations there should be a few write operations -every 5-10 seconds of a few kilobytes each, occasionally up to 1MB. In addition, under heavy load, collectors that +Netdata spreads I/O operations across time. For typical standalone installations, there should be a few write operations +every 5–10 seconds of a few kilobytes each, occasionally up to 1MB. In addition, under a heavy load, collectors that require disk I/O may stop and show gaps in charts. -To optimize your disk footprint in any aspect described below you can: +To optimize your disk footprint in any aspect described below, you can: - -To configure retention, you can: +To configure retention, you can: 1. [Change how long Netdata stores metrics](/docs/netdata-agent/configuration/optimizing-metrics-database/change-metrics-storage.md). @@ -97,7 +96,6 @@ To control disk I/O: 1. [Use a different metric storage database](/src/database/README.md), - Minimize deployment impact on the production system by optimizing disk footprint: 1. [Using streaming and replication](#use-streaming-and-replication) @@ -118,7 +116,7 @@ and makes it easier to configure or disable alerts and agent notifications. The parents by default run health checks for each child, as long as the child is connected (the details are in `stream.conf`). On the child nodes you should add to `netdata.conf` the following: -```conf +```text [health] enabled = no ``` @@ -131,19 +129,18 @@ See [using a different metric storage database](/src/database/README.md). If you know that you don't need an [entire plugin or a specific collector](/src/collectors/README.md#collector-architecture-and-terminology), -you can disable any of them. Keep in mind that if a plugin/collector has nothing to do, it simply shuts down and does -not consume system resources. You will only improve the Agent's performance by disabling plugins/collectors that are +you can disable any of them. Keep in mind that if a plugin/collector has nothing to do, it simply shuts down and doesn’t consume system resources. You will only improve the Agent's performance by disabling plugins/collectors that are actively collecting metrics. Open `netdata.conf` and scroll down to the `[plugins]` section. To disable any plugin, uncomment it and set the value to `no`. For example, to explicitly keep the `proc` and `go.d` plugins enabled while disabling `python.d` and `charts.d`. -```conf +```text [plugins] proc = yes - python.d = no - charts.d = no - go.d = yes + python.d = no + charts.d = no + go.d = yes ``` Disable specific collectors by opening their respective plugin configuration files, uncommenting the line for the @@ -157,11 +154,11 @@ sudo ./edit-config charts.d.conf For example, to disable a few Python collectors: -```conf +```text modules: - apache: no - dockerd: no - fail2ban: no + apache: no + dockerd: no + fail2ban: no ``` ## Reduce collection frequency @@ -181,7 +178,7 @@ If you change this to `2`, Netdata enforces a minimum `update every` setting of other second, which will effectively halve CPU utilization. Set this to `5` or `10` to collect metrics every 5 or 10 seconds, respectively. -```conf +```text [global] update every = 5 ``` @@ -199,7 +196,7 @@ an [internal_plugin/collector](/src/collectors/README.md#collector-architecture- open `netdata.conf` and find the appropriate section. For example, to reduce the frequency of the `apps` plugin, which collects and visualizes metrics on application resource utilization: -```conf +```text [plugin:apps] update every = 5 ``` @@ -208,7 +205,7 @@ To [configure an individual collector](/src/collectors/REFERENCE.md#configure-a- open its specific configuration file with `edit-config` and look for the `update_every` setting. For example, to reduce the frequency of the `nginx` collector, run `sudo ./edit-config go.d/nginx.conf`: -```conf +```text # [ GLOBAL ] update_every: 10 ``` @@ -229,7 +226,7 @@ on [streaming and replication](/docs/observability-centralization-points/README. Automated anomaly detection may be a powerful tool, but we recommend it to only be enabled on Netdata parents that sit outside your production infrastructure, or if you have cpu and memory to spare. You can disable ML with the following: -```conf +```text [ml] enabled = no ``` @@ -251,16 +248,15 @@ looking at the local Agent dashboard. To disable gzip compression, open `netdata.conf` and find the `[web]` section: -```conf +```text [web] enable gzip compression = no ``` Or to lower the default compression level: -```conf +```text [web] enable gzip compression = yes gzip compression level = 1 ``` - diff --git a/docs/netdata-agent/configuration/optimizing-metrics-database/README.md b/docs/netdata-agent/configuration/optimizing-metrics-database/README.md index fdbd3b690..c5769ccd4 100644 --- a/docs/netdata-agent/configuration/optimizing-metrics-database/README.md +++ b/docs/netdata-agent/configuration/optimizing-metrics-database/README.md @@ -1,3 +1,3 @@ # Optimizing Metrics Database Overview -This section contains documentation to help you understand how the metrics DB works, understand the key features and configure them to suit your needs.
\ No newline at end of file +This section contains documentation to help you understand how the metrics DB works, understand the key features and configure them to suit your needs. diff --git a/docs/netdata-agent/configuration/optimizing-metrics-database/change-metrics-storage.md b/docs/netdata-agent/configuration/optimizing-metrics-database/change-metrics-storage.md index 8a8659eff..2282cbc44 100644 --- a/docs/netdata-agent/configuration/optimizing-metrics-database/change-metrics-storage.md +++ b/docs/netdata-agent/configuration/optimizing-metrics-database/change-metrics-storage.md @@ -7,9 +7,9 @@ space**. This provides greater control and helps you optimize storage usage for | Tier | Resolution | Time Limit | Size Limit (min 256 MB) | |:----:|:-------------------:|:----------:|:-----------------------:| -| 0 | high (per second) | 14 days | 1 GiB | -| 1 | middle (per minute) | 3 months | 1 GiB | -| 2 | low (per hour) | 2 years | 1 GiB | +| 0 | high (per second) | 14d | 1 GiB | +| 1 | middle (per minute) | 3mo | 1 GiB | +| 2 | low (per hour) | 2y | 1 GiB | > **Note**: If a user sets a disk space size less than 256 MB for a tier, Netdata will automatically adjust it to 256 MB. @@ -17,7 +17,7 @@ With these defaults, Netdata requires approximately 4 GiB of storage space (incl ## Retention Settings -> **In a parent-child setup**, these settings manage the shared storage space utilized by the Netdata parent agent for +> **In a parent-child setup**, these settings manage the shared storage space used by the Netdata parent agent for > storing metrics collected by both the parent and its child nodes. You can fine-tune retention for each tier by setting a time limit or size limit. Setting a limit to 0 disables it, @@ -32,22 +32,22 @@ retention strategies as shown in the table below: You can change these limits in `netdata.conf`: -``` +```text [db] - mode = dbengine + mode = dbengine storage tiers = 3 # Tier 0, per second data. Set to 0 for no limit. - dbengine tier 0 disk space MB = 1024 - dbengine tier 0 retention days = 14 + dbengine tier 0 retention size = 1GiB + dbengine tier 0 retention time = 14d # Tier 1, per minute data. Set to 0 for no limit. - dbengine tier 1 disk space MB = 1024 - dbengine tier 1 retention days = 90 + dbengine tier 1 retention size = 1GiB + dbengine tier 1 retention time = 3mo # Tier 2, per hour data. Set to 0 for no limit. - dbengine tier 2 disk space MB = 1024 - dbengine tier 2 retention days = 730 + dbengine tier 2 retention size = 1GiB + dbengine tier 2 retention time = 2y ``` ## Monitoring Retention Utilization @@ -58,6 +58,24 @@ your storage space (disk space limits) and time (time limits) are used for metri ## Legacy configuration +### v1.99.0 and prior + +Netdata prior to v2 supports the following configuration options in `netdata.conf`. +They have the same defaults as the latest v2, but the unit of each value is given in the option name, not at the value. + +```text +storage tiers = 3 +# Tier 0, per second data. Set to 0 for no limit. +dbengine tier 0 disk space MB = 1024 +dbengine tier 0 retention days = 14 +# Tier 1, per minute data. Set to 0 for no limit. +dbengine tier 1 disk space MB = 1024 +dbengine tier 1 retention days = 90 +# Tier 2, per hour data. Set to 0 for no limit. +dbengine tier 2 disk space MB = 1024 +dbengine tier 2 retention days = 730 +``` + ### v1.45.6 and prior Netdata versions prior to v1.46.0 relied on a disk space-based retention. @@ -72,17 +90,14 @@ Netdata versions prior to v1.46.0 relied on a disk space-based retention. You can change these limits in `netdata.conf`: -``` +```text [db] - mode = dbengine + mode = dbengine storage tiers = 3 - # Tier 0, per second data dbengine multihost disk space MB = 256 - # Tier 1, per minute data dbengine tier 1 multihost disk space MB = 1024 - # Tier 2, per hour data dbengine tier 2 multihost disk space MB = 1024 ``` @@ -96,7 +111,7 @@ for the parent node and all of its children. To configure the database engine, look for the `page cache size MB` and `dbengine multihost disk space MB` settings in the `[db]` section of your `netdata.conf`. -```conf +```text [db] dbengine page cache size MB = 32 dbengine multihost disk space MB = 256 diff --git a/docs/netdata-agent/configuration/organize-systems-metrics-and-alerts.md b/docs/netdata-agent/configuration/organize-systems-metrics-and-alerts.md index b0094a60f..f7f56279b 100644 --- a/docs/netdata-agent/configuration/organize-systems-metrics-and-alerts.md +++ b/docs/netdata-agent/configuration/organize-systems-metrics-and-alerts.md @@ -1,49 +1,51 @@ # Organize systems, metrics, and alerts When you use Netdata to monitor and troubleshoot an entire infrastructure, you need sophisticated ways of keeping everything organized. -Netdata allows to organize your observability infrastructure with Spaces, Rooms, virtual nodes, host labels, and metric labels. +Netdata allows organizing your observability infrastructure with Spaces, Rooms, virtual nodes, host labels, and metric labels. ## Spaces and Rooms -[Spaces](/docs/netdata-cloud/organize-your-infrastructure-invite-your-team.md#netdata-cloud-spaces) are used for organization-level or infrastructure-level +[Spaces](/docs/netdata-cloud/organize-your-infrastructure-invite-your-team.md#netdata-cloud-spaces) are used for organization-level or infrastructure-level grouping of nodes and people. A node can only appear in a single space, while people can have access to multiple spaces. -The [Rooms](/docs/netdata-cloud/organize-your-infrastructure-invite-your-team.md#netdata-cloud-rooms) in a space bring together nodes and people in -collaboration areas. Rooms can also be used for fine-tuned -[role based access control](/docs/netdata-cloud/authentication-and-authorization/role-based-access-model.md). +The [Rooms](/docs/netdata-cloud/organize-your-infrastructure-invite-your-team.md#netdata-cloud-rooms) in a space bring together nodes and people in +collaboration areas. Rooms can also be used for fine-tuned +[role-based access control](/docs/netdata-cloud/authentication-and-authorization/role-based-access-model.md). ## Virtual nodes -Netdata’s virtual nodes functionality allows you to define nodes in configuration files and have them be treated as regular nodes -in all of the UI, dashboards, tabs, filters etc. For example, you can create a virtual node each for all your Windows machines -and monitor them as discrete entities. Virtual nodes can help you simplify your infrastructure monitoring and focus on the +Netdata’s virtual nodes functionality allows you to define nodes in configuration files and have them be treated as regular nodes +in all the UI, dashboards, tabs, filters, etc. For example, you can create a virtual node each for all your Windows machines +and monitor them as discrete entities. Virtual nodes can help you simplify your infrastructure monitoring and focus on the individual node that matters. To define your windows server as a virtual node you need to: - * Define virtual nodes in `/etc/netdata/vnodes/vnodes.conf` +* Define virtual nodes in `/etc/netdata/vnodes/vnodes.conf` ```yaml - hostname: win_server1 guid: <value> ``` - Just remember to use a valid guid (On Linux you can use `uuidgen` command to generate one, on Windows just use the `[guid]::NewGuid()` command in PowerShell) - - * Add the vnode config to the data collection job. e.g. in `go.d/windows.conf`: + + Just remember to use a valid guid (On Linux you can use `uuidgen` command to generate one, on Windows just use the `[guid]::NewGuid()` command in PowerShell) + +* Add the vnode config to the data collection job. e.g., in `go.d/windows.conf`: + ```yaml jobs: - name: win_server1 vnode: win_server1 url: http://203.0.113.10:9182/metrics ``` - + ## Host labels Host labels can be extremely useful when: -- You need alerts that adapt to the system's purpose -- You need properly-labeled metrics archiving so you can sort, correlate, and mash-up your data to your heart's content. -- You need to keep tabs on ephemeral Docker containers in a Kubernetes cluster. +* You need alerts that adapt to the system's purpose +* You need properly labeled metrics archiving so you can sort, correlate, and mash-up your data to your heart's content. +* You need to keep tabs on ephemeral Docker containers in a Kubernetes cluster. Let's take a peek into how to create host labels and apply them across a few of Netdata's features to give you more organization power over your infrastructure. @@ -56,16 +58,17 @@ parent-child status, and more. They capture the following: -- Kernel version -- Operating system name and version -- CPU architecture, system cores, CPU frequency, RAM, and disk space -- Whether Netdata is running inside of a container, and if so, the OS and hardware details about the container's host -- Whether Netdata is running inside K8s node -- What virtualization layer the system runs on top of, if any -- Whether the system is a streaming parent or child +* Kernel version +* Operating system name and version +* CPU architecture, system cores, CPU frequency, RAM, and disk space +* Whether Netdata is running inside of a container, and if so, the OS and hardware details about the container's host +* Whether Netdata is running inside K8s node +* What virtualization layer the system runs on top of, if any +* Whether the system is a streaming parent or child If you want to organize your systems without manually creating host labels, try the automatic labels in some of the features below. You can see them under `http://HOST-IP:19999/api/v1/info`, beginning with an underscore `_`. + ```json { ... @@ -87,7 +90,7 @@ sudo ./edit-config netdata.conf Create a new `[host labels]` section defining a new host label and its value for the system in question. Make sure not to violate any of the [host label naming rules](/docs/netdata-agent/configuration/common-configuration-changes.md#organize-nodes-with-host-labels). -```conf +```text [host labels] type = webserver location = us-seattle @@ -126,7 +129,6 @@ read the status of your agent. For example, from a VPS system running Debian 10: } ``` - ### Host labels in streaming You may have noticed the `_is_parent` and `_is_child` automatic labels from above. Host labels are also now @@ -134,12 +136,11 @@ streamed from a child to its parent node, which concentrates an entire infrastru and virtualization information in one place: the parent. Now, if you'd like to remind yourself of how much RAM a certain child node has, you can access -`http://localhost:19999/host/CHILD_HOSTNAME/api/v1/info` and reference the automatically-generated host labels from the +`http://localhost:19999/host/CHILD_HOSTNAME/api/v1/info` and reference the automatically generated host labels from the child system. It's a vastly simplified way of accessing critical information about your infrastructure. > ⚠️ Because automatic labels for child nodes are accessible via API calls, and contain sensitive information like -> kernel and operating system versions, you should secure streaming connections with SSL. See the [streaming -> documentation](/src/streaming/README.md#securing-streaming-communications) for details. You may also want to use +> kernel and operating system versions, you should secure streaming connections with SSL. See the [streaming documentation](/src/streaming/README.md#securing-streaming-with-tlsssl) for details. You may also want to use > [access lists](/src/web/server/README.md#access-lists) or [expose the API only to LAN/localhost > connections](/docs/netdata-agent/securing-netdata-agents.md#expose-netdata-only-in-a-private-lan). @@ -153,23 +154,23 @@ alerts to them. For example, let's use configuration example from earlier: -```conf +```text [host labels] type = webserver location = us-seattle installed = 20200218 ``` -You could now create a new health entity (checking if disk space will run out soon) that applies only to any host +You could now create a new health entity (checking if disk space runs out soon) that applies only to any host labeled `webserver`: ```yaml template: disk_fill_rate - on: disk.space - lookup: max -1s at -30m unaligned of avail - calc: ($this - $avail) / (30 * 60) - every: 15s - host labels: type = webserver + on: disk.space + lookup: max -1s at -30m unaligned of avail + calc: ($this - $avail) / (30 * 60) + every: 15s + host labels: type = webserver ``` Or, by using one of the automatic labels, for only webserver systems running a specific OS: @@ -198,9 +199,9 @@ documentation](/src/health/REFERENCE.md#alert-line-host-labels) for more details If you have enabled any metrics exporting via our experimental [exporters](/src/exporting/README.md), any new host labels you created manually are sent to the destination database alongside metrics. You can change this behavior by -editing `exporting.conf`, and you can even send automatically-generated labels on with exported metrics. +editing `exporting.conf`, and you can even send automatically generated labels on with exported metrics. -```conf +```text [exporting:global] enabled = yes send configured labels = yes @@ -209,7 +210,7 @@ send automatic labels = no You can also change this behavior per exporting connection: -```conf +```text [opentsdb:my_instance3] enabled = yes destination = localhost:4242 @@ -227,27 +228,27 @@ more about exporting, read the [documentation](/src/exporting/README.md). The Netdata aggregate charts allow you to filter and group metrics based on label name-value pairs. -All go.d plugin collectors support the specification of labels at the "collection job" level. Some collectors come with out of the box -labels (e.g. generic Prometheus collector, Kubernetes, Docker and more). But you can also add your own custom labels, by configuring -the data collection jobs. +All go.d plugin collectors support the specification of labels at the "collection job" level. Some collectors come without of the box +labels (e.g. generic Prometheus collector, Kubernetes, Docker and more). But you can also add your own custom labels by configuring +the data collection jobs. -For example, suppose we have a single Netdata agent, collecting data from two remote Apache web servers, located in different data centers. +For example, suppose we have a single Netdata agent, collecting data from two remote Apache web servers, located in different data centers. The web servers are load balanced and provide access to the service "Payments". You can define the following in `go.d.conf`, to be able to group the web requests by service or location: -``` +```yaml jobs: - - name: mywebserver1 + - name: my_webserver1 url: http://host1/server-status?auto labels: service: "Payments" location: "Atlanta" - - name: mywebserver2 + - name: my_webserver2 url: http://host2/server-status?auto labels: service: "Payments" location: "New York" ``` -Of course you may define as many custom label/value pairs as you like, in as many data collection jobs you need. +Of course, you may define as many custom label/value pairs as you like, in as many data collection jobs you need. diff --git a/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/README.md b/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/README.md index 00fe63af1..a0810bb51 100644 --- a/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/README.md +++ b/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/README.md @@ -1,7 +1,7 @@ # Running the Netdata Agent behind a reverse proxy If you need to access a Netdata agent's user interface or API in a production environment we recommend you put Netdata behind -another web server and secure access to the dashboard via SSL, user authentication and firewall rules. +another web server and secure access to the dashboard via SSL, user authentication and firewall rules. A dedicated web server also provides more robustness and capabilities than the Agent's [internal web server](/src/web/README.md). @@ -12,7 +12,7 @@ We have documented running behind [Lighttpd](/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-lighttpd.md), [Caddy](/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-caddy.md), and [H2O](/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-h2o.md). -If you prefer a different web server, we suggest you follow the documentation for nginx and tell us how you did it +If you prefer a different web server, we suggest you follow the documentation for nginx and tell us how you did it by adding your own "Running behind webserverX" document. When you run Netdata behind a reverse proxy, we recommend you firewall protect all your Netdata servers, so that only the web server IP will be allowed to directly access Netdata. To do this, run this on each of your servers (or use your firewall manager): @@ -26,9 +26,9 @@ The above will prevent anyone except your web server to access a Netdata dashboa You can also use `netdata.conf`: -``` +```text [web] - allow connections from = localhost 1.2.3.4 + allow connections from = localhost 1.2.3.4 ``` Of course, you can add more IPs. diff --git a/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-apache.md b/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-apache.md index 1f7274d5c..23e4ae233 100644 --- a/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-apache.md +++ b/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-apache.md @@ -1,4 +1,4 @@ -# Netdata via Apache's mod_proxy +# Running Netdata behind Apache's mod_proxy Below you can find instructions for configuring an apache server to: @@ -29,6 +29,7 @@ Also, enable the rewrite module: ```sh sudo a2enmod rewrite ``` + ## Netdata on an existing virtual host On any **existing** and already **working** apache virtual host, you can redirect requests for URL `/netdata/` to one or more Netdata servers. @@ -37,29 +38,29 @@ On any **existing** and already **working** apache virtual host, you can redirec Add the following on top of any existing virtual host. It will allow you to access Netdata as `http://virtual.host/netdata/`. -```conf +```text <VirtualHost *:80> - RewriteEngine On - ProxyRequests Off - ProxyPreserveHost On + RewriteEngine On + ProxyRequests Off + ProxyPreserveHost On + + <Proxy *> + Require all granted + </Proxy> - <Proxy *> - Require all granted - </Proxy> + # Local Netdata server accessed with '/netdata/', at localhost:19999 + ProxyPass "/netdata/" "http://localhost:19999/" connectiontimeout=5 timeout=30 keepalive=on + ProxyPassReverse "/netdata/" "http://localhost:19999/" - # Local Netdata server accessed with '/netdata/', at localhost:19999 - ProxyPass "/netdata/" "http://localhost:19999/" connectiontimeout=5 timeout=30 keepalive=on - ProxyPassReverse "/netdata/" "http://localhost:19999/" + # if the user did not give the trailing /, add it + # for HTTP (if the virtualhost is HTTP, use this) + RewriteRule ^/netdata$ http://%{HTTP_HOST}/netdata/ [L,R=301] + # for HTTPS (if the virtualhost is HTTPS, use this) + #RewriteRule ^/netdata$ https://%{HTTP_HOST}/netdata/ [L,R=301] - # if the user did not give the trailing /, add it - # for HTTP (if the virtualhost is HTTP, use this) - RewriteRule ^/netdata$ http://%{HTTP_HOST}/netdata/ [L,R=301] - # for HTTPS (if the virtualhost is HTTPS, use this) - #RewriteRule ^/netdata$ https://%{HTTP_HOST}/netdata/ [L,R=301] + # rest of virtual host config here - # rest of virtual host config here - </VirtualHost> ``` @@ -67,16 +68,16 @@ Add the following on top of any existing virtual host. It will allow you to acce Add the following on top of any existing virtual host. It will allow you to access multiple Netdata as `http://virtual.host/netdata/HOSTNAME/`, where `HOSTNAME` is the hostname of any other Netdata server you have (to access the `localhost` Netdata, use `http://virtual.host/netdata/localhost/`). -```conf +```text <VirtualHost *:80> - RewriteEngine On - ProxyRequests Off - ProxyPreserveHost On + RewriteEngine On + ProxyRequests Off + ProxyPreserveHost On - <Proxy *> - Require all granted - </Proxy> + <Proxy *> + Require all granted + </Proxy> # proxy any host, on port 19999 ProxyPassMatch "^/netdata/([A-Za-z0-9\._-]+)/(.*)" "http://$1:19999/$2" connectiontimeout=5 timeout=30 keepalive=on @@ -87,8 +88,8 @@ Add the following on top of any existing virtual host. It will allow you to acce # for HTTPS (if the virtualhost is HTTPS, use this) RewriteRule "^/netdata/([A-Za-z0-9\._-]+)$" https://%{HTTP_HOST}/netdata/$1/ [L,R=301] - # rest of virtual host config here - + # rest of virtual host config here + </VirtualHost> ``` @@ -97,7 +98,7 @@ Add the following on top of any existing virtual host. It will allow you to acce If you want to control the servers your users can connect to, replace the `ProxyPassMatch` line with the following. This allows only `server1`, `server2`, `server3` and `server4`. -``` +```text ProxyPassMatch "^/netdata/(server1|server2|server3|server4)/(.*)" "http://$1:19999/$2" connectiontimeout=5 timeout=30 keepalive=on ``` @@ -113,26 +114,28 @@ nano /etc/apache2/sites-available/netdata.conf with this content: -```conf +```text <VirtualHost *:80> - ProxyRequests Off - ProxyPreserveHost On - - ServerName netdata.domain.tld - <Proxy *> - Require all granted - </Proxy> + ProxyRequests Off + ProxyPreserveHost On + + ServerName netdata.domain.tld + + <Proxy *> + Require all granted + </Proxy> - ProxyPass "/" "http://localhost:19999/" connectiontimeout=5 timeout=30 keepalive=on - ProxyPassReverse "/" "http://localhost:19999/" + ProxyPass "/" "http://localhost:19999/" connectiontimeout=5 timeout=30 keepalive=on + ProxyPassReverse "/" "http://localhost:19999/" + + ErrorLog ${APACHE_LOG_DIR}/netdata-error.log + CustomLog ${APACHE_LOG_DIR}/netdata-access.log combined - ErrorLog ${APACHE_LOG_DIR}/netdata-error.log - CustomLog ${APACHE_LOG_DIR}/netdata-access.log combined </VirtualHost> ``` -Enable the VirtualHost: +Enable the VirtualHost: ```sh sudo a2ensite netdata.conf && service apache2 reload @@ -142,15 +145,15 @@ sudo a2ensite netdata.conf && service apache2 reload _Assuming the main goal is to make Netdata running in HTTPS._ -1. Make a subdomain for Netdata on which you enable and force HTTPS - You can use a free Let's Encrypt certificate -2. Go to "Apache & nginx Settings", and in the following section, add: - -```conf -RewriteEngine on -RewriteRule (.*) http://localhost:19999/$1 [P,L] -``` +1. Make a subdomain for Netdata on which you enable and force HTTPS - You can use a free Let's Encrypt certificate +2. Go to "Apache & nginx Settings", and in the following section, add: -3. Optional: If your server is remote, then just replace "localhost" with your actual hostname or IP, it just works. + ```text + RewriteEngine on + RewriteRule (.*) http://localhost:19999/$1 [P,L] + ``` + +3. Optional: If your server is remote, then just replace "localhost" with your actual hostname or IP, it just works. Repeat the operation for as many servers as you need. @@ -165,49 +168,49 @@ Then, generate password for user `netdata`, using `htpasswd -c /etc/apache2/.htp **Apache 2.2 Example:**\ Modify the virtual host with these: -```conf - # replace the <Proxy *> section - <Proxy *> - Order deny,allow - Allow from all - </Proxy> - - # add a <Location /netdata/> section - <Location /netdata/> - AuthType Basic - AuthName "Protected site" - AuthUserFile /etc/apache2/.htpasswd - Require valid-user - Order deny,allow - Allow from all - </Location> +```text + # replace the <Proxy *> section + <Proxy *> + Order deny,allow + Allow from all + </Proxy> + + # add a <Location /netdata/> section + <Location /netdata/> + AuthType Basic + AuthName "Protected site" + AuthUserFile /etc/apache2/.htpasswd + Require valid-user + Order deny,allow + Allow from all + </Location> ``` Specify `Location /` if Netdata is running on dedicated virtual host. **Apache 2.4 (dedicated virtual host) Example:** -```conf +```text <VirtualHost *:80> - RewriteEngine On - ProxyRequests Off - ProxyPreserveHost On - - ServerName netdata.domain.tld - - <Proxy *> - AllowOverride None - AuthType Basic - AuthName "Protected site" - AuthUserFile /etc/apache2/.htpasswd - Require valid-user - </Proxy> - - ProxyPass "/" "http://localhost:19999/" connectiontimeout=5 timeout=30 keepalive=on - ProxyPassReverse "/" "http://localhost:19999/" - - ErrorLog ${APACHE_LOG_DIR}/netdata-error.log - CustomLog ${APACHE_LOG_DIR}/netdata-access.log combined + RewriteEngine On + ProxyRequests Off + ProxyPreserveHost On + + ServerName netdata.domain.tld + + <Proxy *> + AllowOverride None + AuthType Basic + AuthName "Protected site" + AuthUserFile /etc/apache2/.htpasswd + Require valid-user + </Proxy> + + ProxyPass "/" "http://localhost:19999/" connectiontimeout=5 timeout=30 keepalive=on + ProxyPassReverse "/" "http://localhost:19999/" + + ErrorLog ${APACHE_LOG_DIR}/netdata-error.log + CustomLog ${APACHE_LOG_DIR}/netdata-access.log combined </VirtualHost> ``` @@ -217,8 +220,8 @@ Note: Changes are applied by reloading or restarting Apache. If you want to enable CSP within your Apache, you should consider some special requirements of the headers. Modify your configuration like that: -``` - Header always set Content-Security-Policy "default-src http: 'unsafe-inline' 'self' 'unsafe-eval'; script-src http: 'unsafe-inline' 'self' 'unsafe-eval'; style-src http: 'self' 'unsafe-inline'" +```text + Header always set Content-Security-Policy "default-src http: 'unsafe-inline' 'self' 'unsafe-eval'; script-src http: 'unsafe-inline' 'self' 'unsafe-eval'; style-src http: 'self' 'unsafe-inline'" ``` Note: Changes are applied by reloading or restarting Apache. @@ -242,7 +245,7 @@ exceed that threshold, and `mod_evasive` will add your IP address to a blocklist Our users have found success by setting `DOSPageCount` to `30`. Try this, and raise the value if you continue to see 403 errors while accessing the dashboard. -```conf +```text DOSPageCount 30 ``` @@ -255,100 +258,92 @@ To adjust the `DOSPageCount` for a specific virtual host, open your virtual host `/etc/httpd/conf/sites-available/my-domain.conf` or `/etc/apache2/sites-available/my-domain.conf` and add the following: -```conf +```text <VirtualHost *:80> - ... - # Increase the DOSPageCount to prevent 403 errors and IP addresses being blocked. - <IfModule mod_evasive20.c> - DOSPageCount 30 - </IfModule> + ... + # Increase the DOSPageCount to prevent 403 errors and IP addresses being blocked. + <IfModule mod_evasive20.c> + DOSPageCount 30 + </IfModule> </VirtualHost> ``` See issues [#2011](https://github.com/netdata/netdata/issues/2011) and [#7658](https://github.com/netdata/netdata/issues/7568) for more information. -# Netdata configuration +## Netdata configuration You might edit `/etc/netdata/netdata.conf` to optimize your setup a bit. For applying these changes you need to restart Netdata. -## Response compression +### Response compression If you plan to use Netdata exclusively via apache, you can gain some performance by preventing double compression of its output (Netdata compresses its response, apache re-compresses it) by editing `/etc/netdata/netdata.conf` and setting: -``` +```text [web] enable gzip compression = no ``` Once you disable compression at Netdata (and restart it), please verify you receive compressed responses from apache (it is important to receive compressed responses - the charts will be more snappy). -## Limit direct access to Netdata +### Limit direct access to Netdata You would also need to instruct Netdata to listen only on `localhost`, `127.0.0.1` or `::1`. -``` +```text [web] bind to = localhost ``` or -``` +```text [web] bind to = 127.0.0.1 ``` or -``` +```text [web] bind to = ::1 ``` - - You can also use a unix domain socket. This will also provide a faster route between apache and Netdata: -``` +```text [web] bind to = unix:/tmp/netdata.sock ``` Apache 2.4.24+ can not read from `/tmp` so create your socket in `/var/run/netdata` -``` +```text [web] bind to = unix:/var/run/netdata/netdata.sock ``` -_note: Netdata v1.8+ support unix domain sockets_ - At the apache side, prepend the 2nd argument to `ProxyPass` with `unix:/tmp/netdata.sock|`, like this: -``` +```text ProxyPass "/netdata/" "unix:/tmp/netdata.sock|http://localhost:19999/" connectiontimeout=5 timeout=30 keepalive=on ``` - - If your apache server is not on localhost, you can set: -``` +```text [web] bind to = * allow connections from = IP_OF_APACHE_SERVER ``` -*note: Netdata v1.9+ support `allow connections from`* - `allow connections from` accepts [Netdata simple patterns](/src/libnetdata/simple_pattern/README.md) to match against the connection IP address. ## Prevent the double access.log apache logs accesses and Netdata logs them too. You can prevent Netdata from generating its access log, by setting this in `/etc/netdata/netdata.conf`: -``` +```text [logs] access = off ``` @@ -357,7 +352,5 @@ apache logs accesses and Netdata logs them too. You can prevent Netdata from gen Make sure the requests reach Netdata, by examining `/var/log/netdata/access.log`. -1. if the requests do not reach Netdata, your apache does not forward them. -2. if the requests reach Netdata but the URLs are wrong, you have not re-written them properly. - - +1. if the requests do not reach Netdata, your apache does not forward them. +2. if the requests reach Netdata but the URLs are wrong, you have not re-written them properly. diff --git a/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-caddy.md b/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-caddy.md index b7608b309..f43a7a278 100644 --- a/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-caddy.md +++ b/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-caddy.md @@ -1,15 +1,6 @@ -<!-- -title: "Netdata via Caddy" -custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/Running-behind-caddy.md" -sidebar_label: "Netdata via Caddy" -learn_status: "Published" -learn_topic_type: "Tasks" -learn_rel_path: "Configuration/Secure your nodes" ---> +# Running Netdata behind Caddy -# Netdata via Caddy - -To run Netdata via [Caddy v2 proxying,](https://caddyserver.com/docs/caddyfile/directives/reverse_proxy) set your Caddyfile up like this: +To run Netdata via [Caddy v2 reverse proxy,](https://caddyserver.com/docs/caddyfile/directives/reverse_proxy) set your Caddyfile up like this: ```caddyfile netdata.domain.tld { @@ -34,5 +25,3 @@ netdata.domain.tld { You would also need to instruct Netdata to listen only to `127.0.0.1` or `::1`. To limit access to Netdata only from localhost, set `bind socket to IP = 127.0.0.1` or `bind socket to IP = ::1` in `/etc/netdata/netdata.conf`. - - diff --git a/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-h2o.md b/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-h2o.md index 276b72e8b..f2dc45b82 100644 --- a/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-h2o.md +++ b/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-h2o.md @@ -1,12 +1,3 @@ -<!-- -title: "Running Netdata behind H2O" -custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-h2o.md" -sidebar_label: "Running Netdata behind H2O" -learn_status: "Published" -learn_topic_type: "Tasks" -learn_rel_path: "Configuration/Secure your nodes" ---> - # Running Netdata behind H2O [H2O](https://h2o.examp1e.net/) is a new generation HTTP server that provides quicker response to users with less CPU utilization when compared to older generation of web servers. @@ -15,23 +6,23 @@ It is notable for having much simpler configuration than many popular HTTP serve ## Why H2O -- Sane configuration defaults mean that typical configurations are very minimalistic and easy to work with. +- Sane configuration defaults mean that typical configurations are very minimalistic and easy to work with. -- Native support for HTTP/2 provides improved performance when accessing the Netdata dashboard remotely. +- Native support for HTTP/2 provides improved performance when accessing the Netdata dashboard remotely. -- Password protect access to the Netdata dashboard without requiring Netdata Cloud. +- Password protect access to the Netdata dashboard without requiring Netdata Cloud. -## H2O configuration file. +## H2O configuration file -On most systems, the H2O configuration is found under `/etc/h2o`. H2O uses [YAML 1.1](https://yaml.org/spec/1.1/), with a few special extensions, for it’s configuration files, with the main configuration file being `/etc/h2o/h2o.conf`. +On most systems, the H2O configuration is found under `/etc/h2o`. H2O uses [YAML 1.1](https://yaml.org/spec/1.1/), with a few special extensions, for it’s configuration files, with the main configuration file being `/etc/h2o/h2o.conf`. You can edit the H2O configuration file with Nano, Vim or any other text editors with which you are comfortable. After making changes to the configuration files, perform the following: -- Test the configuration with `h2o -m test -c /etc/h2o/h2o.conf` +- Test the configuration with `h2o -m test -c /etc/h2o/h2o.conf` -- Restart H2O to apply tha changes with `/etc/init.d/h2o restart` or `service h2o restart` +- Restart H2O to apply tha changes with `/etc/init.d/h2o restart` or `service h2o restart` ## Ways to access Netdata via H2O @@ -52,7 +43,7 @@ hosts: ### As a subfolder of an existing virtual host -This method is recommended when Netdata is to be served from a subfolder (or directory). +This method is recommended when Netdata is to be served from a subfolder (or directory). In this case, the virtual host `netdata.example.com` already exists and Netdata has to be accessed via `netdata.example.com/netdata/`. ```yaml @@ -72,7 +63,7 @@ hosts: ### As a subfolder for multiple Netdata servers, via one H2O instance -This is the recommended configuration when one H2O instance will be used to manage multiple Netdata servers via subfolders. +This is the recommended configuration when one H2O instance will be used to manage multiple Netdata servers via sub-folders. ```yaml hosts: @@ -100,12 +91,12 @@ Of course you can add as many backend servers as you like. Using the above, you access Netdata on the backend servers, like this: -- `http://netdata.example.com/netdata/server1/` to reach Netdata on `198.51.100.1:19999` -- `http://netdata.example.com/netdata/server2/` to reach Netdata on `198.51.100.2:19999` +- `http://netdata.example.com/netdata/server1/` to reach Netdata on `198.51.100.1:19999` +- `http://netdata.example.com/netdata/server2/` to reach Netdata on `198.51.100.2:19999` ### Encrypt the communication between H2O and Netdata -In case Netdata's web server has been [configured to use TLS](/src/web/server/README.md#enabling-tls-support), it is +In case Netdata's web server has been [configured to use TLS](/src/web/server/README.md#enable-httpstls-support), it is necessary to specify inside the H2O configuration that the final destination is using TLS. To do this, change the `http://` on the `proxy.reverse.url` line in your H2O configuration with `https://` @@ -141,31 +132,27 @@ For more information on using basic authentication with H2O, see [their official If your H2O server is on `localhost`, you can use this to ensure external access is only possible through H2O: -``` +```text [web] bind to = 127.0.0.1 ::1 ``` - - You can also use a unix domain socket. This will provide faster communication between H2O and Netdata as well: -``` +```text [web] bind to = unix:/run/netdata/netdata.sock ``` In the H2O configuration, use a line like the following to connect to Netdata via the unix socket: -```yaml +```text proxy.reverse.url http://[unix:/run/netdata/netdata.sock] ``` - - If your H2O server is not on localhost, you can set: -``` +```text [web] bind to = * allow connections from = IP_OF_H2O_SERVER @@ -181,7 +168,7 @@ the connection IP address. H2O logs accesses and Netdata logs them too. You can prevent Netdata from generating its access log, by setting this in `/etc/netdata/netdata.conf`: -``` +```text [logs] access = off ``` diff --git a/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-haproxy.md b/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-haproxy.md index 9d2aff670..04bd32838 100644 --- a/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-haproxy.md +++ b/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-haproxy.md @@ -1,16 +1,6 @@ -<!-- -title: "Netdata via HAProxy" -custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-haproxy.md" -sidebar_label: "Netdata via HAProxy" -learn_status: "Published" -learn_topic_type: "Tasks" -learn_rel_path: "Configuration/Secure your nodes" ---> - -# Netdata via HAProxy - -> HAProxy is a free, very fast and reliable solution offering high availability, load balancing, -> and proxying for TCP and HTTP-based applications. It is particularly suited for very high traffic websites +# Running Netdata behind HAProxy + +> HAProxy is a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and HTTP-based applications. It is particularly suited for very high traffic websites > and powers quite a number of the world's most visited ones. If Netdata is running on a host running HAProxy, rather than connecting to Netdata from a port number, a domain name can @@ -18,14 +8,14 @@ be pointed at HAProxy, and HAProxy can redirect connections to the Netdata port. Netdata at `https://example.com` or `https://example.com/netdata/`, which is a much nicer experience then `http://example.com:19999`. -To proxy requests from [HAProxy](https://github.com/haproxy/haproxy) to Netdata, +To proxy requests from [HAProxy](https://github.com/haproxy/haproxy) to Netdata, the following configuration can be used: ## Default Configuration For all examples, set the mode to `http` -```conf +```text defaults mode http ``` @@ -38,7 +28,7 @@ A simple example where the base URL, say `http://example.com`, is used with no s Create a frontend to receive the request. -```conf +```text frontend http_frontend ## HTTP ipv4 and ipv6 on all ips ## bind :::80 v4v6 @@ -50,7 +40,7 @@ frontend http_frontend Create the Netdata backend which will send requests to port `19999`. -```conf +```text backend netdata_backend option forwardfor server netdata_local 127.0.0.1:19999 @@ -69,7 +59,7 @@ An example where the base URL is used with a subpath `/netdata/`: To use a subpath, create an ACL, which will set a variable based on the subpath. -```conf +```text frontend http_frontend ## HTTP ipv4 and ipv6 on all ips ## bind :::80 v4v6 @@ -92,7 +82,7 @@ frontend http_frontend Same as simple example, except remove `/netdata/` with regex. -```conf +```text backend netdata_backend option forwardfor server netdata_local 127.0.0.1:19999 @@ -107,14 +97,14 @@ backend netdata_backend ## Using TLS communication -TLS can be used by adding port `443` and a cert to the frontend. +TLS can be used by adding port `443` and a cert to the frontend. This example will only use Netdata if host matches example.com (replace with your domain). ### Frontend This frontend uses a certificate list. -```conf +```text frontend https_frontend ## HTTP ## bind :::80 v4v6 @@ -139,11 +129,11 @@ In the cert list file place a mapping from a certificate file to the domain used `/etc/letsencrypt/certslist.txt`: -```txt +```text example.com /etc/letsencrypt/live/example.com/example.com.pem ``` -The file `/etc/letsencrypt/live/example.com/example.com.pem` should contain the key and +The file `/etc/letsencrypt/live/example.com/example.com.pem` should contain the key and certificate (in that order) concatenated into a `.pem` file.: ```sh @@ -156,7 +146,7 @@ cat /etc/letsencrypt/live/example.com/fullchain.pem \ Same as simple, except set protocol `https`. -```conf +```text backend netdata_backend option forwardfor server netdata_local 127.0.0.1:19999 @@ -172,30 +162,30 @@ backend netdata_backend To use basic HTTP Authentication, create an authentication list: -```conf +```text # HTTP Auth userlist basic-auth-list group is-admin # Plaintext password - user admin password passwordhere groups is-admin + user admin password YOUR_PASSWORD groups is-admin ``` You can create a hashed password using the `mkpassword` utility. ```sh - printf "passwordhere" | mkpasswd --stdin --method=sha-256 + printf "YOUR_PASSWORD" | mkpasswd --stdin --method=sha-256 $5$l7Gk0VPIpKO$f5iEcxvjfdF11khw.utzSKqP7W.0oq8wX9nJwPLwzy1 ``` -Replace `passwordhere` with hash: +Replace `YOUR_PASSWORD` with hash: -```conf +```text user admin password $5$l7Gk0VPIpKO$f5iEcxvjfdF11khw.utzSKqP7W.0oq8wX9nJwPLwzy1 groups is-admin ``` Now add at the top of the backend: -```conf +```text acl devops-auth http_auth_group(basic-auth-list) is-admin http-request auth realm netdata_local unless devops-auth ``` @@ -204,7 +194,7 @@ http-request auth realm netdata_local unless devops-auth Full example configuration with HTTP auth over TLS with subpath: -```conf +```text global maxconn 20000 @@ -293,5 +283,3 @@ backend netdata_backend http-request set-header X-Forwarded-Port %[dst_port] http-request set-header Connection "keep-alive" ``` - - diff --git a/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-lighttpd.md b/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-lighttpd.md index 637bc0642..48b9b2c93 100644 --- a/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-lighttpd.md +++ b/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-lighttpd.md @@ -1,26 +1,17 @@ -<!-- -title: "Netdata via lighttpd v1.4.x" -custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-lighttpd.md" -sidebar_label: "Netdata via lighttpd v1.4.x" -learn_status: "Published" -learn_topic_type: "Tasks" -learn_rel_path: "Configuration/Secure your nodes" ---> - -# Netdata via lighttpd v1.4.x +# Running Netdata behind lighttpd v1.4.x Here is a config for accessing Netdata in a suburl via lighttpd 1.4.46 and newer: -```txt +```text $HTTP["url"] =~ "^/netdata/" { proxy.server = ( "" => ("netdata" => ( "host" => "127.0.0.1", "port" => 19999 ))) proxy.header = ( "map-urlpath" => ( "/netdata/" => "/") ) } ``` -If you have older lighttpd you have to use a chain (such as below), as explained [at this stackoverflow answer](http://stackoverflow.com/questions/14536554/lighttpd-configuration-to-proxy-rewrite-from-one-domain-to-another). +If you have older lighttpd you have to use a chain (such as below), as explained [at this Stack Overflow answer](http://stackoverflow.com/questions/14536554/lighttpd-configuration-to-proxy-rewrite-from-one-domain-to-another). -```txt +```text $HTTP["url"] =~ "^/netdata/" { proxy.server = ( "" => ("" => ( "host" => "127.0.0.1", "port" => 19998 ))) } @@ -31,19 +22,16 @@ $SERVER["socket"] == ":19998" { } ``` - - If the only thing the server is exposing via the web is Netdata (and thus no suburl rewriting required), then you can get away with just -``` +```text proxy.server = ( "" => ( ( "host" => "127.0.0.1", "port" => 19999 ))) ``` -Though if it's public facing you might then want to put some authentication on it. htdigest support -looks like: +Though if it's public facing you might then want to put some authentication on it. `htdigest` support looks like: -``` +```text auth.backend = "htdigest" auth.backend.htdigest.userfile = "/etc/lighttpd/lighttpd.htdigest" auth.require = ( "" => ( "method" => "digest", @@ -55,14 +43,12 @@ auth.require = ( "" => ( "method" => "digest", other auth methods, and more info on htdigest, can be found in lighttpd's [mod_auth docs](http://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_ModAuth). - - It seems that lighttpd (or some versions of it), fail to proxy compressed web responses. To solve this issue, disable web response compression in Netdata. -Open `/etc/netdata/netdata.conf` and set in [global]\: +Open `/etc/netdata/netdata.conf` and set in `[global]`: -``` +```text enable web responses gzip compression = no ``` @@ -71,5 +57,3 @@ enable web responses gzip compression = no You would also need to instruct Netdata to listen only to `127.0.0.1` or `::1`. To limit access to Netdata only from localhost, set `bind socket to IP = 127.0.0.1` or `bind socket to IP = ::1` in `/etc/netdata/netdata.conf`. - - diff --git a/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-nginx.md b/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-nginx.md index f2dd137dd..c0364633a 100644 --- a/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-nginx.md +++ b/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-nginx.md @@ -2,19 +2,19 @@ ## Intro -[Nginx](https://nginx.org/en/) is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server used to host websites and applications of all sizes. +[Nginx](https://nginx.org/en/) is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server used to host websites and applications of all sizes. The software is known for its low impact on memory resources, high scalability, and its modular, event-driven architecture which can offer secure, predictable performance. ## Why Nginx -- By default, Nginx is fast and lightweight out of the box. +- By default, Nginx is fast and lightweight out of the box. -- Nginx is used and useful in cases when you want to access different instances of Netdata from a single server. +- Nginx is used and useful in cases when you want to access different instances of Netdata from a single server. -- Password-protect access to Netdata, until distributed authentication is implemented via the Netdata cloud Sign In mechanism. +- Password-protect access to Netdata, until distributed authentication is implemented via the Netdata cloud Sign In mechanism. -- A proxy was necessary to encrypt the communication to Netdata, until v1.16.0, which provided TLS (HTTPS) support. +- A proxy was necessary to encrypt the communication to Netdata, until v1.16.0, which provided TLS (HTTPS) support. ## Nginx configuration file @@ -22,23 +22,23 @@ All Nginx configurations can be found in the `/etc/nginx/` directory. The main c Configuration options in Nginx are known as directives. Directives are organized into groups known as blocks or contexts. The two terms can be used interchangeably. -Depending on your installation source, you’ll find an example configuration file at `/etc/nginx/conf.d/default.conf` or `etc/nginx/sites-enabled/default`, in some cases you may have to manually create the `sites-available` and `sites-enabled` directories. +Depending on your installation source, you’ll find an example configuration file at `/etc/nginx/conf.d/default.conf` or `etc/nginx/sites-enabled/default`, in some cases you may have to manually create the `sites-available` and `sites-enabled` directories. You can edit the Nginx configuration file with Nano, Vim or any other text editors you are comfortable with. After making changes to the configuration files: -- Test Nginx configuration with `nginx -t`. +- Test Nginx configuration with `nginx -t`. -- Restart Nginx to effect the change with `/etc/init.d/nginx restart` or `service nginx restart`. +- Restart Nginx to effect the change with `/etc/init.d/nginx restart` or `service nginx restart`. ## Ways to access Netdata via Nginx ### As a virtual host -With this method instead of `SERVER_IP_ADDRESS:19999`, the Netdata dashboard can be accessed via a human-readable URL such as `netdata.example.com` used in the configuration below. +With this method instead of `SERVER_IP_ADDRESS:19999`, the Netdata dashboard can be accessed via a human-readable URL such as `netdata.example.com` used in the configuration below. -```conf +```text upstream backend { # the Netdata server server 127.0.0.1:19999; @@ -69,10 +69,10 @@ server { ### As a subfolder to an existing virtual host -This method is recommended when Netdata is to be served from a subfolder (or directory). +This method is recommended when Netdata is to be served from a subfolder (or directory). In this case, the virtual host `netdata.example.com` already exists and Netdata has to be accessed via `netdata.example.com/netdata/`. -```conf +```text upstream netdata { server 127.0.0.1:19999; keepalive 64; @@ -112,9 +112,9 @@ server { ### As a subfolder for multiple Netdata servers, via one Nginx -This is the recommended configuration when one Nginx will be used to manage multiple Netdata servers via subfolders. +This is the recommended configuration when one Nginx will be used to manage multiple Netdata servers via sub-folders. -```conf +```text upstream backend-server1 { server 10.1.1.103:19999; keepalive 64; @@ -159,16 +159,16 @@ Of course you can add as many backend servers as you like. Using the above, you access Netdata on the backend servers, like this: -- `http://netdata.example.com/netdata/server1/` to reach `backend-server1` -- `http://netdata.example.com/netdata/server2/` to reach `backend-server2` +- `http://netdata.example.com/netdata/server1/` to reach `backend-server1` +- `http://netdata.example.com/netdata/server2/` to reach `backend-server2` ### Encrypt the communication between Nginx and Netdata -In case Netdata's web server has been [configured to use TLS](/src/web/server/README.md#enabling-tls-support), it is +In case Netdata's web server has been [configured to use TLS](/src/web/server/README.md#enable-httpstls-support), it is necessary to specify inside the Nginx configuration that the final destination is using TLS. To do this, please, append the following parameters in your `nginx.conf` -```conf +```text proxy_set_header X-Forwarded-Proto https; proxy_pass https://localhost:19999; ``` @@ -189,7 +189,7 @@ printf "yourusername:$(openssl passwd -apr1)" > /etc/nginx/passwords And then enable the authentication inside your server directive: -```conf +```text server { # ... auth_basic "Protected"; @@ -202,40 +202,35 @@ server { If your Nginx is on `localhost`, you can use this to protect your Netdata: -``` +```text [web] bind to = 127.0.0.1 ::1 ``` You can also use a unix domain socket. This will also provide a faster route between Nginx and Netdata: -``` +```text [web] bind to = unix:/var/run/netdata/netdata.sock ``` -*note: Netdata v1.8+ support unix domain sockets* - At the Nginx side, use something like this to use the same unix domain socket: -```conf +```text upstream backend { server unix:/var/run/netdata/netdata.sock; keepalive 64; } ``` - If your Nginx server is not on localhost, you can set: -``` +```text [web] bind to = * allow connections from = IP_OF_NGINX_SERVER ``` -*note: Netdata v1.9+ support `allow connections from`* - `allow connections from` accepts [Netdata simple patterns](/src/libnetdata/simple_pattern/README.md) to match against the connection IP address. @@ -243,7 +238,7 @@ connection IP address. Nginx logs accesses and Netdata logs them too. You can prevent Netdata from generating its access log, by setting this in `/etc/netdata/netdata.conf`: -``` +```text [logs] access = off ``` @@ -252,18 +247,18 @@ Nginx logs accesses and Netdata logs them too. You can prevent Netdata from gene By default, netdata compresses its responses. You can have nginx do that instead, with the following options in the `location /` block: -```conf - location / { - ... - gzip on; - gzip_proxied any; - gzip_types *; - } +```text +location / { + ... + gzip on; + gzip_proxied any; + gzip_types *; +} ``` To disable Netdata's gzip compression, open `netdata.conf` and in the `[web]` section put: -```conf +```text [web] enable gzip compression = no ``` @@ -278,5 +273,3 @@ If you get an 502 Bad Gateway error you might check your Nginx error log: ``` If you see something like the above, chances are high that SELinux prevents nginx from connecting to the backend server. To fix that, just use this policy: `setsebool -P httpd_can_network_connect true`. - - |