diff options
Diffstat (limited to 'docs/netdata-agent/configuration/optimize-the-netdata-agents-performance.md')
| -rw-r--r-- | docs/netdata-agent/configuration/optimize-the-netdata-agents-performance.md | 80 |
1 files changed, 38 insertions, 42 deletions
diff --git a/docs/netdata-agent/configuration/optimize-the-netdata-agents-performance.md b/docs/netdata-agent/configuration/optimize-the-netdata-agents-performance.md index 6acbd4977..ff51fbf78 100644 --- a/docs/netdata-agent/configuration/optimize-the-netdata-agents-performance.md +++ b/docs/netdata-agent/configuration/optimize-the-netdata-agents-performance.md @@ -1,9 +1,9 @@ # How to optimize the Netdata Agent's performance We designed the Netdata Agent to be incredibly lightweight, even when it's collecting a few thousand dimensions every -second and visualizing that data into hundreds of charts. However, the default settings of the Netdata Agent are not -optimized for performance, but for a simple, standalone setup. We want the first install to give you something you can -run without any configuration. Most of the settings and options are enabled, since we want you to experience the full +second and visualizing that data into hundreds of charts. However, the default settings of the Netdata Agent aren’t +optimized for performance, but for a simple, standalone setup. We want the first installation to give you something you can +run without any configuration. Most of the settings and options are enabled since we want you to experience the full thing. By default, Netdata will automatically detect applications running on the node it is installed to start collecting @@ -17,16 +17,16 @@ Netdata for production use. The following table summarizes the effect of each optimization on the CPU, RAM and Disk IO utilization in production. -| Optimization | CPU | RAM | Disk IO | -|-------------------------------------------------------------------------------------------------------------------------------|--------------------|--------------------|--------------------| -| [Use streaming and replication](#use-streaming-and-replication) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | -| [Disable unneeded plugins or collectors](#disable-unneeded-plugins-or-collectors) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | -| [Reduce data collection frequency](#reduce-collection-frequency) | :heavy_check_mark: | | :heavy_check_mark: | +| Optimization | CPU | RAM | Disk IO | +|-----------------------------------------------------------------------------------------------------------------------------------|--------------------|--------------------|--------------------| +| [Use streaming and replication](#use-streaming-and-replication) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | +| [Disable unneeded plugins or collectors](#disable-unneeded-plugins-or-collectors) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | +| [Reduce data collection frequency](#reduce-collection-frequency) | :heavy_check_mark: | | :heavy_check_mark: | | [Change how long Netdata stores metrics](/docs/netdata-agent/configuration/optimizing-metrics-database/change-metrics-storage.md) | | :heavy_check_mark: | :heavy_check_mark: | -| [Use a different metric storage database](/src/database/README.md) | | :heavy_check_mark: | :heavy_check_mark: | -| [Disable machine learning](#disable-machine-learning) | :heavy_check_mark: | | | -| [Use a reverse proxy](#run-netdata-behind-a-proxy) | :heavy_check_mark: | | | -| [Disable/lower gzip compression for the agent dashboard](#disablelower-gzip-compression-for-the-dashboard) | :heavy_check_mark: | | | +| [Use a different metric storage database](/src/database/README.md) | | :heavy_check_mark: | :heavy_check_mark: | +| [Disable machine learning](#disable-machine-learning) | :heavy_check_mark: | | | +| [Use a reverse proxy](#run-netdata-behind-a-proxy) | :heavy_check_mark: | | | +| [Disable/lower gzip compression for the agent dashboard](#disablelower-gzip-compression-for-the-dashboard) | :heavy_check_mark: | | | ## Resources required by a default Netdata installation @@ -39,15 +39,15 @@ You can configure almost all aspects of data collection/retention, and certain a Expect about: - 1-3% of a single core for the netdata core -- 1-3% of a single core for the various collectors (e.g. go.d.plugin, apps.plugin) +- 1-3% of a single core for the various collectors (e.g., go.d.plugin, apps.plugin) - 5-10% of a single core, when ML training runs Your experience may vary depending on the number of metrics collected, the collectors enabled and the specific -environment they run on, i.e. the work they have to do to collect these metrics. +environment they run on, i.e., the work they have to do to collect these metrics. As a general rule, for modern hardware and VMs, the total CPU consumption of a standalone Netdata installation, including all its components, should be below 5 - 15% of a single core. For example, on 8 core server it will use only -0.6% - 1.8% of a total CPU capacity, depending on the CPU characteristics. +0.6% - 1.8% of the total CPU capacity, depending on the CPU characteristics. The Netdata Agent runs with the lowest possible [process scheduling policy](/src/daemon/README.md#netdata-process-scheduling-policy), @@ -55,7 +55,7 @@ which is `nice 19`, and uses the `idle` process scheduler. Together, these setti resources when the node has CPU resources to space. If the node reaches 100% CPU utilization, the Agent is stopped first to ensure your applications get any available resources. -To reduce CPU usage you can (either one or a combination of the following actions): +To reduce CPU usage, you can (either one or a combination of the following actions): 1. [Disable machine learning](#disable-machine-learning), 2. [Use streaming and replication](#use-streaming-and-replication), @@ -77,19 +77,18 @@ To estimate and control memory consumption, you can (either one or a combination ### Disk footprint and I/O -By default, Netdata should not use more than 1GB of disk space, most of which is dedicated for storing metric data and -metadata. For typical installations collecting 2000 - 3000 metrics, this storage should provide a few days of +By default, Netdata shouldn’t use more than 1GB of disk space, most of which is dedicated to storing metric data and +metadata. For typical installations collecting 2000–3000 metrics, this storage should provide a few days of high-resolution retention (per second), about a month of mid-resolution retention (per minute) and more than a year of low-resolution retention (per hour). -Netdata spreads I/O operations across time. For typical standalone installations there should be a few write operations -every 5-10 seconds of a few kilobytes each, occasionally up to 1MB. In addition, under heavy load, collectors that +Netdata spreads I/O operations across time. For typical standalone installations, there should be a few write operations +every 5–10 seconds of a few kilobytes each, occasionally up to 1MB. In addition, under a heavy load, collectors that require disk I/O may stop and show gaps in charts. -To optimize your disk footprint in any aspect described below you can: +To optimize your disk footprint in any aspect described below, you can: - -To configure retention, you can: +To configure retention, you can: 1. [Change how long Netdata stores metrics](/docs/netdata-agent/configuration/optimizing-metrics-database/change-metrics-storage.md). @@ -97,7 +96,6 @@ To control disk I/O: 1. [Use a different metric storage database](/src/database/README.md), - Minimize deployment impact on the production system by optimizing disk footprint: 1. [Using streaming and replication](#use-streaming-and-replication) @@ -118,7 +116,7 @@ and makes it easier to configure or disable alerts and agent notifications. The parents by default run health checks for each child, as long as the child is connected (the details are in `stream.conf`). On the child nodes you should add to `netdata.conf` the following: -```conf +```text [health] enabled = no ``` @@ -131,19 +129,18 @@ See [using a different metric storage database](/src/database/README.md). If you know that you don't need an [entire plugin or a specific collector](/src/collectors/README.md#collector-architecture-and-terminology), -you can disable any of them. Keep in mind that if a plugin/collector has nothing to do, it simply shuts down and does -not consume system resources. You will only improve the Agent's performance by disabling plugins/collectors that are +you can disable any of them. Keep in mind that if a plugin/collector has nothing to do, it simply shuts down and doesn’t consume system resources. You will only improve the Agent's performance by disabling plugins/collectors that are actively collecting metrics. Open `netdata.conf` and scroll down to the `[plugins]` section. To disable any plugin, uncomment it and set the value to `no`. For example, to explicitly keep the `proc` and `go.d` plugins enabled while disabling `python.d` and `charts.d`. -```conf +```text [plugins] proc = yes - python.d = no - charts.d = no - go.d = yes + python.d = no + charts.d = no + go.d = yes ``` Disable specific collectors by opening their respective plugin configuration files, uncommenting the line for the @@ -157,11 +154,11 @@ sudo ./edit-config charts.d.conf For example, to disable a few Python collectors: -```conf +```text modules: - apache: no - dockerd: no - fail2ban: no + apache: no + dockerd: no + fail2ban: no ``` ## Reduce collection frequency @@ -181,7 +178,7 @@ If you change this to `2`, Netdata enforces a minimum `update every` setting of other second, which will effectively halve CPU utilization. Set this to `5` or `10` to collect metrics every 5 or 10 seconds, respectively. -```conf +```text [global] update every = 5 ``` @@ -199,7 +196,7 @@ an [internal_plugin/collector](/src/collectors/README.md#collector-architecture- open `netdata.conf` and find the appropriate section. For example, to reduce the frequency of the `apps` plugin, which collects and visualizes metrics on application resource utilization: -```conf +```text [plugin:apps] update every = 5 ``` @@ -208,7 +205,7 @@ To [configure an individual collector](/src/collectors/REFERENCE.md#configure-a- open its specific configuration file with `edit-config` and look for the `update_every` setting. For example, to reduce the frequency of the `nginx` collector, run `sudo ./edit-config go.d/nginx.conf`: -```conf +```text # [ GLOBAL ] update_every: 10 ``` @@ -229,7 +226,7 @@ on [streaming and replication](/docs/observability-centralization-points/README. Automated anomaly detection may be a powerful tool, but we recommend it to only be enabled on Netdata parents that sit outside your production infrastructure, or if you have cpu and memory to spare. You can disable ML with the following: -```conf +```text [ml] enabled = no ``` @@ -251,16 +248,15 @@ looking at the local Agent dashboard. To disable gzip compression, open `netdata.conf` and find the `[web]` section: -```conf +```text [web] enable gzip compression = no ``` Or to lower the default compression level: -```conf +```text [web] enable gzip compression = yes gzip compression level = 1 ``` - |