diff options
author | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-04-19 02:57:58 +0000 |
---|---|---|
committer | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-04-19 02:57:58 +0000 |
commit | be1c7e50e1e8809ea56f2c9d472eccd8ffd73a97 (patch) | |
tree | 9754ff1ca740f6346cf8483ec915d4054bc5da2d /health/guides/netdev/1min_netdev_backlog_exceeded.md | |
parent | Initial commit. (diff) | |
download | netdata-be1c7e50e1e8809ea56f2c9d472eccd8ffd73a97.tar.xz netdata-be1c7e50e1e8809ea56f2c9d472eccd8ffd73a97.zip |
Adding upstream version 1.44.3.upstream/1.44.3upstream
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'health/guides/netdev/1min_netdev_backlog_exceeded.md')
-rw-r--r-- | health/guides/netdev/1min_netdev_backlog_exceeded.md | 44 |
1 files changed, 44 insertions, 0 deletions
diff --git a/health/guides/netdev/1min_netdev_backlog_exceeded.md b/health/guides/netdev/1min_netdev_backlog_exceeded.md new file mode 100644 index 00000000..dc7b6a2c --- /dev/null +++ b/health/guides/netdev/1min_netdev_backlog_exceeded.md @@ -0,0 +1,44 @@ +### Understand the alert + +The linux kernel contains queues where packets are stored after reception from a network interface controller before being processed by the next protocol stack. There is one netdev backlog queue per CPU core. netdev_max_backlog defines the maximum number of packets that can enter the queue. Queues fill up when an interface receives packets faster than kernel can process them. The default netdev_max_backlog value should be 1000. However this may not be enough in cases such as: + +- Multiple interfaces operating at 1Gbps, or even a single interface at 10Gbps. + +- Lower powered systems process very large amounts of network traffic. + +Netdata monitors the average number of dropped packets in the last minute due to exceeding the netdev backlog queue. + +### Troubleshoot the alert + +- Increase the netdev_max_backlog value + +1. Check your current value: + + ``` + root@netdata~ # sysctl net.core.netdev_max_backlog + net.core.netdev_max_backlog = 1000 + ``` + +2. Try to increase it by a factor of 2. + + ``` + root@netdata~ # sysctl -w net.core.netdev_max_backlog=2000 + ``` + +3. Verify the change and test with the same workload that triggered the alarm originally. + + ``` + root@netdata~ # sysctl net.core.netdev_max_backlog + net.core.netdev_max_backlog = 2000 + ``` + +4. If this change works for your system, you could make it permanently. + + Bump this `net.core.netdev_max_backlog=2000` entry under `/etc/sysctl.conf`. + +5. Reload the sysctl settings. + + ``` + root@netdata~ # sysctl -p + ``` + |