diff options
author | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-05-05 12:08:03 +0000 |
---|---|---|
committer | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-05-05 12:08:18 +0000 |
commit | 5da14042f70711ea5cf66e034699730335462f66 (patch) | |
tree | 0f6354ccac934ed87a2d555f45be4c831cf92f4a /src/health/guides/kubelet/kubelet_1m_pleg_relist_latency_quantile_09.md | |
parent | Releasing debian version 1.44.3-2. (diff) | |
download | netdata-5da14042f70711ea5cf66e034699730335462f66.tar.xz netdata-5da14042f70711ea5cf66e034699730335462f66.zip |
Merging upstream version 1.45.3+dfsg.
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'src/health/guides/kubelet/kubelet_1m_pleg_relist_latency_quantile_09.md')
-rw-r--r-- | src/health/guides/kubelet/kubelet_1m_pleg_relist_latency_quantile_09.md | 45 |
1 files changed, 45 insertions, 0 deletions
diff --git a/src/health/guides/kubelet/kubelet_1m_pleg_relist_latency_quantile_09.md b/src/health/guides/kubelet/kubelet_1m_pleg_relist_latency_quantile_09.md new file mode 100644 index 000000000..6c71f1cf6 --- /dev/null +++ b/src/health/guides/kubelet/kubelet_1m_pleg_relist_latency_quantile_09.md @@ -0,0 +1,45 @@ +### Understand the alert + +This alert calculates the average Pod Lifecycle Event Generator (PLEG) relisting latency over the period of one minute, using the 0.9 quantile. This alert is related to Kubelet, a critical component in the Kubernetes cluster that ensures the correct running of containers inside pods. If you receive this alert, it means that the relisting latency has increased in your Kubernetes cluster, possibly affecting the performance of your workloads. + +### What does PLEG relisting latency mean? + +In Kubernetes, PLEG is responsible for keeping track of container lifecycle events, such as container start, stop, or pause. It periodically relists these events and updates the Kubernetes Pod status, ensuring the scheduler and other components know the correct state of the containers. An increased relisting latency could lead to slower updates on Pod status and overall degraded performance. + +### What does 0.9 quantile mean? + +The 0.9 quantile represents the value below which 90% of the latencies are. An alert based on the 0.9 quantile suggests that 90% of relisting latencies are below the specified threshold, meaning that the remaining 10% are experiencing increased latency, which could lead to issues in your cluster. + +### Troubleshoot the alert + +1. Check Kubelet logs for errors or warnings related to PLEG: + + Access the logs of the Kubelet component running on the affected node: + + ``` + sudo journalctl -u kubelet + ``` + +2. Monitor the overall performance of your Kubernetes cluster: + + Use `kubectl top nodes` to check the resource usage of your nodes and identify any bottlenecks, such as high CPU or memory consumption. + +3. Check the status of Pods: + + Use `kubectl get pods --all-namespaces` to check the status of all Pods in your cluster. Look for Pods in an abnormal state (e.g., Pending, CrashLoopBackOff, or Terminating), which could be related to high PLEG relisting latency. + +4. Analyze Pod logs for issues: + + Investigate the logs of the affected Pods to understand any issues with the container lifecycle events: + + ``` + kubectl logs <pod-name> -n <namespace> + ``` + +5. Review the Kubelet configuration: + + Ensure that your Kubelet configuration is set up correctly to handle your workloads. If necessary, adjust the settings to improve PLEG relisting performance. + +### Useful resources + +1. [Kubernetes Troubleshooting Guide](https://kubernetes.io/docs/tasks/debug-application-cluster/debug-cluster/) |