From b485aab7e71c1625cfc27e0f92c9509f42378458 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Sun, 5 May 2024 13:19:16 +0200 Subject: Adding upstream version 1.45.3+dfsg. Signed-off-by: Daniel Baumann --- .../beanstalk/beanstalk_server_buried_jobs.md | 32 ++++++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 src/health/guides/beanstalk/beanstalk_server_buried_jobs.md (limited to 'src/health/guides/beanstalk/beanstalk_server_buried_jobs.md') diff --git a/src/health/guides/beanstalk/beanstalk_server_buried_jobs.md b/src/health/guides/beanstalk/beanstalk_server_buried_jobs.md new file mode 100644 index 000000000..99d4f5074 --- /dev/null +++ b/src/health/guides/beanstalk/beanstalk_server_buried_jobs.md @@ -0,0 +1,32 @@ +### Understand the alert + +This alert is related to the `Beanstalk` message queue system and is triggered when there are buried jobs in the queue across all tubes. A buried job is one that has encountered an issue during processing by the consumer, so it remains in the queue waiting for manual action. This alert is raised in a warning state if there are more than 0 buried jobs and in a critical state if there are more than 10. + +### What are buried jobs? + +Buried jobs are tasks that have faced an error or issue during processing by the consumer, and as a result, have been `buried`. This means these jobs remain in the queue, awaiting manual intervention for them to be processed again. The presence of buried jobs does not affect the processing of new jobs in the queue. + +### Troubleshoot the alert + +1. Identify the buried jobs: Use the `beanstalk-tool` to inspect the Beanstalk server and list the buried jobs in the tubes. If you don't have `beanstalk-tool`, install it using pip: + + ``` + pip install beanstalkc + beanstalk-tool : stats_tube + ``` + +2. Examine the buried jobs: To investigate the cause of the buried jobs, find related logs, either from the Beanstalk server or from the consumer application. Analyzing the logs can lead to the root cause of the problem. + +3. Fix the issue: Once you identify the cause, resolve the issue in either the consumer application, or if necessary, in the Beanstalk server configuration. + +4. Kick the buried jobs: After resolving the issue, you need to manually kick the buried jobs back into the queue for processing. Use the following command with `beanstalk-tool`: + + ``` + beanstalk-tool : kick --tube= + ``` + +5. Monitor the queue: After kicking the buried jobs, monitor the queue and ensure that the jobs are processed without encountering more errors. + +### Useful resources + +1. [Beanstalk Documentation](https://beanstalkd.github.io/) -- cgit v1.2.3