From 517a443636daa1e8085cb4e5325524a54e8a8fd7 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Tue, 17 Oct 2023 11:30:23 +0200 Subject: Merging upstream version 1.43.0. Signed-off-by: Daniel Baumann --- health/notifications/alerta/README.md | 132 +++++++++++++++++++++++----------- 1 file changed, 92 insertions(+), 40 deletions(-) (limited to 'health/notifications/alerta') diff --git a/health/notifications/alerta/README.md b/health/notifications/alerta/README.md index 237b9a78e..48f9f35a4 100644 --- a/health/notifications/alerta/README.md +++ b/health/notifications/alerta/README.md @@ -1,52 +1,75 @@ -# Alerta Agent alert notifications + -Learn how to send notifications to Alerta using Netdata's Agent alert notification feature, which supports dozens of endpoints, user roles, and more. +# Alerta -> ### Note -> -> This file assumes you have read the [Introduction to Agent alert notifications](https://github.com/netdata/netdata/blob/master/health/notifications/README.md), detailing how the Netdata Agent's alert notification method works. -The [Alerta](https://alerta.io) monitoring system is a tool used to consolidate and de-duplicate alerts from multiple sources for quick ‘at-a-glance’ visualization. -With just one system you can monitor alerts from many other monitoring tools on a single screen. + -![Alerta dashboard showing several alerts](https://docs.alerta.io/_images/alerta-screen-shot-3.png) - -Alerta's advantage is the main view, where you can see all active alert with the most recent state. -You can also view an alert history. +The [Alerta](https://alerta.io/) monitoring system is a tool used to consolidate and de-duplicate alerts from multiple sources for quick ‘at-a-glance’ visualization. With just one system you can monitor alerts from many other monitoring tools on a single screen. You can send Netdata alerts to Alerta to see alerts coming from many Netdata hosts or also from a multi-host Netdata configuration. -## Prerequisites -You need: -- an Alerta instance -- an Alerta API key (if authentication in Alerta is enabled) -- terminal access to the Agent you wish to configure + + +## Setup + +### Prerequisites + +#### + +- A working Alerta instance +- An Alerta API key (if authentication in Alerta is enabled) +- Access to the terminal where Netdata Agent is running + + + +### Configuration + +#### File + +The configuration file name for this integration is `health_alarm_notify.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config health_alarm_notify.conf +``` +#### Options + +The following options can be defined for this notification + +
Config Options + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| SEND_ALERTA | Set `SEND_ALERTA` to YES | | True | +| ALERTA_WEBHOOK_URL | set `ALERTA_WEBHOOK_URL` to the API url you defined when you installed the Alerta server. | | True | +| ALERTA_API_KEY | Set `ALERTA_API_KEY` to your API key. | | True | +| DEFAULT_RECIPIENT_ALERTA | Set `DEFAULT_RECIPIENT_ALERTA` to the default recipient environment you want the alert notifications to be sent to. All roles will default to this variable if left unconfigured. | | True | +| DEFAULT_RECIPIENT_CUSTOM | Set different recipient environments per role, by editing `DEFAULT_RECIPIENT_CUSTOM` with the environment name of your choice | | False | -## Configure Netdata to send alert notifications to Alerta +##### ALERTA_API_KEY -> ### Info -> -> This file mentions editing configuration files. -> -> - To edit configuration files in a safe way, we provide the [`edit config` script](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#use-edit-config-to-edit-configuration-files) located in your [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory) (typically is `/etc/netdata`) that creates the proper file and opens it in an editor automatically. -> Note that to run the script you need to be inside your Netdata config directory. -> -> It is recommended to use this way for configuring Netdata. +You will need an API key to send messages from any source, if Alerta is configured to use authentication (recommended). To create a new API key: +1. Go to Configuration > API Keys. +2. Create a new API key called "netdata" with `write:alerts` permission. -Edit `health_alarm_notify.conf`, changes to this file do not require restarting Netdata: -1. Set `SEND_ALERTA` to `YES`. -2. set `ALERTA_WEBHOOK_URL` to the API url you defined when you installed the Alerta server. -3. Set `ALERTA_API_KEY` to your API key. - You will need an API key to send messages from any source, if Alerta is configured to use authentication (recommended). To create a new API key: - 1. Go to *Configuration* > *API Keys*. - 2. Create a new API key called "netdata" with `write:alerts` permission. -4. Set `DEFAULT_RECIPIENT_ALERTA` to the default recipient environment you want the alert notifications to be sent to. - All roles will default to this variable if left unconfigured. +##### DEFAULT_RECIPIENT_CUSTOM -You can then have different recipient environments per **role**, by editing `DEFAULT_RECIPIENT_CUSTOM` with the environment name you want, in the following entries at the bottom of the same file: +The `DEFAULT_RECIPIENT_CUSTOM` can be edited in the following entries at the bottom of the same file: ```conf role_recipients_alerta[sysadmin]="Systems" @@ -57,11 +80,18 @@ role_recipients_alerta[proxyadmin]="Proxy" role_recipients_alerta[sitemgr]="Sites" ``` -The values you provide should be defined as environments in `/etc/alertad.conf` option `ALLOWED_ENVIRONMENTS`. +The values you provide should be defined as environments in `/etc/alertad.conf` with `ALLOWED_ENVIRONMENTS` option. -An example working configuration would be: -```conf +
+ +#### Examples + +##### Basic Configuration + + + +```yaml #------------------------------------------------------------------------------ # alerta (alerta.io) global notification options @@ -69,8 +99,30 @@ SEND_ALERTA="YES" ALERTA_WEBHOOK_URL="http://yourserver/alerta/api" ALERTA_API_KEY="INSERT_YOUR_API_KEY_HERE" DEFAULT_RECIPIENT_ALERTA="Production" + ``` -## Test the notification method -To test this alert notification method refer to the ["Testing Alert Notifications"](https://github.com/netdata/netdata/blob/master/health/notifications/README.md#testing-alert-notifications) section of the Agent alert notifications page. +## Troubleshooting + +### Test Notification + +You can run the following command by hand, to test alerts configuration: + +```bash +# become user netdata +sudo su -s /bin/bash netdata + +# enable debugging info on the console +export NETDATA_ALARM_NOTIFY_DEBUG=1 + +# send test alarms to sysadmin +/usr/libexec/netdata/plugins.d/alarm-notify.sh test + +# send test alarms to any role +/usr/libexec/netdata/plugins.d/alarm-notify.sh test "ROLE" +``` + +Note that this will test _all_ alert mechanisms for the selected role. + + -- cgit v1.2.3