# PagerDuty PagerDuty is an enterprise incident resolution service that integrates with ITOps and DevOps monitoring stacks to improve operational reliability and agility. From enriching and aggregating events to correlating them into incidents, PagerDuty streamlines the incident management process by reducing alert noise and resolution times. You can send notifications to PagerDuty using Netdata's Agent alert notification feature, which supports dozens of endpoints, user roles, and more. ## Setup ### Prerequisites #### - An installation of the [PagerDuty](https://www.pagerduty.com/docs/guides/agent-install-guide/) agent on the node running the Netdata Agent - A PagerDuty Generic API service using either the `Events API v2` or `Events API v1` - [Add a new service](https://support.pagerduty.com/docs/services-and-integrations#section-configuring-services-and-integrations) to PagerDuty. Click Use our API directly and select either `Events API v2` or `Events API v1`. Once you finish creating the service, click on the Integrations tab to find your Integration Key. - Access to the terminal where Netdata Agent is running ### Configuration #### File The configuration file name for this integration is `health_alarm_notify.conf`. You can edit the configuration file using the `edit-config` script from the Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). ```bash cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata sudo ./edit-config health_alarm_notify.conf ``` #### Options The following options can be defined for this notification
Config Options | Name | Description | Default | Required | |:----|:-----------|:-------|:--------:| | SEND_PD | Set `SEND_PD` to YES | YES | True | | DEFAULT_RECIPIENT_PD | Set `DEFAULT_RECIPIENT_PD` to the PagerDuty service key you want the alert notifications to be sent to. You can define multiple service keys like this: `pd_service_key_1` `pd_service_key_2`. | | True | ##### DEFAULT_RECIPIENT_PD All roles will default to this variable if left unconfigured. The `DEFAULT_RECIPIENT_PD` can be edited in the following entries at the bottom of the same file: ```conf role_recipients_pd[sysadmin]="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxa" role_recipients_pd[domainadmin]="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxb" role_recipients_pd[dba]="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxc" role_recipients_pd[webmaster]="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxd" role_recipients_pd[proxyadmin]="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxe" role_recipients_pd[sitemgr]="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxf" ```
#### Examples ##### Basic Configuration ```yaml #------------------------------------------------------------------------------ # pagerduty.com notification options SEND_PD="YES" DEFAULT_RECIPIENT_PD="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" USE_PD_VERSION="2" ``` ## Troubleshooting ### Test Notification You can run the following command by hand, to test alerts configuration: ```bash # become user netdata sudo su -s /bin/bash netdata # enable debugging info on the console export NETDATA_ALARM_NOTIFY_DEBUG=1 # send test alarms to sysadmin /usr/libexec/netdata/plugins.d/alarm-notify.sh test # send test alarms to any role /usr/libexec/netdata/plugins.d/alarm-notify.sh test "ROLE" ``` Note that this will test _all_ alert mechanisms for the selected role.