blob: 477634a851d88ff15cadf1b310b4ed4443a032fb (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
|
<!--startmeta
custom_edit_url: "https://github.com/netdata/netdata/edit/master/health/notifications/pagerduty/README.md"
meta_yaml: "https://github.com/netdata/netdata/edit/master/health/notifications/pagerduty/metadata.yaml"
sidebar_label: "PagerDuty"
learn_status: "Published"
learn_rel_path: "Alerting/Notifications/Agent Dispatched Notifications"
message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE NOTIFICATION'S metadata.yaml FILE"
endmeta-->
# PagerDuty
<img src="https://netdata.cloud/img/pagerduty.png" width="150"/>
PagerDuty is an enterprise incident resolution service that integrates with ITOps and DevOps monitoring stacks to improve operational reliability and agility. From enriching and aggregating events to correlating them into incidents, PagerDuty streamlines the incident management process by reducing alert noise and resolution times.
You can send notifications to PagerDuty using Netdata's Agent alert notification feature, which supports dozens of endpoints, user roles, and more.
<img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" />
## Setup
### Prerequisites
####
- An installation of the [PagerDuty](https://www.pagerduty.com/docs/guides/agent-install-guide/) agent on the node running the Netdata Agent
- A PagerDuty Generic API service using either the `Events API v2` or `Events API v1`
- [Add a new service](https://support.pagerduty.com/docs/services-and-integrations#section-configuring-services-and-integrations) to PagerDuty. Click Use our API directly and select either `Events API v2` or `Events API v1`. Once you finish creating the service, click on the Integrations tab to find your Integration Key.
- Access to the terminal where Netdata Agent is running
### Configuration
#### File
The configuration file name for this integration is `health_alarm_notify.conf`.
You can edit the configuration file using the `edit-config` script from the
Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory).
```bash
cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata
sudo ./edit-config health_alarm_notify.conf
```
#### Options
The following options can be defined for this notification
<details><summary>Config Options</summary>
| Name | Description | Default | Required |
|:----|:-----------|:-------|:--------:|
| SEND_PD | Set `SEND_PD` to YES | YES | True |
| DEFAULT_RECIPIENT_PD | Set `DEFAULT_RECIPIENT_PD` to the PagerDuty service key you want the alert notifications to be sent to. You can define multiple service keys like this: `pd_service_key_1` `pd_service_key_2`. | | True |
##### DEFAULT_RECIPIENT_PD
All roles will default to this variable if left unconfigured.
The `DEFAULT_RECIPIENT_PD` can be edited in the following entries at the bottom of the same file:
```conf
role_recipients_pd[sysadmin]="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxa"
role_recipients_pd[domainadmin]="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxb"
role_recipients_pd[dba]="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxc"
role_recipients_pd[webmaster]="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxd"
role_recipients_pd[proxyadmin]="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxe"
role_recipients_pd[sitemgr]="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxf"
```
</details>
#### Examples
##### Basic Configuration
```yaml
#------------------------------------------------------------------------------
# pagerduty.com notification options
SEND_PD="YES"
DEFAULT_RECIPIENT_PD="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
USE_PD_VERSION="2"
```
## Troubleshooting
### Test Notification
You can run the following command by hand, to test alerts configuration:
```bash
# become user netdata
sudo su -s /bin/bash netdata
# enable debugging info on the console
export NETDATA_ALARM_NOTIFY_DEBUG=1
# send test alarms to sysadmin
/usr/libexec/netdata/plugins.d/alarm-notify.sh test
# send test alarms to any role
/usr/libexec/netdata/plugins.d/alarm-notify.sh test "ROLE"
```
Note that this will test _all_ alert mechanisms for the selected role.
|
