1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
|
# yamllint disable rule:line-length
---
- id: "notify-cloud-mobile-app"
meta:
name: "Netdata Mobile App"
link: "https://netdata.cloud"
categories:
- notify.cloud
icon_filename: "netdata.png"
keywords:
- mobile-app
- phone
- personal-notifications
setup:
description: |
### Prerequisites
- A Netdata Cloud account
- You need to have the Netdata Mobile App installed on your [Android](https://play.google.com/store/apps/details?id=cloud.netdata.android&pli=1) or [iOS](https://apps.apple.com/in/app/netdata-mobile/id6474659622) phone.
### Netdata Mobile App Configuration and device linking
In order to login to the Netdata Mobile App
1. Download the Netdata Mobile App from [Google Play Store](https://play.google.com/store/apps/details?id=cloud.netdata.android&pli=1) or the [iOS App Store](https://apps.apple.com/in/app/netdata-mobile/id6474659622)
2. Open the App and Choose your Sign-in option
- Email Address: Enter the email address of your registered Netdata Cloud account and click on the verification link received by email on your mobile device.
- Sign-in with QR Code: Scan the QR code from the Netdata Cloud UI under **Profile Picture** --> **Settings** --> **Notifications** --> **Mobile App Notifications** --> **Show QR Code**
### Netdata Configuration
After linking your device, enable the toggle for **Mobile App Notifications** under the same settings panel.
- id: "notify-cloud-discord"
meta:
name: "Discord"
link: "https://discord.com/"
categories:
- notify.cloud
icon_filename: "discord.png"
keywords:
- discord
- community
setup:
description: |
### Prerequisites
- A Netdata Cloud account
- Access to the Space as an **Admin**
### Discord Server Configuration
1. Go to **Server Settings** --> **Integrations**
2. **Create Webhook** or **View Webhooks** if you already have some defined
3. Specify the **Name** and **Channel** on your new webhook
4. Keep note of the **Webhook URL** as you will need it for the configuration of the integration on the Netdata Cloud UI
### Netdata Configuration
1. Click on the **Space settings** cog (located above your profile icon)
2. Click on the **Alerts & Notifications** tab
3. Click on the **+ Add configuration** button
4. Add the Discord Integration
5. A modal will be presented to you to enter the required details to enable the integration:
- **Notification settings**
- Configuration name (optional): A name for your configuration in order to easily refer to it
- Rooms: A list of Rooms for which you want to be notified
- Notifications: The notifications which you want to be notified
- **Integration configuration**
- Webhook URL: The URL you copied from the previous section
- Channel Parameters: Select the channel type which the notifications will be sent to, if it is a Forum channel, you need to specify a thread name
- id: "notify-cloud-pagerduty"
meta:
name: "PagerDuty"
link: "https://www.pagerduty.com/"
categories:
- notify.cloud
icon_filename: "pagerduty.png"
keywords:
- pagerduty
setup:
description: |
### Prerequisites
- A Netdata Cloud account
- Access to the Space as an **Admin**
- The Space needs to be on a paid plan
- You need to have a PagerDuty service to receive events using webhooks.
### PagerDuty Server Configuration
1. Create a service to receive events from your services directory page on PagerDuty
2. On the third step of the service creation, select `Events API V2` Integration
3. Once the service is created, you will be redirected to its configuration page, where you can copy the **Integration Key** and **Integration URL (Alert Events)** in order to add them to your integration configuration in the Netdata Cloud UI
### Netdata Configuration
1. Click on the **Space settings** cog (located above your profile icon)
2. Click on the **Alerts & Notifications** tab
3. Click on the **+ Add configuration** button
4. Add the PagerDuty Integration
5. A modal will be presented to you to enter the required details to enable the integration:
- **Notification settings**
- Configuration name (optional): A name for your configuration in order to easily refer to it
- Rooms: A list of Rooms for which you want to be notified
- Notifications: The notifications which you want to be notified
- **Integration configuration**
- Integration Key: A 32 character key provided by PagerDuty to receive events on your service.
- Integration URL (Alert Events): The URL provided by PagerDuty where Netdata Cloud will send notifications.
- id: "notify-cloud-slack"
meta:
name: "Slack"
link: "https://slack.com/"
categories:
- notify.cloud
icon_filename: "slack.png"
keywords:
- slack
setup:
description: |
### Prerequisites
- A Netdata Cloud account
- Access to the Space as an **Admin**
- The Space needs to be on a paid plan
- You need to have a Slack app on your workspace to receive the Webhooks.
### Slack Server Configuration
1. Create an app to receive webhook integrations. Check the [Slack documentation](https://api.slack.com/apps?new_app=1) for further details
2. Install the app on your workspace
3. Configure Webhook URLs for your workspace
- On your app go to **Incoming Webhooks** and click on **activate incoming webhooks**
- At the bottom of **Webhook URLs for Your Workspace** section you have **Add New Webhook to Workspace**
- Specify the channel where you want your notifications to be delivered
- Once completed, copy the Webhook URL in order to add it to your integration configuration in the Netdata Cloud UI
For more details please check [Incoming webhooks for Slack](https://slack.com/help/articles/115005265063-Incoming-webhooks-for-Slack).
### Netdata Configuration
1. Click on the **Space settings** cog (located above your profile icon)
2. Click on the **Alerts & Notifications** tab
3. Click on the **+ Add configuration** button
4. Add the Slack Integration
5. A modal will be presented to you to enter the required details to enable the integration:
- **Notification settings**
- Configuration name (optional): A name for your configuration in order to easily refer to it
- Rooms: A list of Rooms for which you want to be notified
- Notifications: The notifications which you want to be notified
- **Integration configuration**
- Webhook URL: URL provided on Slack for the channel you want to receive your notifications
- id: "notify-cloud-opsgenie"
meta:
name: "Opsgenie"
link: "https://www.atlassian.com/software/opsgenie"
categories:
- notify.cloud
icon_filename: "opsgenie.png"
keywords:
- opsgenie
- atlassian
setup:
description: |
### Prerequisites
- A Netdata Cloud account
- Access to the Space as an **Admin**
- The Space needs to be on a paid plan
- You need to have permissions on Opsgenie to add new integrations.
### Opsgenie Server Configuration
1. Go to the integrations tab of your team, click **Add integration**
2. Pick **API** from the available integrations and copy the API Key in order to add it to your integration configuration in the Netdata Cloud UI
### Netdata Configuration
1. Click on the **Space settings** cog (located above your profile icon)
2. Click on the **Alerts & Notifications** tab
3. Click on the **+ Add configuration** button
4. Add the Opsgenie Integration
5. A modal will be presented to you to enter the required details to enable the integration:
- **Notification settings**
- Configuration name (optional): A name for your configuration in order to easily refer to it
- Rooms: A list of Rooms for which you want to be notified
- Notifications: The notifications which you want to be notified
- **Integration configuration**
- API Key: The key provided on Opsgenie for the channel you want to receive your notifications
- id: "notify-cloud-mattermost"
meta:
name: "Mattermost"
link: "https://mattermost.com/"
categories:
- notify.cloud
icon_filename: "mattermost.png"
keywords:
- mattermost
setup:
description: |
### Prerequisites
- A Netdata Cloud account
- Access to the Space as an **Admin**
- The Space needs to be on a paid plan
- You need to have permissions on Mattermost to add new integrations.
### Mattermost Server Configuration
1. In Mattermost, go to Product menu > Integrations > Incoming Webhook
- If you don't have the Integrations option, incoming webhooks may not be enabled on your Mattermost server or may be disabled for non-admins. They can be enabled by a System Admin from System Console > Integrations > Integration Management. Once incoming webhooks are enabled, continue with the steps below.
2. Select Add Incoming Webhook and add a name and description for the webhook.
3. Select the channel to receive webhook payloads, then select Add to create the webhook
4. You will end up with a webhook URL that looks like `https://your-mattermost-server.com/hooks/xxx-generatedkey-xxx`, copy it in order to add it to your integration configuration in the Netdata Cloud UI
For more details please check [Incoming webhooks for Mattermost](https://developers.mattermost.com/integrate/webhooks/incoming/).
### Netdata Configuration
1. Click on the **Space settings** cog (located above your profile icon)
2. Click on the **Alerts & Notifications** tab
3. Click on the **+ Add configuration** button
4. Add the Mattermost Integration
5. A modal will be presented to you to enter the required details to enable the integration:
- **Notification settings**
- Configuration name (optional): A name for your configuration in order to easily refer to it
- Rooms: A list of Rooms for which you want to be notified
- Notifications: The notifications which you want to be notified
- **Integration configuration**
- Webhook URL: URL provided on Mattermost for the channel you want to receive your notifications
- id: "notify-cloud-rocketchat"
meta:
name: "RocketChat"
link: "https://www.rocket.chat/"
categories:
- notify.cloud
icon_filename: "rocketchat.png"
keywords:
- rocketchat
setup:
description: |
### Prerequisites
- A Netdata Cloud account
- Access to the Space as an **Admin**
- The Space needs to be on a paid plan
- You need to have permissions on RocketChat to add new integrations.
### RocketChat Server Configuration
Steps to configure your RocketChat server to receive notifications from Netdata Cloud:
1. In RocketChat, Navigate to Administration > Workspace > Integrations
2. Click **+New** at the top right corner
3. For more details about each parameter, check [Create a new incoming webhook](https://docs.rocket.chat/use-rocket.chat/workspace-administration/integrations#create-a-new-incoming-webhook)
4. You will end up with a webhook endpoint that looks like `https://your-server.rocket.chat/hooks/YYYYYYYYYYYYYYYYYYYYYYYY/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`, copy it in order to add it to your integration configuration in the Netdata Cloud UI
For more details please check [Incoming webhooks for RocketChat](https://docs.rocket.chat/use-rocket.chat/workspace-administration/integrations/).
### Netdata Configuration
1. Click on the **Space settings** cog (located above your profile icon)
2. Click on the **Alerts & Notifications** tab
3. Click on the **+ Add configuration** button
4. Add the PagerDuty Integration
5. A modal will be presented to you to enter the required details to enable the integration:
- **Notification settings**
- Configuration name (optional): A name for your configuration in order to easily refer to it
- Rooms: A list of Rooms for which you want to be notified
- Notifications: The notifications which you want to be notified
- **Integration configuration**
- Webhook URL: URL provided on RocketChat for the channel you want to receive your notifications
- id: "notify-cloud-awssns"
meta:
name: "Amazon SNS"
link: "https://aws.amazon.com/sns/"
categories:
- notify.cloud
icon_filename: "awssns.png"
keywords:
- awssns
setup:
description: |
### Prerequisites
- A Netdata Cloud account
- Access to the space as an **Admin**
- The Space needs to be on a paid plan
- An AWS account with AWS SNS access
### AWS SNS Configuration
1. [Setting up access for Amazon SNS](https://docs.aws.amazon.com/sns/latest/dg/sns-setting-up.html)
2. Create a topic
- On AWS SNS management console click on **Create topic**
- On the **Details** section, select the standard type and provide the topic name
- On the **Access policy** section, change the **Publishers** option to **Only the specified AWS accounts** and provide the Netdata AWS account **(123269920060)** that will be used to publish notifications to the topic being created
3. Copy the **Topic ARN** in order to add it to your integration configuration in the Netdata Cloud UI
### Netdata Configuration
1. Click on the **Space settings** cog (located above your profile icon)
2. Click on the **Alerts & Notifications** tab
3. Click on the **+ Add configuration** button
4. Add the AWS SNS Integration
5. A modal will be presented to you to enter the required details to enable the integration:
- **Notification settings**
- Configuration name (optional): A name for your configuration in order to easily refer to it
- Rooms: A list of Rooms for which you want to be notified
- Notifications: The notifications which you want to be notified
- **Integration configuration**
- Topic ARN: The topic provided on AWS SNS (with region) for where to publish your notifications.
- id: "notify-cloud-microsoftteams"
meta:
name: "Microsoft Teams"
link: "https://www.microsoft.com/en-us/microsoft-teams"
categories:
- notify.cloud
icon_filename: "teams.svg"
keywords:
- microsoft
- teams
setup:
description: |
### Prerequisites
- A Netdata Cloud account
- Access to the Space as an **Admin**
- The Space needs to be on a paid plan
- A [Microsoft Teams Essentials subscription](https://www.microsoft.com/en-sg/microsoft-teams/essentials) or higher. Note that this is a **paid** feature
### Microsoft Teams Configuration
1. Navigate to the desired Microsoft Teams channel and hover over the channel name. Click the three dots icon that appears
2. Select "Workflows" from the options, then choose "Post to a channel when a webhook request is received"
3. **Configure Workflow Details**
- Give your workflow a name, such as "Netdata Alerts"
- Select the target team and channel where you will receive notifications
- Click "Add workflow"
4. Once the workflow is created, you will receive a unique Workflow Webhook URL, copy it, in order to add it to your integration configuration in the Netdata Cloud UI
### Netdata Configuration
1. Click on the **Space settings** cog (located above your profile icon)
2. Click on the **Alerts & Notifications** tab
3. Click on the **+ Add configuration** button
4. Add the Microsoft Teams Integration
5. A modal will be presented to you to enter the required details to enable the integration:
- **Notification settings**
- Configuration name (optional): A name for your configuration in order to easily refer to it
- Rooms: A list of Rooms for which you want to be notified
- Notifications: The notifications which you want to be notified
- **Integration configuration**
- Microsoft Teams Incoming Webhook URL: The Incoming Webhook URL that you copied earlier.
- id: "notify-cloud-telegram"
meta:
name: "Telegram"
link: "https://telegram.org/"
categories:
- notify.cloud
icon_filename: "telegram.svg"
keywords:
- Telegram
setup:
description: |
### Prerequisites
- A Netdata Cloud account
- Access to the Space as an **Admin**
- The Space needs to be on a paid plan
- The Telegram bot token, chat ID and optionally the topic ID
### Telegram Configuration
- Bot token: To create one bot, contact the [@BotFather](https://t.me/BotFather) bot and send the command `/newbot` and follow the instructions. **Start a conversation with your bot or invite it into the group where you want it to send notifications**.
- To get the chat ID you have two options:
- Contact the [@myidbot](https://t.me/myidbot) bot and send the `/getid` command to get your personal chat ID, or invite it into a group and use the `/getgroupid` command to get the group chat ID.
- Alternatively, you can get the chat ID directly from the bot API. Send your bot a command in the chat you want to use, then check `https://api.telegram.org/bot{YourBotToken}/getUpdates`, eg. `https://api.telegram.org/bot111122223:7OpFlFFRzRBbrUUmIjj5HF9Ox2pYJZy5/getUpdates`
- To get the topic ID, the easiest way is this: Post a message to that topic, then right-click on it and select `Copy Message Link`. Paste it on a scratchpad and notice that it has the following structure `https://t.me/c/XXXXXXXXXX/YY/ZZ`. The topic ID is `YY` (integer).
### Netdata Configuration
1. Click on the **Space settings** cog (located above your profile icon)
2. Click on the **Alerts & Notifications** tab
3. Click on the **+ Add configuration** button
4. Add the Telegram Integration
5. A modal will be presented to you to enter the required details to enable the integration:
- **Notification settings**
- Configuration name (optional): A name for your configuration in order to easily refer to it
- Rooms: A list of Rooms for which you want to be notified
- Notifications: The notifications which you want to be notified
- **Integration configuration**
- Bot Token: The token of your bot
- Chat ID: The chat id where your bot will deliver messages to
- Topic ID: The identifier of the chat topic to which your bot will send messages. If omitted or 0, messages will be sent to the General topic. If topics are not supported, messages will be sent to the chat.
- id: "notify-cloud-splunk"
meta:
name: "Splunk"
link: "https://splunk.com/"
categories:
- notify.cloud
icon_filename: "splunk-black.svg"
keywords:
- Splunk
setup:
description: |
### Prerequisites
- A Netdata Cloud account
- Access to the Space as an **Admin**
- The Space needs to be on a paid plan
- The URI and token for your Splunk HTTP Event Collector. Refer to the [Splunk documentation](https://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector) for detailed instructions on how to set it up.
### Netdata Configuration
1. Click on the **Space settings** cog (located above your profile icon)
2. Click on the **Alerts & Notifications** tab
3. Click on the **+ Add configuration** button
4. Add the Splunk Integration
5. A modal will be presented to you to enter the required details to enable the integration:
- **Notification settings**
- Configuration name (optional): A name for your configuration in order to easily refer to it
- Rooms: A list of Rooms for which you want to be notified
- Notifications: The notifications which you want to be notified
- **Integration configuration**
- HTTP Event Collector URI: The URI of your HTTP event collector in Splunk
- HTTP Event Collector Token: The token that Splunk provided to you when you created the HTTP Event Collector
- id: "notify-cloud-victorops"
meta:
name: "Splunk VictorOps"
link: "https://www.splunk.com/en_us/about-splunk/acquisitions/splunk-on-call.html"
categories:
- notify.cloud
icon_filename: "victorops.svg"
keywords:
- VictorOps
- Splunk
- On-Call
setup:
description: |
### Prerequisites
- A Netdata Cloud account
- Access to the Space as an **Admin**
- The Space needs to be on a paid plan
- The Destination URL for your Splunk VictorOps REST Endpoint Integration. Refer to the [VictorOps documentation](https://help.victorops.com/knowledge-base/rest-endpoint-integration-guide) for detailed instructions.
### Netdata Configuration
1. Click on the **Space settings** cog (located above your profile icon)
2. Click on the **Alerts & Notifications** tab
3. Click on the **+ Add configuration** button
4. Add the Splunk VictorOps Integration
5. A modal will be presented to you to enter the required details to enable the integration:
- **Notification settings**
- Configuration name (optional): A name for your configuration in order to easily refer to it
- Rooms: A list of Rooms for which you want to be notified
- Notifications: The notifications which you want to be notified
- **Integration configuration**
- Destination URL - The URL provided by VictorOps of your REST endpoint.
- id: "notify-cloud-webhook"
meta:
name: "Webhook"
link: "https://en.wikipedia.org/wiki/Webhook"
categories:
- notify.cloud
icon_filename: "webhook.svg"
keywords:
- generic webhooks
- webhooks
setup:
description: |
### Prerequisites
- A Netdata Cloud account
- Access to the Space as an **Admin**
- The Space needs to be on a paid plan
- You need to have an app that allows you to receive webhooks following a predefined schema.
### Netdata Configuration
1. Click on the **Space settings** cog (located above your profile icon)
2. Click on the **Alerts & Notifications** tab
3. Click on the **+ Add configuration** button
4. Add the Webhook integration
5. A modal will be presented to you to enter the required details to enable the configuration:
- **Notification settings**
- Configuration name (optional): A name for your configuration in order to easily refer to it
- Rooms: A list of Rooms for which you want to be notified
- Notifications: The notifications which you want to be notified
- **Integration configuration**
- Webhook URL: The url of the service that Netdata will send notifications to. In order to keep the communication secured, Netdata only accepts HTTPS urls.
- Extra headers: Optional key-value pairs that you can set to be included in the HTTP requests sent to the webhook URL.
- Authentication Mechanism, Netdata webhook integration supports 3 different authentication mechanisms.
- Mutual TLS (recommended): Default authentication mechanism used if no other method is selected
- Basic: The client sends a request with an Authorization header that includes a base64-encoded string in the format **username:password**.
- Bearer: The client sends a request with an Authorization header that includes a **bearer token**.
### Webhook service
A webhook service allows your application to receive real-time alerts from Netdata by sending HTTP requests to a specified URL.
In this section, we'll go over the steps to set up a generic webhook service, including adding headers, and implementing different types of authorization mechanisms.
#### Netdata webhook integration
Netdata webhook integration service will send alert and reachability notifications to the destination service as soon as they are detected.
For alert notifications, the content sent to the destination service contains a JSON object with the following properties:
| field | type | description |
|:----------------------------------|:--------------------------------|:--------------------------------------------------------------------------|
| message | string | A summary message of the alert. |
| alert | string | The alert the notification is related to. |
| info | string | Additional info related with the alert. |
| chart | string | The chart associated with the alert. |
| context | string | The chart context. |
| space | string | The space where the node that raised the alert is assigned. |
| Rooms | object\[object(string,string)\] | Object with list of Rooms names and urls where the node belongs to. |
| family | string | Context family. |
| class | string | Classification of the alert, e.g. `Error`. |
| severity | string | Alert severity, can be one of `warning`, `critical` or `clear`. |
| date | string | Date of the alert in ISO8601 format. |
| duration | string | Duration the alert has been raised. |
| additional_active_critical_alerts | integer | Number of additional critical alerts currently existing on the same node. |
| additional_active_warning_alerts | integer | Number of additional warning alerts currently existing on the same node. |
| alert_url | string | Netdata Cloud URL for this alert. |
For reachability notifications, the JSON object will contain the following properties:
| field | type | description |
|:-----------------|:--------|:------------------------------------------------------------------------------------------------------------------------------|
| message | string | A summary message of the reachability alert. |
| url | string | Netdata Cloud URL for the host experiencing the reachability alert. |
| host | string | The hostname experiencing the reachability alert. |
| severity | string | Severity for this notification. If host is reachable, severity will be `info`, if host is unreachable, it will be `critical`. |
| status | object | An object with the status information. |
| status.reachable | boolean | `true` if host is reachable, `false` otherwise |
| status.text | string | Can be `reachable` or `unreachable` |
#### Extra headers
When setting up a webhook service, the user can specify a set of headers to be included in the HTTP requests sent to the webhook URL.
By default, the following headers will be sent in the HTTP request
| **Header** | **Value** |
|:------------:|------------------|
| Content-Type | application/json |
#### Authentication mechanisms
Netdata webhook integration supports 3 different authentication mechanisms:
##### Mutual TLS authentication (recommended)
In mutual Transport Layer Security (mTLS) authentication, the client and the server authenticate each other using X.509 certificates. This ensures that the client is connecting to the intended server, and that the server is only accepting connections from authorized clients.
This is the default authentication mechanism used if no other method is selected.
To take advantage of mutual TLS, you can configure your server to verify Netdata's client certificate. In order to achieve this, the Netdata client sending the notification supports mutual TLS (mTLS) to identify itself with a client certificate that your server can validate.
The steps to perform this validation are as follows:
- Store Netdata CA certificate on a file in your disk. The content of this file should be:
<details>
<summary>Netdata CA certificate</summary>
```text
-----BEGIN CERTIFICATE-----
MIIF0jCCA7qgAwIBAgIUDV0rS5jXsyNX33evHEQOwn9fPo0wDQYJKoZIhvcNAQEN
BQAwgYAxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQH
Ew1TYW4gRnJhbmNpc2NvMRYwFAYDVQQKEw1OZXRkYXRhLCBJbmMuMRIwEAYDVQQL
EwlDbG91ZCBTUkUxGDAWBgNVBAMTD05ldGRhdGEgUm9vdCBDQTAeFw0yMzAyMjIx
MjQzMDBaFw0zMzAyMTkxMjQzMDBaMIGAMQswCQYDVQQGEwJVUzETMBEGA1UECBMK
Q2FsaWZvcm5pYTEWMBQGA1UEBxMNU2FuIEZyYW5jaXNjbzEWMBQGA1UEChMNTmV0
ZGF0YSwgSW5jLjESMBAGA1UECxMJQ2xvdWQgU1JFMRgwFgYDVQQDEw9OZXRkYXRh
IFJvb3QgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCwIg7z3R++
ppQYYVVoMIDlhWO3qVTMsAQoJYEvVa6fqaImUBLW/k19LUaXgUJPohB7gBp1pkjs
QfY5dBo8iFr7MDHtyiAFjcQV181sITTMBEJwp77R4slOXCvrreizhTt1gvf4S1zL
qeHBYWEgH0RLrOAqD0jkOHwewVouO0k3Wf2lEbCq3qRk2HeDvkv0LR7sFC+dDms8
fDHqb/htqhk+FAJELGRqLeaFq1Z5Eq1/9dk4SIeHgK5pdYqsjpBzOTmocgriw6he
s7F3dOec1ZZdcBEAxOjbYt4e58JwuR81cWAVMmyot5JNCzYVL9e5Vc5n22qt2dmc
Tzw2rLOPt9pT5bzbmyhcDuNg2Qj/5DySAQ+VQysx91BJRXyUimqE7DwQyLhpQU72
jw29lf2RHdCPNmk8J1TNropmpz/aI7rkperPugdOmxzP55i48ECbvDF4Wtazi+l+
4kx7ieeLfEQgixy4lRUUkrgJlIDOGbw+d2Ag6LtOgwBiBYnDgYpvLucnx5cFupPY
Cy3VlJ4EKUeQQSsz5kVmvotk9MED4sLx1As8V4e5ViwI5dCsRfKny7BeJ6XNPLnw
PtMh1hbiqCcDmB1urCqXcMle4sRhKccReYOwkLjLLZ80A+MuJuIEAUUuEPCwywzU
R7pagYsmvNgmwIIuJtB6mIJBShC7TpJG+wIDAQABo0IwQDAOBgNVHQ8BAf8EBAMC
AQYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQU9IbvOsPSUrpr8H2zSafYVQ9e
Ft8wDQYJKoZIhvcNAQENBQADggIBABQ08aI31VKZs8jzg+y/QM5cvzXlVhcpkZsY
1VVBr0roSBw9Pld9SERrEHto8PVXbadRxeEs4sKivJBKubWAooQ6NTvEB9MHuGnZ
VCU+N035Gq/mhBZgtIs/Zz33jTB2ju3G4Gm9VTZbVqd0OUxFs41Iqvi0HStC3/Io
rKi7crubmp5f2cNW1HrS++ScbTM+VaKVgQ2Tg5jOjou8wtA+204iYXlFpw9Q0qnP
qq6ix7TfLLeRVp6mauwPsAJUgHZluz7yuv3r7TBdukU4ZKUmfAGIPSebtB3EzXfH
7Y326xzv0hEpjvDHLy6+yFfTdBSrKPsMHgc9bsf88dnypNYL8TUiEHlcTgCGU8ts
ud8sWN2M5FEWbHPNYRVfH3xgY2iOYZzn0i+PVyGryOPuzkRHTxDLPIGEWE5susM4
X4bnNJyKH1AMkBCErR34CLXtAe2ngJlV/V3D4I8CQFJdQkn9tuznohUU/j80xvPH
FOcDGQYmh4m2aIJtlNVP6+/92Siugb5y7HfslyRK94+bZBg2D86TcCJWaaZOFUrR
Y3WniYXsqM5/JI4OOzu7dpjtkJUYvwtg7Qb5jmm8Ilf5rQZJhuvsygzX6+WM079y
nsjoQAm6OwpTN5362vE9SYu1twz7KdzBlUkDhePEOgQkWfLHBJWwB+PvB1j/cUA3
5zrbwvQf
-----END CERTIFICATE-----
```
</details>
- Enable client certificate validation on the web server that is doing the TLS termination. Below there are examples on how to perform this configuration in `NGINX` and `Apache`.
**NGINX**
```bash
server {
listen 443 ssl default_server;
# ... existing SSL configuration for server authentication ...
ssl_verify_client on;
ssl_client_certificate /path/to/Netdata_CA.pem;
location / {
if ($ssl_client_s_dn !~ "CN=app.netdata.cloud") {
return 403;
}
# ... existing location configuration ...
}
}
```
**Apache**
```bash
Listen 443
<VirtualHost *:443>
# ... existing SSL configuration for server authentication ...
SSLVerifyClient require
SSLCACertificateFile "/path/to/Netdata_CA.pem"
</VirtualHost>
<Directory /var/www/>
Require expr "%{SSL_CLIENT_S_DN_CN} == 'app.netdata.cloud'"
# ... existing directory configuration ...
</Directory>
```
##### Basic authentication
In basic authorization, the client sends a request with an Authorization header that includes a base64-encoded string in the format username:password. The server then uses this information to authenticate the client. If this authentication method is selected, the user can set the user and password that will be used when connecting to the destination service.
##### Bearer token authentication
In bearer token authentication, the client sends a request with an Authorization header that includes a bearer token. The server then uses this token to authenticate the client. Bearer tokens are typically generated by an authentication service, and are passed to the client after a successful authentication. If this method is selected, the user can set the token to be used for connecting to the destination service.
##### Challenge secret
To validate that you have ownership of the web application that will receive the webhook events, Netdata is using a challenge response check mechanism.
This mechanism works as follows:
- The challenge secret parameter that you provide is a shared secret between only you and Netdata.
- On your request for creating a new Webhook integration, Netdata will make a GET request to the URL of the webhook, adding a query parameter `crc_token`, consisting of a random string.
- You will receive this request on your application and it must construct an encrypted response, consisting of a base64-encoded HMAC SHA-256 hash created from the crc_token and the shared secret. The response will be in the format:
```json
{
"response_token": "sha256=9GKoHJYmcHIkhD+C182QWN79YBd+D+Vkj4snmZrfNi4="
}
```
- Netdata will compare your application's response with the hash that it will generate using the challenge secret, and if they are the same, the integration creation will succeed.
Netdata does this validation every time you update your integration configuration.
- Response requirements:
- A base64 encoded HMAC SHA-256 hash created from the crc_token and the shared secret.
- Valid response_token and JSON format.
- Latency less than 5 seconds.
- 200 HTTP response code.
**Example response token generation in Python:**
Here you can see how to define a handler for a Flask application in python 3:
```python
import base64
import hashlib
import hmac
import json
key ='YOUR_CHALLENGE_SECRET'
@app.route('/webhooks/netdata')
def webhook_challenge():
token = request.args.get('crc_token').encode('ascii')
# creates HMAC SHA-256 hash from incomming token and your consumer secret
sha256_hash_digest = hmac.new(key.encode(),
msg=token,
digestmod=hashlib.sha256).digest()
# construct response data with base64 encoded hash
response = {
'response_token': 'sha256=' + base64.b64encode(sha256_hash_digest).decode('ascii')
}
# returns properly formatted json response
return json.dumps(response)
```
- id: "notify-cloud-ilert"
meta:
name: "ilert"
link: "https://www.ilert.com/"
categories:
- notify.cloud
icon_filename: "ilert.svg"
keywords:
- ilert
setup:
description: |
### Prerequisites
- A Netdata Cloud account
- Access to the Space as an **Admin**
- The Space needs to be on a paid plan
- You need to have permissions on ilert to add new Alert sources.
### ilert Configuration
1. From the navigation bar, open the Alert sources drop down and click "Alert sources"
2. Click on the "+ Create a new alert source" button
3. Configure an Alert source:
- Select "API integration" and click Next
- Provide a name that suits the source's purpose, for example "Netdata"
- Select Escalation policy
- Select Alert grouping (optional)
4. Obtain the API Key:
- Once the Alert source is created, you will be provided with an API key. Copy it in order to add it to your integration configuration in the Netdata Cloud UI
### Netdata Configuration
1. Click on the **Space settings** cog (located above your profile icon)
2. Click on the **Alerts & Notifications** tab
3. Click on the **+ Add configuration** button
4. Add the ilert Integration
5. A modal will be presented to you to enter the required details to enable the integration:
- **Notification settings**
- Configuration name (optional): A name for your configuration in order to easily refer to it
- Rooms: A list of Rooms for which you want to be notified
- Notifications: The notifications which you want to be notified
- **Integration configuration**
- Alert Source API key: The key you copied in the ilert configuration step.
|