1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
|
# Deployment with Centralization Points
An observability centralization point can centralize both metrics and logs. The sending systems are called Children, while the receiving systems are called a Parents.
When metrics and logs are centralized, the Children are never queried for metrics and logs. The Netdata Parents have all the data needed to satisfy queries.
- **Metrics** are centralized by Netdata, with a feature we call **Streaming**. The Parents listen for incoming connections and permit access only to Children that connect to it with the right API key. Children are configured to push their metrics to the Parents and they initiate the connections to do so.
- **Logs** are centralized with methodologies provided by `systemd-journald`. This involves installing `systemd-journal-remote` on both the Parent and the Children, and configuring the keys required for this communication.
| Feature | How it works |
|:---------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|
| Unified infrastructure dashboards for metrics | Yes, at Netdata Cloud |
| Unified infrastructure dashboards for logs | All logs are accessible via the same dashboard at Netdata Cloud, although they are unified per Netdata Parent |
| Centrally configured alerts | Yes, at Netdata Parents |
| Centrally dispatched alert notifications | Yes, at Netdata Cloud |
| Data are exclusively on-prem | Yes, Netdata Cloud queries Netdata Agents to satisfy dashboard queries. |
A configuration with 2 observability centralization points, looks like this:
```mermaid
flowchart LR
WEB[["One unified
dashboard
for all nodes"]]
NC(["<b>Netdata Cloud</b>
decides which agents
need to be queried"])
SA1["Netdata at AWS
A1"]
SA2["Netdata at AWS
A2"]
SAN["Netdata at AWS
AN"]
PA["<b>Netdata Parent A</b>
at AWS
having all metrics & logs
for all Ax nodes"]
SB1["Netdata On-Prem
B1"]
SB2["Netdata On-Prem
B2"]
SBN["Netdata On-Prem
BN"]
PB["<b>Netdata Parent B</b>
On-Prem
having all metrics & logs
for all Bx nodes"]
WEB -->|query| NC -->|query| PA & PB
PA ---|stream| SA1 & SA2 & SAN
PB ---|stream| SB1 & SB2 & SBN
```
Netdata Cloud queries the Netdata Parents to provide aggregated dashboard views.
For alerts, the dispatch of notifications looks like in the following chart:
```mermaid
flowchart LR
NC(["<b>Netdata Cloud</b>
applies silencing
& user settings"])
SA1["Netdata at AWS
A1"]
SA2["Netdata at AWS
A2"]
SAN["Netdata at AWS
AN"]
PA["<b>Netdata Parent A</b>
at AWS
having all metrics & logs
for all Ax nodes"]
SB1["Netdata On-Prem
B1"]
SB2["Netdata On-Prem
B2"]
SBN["Netdata On-Prem
BN"]
PB["<b>Netdata Parent B</b>
On-Prem
having all metrics & logs
for all Bx nodes"]
EMAIL{{"<b>e-mail</b>
notifications"}}
MOBILEAPP{{"<b>Netdata Mobile App</b>
notifications"}}
SLACK{{"<b>Slack</b>
notifications"}}
OTHER{{"Other
notifications"}}
PA & PB -->|alert transitions| NC -->|notification| EMAIL & MOBILEAPP & SLACK & OTHER
SA1 & SA2 & SAN ---|stream| PA
SB1 & SB2 & SBN ---|stream| PB
```
### Configuration steps for deploying Netdata with Observability Centralization Points
For Metrics:
- Install Netdata agents on all systems and the Netdata Parents.
- Configure `stream.conf` at the Netdata Parents to enable streaming access with an API key.
- Configure `stream.conf` at the Netdata Children to enable streaming to the configured Netdata Parents.
For Logs:
- Install `systemd-journal-remote` on all systems and the Netdata Parents.
- Configure `systemd-journal-remote` at the Netdata Parents to enable logs reception.
- Configure `systemd-journal-upload` at the Netdata Children to enable transmission of their logs to the Netdata Parents.
Optionally:
- Disable ML, health checks and dashboard access at Netdata Children to save resources and avoid duplicate notifications.
When using Netdata Cloud:
- Optionally: disable dashboard access on all Netdata agents (including Netdata Parents).
- Optionally: disable alert notifications on all Netdata agents (including Netdata Parents).
|
