summaryrefslogtreecommitdiffstats
path: root/docs/store/distributed-data-architecture.md
blob: c834d710ad8a093c2a11f14b4026d13600cb7adc (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
<!--
title: "Distributed data architecture"
description: "Netdata's distributed data architecture stores metrics on individual nodes for high performance and scalability using all your granular metrics."
custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/store/distributed-data-architecture.md
-->

# Distributed data architecture

Netdata uses a distributed data architecture to help you collect and store per-second metrics from any number of nodes.
Every node in your infrastructure, whether it's one or a thousand, stores the metrics it collects.

Netdata Cloud bridges the gap between many distributed databases by _centralizing the interface_ you use to query and
visualize your nodes' metrics. When you [look at charts in Netdata
Cloud](/docs/visualize/interact-dashboards-charts.md), the metrics values are queried directly from that node's database
and securely streamed to Netdata Cloud, which proxies them to your browser.

Netdata's distributed data architecture has a number of benefits:

-   **Performance**: Every query to a node's database takes only a few milliseconds to complete for responsiveness when
    viewing dashboards or using features like [Metric
    Correlations](https://learn.netdata.cloud/docs/cloud/insights/metric-correlations).
-   **Scalability**: As your infrastructure scales, install the Netdata Agent on every new node to immediately add it to
    your monitoring solution without adding cost or complexity.
-   **1-second granularity**: Without an expensive centralized data lake, you can store all of your nodes' per-second
    metrics, for any period of time, while keeping costs down.
-   **No filtering or selecting of metrics**: Because Netdata's distributed data architecture allows you to store all
    metrics, you don't have to configure which metrics you retain. Keep everything for full visibility during
    troubleshooting and root cause analysis.
-   **Easy maintenance**: There is no centralized data lake to purchase, allocate, monitor, and update, removing
    complexity from your monitoring infrastructure.

## Does Netdata Cloud store my metrics?

Netdata Cloud does not store metric values. 

To enable certain features, such as [viewing active alarms](/docs/monitor/view-active-alarms.md) or [filtering by
hostname/service](https://learn.netdata.cloud/docs/cloud/war-rooms#node-filter), Netdata Cloud does store configured
alarms, their status, and a list of active collectors.

Netdata does not and never will sell your personal data or data about your deployment.

## Long-term metrics storage with Netdata

Any node running the Netdata Agent can store long-term metrics for any retention period, given you allocate the
appropriate amount of RAM and disk space.

Read our document on changing [how long Netdata stores metrics](/docs/store/change-metrics-storage.md) on your nodes for
details.

## Other options for your metrics data

While a distributed data architecture is the default when monitoring infrastructure with Netdata, you can also configure
its behavior based on your needs or the type of infrastructure you manage.

To archive metrics to an external time-series database, such as InfluxDB, Graphite, OpenTSDB, Elasticsearch,
TimescaleDB, and many others, see details on [integrating Netdata via exporting](/docs/export/external-databases.md).

You can also stream between nodes using [streaming](/streaming/README.md), allowing to replicate databases and create
your own centralized data lake of metrics, if you choose to do so.

When you use the database engine to store your metrics, you can always perform a quick backup of a node's
`/var/cache/netdata/dbengine/` folder using the tool of your choice.

## What's next?

You can configure the Netdata Agent to store days, weeks, or months worth of distributed, per-second data by
[configuring the database engine](/docs/store/change-metrics-storage.md). Use our calculator to determine the system
resources required to retain your desired amount of metrics, and expand or contract the database by editing a single
setting.