Merging upstream version 2.0.3+dfsg:

- does not include dygraphs anymore (Closes: #923993)
- does not include pako anymore (Closes: #1042533)
- does not include dashboard binaries anymore (Closes: #1045145)

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

6 files changed, 64 insertions, 78 deletions

diff --git a/src/registry/README.md b/src/registry/README.md
index d976528c7..97db113f7 100644
--- a/src/registry/README.md
+++ b/src/registry/README.md
@@ -1,12 +1,3 @@
-<!--
-title: "Registry"
-description: "Netdata utilizes a central registry of machines/person GUIDs, URLs, and opt-in account information to provide unified cross-server dashboards."
-custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/registry/README.md"
-sidebar_label: "Registry"
-learn_status: "Published"
-learn_rel_path: "Configuration"
--->
-
 # Registry
 
 Netdata provides distributed monitoring.
@@ -14,21 +5,21 @@ Netdata provides distributed monitoring.
 Traditional monitoring solutions centralize all the data to provide unified dashboards across all servers. Before
 Netdata, this was the standard practice. However it has a few issues:
 
-1.  due to the resources required, the number of metrics collected is limited.
-2.  for the same reason, the data collection frequency is not that high, at best it will be once every 10 or 15 seconds,
+1. due to the resources required, the number of metrics collected is limited.
+2. for the same reason, the data collection frequency is not that high, at best it will be once every 10 or 15 seconds,
     at worst every 5 or 10 mins.
-3.  the central monitoring solution needs dedicated resources, thus becoming "another bottleneck" in the whole
+3. the central monitoring solution needs dedicated resources, thus becoming "another bottleneck" in the whole
     ecosystem. It also requires maintenance, administration, etc.
-4.  most centralized monitoring solutions are usually only good for presenting _statistics of past performance_ (i.e.
+4. most centralized monitoring solutions are usually only good for presenting _statistics of past performance_ (i.e.
     cannot be used for real-time performance troubleshooting).
 
 Netdata follows a different approach:
 
-1.  data collection happens per second
-2.  thousands of metrics per server are collected
-3.  data do not leave the server where they are collected
-4.  Netdata servers do not talk to each other
-5.  your browser connects all the Netdata servers
+1. data collection happens per second
+2. thousands of metrics per server are collected
+3. data do not leave the server where they are collected
+4. Netdata servers do not talk to each other
+5. your browser connects all the Netdata servers
 
 Using Netdata, your monitoring infrastructure is embedded on each server, limiting significantly the need of additional
 resources. Netdata is blazingly fast, very resource efficient and utilizes server resources that already exist and are
@@ -46,31 +37,30 @@ etc.) are propagated to the new server, so that the new dashboard will come with
 
 The registry keeps track of 4 entities:
 
-1.  **machines**: i.e. the Netdata installations (a random GUID generated by each Netdata the first time it starts; we
+1. **machines**: i.e. the Netdata installations (a random GUID generated by each Netdata the first time it starts; we
     call this **machine_guid**)
 
-  For each Netdata installation (each `machine_guid`) the registry keeps track of the different URLs it has accessed.
+    For each Netdata installation (each `machine_guid`) the registry keeps track of the different URLs it has accessed.
 
-2.  **persons**: i.e. the web browsers accessing the Netdata installations (a random GUID generated by the registry the
+2. **persons**: i.e. the web browsers accessing the Netdata installations (a random GUID generated by the registry the
     first time it sees a new web browser; we call this **person_guid**)
 
-  For each person, the registry keeps track of the Netdata installations it has accessed and their URLs. 
+    For each person, the registry keeps track of the Netdata installations it has accessed and their URLs.
 
-3.  **URLs** of Netdata installations (as seen by the web browsers)
+3. **URLs** of Netdata installations (as seen by the web browsers)
 
-  For each URL, the registry keeps the URL and nothing more. Each URL is linked to _persons_ and _machines_. The only
+    For each URL, the registry keeps the URL and nothing more. Each URL is linked to _persons_ and _machines_. The only
   way to find a URL is to know its **machine_guid** or have a **person_guid** it is linked to it.
 
-4.  **accounts**: i.e. the information used to sign-in via one of the available sign-in methods. Depending on the
-    method, this may include an email, or an email and a profile picture or avatar.
+4. **accounts**: i.e. the information used to sign-in via one of the available sign-in methods. Depending on the method, this may include an email, or an email and a profile picture or avatar.
 
 For _persons_/_accounts_ and _machines_, the registry keeps links to _URLs_, each link with 2 timestamps (first time
 seen, last time seen) and a counter (number of times it has been seen). *machines_, _persons_ and timestamps are stored
-in the Netdata registry regardless of whether you sign in or not. 
+in the Netdata registry regardless of whether you sign in or not.
 
 ## Who talks to the registry?
 
-Your web browser **only**! If sending this information is against your policies, you 
+Your web browser **only**! If sending this information is against your policies, you
 can [run your own registry](#run-your-own-registry)
 
 Your Netdata servers do not talk to the registry. This is a UML diagram of its operation:
@@ -95,7 +85,7 @@ We believe, it can do it...
 
 **To turn any Netdata into a registry**, edit `/etc/netdata/netdata.conf` and set:
 
-```conf
+```text
 [registry]
     enabled = yes
     registry to announce = http://your.registry:19999
@@ -106,7 +96,7 @@ Restart your Netdata to activate it.
 Then, you need to tell **all your other Netdata servers to advertise your registry**, instead of the default. To do
 this, on each of your Netdata servers, edit `/etc/netdata/netdata.conf` and set:
 
-```conf
+```text
 [registry]
     enabled = no
     registry to announce = http://your.registry:19999
@@ -120,7 +110,7 @@ This is it. You have your registry now.
 You may also want to give your server different names under the node menu (i.e. to have them sorted / grouped). You can
 change its registry name, by setting on each Netdata server:
 
-```conf
+```text
 [registry]
     registry hostname = Group1 - Master DB
 ```
@@ -131,7 +121,7 @@ So this server will appear in the node menu as `Group1 - Master DB`. The max nam
 
 Netdata v1.9+ support limiting access to the registry from given IPs, like this:
 
-```conf
+```text
 [registry]
     allow from = *
 ```
@@ -152,15 +142,16 @@ against the name-pattern.
 Please note that this process can be expensive on a machine that is serving many connections. The behaviour of the
 pattern matching can be controlled with the following setting:
 
-```conf
+```text
 [registry]
     allow by dns = heuristic
 ```
 
 The settings are:
--   `yes` allows the pattern to match DNS names.
--   `no` disables DNS matching for the patterns (they only match IP addresses).
--   `heuristic` will estimate if the patterns should match FQDNs by the presence or absence of `:`s or alpha-characters.
+
+- `yes` allows the pattern to match DNS names.
+- `no` disables DNS matching for the patterns (they only match IP addresses).
+- `heuristic` will estimate if the patterns should match FQDNs by the presence or absence of `:`s or alpha-characters.
 
 ### Where is the registry database stored?
 
@@ -168,14 +159,13 @@ The settings are:
 
 There can be up to 2 files:
 
--   `registry-log.db`, the transaction log
-
-      all incoming requests that affect the registry are saved in this file in real-time.
+- `registry-log.db`, the transaction log
 
--   `registry.db`, the database
+    all incoming requests that affect the registry are saved in this file in real-time.
 
-      every `[registry].registry save db every new entries` entries in `registry-log.db`, Netdata will save its database
-      to `registry.db` and empty `registry-log.db`.
+- `registry.db`, the database
+  
+    every `[registry].registry save db every new entries` entries in `registry-log.db`, Netdata will save its database to `registry.db` and empty `registry-log.db`.
 
 Both files are machine readable text files.
 
@@ -183,9 +173,9 @@ Both files are machine readable text files.
 
 Beginning with `v1.30.0`, when the Netdata Agent's web server processes a request, it delivers the `SameSite=none`
 and `Secure` cookies. If you have problems accessing the local Agent dashboard or Netdata Cloud, disable these
-cookies by [editing `netdata.conf`](/docs/netdata-agent/configuration/README.md#edit-netdataconf):
+cookies by [editing `netdata.conf`](/docs/netdata-agent/configuration/README.md#edit-a-configuration-file-using-edit-config):
 
-```conf
+```text
 [registry]
     enable cookies SameSite and Secure = no
 ```
@@ -208,10 +198,8 @@ redirecting the browser back to itself hoping that it will receive the cookie. I
 registry will keep redirecting your web browser back to itself, which after a few redirects will fail with an error like
 this:
 
-```conf
+```text
 ERROR 409: Cannot ACCESS netdata registry: https://registry.my-netdata.io responded with: {"status":"redirect","registry":"https://registry.my-netdata.io"}
 ```
 
 This error is printed on your web browser console (press F12 on your browser to see it).
-
-
diff --git a/src/registry/registry.c b/src/registry/registry.c
index 803115231..be8d6948f 100644
--- a/src/registry/registry.c
+++ b/src/registry/registry.c
@@ -154,8 +154,8 @@ static inline int registry_person_url_callback_verify_machine_exists(REGISTRY_PE
 // that could make this safe, so try to be as atomic as possible.
 
 void registry_update_cloud_base_url() {
-    registry.cloud_base_url = appconfig_get(&cloud_config, CONFIG_SECTION_GLOBAL, "cloud base url", DEFAULT_CLOUD_BASE_URL);
-    setenv("NETDATA_REGISTRY_CLOUD_BASE_URL", registry.cloud_base_url, 1);
+    registry.cloud_base_url = cloud_config_url_get();
+    nd_setenv("NETDATA_REGISTRY_CLOUD_BASE_URL", registry.cloud_base_url, 1);
 }
 
 // ----------------------------------------------------------------------------
@@ -164,21 +164,19 @@ void registry_update_cloud_base_url() {
 int registry_request_hello_json(RRDHOST *host, struct web_client *w, bool do_not_track) {
     registry_json_header(host, w, "hello", REGISTRY_STATUS_OK);
 
-    if(host->node_id)
-        buffer_json_member_add_uuid(w->response.data, "node_id", host->node_id);
+    if(!UUIDiszero(host->node_id))
+        buffer_json_member_add_uuid(w->response.data, "node_id", host->node_id.uuid);
 
     buffer_json_member_add_object(w->response.data, "agent");
     {
         buffer_json_member_add_string(w->response.data, "machine_guid", localhost->machine_guid);
 
-        if(localhost->node_id)
-            buffer_json_member_add_uuid(w->response.data, "node_id", localhost->node_id);
+        if(!UUIDiszero(localhost->node_id))
+            buffer_json_member_add_uuid(w->response.data, "node_id", localhost->node_id.uuid);
 
-        char *claim_id = get_agent_claimid();
-        if (claim_id) {
-            buffer_json_member_add_string(w->response.data, "claim_id", claim_id);
-            freez(claim_id);
-        }
+        CLAIM_ID claim_id = claim_id_get();
+        if (claim_id_is_set(claim_id))
+            buffer_json_member_add_string(w->response.data, "claim_id", claim_id.str);
 
         buffer_json_member_add_boolean(w->response.data, "bearer_protection", netdata_is_protected_by_bearer);
     }
@@ -198,8 +196,8 @@ int registry_request_hello_json(RRDHOST *host, struct web_client *w, bool do_not
         buffer_json_add_array_item_object(w->response.data);
         buffer_json_member_add_string(w->response.data, "machine_guid", h->machine_guid);
 
-        if(h->node_id)
-            buffer_json_member_add_uuid(w->response.data, "node_id", h->node_id);
+        if(!UUIDiszero(h->node_id))
+            buffer_json_member_add_uuid(w->response.data, "node_id", h->node_id.uuid);
 
         buffer_json_member_add_string(w->response.data, "hostname", rrdhost_registry_hostname(h));
         buffer_json_object_close(w->response.data);
@@ -519,16 +517,16 @@ void registry_statistics(void) {
         rrddim_add(stm, "machines_urls",  NULL,  1, 1024, RRD_ALGORITHM_ABSOLUTE);
     }
 
-    struct aral_statistics *p_aral_stats = aral_statistics(registry.persons_aral);
+    struct aral_statistics *p_aral_stats = aral_get_statistics(registry.persons_aral);
     rrddim_set(stm, "persons",       (collected_number)p_aral_stats->structures.allocated_bytes + (collected_number)p_aral_stats->malloc.allocated_bytes + (collected_number)p_aral_stats->mmap.allocated_bytes);
 
-    struct aral_statistics *m_aral_stats = aral_statistics(registry.machines_aral);
+    struct aral_statistics *m_aral_stats = aral_get_statistics(registry.machines_aral);
     rrddim_set(stm, "machines",      (collected_number)m_aral_stats->structures.allocated_bytes + (collected_number)m_aral_stats->malloc.allocated_bytes + (collected_number)m_aral_stats->mmap.allocated_bytes);
 
-    struct aral_statistics *pu_aral_stats = aral_statistics(registry.person_urls_aral);
+    struct aral_statistics *pu_aral_stats = aral_get_statistics(registry.person_urls_aral);
     rrddim_set(stm, "persons_urls",  (collected_number)pu_aral_stats->structures.allocated_bytes + (collected_number)pu_aral_stats->malloc.allocated_bytes + (collected_number)pu_aral_stats->mmap.allocated_bytes);
 
-    struct aral_statistics *mu_aral_stats = aral_statistics(registry.machine_urls_aral);
+    struct aral_statistics *mu_aral_stats = aral_get_statistics(registry.machine_urls_aral);
     rrddim_set(stm, "machines_urls", (collected_number)mu_aral_stats->structures.allocated_bytes + (collected_number)mu_aral_stats->malloc.allocated_bytes + (collected_number)mu_aral_stats->mmap.allocated_bytes);
 
     rrdset_done(stm);
diff --git a/src/registry/registry.h b/src/registry/registry.h
index 848eb0ac0..b2eb7c00d 100644
--- a/src/registry/registry.h
+++ b/src/registry/registry.h
@@ -76,7 +76,7 @@ void registry_statistics(void);
 
 const char *registry_get_this_machine_guid(void);
 char *registry_get_mgmt_api_key(void);
-char *registry_get_this_machine_hostname(void);
+const char *registry_get_this_machine_hostname(void);
 
 int regenerate_guid(const char *guid, char *result);
 
diff --git a/src/registry/registry_init.c b/src/registry/registry_init.c
index c291c6f82..b98c04bea 100644
--- a/src/registry/registry_init.c
+++ b/src/registry/registry_init.c
@@ -93,7 +93,7 @@ int registry_init(void) {
 
     // configuration options
     registry.save_registry_every_entries = (unsigned long long)config_get_number(CONFIG_SECTION_REGISTRY, "registry save db every new entries", 1000000);
-    registry.persons_expiration = config_get_number(CONFIG_SECTION_REGISTRY, "registry expire idle persons days", 365) * 86400;
+    registry.persons_expiration = config_get_duration_days(CONFIG_SECTION_REGISTRY, "registry expire idle persons", 365) * 86400;
     registry.registry_domain = config_get(CONFIG_SECTION_REGISTRY, "registry domain", "");
     registry.registry_to_announce = config_get(CONFIG_SECTION_REGISTRY, "registry to announce", "https://registry.my-netdata.io");
     registry.hostname = config_get(CONFIG_SECTION_REGISTRY, "registry hostname", netdata_configured_hostname);
@@ -101,8 +101,8 @@ int registry_init(void) {
     registry.enable_cookies_samesite_secure = config_get_boolean(CONFIG_SECTION_REGISTRY, "enable cookies SameSite and Secure", 1);
 
     registry_update_cloud_base_url();
-    setenv("NETDATA_REGISTRY_HOSTNAME", registry.hostname, 1);
-    setenv("NETDATA_REGISTRY_URL", registry.registry_to_announce, 1);
+    nd_setenv("NETDATA_REGISTRY_HOSTNAME", registry.hostname, 1);
+    nd_setenv("NETDATA_REGISTRY_URL", registry.registry_to_announce, 1);
 
     registry.max_url_length = (size_t)config_get_number(CONFIG_SECTION_REGISTRY, "max URL length", 1024);
     if(registry.max_url_length < 10) {
diff --git a/src/registry/registry_internals.c b/src/registry/registry_internals.c
index 54fad4254..51a861866 100644
--- a/src/registry/registry_internals.c
+++ b/src/registry/registry_internals.c
@@ -266,7 +266,7 @@ static inline int is_machine_guid_blacklisted(const char *guid) {
     return 0;
 }
 
-char *registry_get_this_machine_hostname(void) {
+const char *registry_get_this_machine_hostname(void) {
     return registry.hostname;
 }
 
@@ -315,7 +315,7 @@ const char *registry_get_this_machine_guid(void) {
         close(fd);
     }
 
-    setenv("NETDATA_REGISTRY_UNIQUE_ID", guid, 1);
+    nd_setenv("NETDATA_REGISTRY_UNIQUE_ID", guid, 1);
 
     return guid;
 }
diff --git a/src/registry/registry_internals.h b/src/registry/registry_internals.h
index c7f8f43dd..39d37e4f0 100644
--- a/src/registry/registry_internals.h
+++ b/src/registry/registry_internals.h
@@ -30,10 +30,10 @@ struct registry {
 
     // configuration
     unsigned long long save_registry_every_entries;
-    char *registry_domain;
-    char *hostname;
-    char *registry_to_announce;
-    char *cloud_base_url;
+    const char *registry_domain;
+    const char *hostname;
+    const char *registry_to_announce;
+    const char *cloud_base_url;
     time_t persons_expiration; // seconds to expire idle persons
     int verify_cookies_redirects;
     int enable_cookies_samesite_secure;
@@ -42,10 +42,10 @@ struct registry {
     size_t max_name_length;
 
     // file/path names
-    char *pathname;
-    char *db_filename;
-    char *log_filename;
-    char *machine_guid_filename;
+    const char *pathname;
+    const char *db_filename;
+    const char *log_filename;
+    const char *machine_guid_filename;
 
     // open files
     FILE *log_fp;