76 files changed, 67 insertions, 251 deletions

diff --git a/aclk/README.md b/aclk/README.md
deleted file mode 100644
index c853c2324..000000000
--- a/aclk/README.md
+++ /dev/null
@@ -1,133 +0,0 @@
-# Agent-Cloud link (ACLK)
-
-The Agent-Cloud link (ACLK) is the mechanism responsible for securely connecting a Netdata Agent to your web browser
-through Netdata Cloud. The ACLK establishes an outgoing secure WebSocket (WSS) connection to Netdata Cloud on port
-`443`. The ACLK is encrypted, safe, and _is only established if you connect your node_.
-
-The Cloud App lives at app.netdata.cloud which currently resolves to the following list of IPs: 
-
-- 54.198.178.11
-- 44.207.131.212
-- 44.196.50.41 
-
-> ### Caution
->
->This list of IPs can change without notice, we strongly advise you to whitelist following domains `app.netdata.cloud`, `mqtt.netdata.cloud`, if this is not an option in your case always verify the current domain resolution (e.g via the `host` command).
-
-For a guide to connecting a node using the ACLK, plus additional troubleshooting and reference information, read our [connect to Cloud
-documentation](https://github.com/netdata/netdata/blob/master/claim/README.md).
-
-## Data privacy
-
-[Data privacy](https://netdata.cloud/privacy/) is very important to us. We firmly believe that your data belongs to
-you. This is why **we don't store any metric data in Netdata Cloud**.
-
-All the data that you see in the web browser when using Netdata Cloud, is actually streamed directly from the Netdata Agent to the Netdata Cloud dashboard. The data passes through our systems, but it isn't stored.
-
-However, to be able to offer the stunning visualizations and advanced functionality of Netdata Cloud, it does store a limited number of _metadata_. Read more about our [security and privacy design](https://github.com/netdata/netdata/blob/master/docs/netdata-security.md).
-
-## Enable and configure the ACLK
-
-The ACLK is enabled by default, with its settings automatically configured and stored in the Agent's memory. No file is
-created at `/var/lib/netdata/cloud.d/cloud.conf` until you either connect a node or create it yourself. The default
-configuration uses two settings:
-
-```conf
-[global]
-  enabled = yes
-  cloud base url = https://app.netdata.cloud
-```
-
-If your Agent needs to use a proxy to access the internet, you must [set up a proxy for
-connecting to cloud](https://github.com/netdata/netdata/blob/master/claim/README.md#connect-through-a-proxy).
-
-You can configure following keys in the `netdata.conf` section `[cloud]`:
-```
-[cloud]
-  statistics = yes
-  query thread count = 2
-```
-
-- `statistics` enables/disables ACLK related statistics and their charts. You can disable this to save some space in the database and slightly reduce memory usage of Netdata Agent.
-- `query thread count` specifies the number of threads to process cloud queries. Increasing this setting is useful for nodes with many children (streaming), which can expect to handle more queries (and/or more complicated queries).
-
-## Disable the ACLK
-
-You have two options if you prefer to disable the ACLK and not use Netdata Cloud.
-
-### Disable at installation
-
-You can pass the `--disable-cloud` parameter to the Agent installation when using a kickstart script
-([kickstart.sh](https://github.com/netdata/netdata/blob/master/packaging/installer/methods/kickstart.md), or a [manual installation from
-Git](https://github.com/netdata/netdata/blob/master/packaging/installer/methods/manual.md).
-
-When you pass this parameter, the installer does not download or compile any extra libraries. Once running, the Agent
-kills the thread responsible for the ACLK and connecting behavior, and behaves as though the ACLK, and thus Netdata Cloud,
-does not exist.
-
-### Disable at runtime
-
-You can change a runtime setting in your `cloud.conf` file to disable the ACLK. This setting only stops the Agent from
-attempting any connection via the ACLK, but does not prevent the installer from downloading and compiling the ACLK's
-dependencies.
-
-The file typically exists at `/var/lib/netdata/cloud.d/cloud.conf`, but can change if you set a prefix during
-installation. To disable the ACLK, open that file and change the `enabled` setting to `no`:
-
-```conf
-[global]
-    enabled = no
-```
-
-If the file at `/var/lib/netdata/cloud.d/cloud.conf` doesn't exist, you need to create it. 
-
-Copy and paste the first two lines from below, which will change your prompt to `cat`.
-
-```bash
-cd /var/lib/netdata/cloud.d
-cat > cloud.conf << EOF
-```
-
-Copy and paste in lines 3-6, and after the final `EOF`, hit **Enter**. The final line must contain only `EOF`. Hit **Enter** again to return to your normal prompt with the newly-created file.
-
-To get your normal prompt back, the final line
-must contain only `EOF`.
-
-```bash
-[global]
-    enabled = no
-    cloud base url = https://app.netdata.cloud
-EOF
-```
-
-You also need to change the file's permissions. Use `grep "run as user" /etc/netdata/netdata.conf` to figure out which
-user your Agent runs as (typically `netdata`), and replace `netdata:netdata` as shown below if necessary:
-
-```bash
-sudo chmod 0770 cloud.conf
-sudo chown netdata:netdata cloud.conf
-```
-
-Restart your Agent to disable the ACLK.
-
-### Re-enable the ACLK
-
-If you first disable the ACLK and any Cloud functionality and then decide you would like to use Cloud, you must either
-[reinstall Netdata](https://github.com/netdata/netdata/blob/master/packaging/installer/REINSTALL.md) with Cloud enabled or change the runtime setting in your
-`cloud.conf` file.
-
-If you passed `--disable-cloud` to `netdata-installer.sh` during installation, you must
-[reinstall](https://github.com/netdata/netdata/blob/master/packaging/installer/REINSTALL.md) your Agent. Use the same method as before, but pass `--require-cloud` to
-the installer. When installation finishes you can [connect your node](https://github.com/netdata/netdata/blob/master/claim/README.md#how-to-connect-a-node).
-
-If you changed the runtime setting in your `var/lib/netdata/cloud.d/cloud.conf` file, edit the file again and change
-`enabled` to `yes`:
-
-```conf
-[global]
-    enabled = yes
-```
-
-Restart your Agent and [connect your node](https://github.com/netdata/netdata/blob/master/claim/README.md#how-to-connect-a-node).
-
-
diff --git a/aclk/schema-wrappers/schema_wrapper_utils.h b/aclk/schema-wrappers/schema_wrapper_utils.h
deleted file mode 100644
index 2815d0f20..000000000
--- a/aclk/schema-wrappers/schema_wrapper_utils.h
+++ /dev/null
@@ -1,24 +0,0 @@
-// SPDX-License-Identifier: GPL-3.0-or-later
-
-#ifndef SCHEMA_WRAPPER_UTILS_H
-#define SCHEMA_WRAPPER_UTILS_H
-
-#include "database/rrd.h"
-
-#include <sys/time.h>
-#include <google/protobuf/timestamp.pb.h>
-#include <google/protobuf/map.h>
-
-#if GOOGLE_PROTOBUF_VERSION < 3001000
-#define PROTO_COMPAT_MSG_SIZE(msg) (size_t)msg.ByteSize();
-#define PROTO_COMPAT_MSG_SIZE_PTR(msg) (size_t)msg->ByteSize();
-#else
-#define PROTO_COMPAT_MSG_SIZE(msg) msg.ByteSizeLong();
-#define PROTO_COMPAT_MSG_SIZE_PTR(msg) msg->ByteSizeLong();
-#endif
-
-void set_google_timestamp_from_timeval(struct timeval tv, google::protobuf::Timestamp *ts);
-void set_timeval_from_google_timestamp(const google::protobuf::Timestamp &ts, struct timeval *tv);
-int label_add_to_map_callback(const char *name, const char *value, RRDLABEL_SRC ls, void *data);
-
-#endif /* SCHEMA_WRAPPER_UTILS_H */
diff --git a/aclk/aclk-schemas/.gitignore b/src/aclk/aclk-schemas/.gitignore
index bd495e9a7..bd495e9a7 100644
--- a/aclk/aclk-schemas/.gitignore
+++ b/src/aclk/aclk-schemas/.gitignore
diff --git a/aclk/aclk-schemas/.travis.yml b/src/aclk/aclk-schemas/.travis.yml
index 7c99550fe..7c99550fe 100644
--- a/aclk/aclk-schemas/.travis.yml
+++ b/src/aclk/aclk-schemas/.travis.yml
diff --git a/aclk/aclk-schemas/LICENSE b/src/aclk/aclk-schemas/LICENSE
index f288702d2..f288702d2 100644
--- a/aclk/aclk-schemas/LICENSE
+++ b/src/aclk/aclk-schemas/LICENSE
diff --git a/aclk/aclk-schemas/Makefile b/src/aclk/aclk-schemas/Makefile
index 8f4003070..8f4003070 100644
--- a/aclk/aclk-schemas/Makefile
+++ b/src/aclk/aclk-schemas/Makefile
diff --git a/aclk/aclk-schemas/README.md b/src/aclk/aclk-schemas/README.md
index aa6188977..aa6188977 100644
--- a/aclk/aclk-schemas/README.md
+++ b/src/aclk/aclk-schemas/README.md
diff --git a/aclk/aclk-schemas/buf.yml b/src/aclk/aclk-schemas/buf.yml
index 532053d91..532053d91 100644
--- a/aclk/aclk-schemas/buf.yml
+++ b/src/aclk/aclk-schemas/buf.yml
diff --git a/aclk/aclk-schemas/proto/aclk/v1/lib.proto b/src/aclk/aclk-schemas/proto/aclk/v1/lib.proto
index f32c32c6e..f32c32c6e 100644
--- a/aclk/aclk-schemas/proto/aclk/v1/lib.proto
+++ b/src/aclk/aclk-schemas/proto/aclk/v1/lib.proto
diff --git a/aclk/aclk-schemas/proto/agent/v1/cmds.proto b/src/aclk/aclk-schemas/proto/agent/v1/cmds.proto
index c37c00c3a..c37c00c3a 100644
--- a/aclk/aclk-schemas/proto/agent/v1/cmds.proto
+++ b/src/aclk/aclk-schemas/proto/agent/v1/cmds.proto
diff --git a/aclk/aclk-schemas/proto/agent/v1/connection.proto b/src/aclk/aclk-schemas/proto/agent/v1/connection.proto
index 4321b0b90..4321b0b90 100644
--- a/aclk/aclk-schemas/proto/agent/v1/connection.proto
+++ b/src/aclk/aclk-schemas/proto/agent/v1/connection.proto
diff --git a/aclk/aclk-schemas/proto/agent/v1/disconnect.proto b/src/aclk/aclk-schemas/proto/agent/v1/disconnect.proto
index 852ef702a..852ef702a 100644
--- a/aclk/aclk-schemas/proto/agent/v1/disconnect.proto
+++ b/src/aclk/aclk-schemas/proto/agent/v1/disconnect.proto
diff --git a/aclk/aclk-schemas/proto/alarm/v1/config.proto b/src/aclk/aclk-schemas/proto/alarm/v1/config.proto
index 430078fcf..430078fcf 100644
--- a/aclk/aclk-schemas/proto/alarm/v1/config.proto
+++ b/src/aclk/aclk-schemas/proto/alarm/v1/config.proto
diff --git a/aclk/aclk-schemas/proto/alarm/v1/stream.proto b/src/aclk/aclk-schemas/proto/alarm/v1/stream.proto
index 44f190682..44f190682 100644
--- a/aclk/aclk-schemas/proto/alarm/v1/stream.proto
+++ b/src/aclk/aclk-schemas/proto/alarm/v1/stream.proto
diff --git a/aclk/aclk-schemas/proto/chart/v1/config.proto b/src/aclk/aclk-schemas/proto/chart/v1/config.proto
index f0c5e3a35..f0c5e3a35 100644
--- a/aclk/aclk-schemas/proto/chart/v1/config.proto
+++ b/src/aclk/aclk-schemas/proto/chart/v1/config.proto
diff --git a/aclk/aclk-schemas/proto/chart/v1/dimension.proto b/src/aclk/aclk-schemas/proto/chart/v1/dimension.proto
index 8bcb564b8..8bcb564b8 100644
--- a/aclk/aclk-schemas/proto/chart/v1/dimension.proto
+++ b/src/aclk/aclk-schemas/proto/chart/v1/dimension.proto
diff --git a/aclk/aclk-schemas/proto/chart/v1/instance.proto b/src/aclk/aclk-schemas/proto/chart/v1/instance.proto
index 25c99e7c7..25c99e7c7 100644
--- a/aclk/aclk-schemas/proto/chart/v1/instance.proto
+++ b/src/aclk/aclk-schemas/proto/chart/v1/instance.proto
diff --git a/aclk/aclk-schemas/proto/chart/v1/stream.proto b/src/aclk/aclk-schemas/proto/chart/v1/stream.proto
index 9473538f2..9473538f2 100644
--- a/aclk/aclk-schemas/proto/chart/v1/stream.proto
+++ b/src/aclk/aclk-schemas/proto/chart/v1/stream.proto
diff --git a/aclk/aclk-schemas/proto/context/v1/context.proto b/src/aclk/aclk-schemas/proto/context/v1/context.proto
index eb771f8eb..eb771f8eb 100644
--- a/aclk/aclk-schemas/proto/context/v1/context.proto
+++ b/src/aclk/aclk-schemas/proto/context/v1/context.proto
diff --git a/aclk/aclk-schemas/proto/context/v1/stream.proto b/src/aclk/aclk-schemas/proto/context/v1/stream.proto
index a6e7e3abf..a6e7e3abf 100644
--- a/aclk/aclk-schemas/proto/context/v1/stream.proto
+++ b/src/aclk/aclk-schemas/proto/context/v1/stream.proto
diff --git a/aclk/aclk-schemas/proto/nodeinstance/connection/v1/connection.proto b/src/aclk/aclk-schemas/proto/nodeinstance/connection/v1/connection.proto
index f0c02461e..f0c02461e 100644
--- a/aclk/aclk-schemas/proto/nodeinstance/connection/v1/connection.proto
+++ b/src/aclk/aclk-schemas/proto/nodeinstance/connection/v1/connection.proto
diff --git a/aclk/aclk-schemas/proto/nodeinstance/create/v1/creation.proto b/src/aclk/aclk-schemas/proto/nodeinstance/create/v1/creation.proto
index 922337154..922337154 100644
--- a/aclk/aclk-schemas/proto/nodeinstance/create/v1/creation.proto
+++ b/src/aclk/aclk-schemas/proto/nodeinstance/create/v1/creation.proto
diff --git a/aclk/aclk-schemas/proto/nodeinstance/info/v1/info.proto b/src/aclk/aclk-schemas/proto/nodeinstance/info/v1/info.proto
index 7aa9d0448..7aa9d0448 100644
--- a/aclk/aclk-schemas/proto/nodeinstance/info/v1/info.proto
+++ b/src/aclk/aclk-schemas/proto/nodeinstance/info/v1/info.proto
diff --git a/aclk/aclk.c b/src/aclk/aclk.c
index e95d7d6ab..6d583a76a 100644
--- a/aclk/aclk.c
+++ b/src/aclk/aclk.c
@@ -4,7 +4,7 @@
 
 #ifdef ENABLE_ACLK
 #include "aclk_stats.h"
-#include "mqtt_wss_client.h"
+#include "mqtt_websockets/mqtt_wss_client.h"
 #include "aclk_otp.h"
 #include "aclk_tx_msgs.h"
 #include "aclk_query.h"
diff --git a/aclk/aclk.h b/src/aclk/aclk.h
index 72d1a2e11..72d1a2e11 100644
--- a/aclk/aclk.h
+++ b/src/aclk/aclk.h
diff --git a/aclk/aclk_alarm_api.c b/src/aclk/aclk_alarm_api.c
index 664671f70..664671f70 100644
--- a/aclk/aclk_alarm_api.c
+++ b/src/aclk/aclk_alarm_api.c
diff --git a/aclk/aclk_alarm_api.h b/src/aclk/aclk_alarm_api.h
index 4d9d9447a..4d9d9447a 100644
--- a/aclk/aclk_alarm_api.h
+++ b/src/aclk/aclk_alarm_api.h
diff --git a/aclk/aclk_capas.c b/src/aclk/aclk_capas.c
index e8d85a2b9..00102ad4a 100644
--- a/aclk/aclk_capas.c
+++ b/src/aclk/aclk_capas.c
@@ -18,6 +18,7 @@ const struct capability *aclk_get_agent_capas()
         { .name = "http_api_v2", .version = HTTP_API_V2_VERSION, .enabled = 1 },
         { .name = "health",      .version = 1, .enabled = 0 }, // index 7, below
         { .name = "req_cancel",  .version = 1, .enabled = 1 },
+        { .name = "dyncfg",      .version = 2, .enabled = 1 },
         { .name = NULL,          .version = 0, .enabled = 0 }
     };
     agent_capabilities[2].version = ml_capable() ? 1 : 0;
@@ -34,6 +35,7 @@ const struct capability *aclk_get_agent_capas()
 struct capability *aclk_get_node_instance_capas(RRDHOST *host)
 {
     bool functions = (host == localhost || (host->receiver && stream_has_capability(host->receiver, STREAM_CAP_FUNCTIONS)));
+    bool dyncfg = (host == localhost || dyncfg_available_for_rrdhost(host));
 
     struct capability ni_caps[] = {
         { .name = "proto",       .version = 1,                     .enabled = 1 },
@@ -46,6 +48,7 @@ struct capability *aclk_get_node_instance_capas(RRDHOST *host)
         { .name = "http_api_v2", .version = HTTP_API_V2_VERSION,   .enabled = 1 },
         { .name = "health",      .version = 1,                     .enabled = host->health.health_enabled },
         { .name = "req_cancel",  .version = 1,                     .enabled = 1 },
+        { .name = "dyncfg",      .version = 2,                     .enabled = dyncfg },
         { .name = NULL,          .version = 0,                     .enabled = 0 }
     };
 
diff --git a/aclk/aclk_capas.h b/src/aclk/aclk_capas.h
index c39a197b8..c39a197b8 100644
--- a/aclk/aclk_capas.h
+++ b/src/aclk/aclk_capas.h
diff --git a/aclk/aclk_contexts_api.c b/src/aclk/aclk_contexts_api.c
index f3344935e..f3344935e 100644
--- a/aclk/aclk_contexts_api.c
+++ b/src/aclk/aclk_contexts_api.c
diff --git a/aclk/aclk_contexts_api.h b/src/aclk/aclk_contexts_api.h
index f0b5ec77e..f0b5ec77e 100644
--- a/aclk/aclk_contexts_api.h
+++ b/src/aclk/aclk_contexts_api.h
diff --git a/aclk/aclk_otp.c b/src/aclk/aclk_otp.c
index 207ca08cf..e03f8b212 100644
--- a/aclk/aclk_otp.c
+++ b/src/aclk/aclk_otp.c
@@ -7,7 +7,7 @@
 
 #include "daemon/common.h"
 
-#include "mqtt_websockets/c-rbuf/include/ringbuffer.h"
+#include "mqtt_websockets/c-rbuf/cringbuffer.h"
 
 static int aclk_https_request(https_req_t *request, https_req_response_t *response) {
     int rc;
diff --git a/aclk/aclk_otp.h b/src/aclk/aclk_otp.h
index 2d660e5a4..2d660e5a4 100644
--- a/aclk/aclk_otp.h
+++ b/src/aclk/aclk_otp.h
diff --git a/aclk/aclk_proxy.c b/src/aclk/aclk_proxy.c
index 4af46208f..5d1a8b355 100644
--- a/aclk/aclk_proxy.c
+++ b/src/aclk/aclk_proxy.c
@@ -15,20 +15,6 @@ struct {
     { .type = PROXY_TYPE_UNKNOWN,  .url_str = NULL                                      },
 };
 
-const char *aclk_proxy_type_to_s(ACLK_PROXY_TYPE *type)
-{
-    switch (*type) {
-        case PROXY_DISABLED:
-            return "disabled";
-        case PROXY_TYPE_HTTP:
-            return "HTTP";
-        case PROXY_TYPE_SOCKS5:
-            return "SOCKS";
-        default:
-            return "Unknown";
-    }
-}
-
 static inline ACLK_PROXY_TYPE aclk_find_proxy(const char *string)
 {
     int i = 0;
diff --git a/aclk/aclk_proxy.h b/src/aclk/aclk_proxy.h
index b4ceb7df8..6877b526b 100644
--- a/aclk/aclk_proxy.h
+++ b/src/aclk/aclk_proxy.h
@@ -13,7 +13,6 @@ typedef enum aclk_proxy_type {
     PROXY_NOT_SET,
 } ACLK_PROXY_TYPE;
 
-const char *aclk_proxy_type_to_s(ACLK_PROXY_TYPE *type);
 ACLK_PROXY_TYPE aclk_verify_proxy(const char *string);
 const char *aclk_lws_wss_get_proxy_setting(ACLK_PROXY_TYPE *type);
 void safe_log_proxy_censor(char *proxy);
diff --git a/aclk/aclk_query.c b/src/aclk/aclk_query.c
index 5e3574b97..ae8435db7 100644
--- a/aclk/aclk_query.c
+++ b/src/aclk/aclk_query.c
@@ -99,30 +99,48 @@ static int http_api_v2(struct aclk_query_thread *query_thr, aclk_query_t query)
     BUFFER *local_buffer = NULL;
     size_t size = 0;
     size_t sent = 0;
+    usec_t dt_ut = 0;
 
     int z_ret;
     BUFFER *z_buffer = buffer_create(NETDATA_WEB_RESPONSE_INITIAL_SIZE, &netdata_buffers_statistics.buffers_aclk);
-    char *start, *end;
 
     struct web_client *w = web_client_get_from_cache();
-    w->acl = WEB_CLIENT_ACL_ACLK;
-    w->mode = WEB_CLIENT_MODE_GET;
+    web_client_set_conn_cloud(w);
+    w->port_acl = HTTP_ACL_ACLK | HTTP_ACL_ALL_FEATURES;
+    w->acl = w->port_acl;
+    web_client_set_permissions(w, HTTP_ACCESS_MAP_OLD_MEMBER, HTTP_USER_ROLE_MEMBER, WEB_CLIENT_FLAG_AUTH_CLOUD);
+
+    w->mode = HTTP_REQUEST_MODE_GET;
     w->timings.tv_in = query->created_tv;
 
     w->interrupt.callback = aclk_web_client_interrupt_cb;
     w->interrupt.callback_data = pending_req_list_add(query->msg_id);
 
-    usec_t t;
+    buffer_flush(w->response.data);
+    buffer_strcat(w->response.data, query->data.http_api_v2.payload);
+
+    HTTP_VALIDATION validation = http_request_validate(w);
+    if(validation != HTTP_VALIDATION_OK) {
+        nd_log(NDLS_ACCESS, NDLP_ERR, "ACLK received request is not valid, code %d", validation);
+        retval = 1;
+        w->response.code = HTTP_RESP_BAD_REQUEST;
+        w->response.code = (short)aclk_http_msg_v2(query_thr->client, query->callback_topic, query->msg_id,
+                                                   dt_ut, query->created, w->response.code,
+                                                   NULL, 0);
+        goto cleanup;
+    }
+
     web_client_timeout_checkpoint_set(w, query->timeout);
-    if(web_client_timeout_checkpoint_and_check(w, &t)) {
-        nd_log(NDLS_ACCESS, NDLP_ERR, "QUERY CANCELED: QUEUE TIME EXCEEDED %llu ms (LIMIT %d ms)", t / USEC_PER_MS, query->timeout);
+    if(web_client_timeout_checkpoint_and_check(w, &dt_ut)) {
+        nd_log(NDLS_ACCESS, NDLP_ERR,
+               "QUERY CANCELED: QUEUE TIME EXCEEDED %llu ms (LIMIT %d ms)",
+               dt_ut / USEC_PER_MS, query->timeout);
         retval = 1;
         w->response.code = HTTP_RESP_SERVICE_UNAVAILABLE;
         aclk_http_msg_v2_err(query_thr->client, query->callback_topic, query->msg_id, w->response.code, CLOUD_EC_SND_TIMEOUT, CLOUD_EMSG_SND_TIMEOUT, NULL, 0);
         goto cleanup;
     }
 
-    web_client_decode_path_and_query_string(w, query->data.http_api_v2.query);
     char *path = (char *)buffer_tostring(w->url_path_decoded);
 
     if (aclk_stats_enabled) {
@@ -134,41 +152,24 @@ static int http_api_v2(struct aclk_query_thread *query_thr, aclk_query_t query)
     }
 
     w->response.code = (short)web_client_api_request_with_node_selection(localhost, w, path);
-    web_client_timeout_checkpoint_response_ready(w, &t);
+    web_client_timeout_checkpoint_response_ready(w, &dt_ut);
 
     if (aclk_stats_enabled) {
         ACLK_STATS_LOCK;
-        aclk_metrics_per_sample.cloud_q_process_total += t;
+        aclk_metrics_per_sample.cloud_q_process_total += dt_ut;
         aclk_metrics_per_sample.cloud_q_process_count++;
-        if (aclk_metrics_per_sample.cloud_q_process_max < t)
-            aclk_metrics_per_sample.cloud_q_process_max = t;
+        if (aclk_metrics_per_sample.cloud_q_process_max < dt_ut)
+            aclk_metrics_per_sample.cloud_q_process_max = dt_ut;
         ACLK_STATS_UNLOCK;
     }
 
     size = w->response.data->len;
     sent = size;
 
-    // check if gzip encoding can and should be used
-    if ((start = strstr((char *)query->data.http_api_v2.payload, WEB_HDR_ACCEPT_ENC))) {
-        start += strlen(WEB_HDR_ACCEPT_ENC);
-        end = strstr(start, "\x0D\x0A");
-        start = strstr(start, "gzip");
-
-        if (start && start < end) {
-            w->response.zstream.zalloc = Z_NULL;
-            w->response.zstream.zfree = Z_NULL;
-            w->response.zstream.opaque = Z_NULL;
-            if(deflateInit2(&w->response.zstream, web_gzip_level, Z_DEFLATED, 15 + 16, 8, web_gzip_strategy) == Z_OK) {
-                w->response.zinitialized = true;
-                w->response.zoutput = true;
-            } else
-                netdata_log_error("Failed to initialize zlib. Proceeding without compression.");
-        }
-    }
-
     if (w->response.data->len && w->response.zinitialized) {
         w->response.zstream.next_in = (Bytef *)w->response.data->buffer;
         w->response.zstream.avail_in = w->response.data->len;
+
         do {
             w->response.zstream.avail_out = NETDATA_WEB_RESPONSE_ZLIB_CHUNK_SIZE;
             w->response.zstream.next_out = w->response.zbuffer;
@@ -188,6 +189,7 @@ static int http_api_v2(struct aclk_query_thread *query_thr, aclk_query_t query)
             memcpy(&z_buffer->buffer[z_buffer->len], w->response.zbuffer, bytes_to_cpy);
             z_buffer->len += bytes_to_cpy;
         } while(z_ret != Z_STREAM_END);
+
         // so that web_client_build_http_header
         // puts correct content length into header
         buffer_free(w->response.data);
@@ -213,7 +215,9 @@ static int http_api_v2(struct aclk_query_thread *query_thr, aclk_query_t query)
     }
 
     // send msg.
-    w->response.code = aclk_http_msg_v2(query_thr->client, query->callback_topic, query->msg_id, t, query->created, w->response.code, local_buffer->buffer, local_buffer->len);
+    w->response.code = (short)aclk_http_msg_v2(query_thr->client, query->callback_topic, query->msg_id,
+        dt_ut, query->created, w->response.code,
+                                               local_buffer->buffer, local_buffer->len);
 
 cleanup:
     web_client_log_completed_request(w, false);
diff --git a/aclk/aclk_query.h b/src/aclk/aclk_query.h
index dbe6f9e5e..371b8d0a2 100644
--- a/aclk/aclk_query.h
+++ b/src/aclk/aclk_query.h
@@ -5,7 +5,7 @@
 
 #include "libnetdata/libnetdata.h"
 
-#include "mqtt_wss_client.h"
+#include "mqtt_websockets/mqtt_wss_client.h"
 
 #include "aclk_query_queue.h"
 
diff --git a/aclk/aclk_query_queue.c b/src/aclk/aclk_query_queue.c
index 8ca21d456..3edadc002 100644
--- a/aclk/aclk_query_queue.c
+++ b/src/aclk/aclk_query_queue.c
@@ -10,11 +10,9 @@ static netdata_mutex_t aclk_query_queue_mutex = NETDATA_MUTEX_INITIALIZER;
 
 static struct aclk_query_queue {
     aclk_query_t head;
-    aclk_query_t tail;
     int block_push;
 } aclk_query_queue = {
     .head = NULL,
-    .tail = NULL,
     .block_push = 0
 };
 
@@ -31,15 +29,7 @@ static inline int _aclk_queue_query(aclk_query_t query)
         aclk_query_free(query);
         return 1;
     }
-    if (!aclk_query_queue.head) {
-        aclk_query_queue.head = query;
-        aclk_query_queue.tail = query;
-        ACLK_QUEUE_UNLOCK;
-        return 0;
-    }
-    // TODO deduplication
-    aclk_query_queue.tail->next = query;
-    aclk_query_queue.tail = query;
+    DOUBLE_LINKED_LIST_APPEND_ITEM_UNSAFE(aclk_query_queue.head, query, prev, next);
     ACLK_QUEUE_UNLOCK;
     return 0;
 
@@ -77,9 +67,7 @@ aclk_query_t aclk_queue_pop(void)
         return ret;
     }
 
-    aclk_query_queue.head = ret->next;
-    if (unlikely(!aclk_query_queue.head))
-        aclk_query_queue.tail = aclk_query_queue.head;
+    DOUBLE_LINKED_LIST_REMOVE_ITEM_UNSAFE(aclk_query_queue.head, ret, prev, next);
     ACLK_QUEUE_UNLOCK;
 
     ret->next = NULL;
@@ -92,7 +80,7 @@ void aclk_queue_flush(void)
     while (query) {
         aclk_query_free(query);
         query = aclk_queue_pop();
-    };
+    }
 }
 
 aclk_query_t aclk_query_new(aclk_query_type_t type)
diff --git a/aclk/aclk_query_queue.h b/src/aclk/aclk_query_queue.h
index 5983561a6..4a4a36a3f 100644
--- a/aclk/aclk_query_queue.h
+++ b/src/aclk/aclk_query_queue.h
@@ -55,7 +55,7 @@ struct aclk_query {
     struct timeval created_tv;
     usec_t created;
     int timeout;
-    aclk_query_t next;
+    aclk_query_t prev, next;
 
     // TODO maybe remove?
     int version;
@@ -75,12 +75,13 @@ void aclk_queue_flush(void);
 void aclk_queue_lock(void);
 void aclk_queue_unlock(void);
 
-#define QUEUE_IF_PAYLOAD_PRESENT(query)                                                                                \
+#define QUEUE_IF_PAYLOAD_PRESENT(query) do {                                                                           \
     if (likely(query->data.bin_payload.payload)) {                                                                     \
         aclk_queue_query(query);                                                                                       \
     } else {                                                                                                           \
-        netdata_log_error("Failed to generate payload (%s)", __FUNCTION__);                                            \
+        nd_log(NDLS_DAEMON, NDLP_ERR, "Failed to generate payload");                                                   \
         aclk_query_free(query);                                                                                        \
-    }
+    }                                                                                                                  \
+} while(0)
 
 #endif /* NETDATA_ACLK_QUERY_QUEUE_H */
diff --git a/aclk/aclk_rrdhost_state.h b/src/aclk/aclk_rrdhost_state.h
index 5c8a2ddc9..5c8a2ddc9 100644
--- a/aclk/aclk_rrdhost_state.h
+++ b/src/aclk/aclk_rrdhost_state.h
diff --git a/aclk/aclk_rx_msgs.c b/src/aclk/aclk_rx_msgs.c
index 0e91e28c0..fb6a2b902 100644
--- a/aclk/aclk_rx_msgs.c
+++ b/src/aclk/aclk_rx_msgs.c
@@ -11,7 +11,6 @@
 #include "schema-wrappers/proto_2_json.h"
 
 #define ACLK_V2_PAYLOAD_SEPARATOR "\x0D\x0A\x0D\x0A"
-#define ACLK_CLOUD_REQ_V2_PREFIX "GET /"
 
 #define ACLK_V_COMPRESSION 2
 
@@ -100,13 +99,17 @@ static inline int aclk_v2_payload_get_query(const char *payload, char **query_ur
 {
     const char *start, *end;
 
-    // TODO better check of URL
-    if(strncmp(payload, ACLK_CLOUD_REQ_V2_PREFIX, strlen(ACLK_CLOUD_REQ_V2_PREFIX))) {
+    if(strncmp(payload, "GET /", 5) == 0 || strncmp(payload, "PUT /", 5) == 0)
+        start = payload + 4;
+    else if(strncmp(payload, "POST /", 6) == 0)
+        start = payload + 5;
+    else if(strncmp(payload, "DELETE /", 8) == 0)
+        start = payload + 7;
+    else {
         errno = 0;
-        netdata_log_error("Only accepting requests that start with \"%s\" from CLOUD.", ACLK_CLOUD_REQ_V2_PREFIX);
+        netdata_log_error("Only accepting requests that start with GET, POST, PUT, DELETE from CLOUD.");
         return 1;
     }
-    start = payload + 4;
 
     if(!(end = strstr(payload, HTTP_1_1 HTTP_ENDL))) {
         errno = 0;
@@ -196,7 +199,7 @@ int aclk_handle_cloud_cmd_message(char *payload)
 
     // Originally we were expecting to have multiple types of 'cmd' message,
     // but after the new protocol was designed we will ever only have 'http'
-    if (strcmp(cloud_to_agent.type_id, "http")) {
+    if (strcmp(cloud_to_agent.type_id, "http") != 0) {
         error_report("Only 'http' cmd message is supported");
         goto err_cleanup;
     }
diff --git a/aclk/aclk_rx_msgs.h b/src/aclk/aclk_rx_msgs.h
index 61921faec..61921faec 100644
--- a/aclk/aclk_rx_msgs.h
+++ b/src/aclk/aclk_rx_msgs.h
diff --git a/aclk/aclk_stats.c b/src/aclk/aclk_stats.c
index f4672882b..71c56f604 100644
--- a/aclk/aclk_stats.c
+++ b/src/aclk/aclk_stats.c
@@ -1,6 +1,8 @@
 // SPDX-License-Identifier: GPL-3.0-or-later
 
+#ifndef MQTT_WSS_CPUSTATS
 #define MQTT_WSS_CPUSTATS
+#endif
 
 #include "aclk_stats.h"
 
diff --git a/aclk/aclk_stats.h b/src/aclk/aclk_stats.h
index 002ebcfa6..758347ae2 100644
--- a/aclk/aclk_stats.h
+++ b/src/aclk/aclk_stats.h
@@ -6,7 +6,7 @@
 #include "daemon/common.h"
 #include "libnetdata/libnetdata.h"
 #include "aclk_query_queue.h"
-#include "mqtt_wss_client.h"
+#include "mqtt_websockets/mqtt_wss_client.h"
 
 extern netdata_mutex_t aclk_stats_mutex;
 
diff --git a/aclk/aclk_tx_msgs.c b/src/aclk/aclk_tx_msgs.c
index 0e4182a72..0e4182a72 100644
--- a/aclk/aclk_tx_msgs.c
+++ b/src/aclk/aclk_tx_msgs.c
diff --git a/aclk/aclk_tx_msgs.h b/src/aclk/aclk_tx_msgs.h
index 9e7d89077..86ed20c38 100644
--- a/aclk/aclk_tx_msgs.h
+++ b/src/aclk/aclk_tx_msgs.h
@@ -5,7 +5,7 @@
 #include <json-c/json.h>
 #include "libnetdata/libnetdata.h"
 #include "daemon/common.h"
-#include "mqtt_wss_client.h"
+#include "mqtt_websockets/mqtt_wss_client.h"
 #include "schema-wrappers/schema_wrappers.h"
 #include "aclk_util.h"
 
diff --git a/aclk/aclk_util.c b/src/aclk/aclk_util.c
index 3bf2e3f18..3bf2e3f18 100644
--- a/aclk/aclk_util.c
+++ b/src/aclk/aclk_util.c
diff --git a/aclk/aclk_util.h b/src/aclk/aclk_util.h
index 38ef5b0bc..6c0239cc3 100644
--- a/aclk/aclk_util.h
+++ b/src/aclk/aclk_util.h
@@ -5,7 +5,7 @@
 #include "libnetdata/libnetdata.h"
 
 #ifdef ENABLE_ACLK
-#include "mqtt_wss_client.h"
+#include "mqtt_websockets/mqtt_wss_client.h"
 
 #define CLOUD_EC_MALFORMED_NODE_ID   1
 #define CLOUD_EMSG_MALFORMED_NODE_ID "URL requests node_id but there is not enough chars following (for it to be valid uuid)."
diff --git a/aclk/helpers/mqtt_wss_pal.h b/src/aclk/helpers/mqtt_wss_pal.h
index 5c89f8bb7..5c89f8bb7 100644
--- a/aclk/helpers/mqtt_wss_pal.h
+++ b/src/aclk/helpers/mqtt_wss_pal.h
diff --git a/aclk/helpers/ringbuffer_pal.h b/src/aclk/helpers/ringbuffer_pal.h
index 2f7e1cb93..2f7e1cb93 100644
--- a/aclk/helpers/ringbuffer_pal.h
+++ b/src/aclk/helpers/ringbuffer_pal.h
diff --git a/aclk/https_client.c b/src/aclk/https_client.c
index 5385786b8..7fed1962a 100644
--- a/aclk/https_client.c
+++ b/src/aclk/https_client.c
@@ -8,8 +8,6 @@
 
 #include "daemon/global_statistics.h"
 
-#define DEFAULT_CHUNKED_RESPONSE_BUFFER_SIZE (4096)
-
 static const char *http_req_type_to_str(http_req_type_t req) {
     switch (req) {
         case HTTP_REQ_GET:
@@ -25,7 +23,6 @@ static const char *http_req_type_to_str(http_req_type_t req) {
 
 #define TRANSFER_ENCODING_CHUNKED (-2)
 
-#define HTTP_PARSE_CTX_INITIALIZER { .state = HTTP_PARSE_INITIAL, .content_length = -1, .http_code = 0 }
 void http_parse_ctx_create(http_parse_ctx *ctx)
 {
     ctx->state = HTTP_PARSE_INITIAL;
@@ -316,8 +313,6 @@ typedef struct https_req_ctx {
 
     size_t written;
 
-    int self_signed_allowed;
-
     http_parse_ctx parse_ctx;
 
     time_t req_start_time;
@@ -762,12 +757,6 @@ void https_req_response_free(https_req_response_t *res) {
     freez(res->payload);
 }
 
-void https_req_response_init(https_req_response_t *res) {
-    res->http_code = 0;
-    res->payload = NULL;
-    res->payload_size = 0;
-}
-
 static inline char *UNUSED_FUNCTION(min_non_null)(char *a, char *b) {
     if (!a)
         return b;
@@ -816,12 +805,11 @@ static inline void port_by_proto(url_t *url) {
     }
 }
 
-#define STRDUPZ_2PTR(dest, start, end)                                                                                 \
-    {                                                                                                                  \
+#define STRDUPZ_2PTR(dest, start, end) do {                                                                            \
         dest = mallocz(1 + end - start);                                                                               \
         memcpy(dest, start, end - start);                                                                              \
         dest[end - start] = 0;                                                                                         \
-    }
+    } while(0)
 
 int url_parse(const char *url, url_t *parsed) {
     const char *start = url;
@@ -833,7 +821,7 @@ int url_parse(const char *url, url_t *parsed) {
             return 1;
         }
 
-        STRDUPZ_2PTR(parsed->proto, start, end)
+        STRDUPZ_2PTR(parsed->proto, start, end);
         start = end + strlen(URI_PROTO_SEPARATOR);
     }
 
diff --git a/aclk/https_client.h b/src/aclk/https_client.h
index 0b97fbb02..79d0a42b7 100644
--- a/aclk/https_client.h
+++ b/src/aclk/https_client.h
@@ -5,8 +5,8 @@
 
 #include "libnetdata/libnetdata.h"
 
-#include "mqtt_websockets/c-rbuf/include/ringbuffer.h"
-#include "mqtt_websockets/c_rhash/include/c_rhash.h"
+#include "mqtt_websockets/c-rbuf/cringbuffer.h"
+#include "mqtt_websockets/c_rhash/c_rhash.h"
 
 typedef enum http_req_type {
     HTTP_REQ_GET = 0,
@@ -56,7 +56,6 @@ int url_parse(const char *url, url_t *parsed);
 void url_t_destroy(url_t *url);
 
 void https_req_response_free(https_req_response_t *res);
-void https_req_response_init(https_req_response_t *res);
 
 #define HTTPS_REQ_RESPONSE_T_INITIALIZER            \
     {                                               \
diff --git a/aclk/schema-wrappers/agent_cmds.cc b/src/aclk/schema-wrappers/agent_cmds.cc
index 6950f4025..6950f4025 100644
--- a/aclk/schema-wrappers/agent_cmds.cc
+++ b/src/aclk/schema-wrappers/agent_cmds.cc
diff --git a/aclk/schema-wrappers/agent_cmds.h b/src/aclk/schema-wrappers/agent_cmds.h
index 7e01f86c5..7e01f86c5 100644
--- a/aclk/schema-wrappers/agent_cmds.h
+++ b/src/aclk/schema-wrappers/agent_cmds.h
diff --git a/aclk/schema-wrappers/alarm_config.cc b/src/aclk/schema-wrappers/alarm_config.cc
index 64d28f324..64d28f324 100644
--- a/aclk/schema-wrappers/alarm_config.cc
+++ b/src/aclk/schema-wrappers/alarm_config.cc
diff --git a/aclk/schema-wrappers/alarm_config.h b/src/aclk/schema-wrappers/alarm_config.h
index 3c9a5d9a8..3c9a5d9a8 100644
--- a/aclk/schema-wrappers/alarm_config.h
+++ b/src/aclk/schema-wrappers/alarm_config.h
diff --git a/aclk/schema-wrappers/alarm_stream.cc b/src/aclk/schema-wrappers/alarm_stream.cc
index 29d80e39e..29d80e39e 100644
--- a/aclk/schema-wrappers/alarm_stream.cc
+++ b/src/aclk/schema-wrappers/alarm_stream.cc
diff --git a/aclk/schema-wrappers/alarm_stream.h b/src/aclk/schema-wrappers/alarm_stream.h
index 3c81ff445..3c81ff445 100644
--- a/aclk/schema-wrappers/alarm_stream.h
+++ b/src/aclk/schema-wrappers/alarm_stream.h
diff --git a/aclk/schema-wrappers/capability.cc b/src/aclk/schema-wrappers/capability.cc
index af45740a9..af45740a9 100644
--- a/aclk/schema-wrappers/capability.cc
+++ b/src/aclk/schema-wrappers/capability.cc
diff --git a/aclk/schema-wrappers/capability.h b/src/aclk/schema-wrappers/capability.h
index c6085a44b..c6085a44b 100644
--- a/aclk/schema-wrappers/capability.h
+++ b/src/aclk/schema-wrappers/capability.h
diff --git a/aclk/schema-wrappers/connection.cc b/src/aclk/schema-wrappers/connection.cc
index 20b40ece2..20b40ece2 100644
--- a/aclk/schema-wrappers/connection.cc
+++ b/src/aclk/schema-wrappers/connection.cc
diff --git a/aclk/schema-wrappers/connection.h b/src/aclk/schema-wrappers/connection.h
index 0356c7d78..0356c7d78 100644
--- a/aclk/schema-wrappers/connection.h
+++ b/src/aclk/schema-wrappers/connection.h
diff --git a/aclk/schema-wrappers/context.cc b/src/aclk/schema-wrappers/context.cc
index b04c9d20c..b04c9d20c 100644
--- a/aclk/schema-wrappers/context.cc
+++ b/src/aclk/schema-wrappers/context.cc
diff --git a/aclk/schema-wrappers/context.h b/src/aclk/schema-wrappers/context.h
index cbb7701a8..cbb7701a8 100644
--- a/aclk/schema-wrappers/context.h
+++ b/src/aclk/schema-wrappers/context.h
diff --git a/aclk/schema-wrappers/context_stream.cc b/src/aclk/schema-wrappers/context_stream.cc
index 3bb1956cb..3bb1956cb 100644
--- a/aclk/schema-wrappers/context_stream.cc
+++ b/src/aclk/schema-wrappers/context_stream.cc
diff --git a/aclk/schema-wrappers/context_stream.h b/src/aclk/schema-wrappers/context_stream.h
index 8c691d2cc..8c691d2cc 100644
--- a/aclk/schema-wrappers/context_stream.h
+++ b/src/aclk/schema-wrappers/context_stream.h
diff --git a/aclk/schema-wrappers/node_connection.cc b/src/aclk/schema-wrappers/node_connection.cc
index db1fa6449..db1fa6449 100644
--- a/aclk/schema-wrappers/node_connection.cc
+++ b/src/aclk/schema-wrappers/node_connection.cc
diff --git a/aclk/schema-wrappers/node_connection.h b/src/aclk/schema-wrappers/node_connection.h
index dac0d8fe0..dac0d8fe0 100644
--- a/aclk/schema-wrappers/node_connection.h
+++ b/src/aclk/schema-wrappers/node_connection.h
diff --git a/aclk/schema-wrappers/node_creation.cc b/src/aclk/schema-wrappers/node_creation.cc
index 5ad25b7e5..5ad25b7e5 100644
--- a/aclk/schema-wrappers/node_creation.cc
+++ b/src/aclk/schema-wrappers/node_creation.cc
diff --git a/aclk/schema-wrappers/node_creation.h b/src/aclk/schema-wrappers/node_creation.h
index 7a8c7f7c7..7a8c7f7c7 100644
--- a/aclk/schema-wrappers/node_creation.h
+++ b/src/aclk/schema-wrappers/node_creation.h
diff --git a/aclk/schema-wrappers/node_info.cc b/src/aclk/schema-wrappers/node_info.cc
index 5e321f688..5e321f688 100644
--- a/aclk/schema-wrappers/node_info.cc
+++ b/src/aclk/schema-wrappers/node_info.cc
diff --git a/aclk/schema-wrappers/node_info.h b/src/aclk/schema-wrappers/node_info.h
index 4f57601df..4f57601df 100644
--- a/aclk/schema-wrappers/node_info.h
+++ b/src/aclk/schema-wrappers/node_info.h
diff --git a/aclk/schema-wrappers/proto_2_json.cc b/src/aclk/schema-wrappers/proto_2_json.cc
index 854396510..854396510 100644
--- a/aclk/schema-wrappers/proto_2_json.cc
+++ b/src/aclk/schema-wrappers/proto_2_json.cc
diff --git a/aclk/schema-wrappers/proto_2_json.h b/src/aclk/schema-wrappers/proto_2_json.h
index 3bd98478c..3bd98478c 100644
--- a/aclk/schema-wrappers/proto_2_json.h
+++ b/src/aclk/schema-wrappers/proto_2_json.h
diff --git a/aclk/schema-wrappers/schema_wrapper_utils.cc b/src/aclk/schema-wrappers/schema_wrapper_utils.cc
index 96a4b9bf1..96a4b9bf1 100644
--- a/aclk/schema-wrappers/schema_wrapper_utils.cc
+++ b/src/aclk/schema-wrappers/schema_wrapper_utils.cc
diff --git a/aclk/schema-wrappers/schema_wrappers.h b/src/aclk/schema-wrappers/schema_wrappers.h
index b651b8845..b651b8845 100644
--- a/aclk/schema-wrappers/schema_wrappers.h
+++ b/src/aclk/schema-wrappers/schema_wrappers.h