Merging upstream version 1.11.1+dfsg

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

5 files changed, 452 insertions, 38 deletions

diff --git a/daemon/Makefile.am b/daemon/Makefile.am
index bdd02774..bffc864d 100644
--- a/daemon/Makefile.am
+++ b/daemon/Makefile.am
@@ -5,4 +5,5 @@ MAINTAINERCLEANFILES= $(srcdir)/Makefile.in
 
 dist_noinst_DATA = \
 	README.md \
+	config/README.md \
 	$(NULL)
diff --git a/daemon/Makefile.in b/daemon/Makefile.in
index 8c7ad21a..7111dfa1 100644
--- a/daemon/Makefile.in
+++ b/daemon/Makefile.in
@@ -272,6 +272,7 @@ AUTOMAKE_OPTIONS = subdir-objects
 MAINTAINERCLEANFILES = $(srcdir)/Makefile.in
 dist_noinst_DATA = \
 	README.md \
+	config/README.md \
 	$(NULL)
 
 all: all-am
diff --git a/daemon/README.md b/daemon/README.md
index 2a5ebfa0..305fc961 100644
--- a/daemon/README.md
+++ b/daemon/README.md
@@ -1,6 +1,87 @@
-# Netdata daemon
+# Running the Netdata Daemon
 
+## Starting netdata
 
+- You can start netdata by executing it with `/usr/sbin/netdata` (the installer will also start it).
+
+- You can stop netdata by killing it with `killall netdata`.
+    You can stop and start netdata at any point. Netdata saves on exit its round robbin
+    database to `/var/cache/netdata` so that it will continue from where it stopped the last time.
+
+Access to the web site, for all graphs, is by default on port `19999`, so go to:
+
+ ```
+ http://127.0.0.1:19999/
+ ```
+
+You can get the running config file at any time, by accessing `http://127.0.0.1:19999/netdata.conf`.
+
+### Starting netdata at boot
+
+In the `system` directory you can find scripts and configurations for the various distros.
+
+#### systemd
+
+The installer already installs `netdata.service` if it detects a systemd system.
+
+To install `netdata.service` by hand, run:
+
+```sh
+# stop netdata
+killall netdata
+
+# copy netdata.service to systemd
+cp system/netdata.service /etc/systemd/system/
+
+# let systemd know there is a new service
+systemctl daemon-reload
+
+# enable netdata at boot
+systemctl enable netdata
+
+# start netdata
+systemctl start netdata
+```
+
+#### init.d
+
+In the system directory you can find `netdata-lsb`. Copy it to the proper place according to your distribution documentation. For Ubuntu, this can be done via running the following commands as root.
+
+```sh
+# copy the netdata startup file to /etc/init.d
+cp system/netdata-lsb /etc/init.d/netdata
+
+# make sure it is executable
+chmod +x /etc/init.d/netdata
+
+# enable it
+update-rc.d netdata defaults
+```
+
+#### openrc (gentoo)
+
+In the `system` directory you can find `netdata-openrc`. Copy it to the proper place according to your distribution documentation.
+
+#### CentOS / Red Hat Enterprise Linux
+
+For older versions of RHEL/CentOS that don't have systemd, an init script is included in the system directory. This can be installed by running the following commands as root.
+
+```sh
+# copy the netdata startup file to /etc/init.d
+cp system/netdata-init-d /etc/init.d/netdata
+
+# make sure it is executable
+chmod +x /etc/init.d/netdata
+
+# enable it
+chkconfig --add netdata
+```
+
+_There have been some recent work on the init script, see PR https://github.com/netdata/netdata/pull/403_
+
+#### other systems
+
+You can start netdata by running it from `/etc/rc.local` or equivalent.
 
 ## Command line options
 
@@ -32,7 +113,7 @@ The command line options of the netdata 1.10.0 version are the following:
  Source Code: https://github.com/netdata/netdata
  Wiki / Docs: https://github.com/netdata/netdata/wiki
  Support    : https://github.com/netdata/netdata/issues
- License    : https://github.com/netdata/netdata/blob/master/LICENSE.md
+ License    : https://github.com/netdata/netdata/blob/master/LICENSE
 
  Twitter    : https://twitter.com/linuxnetdata
  Facebook   : https://www.facebook.com/linuxnetdata/
@@ -387,7 +468,7 @@ When you compile netdata with debugging:
 
 1. compiler optimizations for your CPU are disabled (netdata will run somewhat slower)
 
-2. a lot of code is added all over netdata, to log debug messages to `/var/log/netdata/debug.log`. However, nothing is printed by default. netdata allows you to select which sections of netdata you want to trace. Tracing is activated via the config option `debug flags`. It accepts a hex number, to enable or disable specific sections. You can find the options supported at [log.h](https://github.com/netdata/netdata/blob/master/libnetdata/log/log.h). They are the `D_*` defines. The value `0xffffffffffffffff` will enable all possible debug flags.
+2. a lot of code is added all over netdata, to log debug messages to `/var/log/netdata/debug.log`. However, nothing is printed by default. netdata allows you to select which sections of netdata you want to trace. Tracing is activated via the config option `debug flags`. It accepts a hex number, to enable or disable specific sections. You can find the options supported at [log.h](../libnetdata/log/log.h). They are the `D_*` defines. The value `0xffffffffffffffff` will enable all possible debug flags.
 
 Once netdata is compiled with debugging and tracing is enabled for a few sections, the file `/var/log/netdata/debug.log` will contain the messages.
 
diff --git a/daemon/config/README.md b/daemon/config/README.md
new file mode 100755
index 00000000..5cd7844a
--- /dev/null
+++ b/daemon/config/README.md
@@ -0,0 +1,175 @@
+# Configuration Guide
+
+Configuration files are placed in `/etc/netdata`.
+
+## Netdata Daemon
+
+The daemon configuration file is read from `/etc/netdata/netdata.conf`.
+
+In this file you can configure all aspects of netdata. Netdata provides configuration settings for plugins and charts found when started. You can find all these settings, with their default values, by accessing the URL `https://netdata.server.hostname:19999/netdata.conf`. For example check the configuration file of [netdata.firehol.org](http://netdata.firehol.org/netdata.conf).
+
+The configuration file has sections stated with `[section]`. There will be the following sections:
+
+1. `[global]` for global netdata daemon options
+2. `[plugins]` for controlling which plugins the netdata will use
+3. `[plugin:NAME]` one such section for each plugin enabled
+4. `[CHART_NAME]` once such section for each chart defined
+
+The configuration file is a `name = value` dictionary. Netdata will not complain if you set options unknown to it. When you check the running configuration by accessing the URL `/netdata.conf` on your netdata server, netdata will add a comment on settings it does not currently use.
+
+### [global] section options
+
+
+setting | default | info
+:------:|:-------:|:----
+hostname|auto-detected|The hostname of the computer running netdata.
+history|3600|The number of entries the netdata daemon will by default keep in memory for each chart dimension. This setting can also be configured per chart. Check [Memory Requirements](../../database/#netdata-database) for more information.
+config directory|`/etc/netdata`|The directory configuration files are kept.
+plugins directory|`/usr/libexec/netdata/plugins.d`|The directory plugin programs are kept. This setting supports multiple directories, space separated. If any directory path contains spaces, enclose it in single or double quotes.
+web files directory|`/usr/share/netdata/web`|The directory the web static files are kept.
+cache directory|`/var/cache/netdata`|The directory the memory database will be stored if and when netdata exits. Netdata will re-read the database when it will start again, to continue from the same point.
+log directory|`/var/log/netdata`|The directory in which the [log files](../#log-files) are kept.
+host access prefix|*empty*|This is used in docker environments where /proc, /sys, etc have to be accessed via another path. You may also have to set SYS_PTRACE capability on the docker for this work. Check [issue 43](https://github.com/netdata/netdata/issues/43).
+debug flags|0x00000000|Bitmap of debug options to enable. For more information check [Tracing Options](../#debugging).
+memory deduplication (ksm)|yes|When set to `yes`, netdata will offer its in-memory round robin database to kernel same page merging (KSM) for deduplication. For more information check [[Memory Deduplication - Kernel Same Page Merging - KSM]]
+debug log|`/var/log/netdata/debug.log`|The filename to save debug information. This file will not be created is debugging is not enabled. You can also set it to `syslog` to send the debug messages to syslog, or `none` to disable this log. For more information check [Tracing Options](../#debugging).
+error log|`/var/log/netdata/error.log`|The filename to save error messages for netdata daemon and all plugins (`stderr` is sent here for all netdata programs, including the plugins). You can also set it to `syslog` to send the errors to syslog, or `none` to disable this log.
+access log|`/var/log/netdata/access.log`|The filename to save the log of web clients accessing netdata charts. You can also set it to `syslog` to send the access log to syslog, or `none` to disable this log.
+memory mode|save|When set to `save` netdata will save its round robin database on exit and load it on startup. When set to `map` the cache files will be updated in real time (check `man mmap` - do not set this on systems with heavy load or slow disks - the disks will continuously sync the in-memory database of netdata). When set to `ram` the round robin database will be temporary and it will be lost when netdata exits.
+update every|1|The frequency in seconds, for data collection. For more information see [Performance](../../doc/Performance.md#netdata-performance).
+run as user|`netdata`|The user netdata will run as.
+web files owner|`netdata`|The user that owns the web static files. Netdata will refuse to serve a file that is not owned by this user, even if it has read access to that file. If the user given is not found, netdata will only serve files owned by user given in `run as user`.
+http port listen backlog|100|The port backlog. Check `man 2 listen`.
+default port|19999|The default port to listen for web clients.
+bind to|`*`|The IP address and port to listen to. This is a space separated list of IPv4 or IPv6 address and ports. The default will bind to all IP addresses. Example: `bind to = 127.0.0.1:19999 10.11.12.1:19998 [::1]:19999`.
+disconnect idle web clients after seconds|60|The time in seconds to disconnect web clients after being totally idle.
+enable web responses gzip compression|yes|When set to `yes`, netdata web responses will be GZIP compressed, if the web client accepts such responses.
+
+##### netdata process priority
+
+By default, netdata runs with the `idle` process scheduler, which assigns CPU resources to netdata, only when the system has such resources to spare.
+
+The following `netdata.conf` settings control this:
+
+```
+[global]
+    process scheduling policy = idle
+    process scheduling priority = 0
+    process nice level = 19
+```
+
+The policies supported by netdata are `idle` (the netdata default), `other` (also as `nice`), `batch`, `rr`, `fifo`. netdata also recognizes `keep` and `none` to keep the current settings without changing them.
+
+For `other`, `nice` and `batch`, the setting `process nice level = 19` is activated to configure the nice level of netdata. Nice gets values -20 (highest) to 19 (lowest).
+
+For `rr` and `fifo`, the setting `process scheduling priority = 0` is activated to configure the priority of the relative scheduling policy. Priority gets values 1 (lowest) to 99 (highest).
+
+For the details of each scheduler, see `man sched_setscheduler` and `man sched`.
+
+When netdata is running under systemd, it can only lower its priority (the default is `other` with `nice level = 0`). If you want to make netdata to get more CPU than that, you will need to set in `netdata.conf`:
+
+```
+[global]
+    process scheduling policy = keep
+```
+
+and edit `/etc/systemd/system/netdata.service` and add:
+
+```
+CPUSchedulingPolicy=other | batch | idle | fifo | rr
+CPUSchedulingPriority=99
+Nice=-10
+```
+
+
+### [plugins] section options
+
+In this section there will be a boolean (`yes`/`no`) option for each plugin. Additionally, there will be the following options:
+
+setting | default | info
+:------:|:-------:|:----
+checks|no|This is a debugging plugin for the internal latency of netdata.
+enable running new plugins|yes|When set to `yes`, netdata will enable plugins not configured specifically for them. Setting this to `no` will disable all plugins you have not set to `yes` explicitly.
+check for new plugins every|60|The time in seconds to check for new plugins in the plugins directory. This allows having other applications dynamically creating plugins for netdata.
+
+## Netdata Plugins
+
+The configuration options for plugins appear in sections following the pattern `[plugin:NAME]`.
+
+### Internal Plugins
+
+Most internal plugins will provide additional options. Check [Internal Plugins](../../collectors/) for more information.
+
+
+### External Plugins
+
+External plugins will have only 2 options at `netdata.conf`:
+
+setting | default | info
+:------:|:-------:|:----
+update every|the value of `[global].update every` setting|The frequency in seconds the plugin should collect values. For more information check [Performance](../../doc/Performance.md#netdata-performance).
+command options|*empty*|Additional command line options to pass to the plugin. 
+
+External plugins that need additional configuration may support a dedicated file in `/etc/netdata`. Check their documentation.
+
+---
+
+## A note about netdata.conf
+
+This config file is not needed by default. You can just touch it (to be empty) to get rid of the error message displayed when missing.
+
+The whole idea came up when I was evaluating the documentation involved in maintaining a complex configuration system. My intention was to give configuration options for everything imaginable. But then, documenting all these options would require a tremendous amount of time, users would have to search through endless pages for the option they need, etc.
+
+I concluded then that configuring software like that is a waste for time and effort. Of course there must be plenty of configuration options, but the implementation itself should require a lot less effort for both the devs and the users.
+
+So, I did this:
+
+1. No configuration is required to run netdata
+2. There are plenty of options to tweak
+3. There is minimal documentation (or no at all)
+
+### Why this works?
+
+The configuration file is a `name = value` dictionary with `[sections]`. Write whatever you like there as long as it follows this simple format.
+
+Netdata loads this dictionary and then when the code needs a value from it, it just looks up the `name` in the dictionary at the proper `section`. In all places, in the code, there are both the `names` and their `default values`, so if something is not found in the configuration file, the default is used. The lookup is made using B-Trees and hashes (no string comparisons), so they are super fast. Also the `names` of the settings can be `my super duper setting that once set to yes, will turn the world upside down = no` - so goodbye to most of the documentation involved.
+
+Next, netdata can generate a valid configuration for the user to edit. No need to remember anything. Just get the configuration from the server (`/netdata.conf` on your netdata server), edit it and save it.
+
+Last, what about options you believe you have set, but you misspelled? When you get the configuration file from the server, there will be a comment above all `name = value` pairs the server does not use. So you know that whatever you wrote there, is not used.
+
+### limiting access to netdata.conf
+
+netdata v1.9+ limit by default access to `http://your.netdata.ip:19999/netdata.conf` to private IP addresses. This is controlled by this settings:
+
+```
+[web]
+	allow netdata.conf from = localhost fd* 10.* 192.168.* 172.16.* 172.17.* 172.18.* 172.19.* 172.20.* 172.21.* 172.22.* 172.23.* 172.24.* 172.25.* 172.26.* 172.27.* 172.28.* 172.29.* 172.30.* 172.31.*
+```
+
+The IPs listed are all the private IPv4 addresses, including link local IPv6 addresses.
+
+> Keep in mind that connections to netdata API ports are filtered by `[web].allow connections from`. So, IPs allowed by `[web].allow netdata.conf from` should also be allowed by `[web].allow connections from`.
+
+
+## netdata simple patterns
+
+Unix prefers regular expressions. But they are just too hard, too cryptic to use, write and understand.
+
+So, netdata supports [simple patterns](../../libnetdata/simple_pattern/). 
+
+## Applying changes
+
+After `netdata.conf` has been modified, netdata needs to be restarted for changes to apply:
+
+```bash
+sudo service netdata restart
+```
+
+If the above does not work, try the following:
+
+```bash
+sudo killall netdata; sleep 10; sudo netdata
+```
+
+Please note that your data history will be lost if you have modified `history` parameter in section `[global]`.
diff --git a/daemon/unit_test.c b/daemon/unit_test.c
index 9978647b..a92a50a1 100644
--- a/daemon/unit_test.c
+++ b/daemon/unit_test.c
@@ -130,13 +130,17 @@ int check_storage_number(calculated_number n, int debug) {
             p, pdiff, pcdiff
         );
         if(len != strlen(buffer)) fprintf(stderr, "ERROR: printed number %s is reported to have length %zu but it has %zu\n", buffer, len, strlen(buffer));
-        if(dcdiff > ACCURACY_LOSS) fprintf(stderr, "WARNING: packing number " CALCULATED_NUMBER_FORMAT " has accuracy loss " CALCULATED_NUMBER_FORMAT " %%\n", n, dcdiff);
-        if(pcdiff > ACCURACY_LOSS) fprintf(stderr, "WARNING: re-parsing the packed, unpacked and printed number " CALCULATED_NUMBER_FORMAT " has accuracy loss " CALCULATED_NUMBER_FORMAT " %%\n", n, pcdiff);
+
+        if(dcdiff > ACCURACY_LOSS_ACCEPTED_PERCENT)
+            fprintf(stderr, "WARNING: packing number " CALCULATED_NUMBER_FORMAT " has accuracy loss " CALCULATED_NUMBER_FORMAT " %%\n", n, dcdiff);
+
+        if(pcdiff > ACCURACY_LOSS_ACCEPTED_PERCENT)
+            fprintf(stderr, "WARNING: re-parsing the packed, unpacked and printed number " CALCULATED_NUMBER_FORMAT " has accuracy loss " CALCULATED_NUMBER_FORMAT " %%\n", n, pcdiff);
     }
 
     if(len != strlen(buffer)) return 1;
-    if(dcdiff > ACCURACY_LOSS) return 3;
-    if(pcdiff > ACCURACY_LOSS) return 4;
+    if(dcdiff > ACCURACY_LOSS_ACCEPTED_PERCENT) return 3;
+    if(pcdiff > ACCURACY_LOSS_ACCEPTED_PERCENT) return 4;
     return 0;
 }
 
@@ -159,6 +163,9 @@ void benchmark_storage_number(int loop, int multiplier) {
     storage_number s;
     unsigned long long user, system, total, mine, their;
 
+    calculated_number storage_number_positive_min = unpack_storage_number(STORAGE_NUMBER_POSITIVE_MIN_RAW);
+    calculated_number storage_number_positive_max = unpack_storage_number(STORAGE_NUMBER_POSITIVE_MAX_RAW);
+
     char buffer[100];
 
     struct rusage now, last;
@@ -181,11 +188,11 @@ void benchmark_storage_number(int loop, int multiplier) {
     }
 
     fprintf(stderr, "\nNETDATA FLOATING POINT\n");
-    fprintf(stderr, "MIN POSITIVE VALUE " CALCULATED_NUMBER_FORMAT "\n", storage_number_min(1));
-    fprintf(stderr, "MAX POSITIVE VALUE " CALCULATED_NUMBER_FORMAT "\n", (calculated_number)STORAGE_NUMBER_POSITIVE_MAX);
-    fprintf(stderr, "MIN NEGATIVE VALUE " CALCULATED_NUMBER_FORMAT "\n", (calculated_number)STORAGE_NUMBER_NEGATIVE_MIN);
-    fprintf(stderr, "MAX NEGATIVE VALUE " CALCULATED_NUMBER_FORMAT "\n", -storage_number_min(1));
-    fprintf(stderr, "Maximum accuracy loss: " CALCULATED_NUMBER_FORMAT "%%\n\n\n", (calculated_number)ACCURACY_LOSS);
+    fprintf(stderr, "MIN POSITIVE VALUE " CALCULATED_NUMBER_FORMAT "\n", unpack_storage_number(STORAGE_NUMBER_POSITIVE_MIN_RAW));
+    fprintf(stderr, "MAX POSITIVE VALUE " CALCULATED_NUMBER_FORMAT "\n", unpack_storage_number(STORAGE_NUMBER_POSITIVE_MAX_RAW));
+    fprintf(stderr, "MIN NEGATIVE VALUE " CALCULATED_NUMBER_FORMAT "\n", unpack_storage_number(STORAGE_NUMBER_NEGATIVE_MIN_RAW));
+    fprintf(stderr, "MAX NEGATIVE VALUE " CALCULATED_NUMBER_FORMAT "\n", unpack_storage_number(STORAGE_NUMBER_NEGATIVE_MAX_RAW));
+    fprintf(stderr, "Maximum accuracy loss accepted: " CALCULATED_NUMBER_FORMAT "%%\n\n\n", (calculated_number)ACCURACY_LOSS_ACCEPTED_PERCENT);
 
     // ------------------------------------------------------------------------
 
@@ -194,11 +201,11 @@ void benchmark_storage_number(int loop, int multiplier) {
 
     // do the job
     for(j = 1; j < 11 ;j++) {
-        n = STORAGE_NUMBER_POSITIVE_MIN * j;
+        n = storage_number_positive_min * j;
 
         for(i = 0; i < loop ;i++) {
             n *= multiplier;
-            if(n > STORAGE_NUMBER_POSITIVE_MAX) n = STORAGE_NUMBER_POSITIVE_MIN;
+            if(n > storage_number_positive_max) n = storage_number_positive_min;
 
             print_calculated_number(buffer, n);
         }
@@ -219,11 +226,11 @@ void benchmark_storage_number(int loop, int multiplier) {
 
     // do the job
     for(j = 1; j < 11 ;j++) {
-        n = STORAGE_NUMBER_POSITIVE_MIN * j;
+        n = storage_number_positive_min * j;
 
         for(i = 0; i < loop ;i++) {
             n *= multiplier;
-            if(n > STORAGE_NUMBER_POSITIVE_MAX) n = STORAGE_NUMBER_POSITIVE_MIN;
+            if(n > storage_number_positive_max) n = storage_number_positive_min;
             snprintfz(buffer, 100, CALCULATED_NUMBER_FORMAT, n);
         }
     }
@@ -250,13 +257,13 @@ void benchmark_storage_number(int loop, int multiplier) {
 
     // do the job
     for(j = 1; j < 11 ;j++) {
-        n = STORAGE_NUMBER_POSITIVE_MIN * j;
+        n = storage_number_positive_min * j;
 
         for(i = 0; i < loop ;i++) {
             n *= multiplier;
-            if(n > STORAGE_NUMBER_POSITIVE_MAX) n = STORAGE_NUMBER_POSITIVE_MIN;
+            if(n > storage_number_positive_max) n = storage_number_positive_min;
 
-            s = pack_storage_number(n, 1);
+            s = pack_storage_number(n, SN_EXISTS);
             d = unpack_storage_number(s);
             print_calculated_number(buffer, d);
         }
@@ -282,7 +289,7 @@ void benchmark_storage_number(int loop, int multiplier) {
 }
 
 static int check_storage_number_exists() {
-    uint32_t flags = SN_EXISTS;
+    uint32_t flags;
 
 
     for(flags = 0; flags < 7 ; flags++) {
@@ -309,10 +316,12 @@ static int check_storage_number_exists() {
     return 0;
 }
 
-int unit_test_storage()
-{
+int unit_test_storage() {
     if(check_storage_number_exists()) return 0;
 
+    calculated_number storage_number_positive_min = unpack_storage_number(STORAGE_NUMBER_POSITIVE_MIN_RAW);
+    calculated_number storage_number_negative_max = unpack_storage_number(STORAGE_NUMBER_NEGATIVE_MAX_RAW);
+
     calculated_number c, a = 0;
     int i, j, g, r = 0;
 
@@ -325,14 +334,15 @@ int unit_test_storage()
             a += 0.0000001;
             c = a * g;
             for(i = 0; i < 21 ;i++, c *= 10) {
-                if(c > 0 && c < STORAGE_NUMBER_POSITIVE_MIN) continue;
-                if(c < 0 && c > STORAGE_NUMBER_NEGATIVE_MAX) continue;
+                if(c > 0 && c < storage_number_positive_min) continue;
+                if(c < 0 && c > storage_number_negative_max) continue;
 
                 if(check_storage_number(c, 1)) return 1;
             }
         }
     }
 
+    // if(check_storage_number(858993459.1234567, 1)) return 1;
     benchmark_storage_number(1000000, 2);
     return r;
 }
@@ -575,28 +585,36 @@ struct test test4 = {
 };
 
 // --------------------------------------------------------------------------------------------------------------------
-// test5
+// test5 - 32 bit overflows
 
 struct feed_values test5_feed[] = {
-        { 500000, 1000 },
-        { 1000000, 2000 },
-        { 1000000, 2000 },
-        { 1000000, 2000 },
-        { 1000000, 3000 },
-        { 1000000, 2000 },
-        { 1000000, 2000 },
-        { 1000000, 2000 },
-        { 1000000, 2000 },
-        { 1000000, 2000 },
+        { 0,       0x00000000FFFFFFFFULL / 3 * 0 },
+        { 1000000, 0x00000000FFFFFFFFULL / 3 * 1 },
+        { 1000000, 0x00000000FFFFFFFFULL / 3 * 2 },
+        { 1000000, 0x00000000FFFFFFFFULL / 3 * 0 },
+        { 1000000, 0x00000000FFFFFFFFULL / 3 * 1 },
+        { 1000000, 0x00000000FFFFFFFFULL / 3 * 2 },
+        { 1000000, 0x00000000FFFFFFFFULL / 3 * 0 },
+        { 1000000, 0x00000000FFFFFFFFULL / 3 * 1 },
+        { 1000000, 0x00000000FFFFFFFFULL / 3 * 2 },
+        { 1000000, 0x00000000FFFFFFFFULL / 3 * 0 },
 };
 
 calculated_number test5_results[] = {
-        1000, 500, 0, 500, 500, 0, 0, 0, 0
+        0x00000000FFFFFFFFULL / 3,
+        0x00000000FFFFFFFFULL / 3,
+        0x00000000FFFFFFFFULL / 3,
+        0x00000000FFFFFFFFULL / 3,
+        0x00000000FFFFFFFFULL / 3,
+        0x00000000FFFFFFFFULL / 3,
+        0x00000000FFFFFFFFULL / 3,
+        0x00000000FFFFFFFFULL / 3,
+        0x00000000FFFFFFFFULL / 3,
 };
 
 struct test test5 = {
         "test5",            // name
-        "test incremental values ups and downs",
+        "test 32-bit incremental values overflow",
         1,                  // update_every
         1,                  // multiplier
         1,                  // divisor
@@ -610,6 +628,135 @@ struct test test5 = {
 };
 
 // --------------------------------------------------------------------------------------------------------------------
+// test5b - 16 bit overflows
+
+struct feed_values test5b_feed[] = {
+        { 0,       0x000000000000FFFFULL / 3 * 0 },
+        { 1000000, 0x000000000000FFFFULL / 3 * 1 },
+        { 1000000, 0x000000000000FFFFULL / 3 * 2 },
+        { 1000000, 0x000000000000FFFFULL / 3 * 0 },
+        { 1000000, 0x000000000000FFFFULL / 3 * 1 },
+        { 1000000, 0x000000000000FFFFULL / 3 * 2 },
+        { 1000000, 0x000000000000FFFFULL / 3 * 0 },
+        { 1000000, 0x000000000000FFFFULL / 3 * 1 },
+        { 1000000, 0x000000000000FFFFULL / 3 * 2 },
+        { 1000000, 0x000000000000FFFFULL / 3 * 0 },
+};
+
+calculated_number test5b_results[] = {
+        0x000000000000FFFFULL / 3,
+        0x000000000000FFFFULL / 3,
+        0x000000000000FFFFULL / 3,
+        0x000000000000FFFFULL / 3,
+        0x000000000000FFFFULL / 3,
+        0x000000000000FFFFULL / 3,
+        0x000000000000FFFFULL / 3,
+        0x000000000000FFFFULL / 3,
+        0x000000000000FFFFULL / 3,
+};
+
+struct test test5b = {
+        "test5b",            // name
+        "test 16-bit incremental values overflow",
+        1,                  // update_every
+        1,                  // multiplier
+        1,                  // divisor
+        RRD_ALGORITHM_INCREMENTAL, // algorithm
+        10,                 // feed entries
+        9,                  // result entries
+        test5b_feed,        // feed
+        test5b_results,     // results
+        NULL,               // feed2
+        NULL                // results2
+};
+
+// --------------------------------------------------------------------------------------------------------------------
+// test5c - 8 bit overflows
+
+struct feed_values test5c_feed[] = {
+        { 0,       0x00000000000000FFULL / 3 * 0 },
+        { 1000000, 0x00000000000000FFULL / 3 * 1 },
+        { 1000000, 0x00000000000000FFULL / 3 * 2 },
+        { 1000000, 0x00000000000000FFULL / 3 * 0 },
+        { 1000000, 0x00000000000000FFULL / 3 * 1 },
+        { 1000000, 0x00000000000000FFULL / 3 * 2 },
+        { 1000000, 0x00000000000000FFULL / 3 * 0 },
+        { 1000000, 0x00000000000000FFULL / 3 * 1 },
+        { 1000000, 0x00000000000000FFULL / 3 * 2 },
+        { 1000000, 0x00000000000000FFULL / 3 * 0 },
+};
+
+calculated_number test5c_results[] = {
+        0x00000000000000FFULL / 3,
+        0x00000000000000FFULL / 3,
+        0x00000000000000FFULL / 3,
+        0x00000000000000FFULL / 3,
+        0x00000000000000FFULL / 3,
+        0x00000000000000FFULL / 3,
+        0x00000000000000FFULL / 3,
+        0x00000000000000FFULL / 3,
+        0x00000000000000FFULL / 3,
+};
+
+struct test test5c = {
+        "test5c",            // name
+        "test 8-bit incremental values overflow",
+        1,                  // update_every
+        1,                  // multiplier
+        1,                  // divisor
+        RRD_ALGORITHM_INCREMENTAL, // algorithm
+        10,                 // feed entries
+        9,                  // result entries
+        test5c_feed,        // feed
+        test5c_results,     // results
+        NULL,               // feed2
+        NULL                // results2
+};
+
+// --------------------------------------------------------------------------------------------------------------------
+// test5d - 64 bit overflows
+
+struct feed_values test5d_feed[] = {
+        { 0,       0xFFFFFFFFFFFFFFFFULL / 3 * 0 },
+        { 1000000, 0xFFFFFFFFFFFFFFFFULL / 3 * 1 },
+        { 1000000, 0xFFFFFFFFFFFFFFFFULL / 3 * 2 },
+        { 1000000, 0xFFFFFFFFFFFFFFFFULL / 3 * 0 },
+        { 1000000, 0xFFFFFFFFFFFFFFFFULL / 3 * 1 },
+        { 1000000, 0xFFFFFFFFFFFFFFFFULL / 3 * 2 },
+        { 1000000, 0xFFFFFFFFFFFFFFFFULL / 3 * 0 },
+        { 1000000, 0xFFFFFFFFFFFFFFFFULL / 3 * 1 },
+        { 1000000, 0xFFFFFFFFFFFFFFFFULL / 3 * 2 },
+        { 1000000, 0xFFFFFFFFFFFFFFFFULL / 3 * 0 },
+};
+
+calculated_number test5d_results[] = {
+        0xFFFFFFFFFFFFFFFFULL / 3,
+        0xFFFFFFFFFFFFFFFFULL / 3,
+        0xFFFFFFFFFFFFFFFFULL / 3,
+        0xFFFFFFFFFFFFFFFFULL / 3,
+        0xFFFFFFFFFFFFFFFFULL / 3,
+        0xFFFFFFFFFFFFFFFFULL / 3,
+        0xFFFFFFFFFFFFFFFFULL / 3,
+        0xFFFFFFFFFFFFFFFFULL / 3,
+        0xFFFFFFFFFFFFFFFFULL / 3,
+};
+
+struct test test5d = {
+        "test5d",            // name
+        "test 64-bit incremental values overflow",
+        1,                  // update_every
+        1,                  // multiplier
+        1,                  // divisor
+        RRD_ALGORITHM_INCREMENTAL, // algorithm
+        10,                 // feed entries
+        9,                  // result entries
+        test5d_feed,        // feed
+        test5d_results,     // results
+        NULL,               // feed2
+        NULL                // results2
+};
+
+// --------------------------------------------------------------------------------------------------------------------
 // test6
 
 struct feed_values test6_feed[] = {
@@ -1131,7 +1278,7 @@ int run_test(struct test *test)
     unsigned long max = (st->counter < test->result_entries)?st->counter:test->result_entries;
     for(c = 0 ; c < max ; c++) {
         calculated_number v = unpack_storage_number(rd->values[c]);
-        calculated_number n = test->results[c];
+        calculated_number n = unpack_storage_number(pack_storage_number(test->results[c], SN_EXISTS));
         int same = (calculated_number_round(v * 10000000.0) == calculated_number_round(n * 10000000.0))?1:0;
         fprintf(stderr, "    %s/%s: checking position %lu (at %lu secs), expecting value " CALCULATED_NUMBER_FORMAT ", found " CALCULATED_NUMBER_FORMAT ", %s\n",
             test->name, rd->name, c+1,
@@ -1267,6 +1414,15 @@ int run_all_mockup_tests(void)
     if(run_test(&test5))
         return 1;
 
+    if(run_test(&test5b))
+        return 1;
+
+    if(run_test(&test5c))
+        return 1;
+
+    if(run_test(&test5d))
+        return 1;
+
     if(run_test(&test6))
         return 1;