Adding upstream version 5.7.1.upstream/5.7.1

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 338 insertions, 0 deletions

diff --git a/daemon/bindings/cache.rst b/daemon/bindings/cache.rst
new file mode 100644
index 0000000..36114d2
--- /dev/null
+++ b/daemon/bindings/cache.rst
@@ -0,0 +1,338 @@
+.. SPDX-License-Identifier: GPL-3.0-or-later
+
+Cache
+=====
+
+Cache in Knot Resolver is stored on disk and also shared between
+:ref:`systemd-multiple-instances` so resolver doesn't lose the cached data on
+restart or crash.
+
+To improve performance even further the resolver implements so-called aggressive caching
+for DNSSEC-validated data (:rfc:`8198`), which improves performance and also protects
+against some types of Random Subdomain Attacks.
+
+
+.. _`cache_sizing`:
+
+Sizing
+------
+
+For personal and small office use-cases cache size around 100 MB is more than enough.
+
+For large deployments we recommend to run Knot Resolver on a dedicated machine,
+and to allocate 90% of machine's free memory for resolver's cache.
+
+.. note:: Choosing a cache size that can fit into RAM is important even if the
+   cache is stored on disk (default). Otherwise, the extra I/O caused by disk
+   access for missing pages can cause performance issues.
+
+For example, imagine you have a machine with 16 GB of memory.
+After machine restart you use command ``free -m`` to determine
+amount of free memory (without swap):
+
+.. code-block:: bash
+
+  $ free -m
+                total        used        free
+  Mem:          15907         979       14928
+
+Now you can configure cache size to be 90% of the free memory 14 928 MB, i.e. 13 453 MB:
+
+.. code-block:: lua
+
+   -- 90 % of free memory after machine restart
+   cache.size = 13453 * MB
+
+It is also possible to set the cache size based on the file system size. This is useful
+if you use a dedicated partition for cache (e.g. non-persistent tmpfs). It is recommended
+to leave some free space for special files, such as locks.:
+
+.. code-block:: lua
+
+   cache.size = cache.fssize() - 10*MB
+
+.. note:: The :ref:`garbage-collector` can be used to periodically trim the
+   cache. It is enabled and configured by default when running kresd with
+   systemd integration.
+
+.. _`cache_persistence`:
+
+Persistence
+-----------
+.. tip:: Using tmpfs for cache improves performance and reduces disk I/O.
+
+By default the cache is saved on a persistent storage device
+so the content of the cache is persisted during system reboot.
+This usually leads to smaller latency after restart etc.,
+however in certain situations a non-persistent cache storage might be preferred, e.g.:
+
+  - Resolver handles high volume of queries and I/O performance to disk is too low.
+  - Threat model includes attacker getting access to disk content in power-off state.
+  - Disk has limited number of writes (e.g. flash memory in routers).
+
+If non-persistent cache is desired configure cache directory to be on
+tmpfs_ filesystem, a temporary in-memory file storage.
+The cache content will be saved in memory, and thus have faster access
+and will be lost on power-off or reboot.
+
+
+.. note:: In most of the Unix-like systems ``/tmp`` and ``/var/run`` are
+   commonly mounted as tmpfs.  While it is technically possible to move the
+   cache to an existing tmpfs filesystem, it is *not recommended*, since the
+   path to cache is configured in multiple places.
+
+Mounting the cache directory as tmpfs_ is the recommended approach.  Make sure
+to use appropriate ``size=`` option and don't forget to adjust the size in the
+config file as well.
+
+.. code-block:: none
+
+   # /etc/fstab
+   tmpfs	/var/cache/knot-resolver	tmpfs	rw,size=2G,uid=knot-resolver,gid=knot-resolver,nosuid,nodev,noexec,mode=0700 0 0
+
+.. code-block:: lua
+
+   -- /etc/knot-resolver/kresd.conf
+   cache.size = cache.fssize() - 10*MB
+
+.. _tmpfs: https://en.wikipedia.org/wiki/Tmpfs
+
+Configuration reference
+-----------------------
+
+.. function:: cache.open(max_size[, config_uri])
+
+   :param number max_size: Maximum cache size in bytes.
+   :return: ``true`` if cache was opened
+
+   Open cache with a size limit. The cache will be reopened if already open.
+   Note that the max_size cannot be lowered, only increased due to how cache is implemented.
+
+   .. tip:: Use ``kB, MB, GB`` constants as a multiplier, e.g. ``100*MB``.
+
+   The URI ``lmdb://path`` allows you to change the cache directory.
+
+   Example:
+
+   .. code-block:: lua
+
+      cache.open(100 * MB, 'lmdb:///var/cache/knot-resolver')
+
+.. envvar:: cache.size
+
+   Set the cache maximum size in bytes. Note that this is only a hint to the backend,
+   which may or may not respect it. See :func:`cache.open()`.
+
+   .. code-block:: lua
+
+	cache.size = 100 * MB -- equivalent to `cache.open(100 * MB)`
+
+.. envvar:: cache.current_size
+
+   Get the maximum size in bytes.
+
+   .. code-block:: lua
+
+	print(cache.current_size)
+
+.. envvar:: cache.storage
+
+   Set the cache storage backend configuration, see :func:`cache.backends()` for
+   more information. If the new storage configuration is invalid, it is not set.
+
+   .. code-block:: lua
+
+	cache.storage = 'lmdb://.'
+
+.. envvar:: cache.current_storage
+
+   Get the storage backend configuration.
+
+   .. code-block:: lua
+
+	print(cache.current_storage)
+
+.. function:: cache.backends()
+
+   :return: map of backends
+
+   .. note:: For now there is only one backend implementation, even though the APIs are ready for different (synchronous) backends.
+
+   The cache supports runtime-changeable backends, using the optional :rfc:`3986` URI, where the scheme
+   represents backend protocol and the rest of the URI backend-specific configuration. By default, it
+   is a ``lmdb`` backend in working directory, i.e. ``lmdb://``.
+
+   Example output:
+
+   .. code-block:: lua
+
+   	[lmdb://] => true
+
+.. function:: cache.count()
+
+   :return: Number of entries in the cache. Meaning of the number is an implementation detail and is subject of change.
+
+.. function:: cache.close()
+
+   :return: ``true`` if cache was closed
+
+   Close the cache.
+
+   .. note:: This may or may not clear the cache, depending on the cache backend.
+
+.. function:: cache.fssize()
+
+   :return: Partition size of cache storage.
+
+.. function:: cache.stats()
+
+   Return table with low-level statistics for internal cache operation and storage.
+   This counts each access to cache and does not directly map to individual
+   DNS queries or resource records.
+   For query-level statistics see :ref:`stats module <mod-stats>`.
+
+   Example:
+
+   .. code-block:: lua
+
+       > cache.stats()
+       [clear] => 0
+       [close] => 0
+       [commit] => 117
+       [count] => 2
+       [count_entries] => 6187
+       [match] => 21
+       [match_miss] => 2
+       [open] => 0
+       [read] => 4313
+       [read_leq] => 9
+       [read_leq_miss] => 4
+       [read_miss] => 1143
+       [remove] => 17
+       [remove_miss] => 0
+       [usage_percent] => 15.625
+       [write] => 189
+
+
+   Cache operation `read_leq` (*read less or equal*, i.e. range search) was requested 9 times,
+   and 4 out of 9 operations were finished with *cache miss*.
+   Cache contains 6187 internal entries which occupy 15.625 % cache size.
+
+
+.. function:: cache.max_ttl([ttl])
+
+  :param number ttl: maximum TTL in seconds (default: 1 day)
+
+  .. KR_CACHE_DEFAULT_TTL_MAX ^^
+
+  :return: current maximum TTL
+
+  Get or set upper TTL bound applied to all received records.
+
+  .. note:: The `ttl` value must be in range `(min_ttl, 2147483647)`.
+
+  .. code-block:: lua
+
+     -- Get maximum TTL
+     cache.max_ttl()
+     518400
+     -- Set maximum TTL
+     cache.max_ttl(172800)
+     172800
+
+.. function:: cache.min_ttl([ttl])
+
+  :param number ttl: minimum TTL in seconds (default: 5 seconds)
+
+  .. KR_CACHE_DEFAULT_TTL_MIN ^^
+
+  :return: current minimum TTL
+
+  Get or set lower TTL bound applied to all received records.
+  Forcing TTL higher than specified violates DNS standards, so use higher values with care.
+  TTL still won't be extended beyond expiration of the corresponding DNSSEC signature.
+
+  .. note:: The `ttl` value must be in range `<0, max_ttl)`.
+
+  .. code-block:: lua
+
+     -- Get minimum TTL
+     cache.min_ttl()
+     0
+     -- Set minimum TTL
+     cache.min_ttl(5)
+     5
+
+.. function:: cache.ns_tout([timeout])
+
+  :param number timeout: NS retry interval in milliseconds (default: :c:macro:`KR_NS_TIMEOUT_RETRY_INTERVAL`)
+  :return: current timeout
+
+  Get or set time interval for which a nameserver address will be ignored after determining that it doesn't return (useful) answers.
+  The intention is to avoid waiting if there's little hope; instead, kresd can immediately SERVFAIL or immediately use stale records (with :ref:`serve_stale <mod-serve_stale>` module).
+
+  .. warning:: This settings applies only to the current kresd process.
+
+.. function:: cache.get([domain])
+
+  This function is not implemented at this moment.
+  We plan to re-introduce it soon, probably with a slightly different API.
+
+.. function:: cache.clear([name], [exact_name], [rr_type], [chunk_size], [callback], [prev_state])
+
+     Purge cache records matching specified criteria. There are two specifics:
+
+     * To reliably remove **negative** cache entries you need to clear subtree with the whole zone. E.g. to clear negative cache entries for (formerly non-existing) record `www.example.com. A` you need to flush whole subtree starting at zone apex, e.g. `example.com.` [#]_.
+     * This operation is asynchronous and might not be yet finished when call to ``cache.clear()`` function returns. Return value indicates if clearing continues asynchronously or not.
+
+  :param string name: subtree to purge; if the name isn't provided, whole cache is purged
+        (and any other parameters are disregarded).
+  :param bool exact_name: if set to ``true``, only records with *the same* name are removed;
+                          default: false.
+  :param kres.type rr_type: you may additionally specify the type to remove,
+        but that is only supported with ``exact_name == true``; default: nil.
+  :param integer chunk_size: the number of records to remove in one round; default: 100.
+        The purpose is not to block the resolver for long.
+        The default ``callback`` repeats the command after one millisecond
+        until all matching data are cleared.
+  :param function callback: a custom code to handle result of the underlying C call.
+        Its parameters are copies of those passed to `cache.clear()` with one additional
+        parameter ``rettable`` containing table with return value from current call.
+        ``count`` field contains a return code from :func:`kr_cache_remove_subtree()`.
+  :param table prev_state: return value from previous run (can be used by callback)
+
+  :rtype: table
+  :return: ``count`` key is always present. Other keys are optional and their presence indicate special conditions.
+
+   * **count** *(integer)* - number of items removed from cache by this call (can be 0 if no entry matched criteria)
+   * **not_apex** - cleared subtree is not cached as zone apex; proofs of non-existence were probably not removed
+   * **subtree** *(string)* - hint where zone apex lies (this is estimation from cache content and might not be accurate)
+   * **chunk_limit** - more than ``chunk_size`` items needs to be cleared, clearing will continue asynchronously
+
+
+  Examples:
+
+  .. code-block:: lua
+
+     -- Clear whole cache
+     > cache.clear()
+     [count] => 76
+
+     -- Clear records at and below 'com.'
+     > cache.clear('com.')
+     [chunk_limit] => chunk size limit reached; the default callback will continue asynchronously
+     [not_apex] => to clear proofs of non-existence call cache.clear('com.')
+     [count] => 100
+     [round] => 1
+     [subtree] => com.
+     > worker.sleep(0.1)
+     [cache] asynchronous cache.clear('com', false) finished
+
+     -- Clear only 'www.example.com.'
+     > cache.clear('www.example.com.', true)
+     [round] => 1
+     [count] => 1
+     [not_apex] => to clear proofs of non-existence call cache.clear('example.com.')
+     [subtree] => example.com.
+
+.. [#] This is a consequence of DNSSEC negative cache which relies on proofs of non-existence on various owner nodes. It is impossible to efficiently flush part of DNS zones signed with NSEC3.