path: root/doc/operation.rst

.. highlight:: none
.. _Operation:

*********
Operation
*********

The Knot DNS server part :doc:`knotd<man_knotd>` can run either in the foreground,
or in the background using the ``-d`` option. When run in the foreground, it
doesn't create a PID file.  Other than that, there are no differences and you
can control both the same way.

The tool :doc:`knotc<man_knotc>` is designed as a user front-end, making it easier
to control a running server daemon. If you want to control the daemon directly,
use ``SIGINT`` to quit the process or ``SIGHUP`` to reload the configuration.

If you pass neither configuration file (``-c`` parameter) nor configuration
database (``-C`` parameter), the server will first attempt to use the default
configuration database stored in ``/var/lib/knot/confdb`` or the
default configuration file stored in ``/etc/knot/knot.conf``. Both the
default paths can be reconfigured with ``--with-storage=path`` or
``--with-configdir=path`` respectively.

Example of server start as a daemon::

    $ knotd -d -c knot.conf

Example of server shutdown::

    $ knotc -c knot.conf stop

For a complete list of actions refer to the program help (``-h`` parameter)
or to the corresponding manual page.

Also, the server needs to create :ref:`server_rundir` and :ref:`zone_storage`
directories in order to run properly.

.. _Configuration database:

Configuration database
======================

In the case of a huge configuration file, the configuration can be stored
in a binary database. Such a database can be simply initialized::

    $ knotc conf-init

or preloaded from a file::

    $ knotc conf-import input.conf

Also the configuration database can be exported into a textual file::

    $ knotc conf-export output.conf

.. WARNING::
   The import and export commands access the configuration database
   directly, without any interaction with the server. Therefore, any data
   not yet committed to the database won't be exported. And the server won't
   reflect imported configuration correctly. So it is strictly recommended to
   import new configuration when the server is not running.

.. _Dynamic configuration:

Dynamic configuration
=====================

The configuration database can be accessed using the server control interface
while the server is running. To get the full power of the dynamic configuration,
the server must be started with a specified configuration database location
or with the default database initialized. Otherwise all the changes to the
configuration will be temporary (until the server is stopped).

.. NOTE::
   The database can be :ref:`imported<Configuration database>` in advance.

Most of the commands get an item name and value parameters. The item name is
in the form of ``section[identifier].name``. If the item is multivalued,
more values can be specified as individual (command line) arguments.

.. CAUTION::
   Beware of the possibility of pathname expansion by the shell. For this reason,
   it is advisable to escape (with backslash) square brackets or to quote command parameters if
   not executed in the interactive mode.

To get the list of configuration sections or to get the list of section items::

    $ knotc conf-list
    $ knotc conf-list 'server'

To get the whole configuration or to get the whole configuration section or
to get all section identifiers or to get a specific configuration item::

    $ knotc conf-read
    $ knotc conf-read 'remote'
    $ knotc conf-read 'zone.domain'
    $ knotc conf-read 'zone[example.com].master'

.. WARNING::
   The following operations don't work on OpenBSD!

Modifying operations require an active configuration database transaction.
Just one transaction can be active at a time. Such a transaction then can
be aborted or committed. A semantic check is executed automatically before
every commit::

    $ knotc conf-begin
    $ knotc conf-abort
    $ knotc conf-commit

To set a configuration item value or to add more values or to add a new
section identifier or to add a value to all identified sections::

    $ knotc conf-set 'server.identity' 'Knot DNS'
    $ knotc conf-set 'server.listen' '0.0.0.0@53' '::@53'
    $ knotc conf-set 'zone[example.com]'
    $ knotc conf-set 'zone.slave' 'slave2'

.. NOTE::
   Also the include operation can be performed. A non-absolute file
   location is relative to the server binary path, not to the control binary
   path!

   ::

      $ knotc conf-set 'include' '/tmp/new_zones.conf'

To unset the whole configuration or to unset the whole configuration section
or to unset an identified section or to unset an item or to unset a specific
item value::

    $ knotc conf-unset
    $ knotc conf-unset 'zone'
    $ knotc conf-unset 'zone[example.com]'
    $ knotc conf-unset 'zone[example.com].master'
    $ knotc conf-unset 'zone[example.com].master' 'remote2' 'remote5'

To get the change between the current configuration and the active transaction
for the whole configuration or for a specific section or for a specific
identified section or for a specific item::

    $ knotc conf-diff
    $ knotc conf-diff 'zone'
    $ knotc conf-diff 'zone[example.com]'
    $ knotc conf-diff 'zone[example.com].master'

.. CAUTION::
   While it is possible to change most of the configuration parameters
   dynamically or via configuration file reload, a few of the parameters
   in the section ``server`` require restarting the server, such that the changes
   take effect. These parameters are:
   :ref:`rundir<server_rundir>`,
   :ref:`user<server_user>`,
   :ref:`pidfile<server_pidfile>`,
   :ref:`tcp-reuseport<server_tcp-reuseport>`,
   :ref:`udp-workers<server_udp-workers>`,
   :ref:`tcp-workers<server_tcp-workers>`,
   :ref:`background-workers<server_background-workers>`, and
   :ref:`listen<server_listen>`.

An example of possible configuration initialization::

    $ knotc conf-begin
    $ knotc conf-set 'server.listen' '0.0.0.0@53' '::@53'
    $ knotc conf-set 'remote[master_server]'
    $ knotc conf-set 'remote[master_server].address' '192.168.1.1'
    $ knotc conf-set 'template[default]'
    $ knotc conf-set 'template[default].storage' '/var/lib/knot/zones/'
    $ knotc conf-set 'template[default].master' 'master_server'
    $ knotc conf-set 'zone[example.com]'
    $ knotc conf-diff
    $ knotc conf-commit

.. _Secondary mode:

Secondary (slave) mode
======================

Running the server as a secondary is very straightforward as the zone
is transfered automatically from a remote server. The received zone is
usually stored in a zone file after the :ref:`zone_zonefile-sync` period
elapses. Zone differences are stored in the :ref:`zone journal <Journal behaviour>`.

.. _Primary mode:

Primary (master) mode
=====================

If you just want to check the zone files before starting, you can use::

    $ knotc zone-check example.com

.. _Editing zones:

Reading and editing zones
=========================

Knot DNS allows you to read or change zone contents online using the server
control interface.

.. WARNING::
   Avoid concurrent zone access when a zone event (zone file load, refresh,
   DNSSEC signing, dynamic update) is in progress or pending. In such a case
   zone events must be frozen before. For more information on how to freeze the
   zone read :ref:`Editing zone file`.

To get contents of all configured zones, or a specific zone contents, or zone
records with a specific owner, or even with a specific record type::

    $ knotc zone-read --
    $ knotc zone-read example.com
    $ knotc zone-read example.com ns1
    $ knotc zone-read example.com ns1 NS

.. NOTE::
   If the record owner is not a fully qualified domain name, then it is
   considered as a relative name to the zone name.

To start a writing transaction on all zones or on specific zones::

    $ knotc zone-begin --
    $ knotc zone-begin example.com example.net

Now you can list all nodes within the transaction using the ``zone-get``
command, which always returns current data with all changes included. The
command has the same syntax as ``zone-read``.

Within the transaction, you can add a record to a specific zone or to all
zones with an open transaction::

    $ knotc zone-set example.com ns1 3600 A 192.168.0.1
    $ knotc zone-set -- ns1 3600 A 192.168.0.1

To remove all records with a specific owner, or a specific rrset, or
specific record data::

    $ knotc zone-unset example.com ns1
    $ knotc zone-unset example.com ns1 A
    $ knotc zone-unset example.com ns1 A 192.168.0.2

To see the difference between the original zone and the current version::

    $ knotc zone-diff example.com

Finally, either commit or abort your transaction::

    $ knotc zone-commit example.com
    $ knotc zone-abort example.com

A full example of setting up a completely new zone from scratch::

    $ knotc conf-begin
    $ knotc conf-set zone.domain example.com
    $ knotc conf-commit
    $ knotc zone-begin example.com
    $ knotc zone-set example.com  @ 3600 SOA  ns admin 1 86400 900 691200 3600
    $ knotc zone-set example.com  @ 3600 NS   ns
    $ knotc zone-set example.com ns 3600 A    192.168.0.1
    $ knotc zone-set example.com ns 3600 AAAA 2001:DB8::1
    $ knotc zone-commit example.com

.. NOTE::
    If quotes are necessary for record data specification, remember to escape them::

       $ knotc zone-set example.com @ 3600 TXT \"v=spf1 a:mail.example.com -all\"

.. _Editing zone file:

Reading and editing the zone file safely
========================================

It's always possible to read and edit zone contents via zone file manipulation.
It may lead to confusion, however, if the zone contents are continuously being
changed by DDNS, DNSSEC signing and the like. In such a case, the safe way to
modify the zone file is to freeze zone events first::

    $ knotc -b zone-freeze example.com.
    $ knotc -b zone-flush example.com.

After calling freeze on the zone, there still may be running zone operations (e.g. signing),
causing freeze pending. Because of this, the blocking mode is used to ensure
the operation was finished. Then the zone can be flushed to a file.

Now the zone file can be safely modified (e.g. using a text editor).
If :ref:`zone_zonefile-load` is not set to `difference-no-serial`, it's also necessary to
**increase SOA serial** in this step to keep consistency. Finally, we can load the
modified zone file and if successful, thaw the zone::

    $ knotc -b zone-reload example.com.
    $ knotc zone-thaw example.com.

.. _Zone loading:

Zone loading
============

The process of how the server loads a zone is influenced by the configuration of the
:ref:`zonefile-load <zone_zonefile-load>` and :ref:`journal-content <zone_journal-content>`
parameters (also DNSSEC signing applies), the existence of a zone file and journal
(and their relative out-of-dateness), and whether it is a cold start of the server
or a zone reload (e.g. invoked by the :doc:`knotc<man_knotc>` interface). Please note
that zone transfers are not taken into account here – they are planned after the zone
is loaded (including :ref:`zone bootstrap<Zone bootstrap>`).

If the zone file exists and is not excluded by the configuration, it is first loaded
and according to its SOA serial number, relevant journal changesets are applied.
If this is a zone reload and we have :ref:`zone_zonefile-load` set to `difference`, the difference
between old and new contents is computed and stored in the journal like an update.
The zone file should be either unchanged since last load or changed with incremented
SOA serial. In the case of a decreased SOA serial, the load is interrupted with
an error; if unchanged, it is increased by the server.

If the procedure described above succeeds without errors, the resulting zone contents are (after potential DNSSEC signing)
used as the new zone.

The option :ref:`zone_journal-content` set to `all` lets the server, beside better performance, keep
track of the zone contents also across server restarts. It makes the cold start
effectively work like a zone reload with the old contents loaded from the journal
(unless this is the very first start with the zone not yet saved into the journal).

.. _Journal behaviour:

Journal behaviour
=================

The zone journal keeps some history of changes made to the zone. It is useful for
responding to IXFR queries. Also if :ref:`zone file flush <zone_zonefile-sync>` is disabled, the
journal keeps the difference between the zone file and the current zone in case of server shutdown.
The history is stored in changesets – differences of zone contents between two
(usually subsequent) zone versions (specified by SOA serials).

Journals of all zones are stored in a common LMDB database. Huge changesets are
split into 15-70 KiB [#fn-hc]_ blocks to prevent fragmentation of the DB. The
journal does each operation in one transaction to keep consistency of the DB and performance.

Each zone journal has its own occupation limits :ref:`maximum usage <zone_journal-max-usage>`
and :ref:`maximum depth <zone_journal-max-depth>`. Changesets are stored in the journal
one by one. When hitting any of the limits, the zone is flushed into the zone file
if there are no redundant changesets to delete, and the oldest changesets are deleted.
In the case of the size limit, twice [#fn-hc]_ the needed amount of space is purged
to prevent overly frequent deletes.

If :ref:`zone file flush <zone_zonefile-sync>` is disabled, then instead of flushing
the zone, the journal tries to save space by merging the changesets into a special one.
This approach is effective if the changes rewrite each other, e.g. periodically
changing the same zone records, re-signing the whole zone etc. Thus the difference between the zone
file and the zone is still preserved even if the journal deletes some older changesets.

If the journal is used to store both zone history and contents, a special changeset
is present with zone contents. When the journal gets full, the changes are merged into this
special changeset.

There is also a :ref:`safety hard limit <database_journal-db-max-size>` for overall
journal database size, but it's strongly recommended to set the per-zone limits in
a way to prevent hitting this one. For LMDB, it's hard to recover from the
database-full state. For wiping one zone's journal, see *knotc zone-purge +journal*
command.

.. [#fn-hc] This constant is hardcoded.

.. _Handling, zone file, journal, changes, serials:

Handling zone file, journal, changes, serials
=============================================

Some configuration options regarding the zone file and journal, together with operation
procedures, might lead to unexpected results. This chapter points out
potential interference and both recommends and warns before some combinations thereof.
Unfortunately, there is no optimal combination of configuration options,
every approach has some disadvantages.

Example 1
---------

Keep the zone file updated::

   zonefile-sync: 0
   zonefile-load: whole
   journal-content: changes

These are default values. The user can always check the current zone
contents in the zone file, and also modify it (recommended with server turned-off or
taking the :ref:`safe way<Editing zone file>`). The journal serves here just as a source of
history for secondary servers' IXFR. Some users dislike that the server overwrites their
prettily prepared zone file.

Example 2
---------

Zonefileless setup::

   zonefile-sync: -1
   zonefile-load: none
   journal-content: all

Zone contents are stored only in the journal. The zone is updated by DDNS,
zone transfer, or via the control interface. The user might have filled the
zone contents initially from a zone file by setting :ref:`zone_zonefile-load` to
`whole` temporarily.
It's also a good setup for secondary servers. Anyway, it's recommended to carefully tune
the journal-size-related options to avoid surprises like the journal getting full
(see :ref:`Journal behaviour`).

Example 3
---------

Input-only zone file::

   zonefile-sync: -1
   zonefile-load: difference
   journal-content: changes

The user can make changes to the zone by editing the zone file, and his pretty zone file
is never overwritten or filled with DNSSEC-related autogenerated records – they are
only stored in the journal.

.. WARNING::
   The zone file's SOA serial must be properly set to a number which is higher than the
   current SOA serial in the zone (not in the zone file) if manually updated!
   This is important to ensure consistency of the journal and outgoing IXFR.

Example 4
---------

Auto-increment SOA serial::

   zonefile-sync: -1
   zonefile-load: difference-no-serial
   journal-content: all

This is similar to the previous setup, but the SOA serial is handled by the server
automatically. So the user no longer needs to care about it in the zone file.

However, this requires setting :ref:`zone_journal-content` to `all` so that
the information about the last real SOA serial is preserved in case of server re-start.
The sizing of journal limits needs to be taken into consideration
(see :ref:`Journal behaviour`).

.. _Zone bootstrap:

Zone bootstrapping on secondary
===============================

When zone refresh from the primary fails, the ``retry`` value from SOA is used
as the interval between refresh attempts. In a case that SOA isn't known to the
secondary (either because the zone hasn't been retrieved from the primary yet,
or the zone has expired), a backoff is used for repeated retry attempts.

With every retry, the delay rises as a quadratic polynomial (5 * n^2, where n
represents the sequence number of the retry attempt) up to two hours, each time
with a random delay of 0 to 30 seconds added to spread the load on the primary.
In each attempt, the retry interval is subject to :ref:`zone_retry-min-interval`
and :ref:`zone_retry-max-interval`.

Until the refresh has been successfully completed, the backoff is restarted from
the beginning by every ``zone-refresh`` or ``zone-retransfer`` of the zone
triggered manually via :doc:`knotc<man_knotc>`, by ``zone-purge`` or
``zone-restore`` of the zone's timers, or by a restart of :doc:`knotd<man_knotd>`.

.. _Zone expiration:

Zone expiration
===============

On a primary, zone normally never expires. On a secondary, zone expiration results
in removal of the current zone contents and a trigger of immediate zone refresh.
The zone file and zone's journal are kept, but not used for answering requests
until the refresh is successfully completed.

The zone expire timer is set according to the zone's SOA expire field. In addition
to it, Knot DNS also supports EDNS EXPIRE extension of the expire timer in both
primary and secondary roles as described in :rfc:`7314`.

When Knot DNS is configured as a secondary, EDNS EXPIRE option present in a SOA,
IXFR, or AFXR response from the primary is processed and used to update the zone
timer when necessary. This functionality (together with requests of any other EDNS
options) for a specified primary may be disabled using the :ref:`remote_no-edns`
configuration parameter.

If it's necessary, any zone may be expired manually using the ``zone-purge``
command of the :doc:`knotc<man_knotc>` utility. Manual expiration is applicable
to any zone, including a catalog zone or a zone on a primary. Beware, a manually
expired zone on a primary or a manually expired catalog zone becomes valid again
after a server configuration is reloaded or the :doc:`knotd<man_knotd>` process
is restarted, provided that the zone data hasn't been removed.

.. _DNSSEC Key states:

DNSSEC key states
=================

During its lifetime, a DNSSEC key finds itself in different states. Most of the time it
is used for signing the zone and published in the zone. In order to exchange
the key, one type of a key rollover is necessary, and during this rollover,
the key goes through various states with respect to the rollover type and also the
state of the other key being rolled-over.

First, let's list the states of the key being rolled-in.

Standard states:

- ``active`` — The key is used for signing.
- ``published`` — The key is published in the zone, but not used for signing. If the key is
  a KSK or CSK, it is used for signing the DNSKEY RRSet.
- ``ready`` (only for KSK) — The key is published in the zone and used for signing. The
  old key is still active, since we are waiting for the DS records in the parent zone to be
  updated (i.e. "KSK submission").

Special states for algorithm rollover:

- ``pre-active`` — The key is not yet published in the zone, but it's used for signing the zone.
- ``published`` — The key is published in the zone, and it's still used for signing since the
  pre-active state.

Second, we list the states of the key being rolled-out.

Standard states:

- ``retire-active`` — The key is still used for signing, and is published in the zone, waiting for
  the updated DS records in parent zone to be acked by resolvers (KSK case) or synchronizing
  with KSK during algorithm rollover (ZSK case).
- ``retired`` — The key is no longer used for signing. If ZSK, the key is still published in the zone.
- ``removed`` — The key is not used in any way (in most cases such keys are deleted immediately).

Special states for algorithm rollover:

- ``post-active`` — The key is no longer published in the zone, but still used for signing.

Special states for :rfc:`5011` trust anchor roll-over

- ``revoke`` (only for KSK) — The key is published and used for signing, and the Revoked flag is set.

.. NOTE::
   Trust anchor roll-over is not implemented with automatic key management.

   The ``revoke`` state can only be established using :doc:`keymgr<man_keymgr>` when using
   :ref:`dnssec-manual-key-management`.

The states listed above are relevant for :doc:`keymgr<man_keymgr>` operations like generating
a key, setting its timers and listing KASP database.

Note that the key "states" displayed in the server log lines while zone signing
are not according to those listed above, but just a hint as to what the key is currently used for
(e.g. "public, active" = key is published in the zone and used for signing).

.. _DNSSEC Key rollovers:

DNSSEC key rollovers
====================

This section describes the process of DNSSEC key rollover and its implementation
in Knot DNS, and how the operator might watch and check that it's working correctly.
The prerequisite is automatic zone signing with enabled
:ref:`automatic key management<dnssec-automatic-ksk-management>`.

The KSK and ZSK rollovers are triggered by the respective zone key getting old according
to the settings (see :ref:`KSK<policy_ksk-lifetime>` and :ref:`ZSK<policy_zsk-lifetime>` lifetimes).

The algorithm rollover starts when the policy :ref:`algorithm<policy_algorithm>`
field is updated to a different value.

The signing scheme rollover happens when the policy :ref:`signing scheme<policy_single-type-signing>`
field is changed.

It's also possible to change the algorithm and signing scheme in one rollover.

The operator may check the next rollover phase time by watching the next zone signing time,
either in the log or via ``knotc zone-status``. There is no special log for finishing a rollover.

.. NOTE::
   There are never two key rollovers running in parallel for one zone. If
   a rollover is triggered while another is in progress, it waits until the
   first one is finished. Note that a rollover might be considered finished
   when the old key is retired or waiting to be deleted.

The ZSK rollover is performed with Pre-publish method, KSK rollover uses Double-Signature
scheme, as described in :rfc:`6781`.

.. _Automatic KSK and ZSK rollovers example:

Automatic KSK and ZSK rollovers example
---------------------------------------

Let's start with the following set of keys::

  2021-05-10T20:50:00+0200 info: [example.com.] DNSSEC, key, tag 50613, algorithm ECDSAP256SHA256, KSK, public, active
  2021-05-10T20:50:00+0200 info: [example.com.] DNSSEC, key, tag 62932, algorithm ECDSAP256SHA256, public, active

The last fields hint the key state: ``public`` denotes a key that will be presented
as the DNSKEY record, ``ready`` means that CDS/CDNSKEY records were created,
``active`` tells us that the key is used for signing, while ``active+`` is an
active key undergoing a roll-over or roll-in.

For demonstration purposes, the following configuration is used::

  submission:
   - id: test_submission
     check-interval: 2s
     parent: dnssec_validating_resolver

  policy:
   - id: test_policy
     ksk-lifetime: 5m
     zsk-lifetime: 2m
     propagation-delay: 2s
     dnskey-ttl: 10s
     zone-max-ttl: 15s
     ksk-submission: test_submission

Upon the zone's KSK lifetime expiration, a new KSK is generated and the rollover
continues along the lines of :rfc:`6781#section-4.1.2`::

  # KSK Rollover (50613 -> 9081)

  2021-05-10T20:50:00+0200 info: [example.com.] DNSSEC, signing zone
  2021-05-10T20:50:00+0200 info: [example.com.] DNSSEC, KSK rollover started
  2021-05-10T20:50:00+0200 info: [example.com.] DNSSEC, key, tag 50613, algorithm ECDSAP256SHA256, KSK, public, active
  2021-05-10T20:50:00+0200 info: [example.com.] DNSSEC, key, tag 62932, algorithm ECDSAP256SHA256, public, active
  2021-05-10T20:50:00+0200 info: [example.com.] DNSSEC, key, tag  9081, algorithm ECDSAP256SHA256, KSK, public, active+
  2021-05-10T20:50:00+0200 info: [example.com.] DNSSEC, signing started
  2021-05-10T20:50:00+0200 info: [example.com.] DNSSEC, successfully signed
  2021-05-10T20:50:00+0200 info: [example.com.] DNSSEC, next signing at 2021-05-10T20:50:12+0200

  ... (propagation-delay + dnskey-ttl) ...

  2021-05-10T20:50:12+0200 info: [example.com.] DNSSEC, signing zone
  2021-05-10T20:50:12+0200 notice: [example.com.] DNSSEC, KSK submission, waiting for confirmation
  2021-05-10T20:50:12+0200 info: [example.com.] DNSSEC, key, tag 50613, algorithm ECDSAP256SHA256, KSK, public, active
  2021-05-10T20:50:12+0200 info: [example.com.] DNSSEC, key, tag 62932, algorithm ECDSAP256SHA256, public, active
  2021-05-10T20:50:12+0200 info: [example.com.] DNSSEC, key, tag  9081, algorithm ECDSAP256SHA256, KSK, public, ready, active+
  2021-05-10T20:50:12+0200 info: [example.com.] DNSSEC, signing started
  2021-05-10T20:50:12+0200 info: [example.com.] DNSSEC, successfully signed
  2021-05-10T20:50:12+0200 info: [example.com.] DNSSEC, next signing at 2021-05-17T20:49:56+0200

At this point the new KSK has to be submitted to the parent zone. Knot detects the updated parent's DS
record automatically (and waits for additional period of the DS's TTL before retiring the old key)
if :ref:`parent DS check<Submission section>` is configured, otherwise the
operator must confirm it manually (using ``knotc zone-ksk-submitted``)::

  2021-05-10T20:50:12+0200 info: [example.com.] DS check, outgoing, remote 127.0.0.1@5300, KSK submission check: negative
  2021-05-10T20:50:14+0200 info: [example.com.] DS check, outgoing, remote 127.0.0.1@5300, KSK submission check: negative
  2021-05-10T20:50:16+0200 info: [example.com.] DS check, outgoing, remote 127.0.0.1@5300, KSK submission check: positive
  2021-05-10T20:50:16+0200 notice: [example.com.] DNSSEC, KSK submission, confirmed
  2021-05-10T20:50:16+0200 info: [example.com.] DNSSEC, signing zone
  2021-05-10T20:50:16+0200 info: [example.com.] DNSSEC, key, tag 50613, algorithm ECDSAP256SHA256, KSK, public, active+
  2021-05-10T20:50:16+0200 info: [example.com.] DNSSEC, key, tag 62932, algorithm ECDSAP256SHA256, public, active
  2021-05-10T20:50:16+0200 info: [example.com.] DNSSEC, key, tag  9081, algorithm ECDSAP256SHA256, KSK, public, active
  2021-05-10T20:50:16+0200 info: [example.com.] DNSSEC, signing started
  2021-05-10T20:50:16+0200 info: [example.com.] DNSSEC, successfully signed
  2021-05-10T20:50:16+0200 info: [example.com.] DNSSEC, next signing at 2021-05-10T20:50:23+0200

  ... (parent's DS TTL is 7 seconds) ...

  2021-05-10T20:50:23+0200 info: [example.com.] DNSSEC, signing zone
  2021-05-10T20:50:23+0200 info: [example.com.] DNSSEC, key, tag 62932, algorithm ECDSAP256SHA256, public, active
  2021-05-10T20:50:23+0200 info: [example.com.] DNSSEC, key, tag  9081, algorithm ECDSAP256SHA256, KSK, public, active
  2021-05-10T20:50:23+0200 info: [example.com.] DNSSEC, signing started
  2021-05-10T20:50:23+0200 info: [example.com.] DNSSEC, successfully signed
  2021-05-10T20:50:23+0200 info: [example.com.] DNSSEC, next signing at 2021-05-10T20:51:56+0200

Upon the zone's ZSK lifetime expiration, a new ZSK is generated and the rollover
continues along the lines of :rfc:`6781#section-4.1.1`::

  # ZSK Rollover (62932 -> 33255)

  2021-05-10T20:51:56+0200 info: [example.com.] DNSSEC, signing zone
  2021-05-10T20:51:56+0200 info: [example.com.] DNSSEC, ZSK rollover started
  2021-05-10T20:51:56+0200 info: [example.com.] DNSSEC, key, tag 62932, algorithm ECDSAP256SHA256, public, active
  2021-05-10T20:51:56+0200 info: [example.com.] DNSSEC, key, tag  9081, algorithm ECDSAP256SHA256, KSK, public, active
  2021-05-10T20:51:56+0200 info: [example.com.] DNSSEC, key, tag 33255, algorithm ECDSAP256SHA256, public
  2021-05-10T20:51:56+0200 info: [example.com.] DNSSEC, signing started
  2021-05-10T20:51:56+0200 info: [example.com.] DNSSEC, successfully signed
  2021-05-10T20:51:56+0200 info: [example.com.] DNSSEC, next signing at 2021-05-10T20:52:08+0200

  ... (propagation-delay + dnskey-ttl) ...

  2021-05-10T20:52:08+0200 info: [example.com.] DNSSEC, signing zone
  2021-05-10T20:52:08+0200 info: [example.com.] DNSSEC, key, tag 62932, algorithm ECDSAP256SHA256, public
  2021-05-10T20:52:08+0200 info: [example.com.] DNSSEC, key, tag  9081, algorithm ECDSAP256SHA256, KSK, public, active
  2021-05-10T20:52:08+0200 info: [example.com.] DNSSEC, key, tag 33255, algorithm ECDSAP256SHA256, public, active
  2021-05-10T20:52:08+0200 info: [example.com.] DNSSEC, signing started
  2021-05-10T20:52:08+0200 info: [example.com.] DNSSEC, successfully signed
  2021-05-10T20:52:08+0200 info: [example.com.] DNSSEC, next signing at 2021-05-10T20:52:25+0200

  ... (propagation-delay + zone-max-ttl) ...

  2021-05-10T20:52:25+0200 info: [example.com.] DNSSEC, signing zone
  2021-05-10T20:52:25+0200 info: [example.com.] DNSSEC, key, tag  9081, algorithm ECDSAP256SHA256, KSK, public, active
  2021-05-10T20:52:25+0200 info: [example.com.] DNSSEC, key, tag 33255, algorithm ECDSAP256SHA256, public, active
  2021-05-10T20:52:25+0200 info: [example.com.] DNSSEC, signing started
  2021-05-10T20:52:25+0200 info: [example.com.] DNSSEC, successfully signed
  2021-05-10T20:52:25+0200 info: [example.com.] DNSSEC, next signing at 2021-05-10T20:54:08+0200
  2021-05-10T20:54:08+0200 info: [example.com.] DNSSEC, signing zone

Further rollovers::

  ... (zsk-lifetime - propagation-delay - zone-max-ttl) ...

  # Another ZSK Rollover (33255 -> 49526)

  2021-05-10T20:54:08+0200 info: [example.com.] DNSSEC, signing zone
  2021-05-10T20:54:08+0200 info: [example.com.] DNSSEC, ZSK rollover started
  2021-05-10T20:54:08+0200 info: [example.com.] DNSSEC, key, tag  9081, algorithm ECDSAP256SHA256, KSK, public, active
  2021-05-10T20:54:08+0200 info: [example.com.] DNSSEC, key, tag 33255, algorithm ECDSAP256SHA256, public, active
  2021-05-10T20:54:08+0200 info: [example.com.] DNSSEC, key, tag 49526, algorithm ECDSAP256SHA256, public
  2021-05-10T20:54:08+0200 info: [example.com.] DNSSEC, signing started
  2021-05-10T20:54:08+0200 info: [example.com.] DNSSEC, successfully signed
  2021-05-10T20:54:08+0200 info: [example.com.] DNSSEC, next signing at 2021-05-10T20:54:20+0200

  ...

  # Another KSK Rollover (9081 -> 9179)

  2021-05-10T20:55:00+0200 info: [example.com.] DNSSEC, signing zone
  2021-05-10T20:55:00+0200 info: [example.com.] DNSSEC, KSK rollover started
  2021-05-10T20:55:00+0200 info: [example.com.] DNSSEC, key, tag  9081, algorithm ECDSAP256SHA256, KSK, public, active
  2021-05-10T20:55:00+0200 info: [example.com.] DNSSEC, key, tag 49526, algorithm ECDSAP256SHA256, public, active
  2021-05-10T20:55:00+0200 info: [example.com.] DNSSEC, key, tag  9179, algorithm ECDSAP256SHA256, KSK, public, active+
  2021-05-10T20:55:00+0200 info: [example.com.] DNSSEC, signing started
  2021-05-10T20:55:00+0200 info: [example.com.] DNSSEC, successfully signed
  2021-05-10T20:55:00+0200 info: [example.com.] DNSSEC, next signing at 2021-05-10T20:55:12+0200

  ...

.. TIP::
   If systemd is available, the KSK submission event is logged into journald
   in a structured way. The intended use case is to trigger a user-created script.
   Example::

     journalctl -f -t knotd -o json | python3 -c '
     import json, sys
     for line in sys.stdin:
       k = json.loads(line);
       if "KEY_SUBMISSION" in k:
         print("%s, zone=%s, keytag=%s" % (k["__REALTIME_TIMESTAMP"], k["ZONE"], k["KEY_SUBMISSION"]))
     '

   Alternatively, the :ref:`D-Bus signaling<server_dbus-event>` can be utilized for the same use.

.. _DNSSEC Shared KSK:

DNSSEC shared KSK
=================

Knot DNS allows, with automatic DNSSEC key management, to configure a shared KSK for multiple zones.
By enabling :ref:`policy_ksk-shared`, we tell Knot to share all newly-created KSKs
among all the zones with the same :ref:`DNSSEC signing policy<Policy section>` assigned.

The feature works as follows. Each zone still manages its keys separately. If a new KSK shall be
generated for the zone, it first checks if it can grab another zone's shared KSK instead -
that is the last generated KSK in any of the zones with the same policy assigned.
Anyway, only the cryptographic material is shared, the key may have different timers
in each zone.

.. rubric:: Consequences:

If we have an initial setting with brand new zones without any DNSSEC keys,
the initial keys for all zones are generated. With shared KSK, they will all have the same KSK,
but different ZSKs. The KSK rollovers may take place at slightly different times for each of the zones,
but the resulting new KSK will be shared again among all of them.

If we have zones which already have their keys, turning on the shared KSK feature triggers no action.
But when a KSK rollover takes place, they will use the same new key afterwards.

.. WARNING::
   Changing the policy :ref:`id<policy_id>` must be done carefully if shared
   KSK is in use.

.. _DNSSEC Delete algorithm:

DNSSEC delete algorithm
=======================

This is how to "disconnect" a signed zone from a DNSSEC-aware parent zone.
More precisely, we tell the parent zone to remove our zone's DS record by
publishing a special formatted CDNSKEY and CDS record. This is mostly useful
if we want to turn off DNSSEC on our zone so it becomes insecure, but not bogus.

With automatic DNSSEC signing and key management by Knot, this is as easy as
configuring :ref:`policy_cds-cdnskey-publish` option and reloading the configuration.
We check if the special CDNSKEY and CDS records with the rdata "0 3 0 AA==" and "0 0 0 00",
respectively, appeared in the zone.

After the parent zone notices and reflects the change, we wait for TTL expire
(so all resolvers' caches get updated), and finally we may do anything with the
zone, e.g. turning off DNSSEC, removing all the keys and signatures as desired.

.. _DNSSEC Offline KSK:

DNSSEC Offline KSK
==================

Knot DNS allows a special mode of operation where the private part of the Key Signing Key is
not available to the daemon, but it is rather stored securely in an offline storage. This requires
that the KSK/ZSK signing scheme is used (i.e. :ref:`policy_single-type-signing` is off).
The Zone Signing Key is always fully available to the daemon in order to sign common changes to the zone contents.

The server (or the "ZSK side") only uses ZSK to sign zone contents and its changes. Before
performing a ZSK rollover, the DNSKEY records will be pre-generated and signed by the
signer (the "KSK side"). Both sides exchange keys in the form of human-readable messages with the help
of the :doc:`keymgr<man_keymgr>` utility.

Prerequisites
-------------

For the ZSK side (i.e. the operator of the DNS server), the zone has to be configured with:

- Enabled :ref:`DNSSEC signing <zone_dnssec-signing>`
- Properly configured and assigned :ref:`DNSSEC policy <Policy section>`:

  - Enabled :ref:`policy_manual`
  - Enabled :ref:`policy_offline-ksk`
  - Explicit :ref:`policy_dnskey-ttl`
  - Explicit :ref:`policy_zone-max-ttl`
  - Other options are optional
- KASP DB may contain a ZSK (the present or some previous one(s))

For the KSK side (i.e. the operator of the KSK signer), the zone has to be configured with:

- Properly configured and assigned :ref:`DNSSEC policy <Policy section>`:

  - Enabled :ref:`policy_manual`
  - Enabled :ref:`policy_offline-ksk`
  - Explicit :ref:`policy_rrsig-refresh`
  - Optional :ref:`policy_rrsig-lifetime`, :ref:`policy_rrsig-pre-refresh`,
    :ref:`policy_algorithm`, :ref:`policy_reproducible-signing`,
    and :ref:`policy_cds-cdnskey-publish`
  - Other options are ignored
- KASP DB contains a KSK (the present or a newly generated one)

Generating and signing future ZSKs
----------------------------------

1.  Use the ``keymgr pregenerate`` command on the ZSK side to prepare the ZSKs for a specified period of time in the future. The following example
    generates ZSKs for the *example.com* zone for 6 months ahead starting from now::

     $ keymgr -c /path/to/ZSK/side.conf example.com. pregenerate +6mo

    If the time period is selected as e.g. *2 x* :ref:`policy_zsk-lifetime` *+ 4 x* :ref:`policy_propagation-delay`, it will
    prepare roughly two complete future key rollovers. The newly-generated
    ZSKs remain in non-published state until their rollover starts, i.e. the time
    they would be generated in case of automatic key management.

2.  Use the ``keymgr generate-ksr`` command on the ZSK side to export the public parts of the future ZSKs in a form
    similar to DNSKEY records. You might use the same time period as in the first step::

     $ keymgr -c /path/to/ZSK/side.conf example.com. generate-ksr +0 +6mo > /path/to/ksr/file

    Save the output of the command (called the Key Signing Request or KSR) to a file and transfer it to the KSK side e.g. via e-mail.

3.  Use the ``keymgr sign-ksr`` command on the KSK side with the KSR file from the previous step as a parameter::

     $ keymgr -c /path/to/KSK/side.conf example.com. sign-ksr /path/to/ksr/file > /path/to/skr/file

    This creates all the future forms of the DNSKEY, CDNSKEY and CSK records and all the respective RRSIGs and prints them on output. Save
    the output of the command (called the Signed Key Response or SKR) to a file and transfer it back to the ZSK side.

4.  Use the ``keymgr import-skr`` command to import the records and signatures from the SKR file generated in the last step
    into the KASP DB on the ZSK side::

     $ keymgr -c /path/to/ZSK/side.conf example.com. import-skr /path/to/skr/file

5. Use the ``knotc zone-keys-load`` command to trigger a zone re-sign on the ZSK-side and set up the future re-signing events correctly.::

    $ knotc -c /path/to/ZSK/side.conf zone-keys-load example.com.

6. Now the future ZSKs and DNSKEY records with signatures are ready in KASP DB for later usage.
   Knot automatically uses them at the correct time intervals.
   The entire procedure must be repeated before the time period selected at the beginning passes,
   or whenever a configuration is changed significantly. Importing new SKR over some previously-imported
   one leads to deleting the old offline records.

Offline KSK and manual ZSK management
-------------------------------------

If the automatically preplanned ZSK roll-overs (first step) are not desired, just set the :ref:`policy_zsk-lifetime`
to zero, and manually pregenerate ZSK keys and set their timers. Then follow the steps
``generate-ksr — sign-ksr — import-skr — zone-keys-load`` and repeat the ceremony when necessary.

Offline KSK roll-over
---------------------

The KSKs (on the KSK side) must be managed manually, but manual KSK roll-over is possible. Just plan the steps
of the KSK roll-over in advance, and whenever the KSK set or timers are changed, re-perform the relevant rest of the ceremony
``sign-ksr — import-skr — zone-keys-load``.

Emergency SKR
-------------

A general recommendation for large deployments is to have some backup pre-published keys, so that if the current ones are
compromised, they can be rolled-over to the backup ones without any delay. But in the case of Offline KSK, according to
the procedures above, both ZSK and KSK immediate rollovers require the KSR-SKR ceremony.

However, a trick can be done to achieve really immediate key substitution. This is no longer about Knot DNS functionality,
just a hint for the operator.

The idea is to perform every KSR-SKR ceremony twice: once with normal state of the keys (the backup key is only published),
and once with the keys already exchanged (the backup key is temporarily marked as active and the standard key temporarily
as public only). The second (backup) SKR should be saved for emergency key replacement.

Summary of the steps:

* Prepare KSK and ZSK side as usual, including public-only emergency key
* Perform normal Offline KSK ceremony:

  * Pre-generate ZSKs (only in the case of automatic ZSK management)
  * Generate KSR
  * Sign KSR on the KSK side
  * Import SKR
  * Re-sign the zone

* Freeze the zone on the ZSK side
* Temporarily set the backup key as active and the normal key as publish-only
* Perform backup Offline KSK ceremony:

  * Generate KSR (only if the backup key is a replacement for ZSK)
  * Sign the KSR on the KSK side
  * Save the SKR to a backup storage, don't import it yet

* Return the keys to the previous state
* Thaw the zone on the ZSK side

Emergency key replacement:

* Import the backup SKR
* Align the keys with the new states (backup key as active, compromised key as public)
* Re-sign the zone

.. _DNSSEC multi-signer:

DNSSEC multi-signer
===================

`Multi-signer` is a general term that refers to any mode of operation in which
a DNS zone is signed by multiple servers (usually two) in parallel.
Knot DNS offers various multi-signer modes, which are recommended for redundancy
within an organization. For multi-signer operations involving multiple
"DNSSEC providers" and the ability to switch between them, you can also refer to
`MUSIC <https://github.com/DNSSEC-Provisioning/music>`_.


Regardless of the chosen mode from the following options, any secondary that has multiple signers
configured as primaries must prevent interchanged IXFR from them. This can be achieved
either by setting :ref:`master pinning <zone_master-pin-tolerance>` on every secondary or
by setting distinct :ref:`zone_serial-modulo` on each signer. It is recommended to combine
both approaches. Alternatively, if any of the secondaries is not Knot DNS,
:ref:`zone_provide-ixfr` can be disabled on the signers.

Sharing private keys, manual policy
-----------------------------------

When DNSSEC keys are shared among zone signing servers (signers), one challenge
is automatic key management (roll-overs) and synchronization among the signers.
In this example mode of operation, it is expected that key management is maintained
outside of Knot, and the generated keys, including private keys and metadata
(timers), are available in Bind9 format.

Every new key is then imported into each Knot using the :doc:`keymgr <man_keymgr>`
``import-bind`` command, after which :doc:`knotc <man_knotc>` ``zone-keys-load``
is invoked. With :ref:`policy_manual` policy configured, the signers simply
follow prescribed key timers, maintaining the same key set at each signer.
For more useful commands like ``list``, ``set``, and ``delete``, refer
to :doc:`keymgr <man_keymgr>`.

Sharing private keys, automatic policy
--------------------------------------

Knot handles automatic key management very well, but enabling it on multiple
instances would lead to redundant key generation. However, it's possible to enable it on one signer
and keep synchronizing the keys to all others. The managing signer shall be configured with
:ref:`automatic ZSK/KSK management <dnssec-automatic-zsk-management>`, all others
with :ref:`policy_manual` policy.

The key set changes on the managing signer can be monitored by periodic queries
with :doc:`keymgr <man_keymgr>` ``list``, or by listening to
:ref:`D-Bus <server_dbus-event>` interface and watching for the ``keys_updated`` event.

Whenever the key set is changed, key synchronization can be safely performed with
:ref:`Data and metadata backup` feature. Dump the KASP
database on the managing signer with :doc:`knotc <man_knotc>` ``zone-backup +kaspdb``,
transfer the backup directory to each other signer, and import the keys by
:doc:`knotc <man_knotc>` ``zone-restore +kaspdb``, followed by ``zone-keys-load``
on them.

This way, the full key set, including private keys and all metadata, is always
synchronized between signers. The method of transporting the backup directory
is beyond the scope of Knot and this documentation. An eventual loss of the managing
signer results in the automatic key management being halted, but DNSSEC signing continues
to function. The synchronization delay for keys between the managing signer and other
signers must be accounted for in :ref:`policy_propagation-delay`.

Distinct keys, DNSKEY record synchronization
--------------------------------------------

When the DNSSEC keys are not shared among signers, each server can manage its own keys separately.
However, the DNSKEY (including CDNSKEY and CDS) records (with public keys) must be synchronized
for full validity of the signed zone. :ref:`Dynamic updates` can be used to achieve this sharing.

The following configuration options should be used:

  - Set :ref:`policy_dnskey-management` to ``incremental`` on each signer to ensure
    it retains the other's DNSKEY records in the zone during signing.
  - Set :ref:`policy_delete-delay` to a reasonable time interval, which ensures that
    all signers get synchronized during this period.
  - Set :ref:`policy_cds-cdnskey-publish` to either ``none`` or ``always``, otherwise
    the parent DS record might configure itself to point only to one signer's KSK.
  - Configure :ref:`policy_dnskey-sync` to all other signers so that this signer's
    public keys appear in each other's DNSKEY (also applies to CDNSKEY and CDS) RRSets.
  - Configure :ref:`ACL` so that DDNS from all other signers is allowed.
  - Set :ref:`zone_ddns-master` to empty value (`""`) so that DDNS from other signers
    is not forwarded to the primary master if any.
  - Additionally, the synchronization delay between all signers must be accounted
    for in :ref:`policy_propagation-delay`.

With careful configuration, all signers automatically synchronize their DNSKEY (and eventually
CDNSKEY and CDS) RRSets, keeping them synchronized during roll-overs. Nevertheless,
it is recommended to monitor their logs.

.. NOTE::
   It is highly recommended to use this mode with only two signers. With three or more signers,
   it often happens that they continuously overwrite each other's DNSKEYs for a long time before
   settling down. This can be mitigated by configuring :ref:`policy_dnskey-sync` in a cyclic maner,
   such that they form a cycle (i.e. signer1 synchronizes only to signer2, signer2 to signer3 and so on).
   However, this in turn leads to a breakage in DNSKEY synchronization whenever any signer goes offline.
   A practical compromise is carefully configuring the order of each signer's :ref:`policy_dnskey-sync`
   values in the way that the "cycling" signer is at the first position and the remaining signers follow.

An illustrative example of the second of three signers::

   remote:
       - id: signer1
         address: 10.20.30.1
       - id: signer3
         address: 10.20.30.3

    acl:
       - id: signers
         remote: [ signer1, signer3 ]
         action: [ query, update ]
         # TODO configure TSIGs!

   dnskey-sync:
       - id: sync
         remote: [ signer3, signer1 ] # the order matters here!

   policy:
       - id: multisigner
         single-type-signing: on
         ksk-lifetime: 60d
         ksk-submission: ... # TODO see Automatic KSK management
         propagation-delay: 14h
         delete-delay: 2h
         cds-cdnskey-publish: always
         dnskey-management: incremental
         dnskey-sync: sync

   zone:
       - domain: example.com.
         # TODO configure zonefile and journal
         # TODO configure transfers in/out: master, NOTIFY, ACLs...
         dnssec-signing: on
         dnssec-policy: multisigner
         ddns-master: ""
         serial-modulo: 1/3
         acl: signers

Distinct keys, DNSKEY at common unsigned primary
------------------------------------------------

The same approach and configuration can be used, with the difference that the signers
do not send updated DNSKEYs (along with CDNSKEYs and CDSs) to each other. Instead, they
send the updates to their common primary, which holds the unsigned version of zone.
The only configuration change involves redirecting
:ref:`policy_dnskey-sync` to the common primary and adjusting its ACL to allow DDNS
from the signers.

It is also necessary to configure :ref:`zone_ixfr-benevolent` on each signer so that
they accept incremental zone transfers from the primary with additions (or removals)
of their own's DNSKEYs.

This setup should work nicely with any number of signers, however, due to the size
of DNSKEY RRSet, at most three are advisable.

.. _DNSSEC keys import to HSM:

DNSSEC keys import to HSM
=========================

Knot DNS stores DNSSEC keys in textual PEM format (:rfc:`7468`),
while many HSM management software require the keys for import to be in binary
DER format (`Rec. ITU-T X.690 <https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=x.690>`_).
Keys can be converted from one format to another by software tools such as
``certtool`` from `GnuTLS <https://www.gnutls.org/>`_ suite or
``openssl`` from `OpenSSL <https://www.openssl.org/>`_ suite.

In the examples below, ``c4eae5dea3ee8c15395680085c515f2ad41941b6`` is used as the key ID,
``c4eae5dea3ee8c15395680085c515f2ad41941b6.pem`` represents the filename of the key in PEM format
as copied from the Knot DNS zone's :ref:`KASP database directory <database_kasp-db>`,
``c4eae5dea3ee8c15395680085c515f2ad41941b6.priv.der`` represents the file containing the private
key in DER format as generated by the conversion tool, and
``c4eae5dea3ee8c15395680085c515f2ad41941b6.pub.der`` represents the file containing the public
key in DER format as generated by the conversion tool.

.. code-block:: console

   $ certtool -V -k --outder --infile c4eae5dea3ee8c15395680085c515f2ad41941b6.pem \
     --outfile c4eae5dea3ee8c15395680085c515f2ad41941b6.priv.der

   $ certtool -V --pubkey-info --outder --load-privkey c4eae5dea3ee8c15395680085c515f2ad41941b6.pem \
     --outfile c4eae5dea3ee8c15395680085c515f2ad41941b6.pub.der

As an alternative, ``openssl`` can be used instead. It is necessary to specify either ``rsa`` or ``ec``
command according to the algorithm used by the key.

.. code-block:: console

   $ openssl rsa -outform DER -in c4eae5dea3ee8c15395680085c515f2ad41941b6.pem \
     -out c4eae5dea3ee8c15395680085c515f2ad41941b6.priv.der

   $ openssl rsa -outform DER -in c4eae5dea3ee8c15395680085c515f2ad41941b6.pem \
     -out c4eae5dea3ee8c15395680085c515f2ad41941b6.pub.der -pubout

Actual import of keys (both public and private keys from the same key pair) to an HSM can be done
via PKCS #11 interface, by ``pkcs11-tool`` from `OpenSC <https://github.com/OpenSC/OpenSC/wiki>`_ toolkit
for example.  In the example below, ``/usr/local/lib/pkcs11.so`` is used as a name of the PKCS #11 library
or module used for communication with the HSM.

.. code-block:: console

   $ pkcs11-tool --module /usr/local/lib/pkcs11.so --login \
     --write-object c4eae5dea3ee8c15395680085c515f2ad41941b6.priv.der --type privkey \
     --usage-sign --id c4eae5dea3ee8c15395680085c515f2ad41941b6

   $ pkcs11-tool --module /usr/local/lib/pkcs11.so -login \
     --write-object c4eae5dea3ee8c15395680085c515f2ad41941b6.pub.der --type pubkey \
     --usage-sign --id c4eae5dea3ee8c15395680085c515f2ad41941b6

.. _Controlling a running daemon:

Daemon controls
===============

Knot DNS was designed to allow server reconfiguration on-the-fly
without interrupting its operation. Thus it is possible to change
both configuration and zone files and also add or remove zones without
restarting the server. This can be done with::

    $ knotc reload

If you want to refresh the secondary zones, you can do this with::

    $ knotc zone-refresh

.. _Data and metadata backup:

Data and metadata backup
========================

Some of the zone-related data, such as zone contents or DNSSEC signing keys,
and metadata, like zone timers, might be worth backing up. For the sake of
consistency, it's usually necessary to shut down the server, or at least freeze all
the zones, before copying the data like zone files, KASP database, etc, to
a backup location. To avoid this necessity, Knot DNS provides a feature to
back up some or all of the zones seamlessly.

Online backup
-------------

While the server is running and the zones normally loaded (even when they are
constantly/frequently being updated), the user can manually trigger the
backup by calling::

    $ knotc zone-backup +backupdir /path/of/backup

To back up just some of the zones (instead of all), the user might provide
their list::

    $ knotc zone-backup +backupdir /path/to/backup zone1.com. zone2.com. ...

The backup directory should be empty or non-existing and it must be accessible
and writable for the :ref:`server_user` account under which knotd is running.
The backup procedure will begin soon and will happen zone-by-zone
(partially in parallel if more :ref:`server_background-workers` are configured).
**The user shall check the logs for the outcome of each zone's backup attempt.**
The knotc's ``-b`` parameter might be used if the user desires to wait until
the backup work is done and a simple result status is printed out.

.. TIP::
   There is a plain ASCII text file in the backup directory,
   ``knot_backup.label``, that contains some useful information about the
   backup, such as the backup creation date & time, the server identity, etc.
   Care must always be taken **not to remove this file** from the backup nor to
   damage it.

If a backup fails, the backup directory containing incomplete backup is retained.
For repeated backup attempts to the same directory, it must be removed or renamed
manually first.

.. NOTE::
   When backing up or restoring a catalog zone, it's necessary to make sure that
   the contents of the catalog doesn't change during the backup or restore.
   An easy solution is to use ``knotc zone-freeze`` on the catalog zone for the
   time of backup and restore.

Offline restore
---------------

If the Online backup was performed for all zones, it's possible to
restore the backed up data by simply copying them to their normal locations,
since they're simply copies. For example, the user can copy (overwrite)
the backed up KASP database files to their configured location.

This restore of course must be done when the server is stopped. After starting up
the server, it should run in the same state as at the time of backup.

This method is recommended in the case of complete data loss, for example
physical server failure.

.. NOTE::
   The online backup procedure stores all zone files in a single directory
   using their default file names. If the original directory layout was
   different, then the required directory structure must be created manually
   for offline restore and zone files must be placed individually to their
   respective directories. If the zone file names don't follow the default
   pattern, they must be renamed manually to match the configuration. These
   limitations don't apply to the online restore procedure.

Online restore
--------------

This procedure is symmetrical to Online backup. By calling::

    $ knotc zone-restore +backupdir /path/of/backup

the user triggers a one-by-one zone restore from the backup on a running
server. Again, a subset of zones might be specified. It must be specified
if the backup was created for only a subset of zones.

.. NOTE::
   For restore of backups that have been created by Knot DNS releases prior
   to 3.1, it's necessary to use the ``-f`` option. Since this option also
   turns off some verification checks, it shouldn't be used in other cases.

.. NOTE::
   For QUIC, only the auto-generated key is restored. The ``zone-restore``
   command doesn't restore a user-defined QUIC key and certificate so as to
   avoid possible configuration management conflicts and they must be restored
   from the backup (its subdirectory ``quic``) manually. In all cases,
   restart of the Knot server after the restore is necessary for the restored
   QUIC key/certificate to take effect.

Limitations
-----------

Neither configuration file nor :ref:`Configuration database` is backed up
by zone backup. The configuration has to be synchronized before zone restore
is performed!

If the private keys are stored in a HSM (anything using a PKCS#11 interface),
they are not backed up. This includes internal metadata of the PKCS#11 provider
software, such as key mappings, authentication information, and the configuration
of the provider. Details are vendor-specific.

The restore procedure does not care for keys deleted after taking the snapshot.
Thus, after restore, there might remain some redundant ``.pem`` files
of obsolete signing keys.

.. TIP::
   In order to seamlessly deploy a restored backup of KASP DB with respect to
   a possibly ongoing DNSSEC key rollover, it's recommended to set
   :ref:`propagation-delay <policy_propagation-delay>` to the sum of:

   - The maximum delay between beginning of the zone signing and publishing
     re-signed zone on all public secondary servers.
   - How long it takes for the backup server to start up with the restored data.
   - The period between taking backup snapshots of the live environment.

.. _Statistics:

Statistics
==========

The server provides some general statistics and optional query module statistics
(see :ref:`mod-stats<mod-stats>`).

Server statistics or global module statistics can be shown by::

    $ knotc stats
    $ knotc stats server             # Show all server counters
    $ knotc stats mod-stats          # Show all mod-stats counters
    $ knotc stats server.zone-count  # Show specific server counter

Per zone statistics can be shown by::

    $ knotc zone-stats example.com.                       # Show all zone counters
    $ knotc zone-stats example.com. mod-stats             # Show all zone mod-stats counters
    $ knotc zone-stats example.com. mod-stats.query-type  # Show specific zone counter
    $ knotc zone-stats --                                 # Show all zone counters for all zones
    $ knotc zone-stats -- mod-stats.request-protocol      # Show specific zone counter for all zones

To show all supported counters even with 0 value, use the force option.

A simple periodic statistic dump to a YAML file can also be enabled. See
:ref:`stats section` for the configuration details.

As the statistics data can be accessed over the server control socket,
it is possible to create an arbitrary script (Python is supported at the moment)
which could, for example, publish the data in JSON format via HTTP(S)
or upload the data to a more efficient time series database. Take a look into
the python folder of the project for these scripts.

.. _Mode XDP:

Mode XDP
========

Thanks to recent Linux kernel capabilities, namely eXpress Data Path and AF_XDP
address family, Knot DNS offers a high-performance DNS over UDP packet processing
mode. The basic idea is to filter DNS messages close to the network device and
effectively forward them to the nameserver without touching the network stack
of the operating system. Other messages (including DNS over TCP) are processed
as usual.

If :ref:`xdp_listen` is configured, the server creates
additional XDP workers, listening on specified interface(s) and port(s) for DNS
over UDP queries. Each XDP worker handles one RX and TX network queue pair.

.. _Mode XDP_pre-requisites:

Pre-requisites
--------------

* Linux kernel 4.18+ (5.x+ is recommended for optimal performance) compiled with
  the `CONFIG_XDP_SOCKETS=y` option. The XDP mode isn't supported in other operating systems.
* A multiqueue network card, which offers enough Combined RX/TX channels, with
  native XDP support is highly recommended. Successfully tested cards:

  * Intel series 700 (driver `i40e`), maximum number of channels per interface is 64.
  * Intel series 500 (driver `ixgbe`), maximum number of channels per interface is 64.
    The number of CPUs available has to be at most 64!

* If the `knotd` service is not directly executed in the privileged mode, some
  additional Linux capabilities have to be set:

  Execute command::

    systemctl edit knot

  And insert these lines::

    [Service]
    CapabilityBoundingSet=CAP_NET_RAW CAP_NET_ADMIN CAP_SYS_ADMIN CAP_IPC_LOCK CAP_SYS_RESOURCE
    AmbientCapabilities=CAP_NET_RAW CAP_NET_ADMIN CAP_SYS_ADMIN CAP_IPC_LOCK CAP_SYS_RESOURCE

  The `CAP_SYS_RESOURCE` is needed on Linux < 5.11.

  All the capabilities are dropped upon the service is started.

* For proper processing of VLAN traffic, VLAN offloading should be disabled. E.g.::

    ethtool -K <interface> tx-vlan-offload off rx-vlan-offload off


Optimizations
-------------

Some helpful commands::

 ethtool -N <interface> rx-flow-hash udp4 sdfn
 ethtool -N <interface> rx-flow-hash udp6 sdfn
 ethtool -L <interface> combined <?>
 ethtool -G <interface> rx <?> tx <?>
 renice -n 19 -p $(pgrep '^ksoftirqd/[0-9]*$')

Limitations
-----------

* Request and its response must go through the same physical network device.
* Dynamic DNS over XDP is not supported.
* MTU higher than 1790 bytes is not supported.
* Multiple BPF filters per one network device are not supported.
* Systems with big-endian byte ordering require special recompilation of the nameserver.
* IPv4 header and UDP checksums are not verified on received DNS messages.
* DNS over XDP traffic is not visible to common system tools (e.g. firewall, tcpdump etc.).
* BPF filter is not automatically unloaded from the network device. Manual filter unload::

   ip link set dev <interface> xdp off