summaryrefslogtreecommitdiffstats
path: root/daemon/bindings/net_xdpsrv.rst
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-07 15:26:00 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-07 15:26:00 +0000
commit830407e88f9d40d954356c3754f2647f91d5c06a (patch)
treed6a0ece6feea91f3c656166dbaa884ef8a29740e /daemon/bindings/net_xdpsrv.rst
parentInitial commit. (diff)
downloadknot-resolver-830407e88f9d40d954356c3754f2647f91d5c06a.tar.xz
knot-resolver-830407e88f9d40d954356c3754f2647f91d5c06a.zip
Adding upstream version 5.6.0.upstream/5.6.0upstream
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'daemon/bindings/net_xdpsrv.rst')
-rw-r--r--daemon/bindings/net_xdpsrv.rst140
1 files changed, 140 insertions, 0 deletions
diff --git a/daemon/bindings/net_xdpsrv.rst b/daemon/bindings/net_xdpsrv.rst
new file mode 100644
index 0000000..e3014fe
--- /dev/null
+++ b/daemon/bindings/net_xdpsrv.rst
@@ -0,0 +1,140 @@
+.. SPDX-License-Identifier: GPL-3.0-or-later
+
+.. _dns-over-xdp:
+
+XDP for higher UDP performance
+------------------------------
+
+.. warning::
+ As of version 5.2.0, XDP support in Knot Resolver is considered
+ experimental. The impact on overall throughput and performance may not
+ always be beneficial.
+
+Using XDP allows significant speedup of UDP packet processing in recent Linux kernels,
+especially with some network drivers that implement good support.
+The basic idea is that for selected packets the Linux networking stack is bypassed,
+and some drivers can even directly use the user-space buffers for reading and writing.
+
+.. TODO perhaps some hint/link about how significant speedup one might get? (link to some talk video?)
+
+Prerequisites
+^^^^^^^^^^^^^
+.. this is mostly copied from knot-dns doc/operations.rst
+
+.. warning::
+ Bypassing the network stack has significant implications, such as bypassing the firewall
+ and monitoring solutions.
+ Make sure you're familiar with the trade-offs before using this feature.
+ Read more in :ref:`dns-over-xdp_limitations`.
+
+* Linux kernel 4.18+ (5.x+ is recommended for optimal performance) compiled with
+ the `CONFIG_XDP_SOCKETS=y` option. XDP isn't supported in other operating systems.
+* libknot compiled with XDP support
+* **A multiqueue network card with native XDP support is highly recommended**,
+ otherwise the performance gain will be much lower and you may encounter
+ issues due to XDP emulation.
+ Successfully tested cards:
+
+ * Intel series 700 (driver `i40e`), maximum number of queues per interface is 64.
+ * Intel series 500 (driver `ixgbe`), maximum number of queues per interface is 64.
+ The number of CPUs available has to be at most 64!
+
+
+Set up
+^^^^^^
+.. first parts are mostly copied from knot-dns doc/operations.rst
+
+The server instances need additional Linux **capabilities** during startup.
+(Or you could start them as `root`.)
+Execute command
+
+.. code-block:: bash
+
+ systemctl edit kresd@.service
+
+And insert these lines:
+
+.. code-block:: ini
+
+ [Service]
+ CapabilityBoundingSet=CAP_NET_RAW CAP_NET_ADMIN CAP_SYS_ADMIN CAP_IPC_LOCK CAP_SYS_RESOURCE
+ AmbientCapabilities=CAP_NET_RAW CAP_NET_ADMIN CAP_SYS_ADMIN CAP_IPC_LOCK CAP_SYS_RESOURCE
+
+The ``CAP_SYS_RESOURCE`` is only needed on Linux < 5.11.
+
+.. TODO suggest some way for ethtool -L? Perhaps via systemd units?
+
+You want the same number of kresd instances and network **queues** on your card;
+you can use ``ethtool -L`` before the services start.
+With XDP this is more important than with vanilla UDP, as we only support one instance
+per queue and unclaimed queues will fall back to vanilla UDP.
+Ideally you can set these numbers as high as the number of CPUs that you want kresd to use.
+
+Modification of ``/etc/knot-resolver/kresd.conf`` may often be quite simple, for example:
+
+.. code-block:: lua
+
+ net.listen('eth2', 53, { kind = 'xdp' })
+ net.listen('203.0.113.53', 53, { kind = 'dns' })
+
+Note that you want to also keep the vanilla DNS line to service TCP
+and possibly any fallback UDP (e.g. from unclaimed queues).
+XDP listening is in principle done on queues of whole network interfaces
+and the target addresses of incoming packets aren't checked in any way,
+but you are still allowed to specify interface by an address
+(if it's unambiguous at that moment):
+
+.. code-block:: lua
+
+ net.listen('203.0.113.53', 53, { kind = 'xdp' })
+ net.listen('203.0.113.53', 53, { kind = 'dns' })
+
+The default selection of queues is tailored for the usual naming convention:
+``kresd@1.service``, ``kresd@2.service``, ...
+but you can still specify them explicitly, e.g. the default is effectively the same as:
+
+.. code-block:: lua
+
+ net.listen('eth2', 53, { kind = 'xdp', nic_queue = env.SYSTEMD_INSTANCE - 1 })
+
+
+Optimizations
+^^^^^^^^^^^^^
+.. this is basically copied from knot-dns doc/operations.rst
+
+Some helpful commands:
+
+.. code-block:: text
+
+ ethtool -N <interface> rx-flow-hash udp4 sdfn
+ ethtool -N <interface> rx-flow-hash udp6 sdfn
+ ethtool -L <interface> combined <queue-number>
+ ethtool -G <interface> rx <ring-size> tx <ring-size>
+ renice -n 19 -p $(pgrep '^ksoftirqd/[0-9]*$')
+
+.. TODO CPU affinities? `CPUAffinity=%i` in systemd unit sounds good.
+
+
+.. _dns-over-xdp_limitations:
+
+Limitations
+^^^^^^^^^^^
+.. this is basically copied from knot-dns doc/operations.rst
+
+* VLAN segmentation is not supported.
+* MTU higher than 1792 bytes is not supported.
+* Multiple BPF filters per one network device are not supported.
+* Symmetrical routing is required (query source MAC/IP addresses and
+ reply destination MAC/IP addresses are the same).
+* Systems with big-endian byte ordering require special recompilation of libknot.
+* IPv4 header and UDP checksums are not verified on received DNS messages.
+* DNS over XDP traffic is not visible to common system tools (e.g. firewall, tcpdump etc.).
+* BPF filter is not automatically unloaded from the network device. Manual filter unload::
+
+ ip link set dev <interface> xdp off
+
+* Knot Resolver only supports using XDP towards clients currently (not towards upstreams).
+* When starting up an XDP socket you may get a harmless warning::
+
+ libbpf: Kernel error message: XDP program already attached
+