Adding upstream version 1:7.0.3.upstream/1%7.0.3

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 434 insertions, 0 deletions

diff --git a/doc/userguide/setting-up-ipsinline-for-linux.rst b/doc/userguide/setting-up-ipsinline-for-linux.rst
new file mode 100644
index 0000000..fd4fcb6
--- /dev/null
+++ b/doc/userguide/setting-up-ipsinline-for-linux.rst
@@ -0,0 +1,434 @@
+Setting up IPS/inline for Linux
+================================
+
+Setting up IPS with Netfilter
+-----------------------------
+
+In this guide, we'll discuss how to work with Suricata in layer3 `inline
+mode` using ``iptables``.
+
+First, start by compiling Suricata with NFQ support. For instructions
+see `Ubuntu Installation
+<https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Ubuntu_Installation>`_.
+For more information about NFQ and ``iptables``, see
+:ref:`suricata-yaml-nfq`.
+
+To check if you have NFQ enabled in your Suricata build, enter the following command: ::
+
+  suricata --build-info
+
+and make sure that NFQ is listed in the output.
+
+To run Suricata with the NFQ mode, you have to make use of the ``-q`` option. This
+option tells Suricata which queue numbers it should use.
+
+::
+
+  sudo suricata -c /etc/suricata/suricata.yaml -q 0
+
+
+Iptables configuration
+~~~~~~~~~~~~~~~~~~~~~~
+
+First of all, it is important to know which traffic you would like to send
+to Suricata. There are two choices:
+
+1.  Traffic that passes your computer
+2.  Traffic that is generated by your computer.
+
+.. image:: setting-up-ipsinline-for-linux/IPtables.png
+
+.. image:: setting-up-ipsinline-for-linux/iptables1.png
+
+If Suricata is running on a gateway and is meant to protect the computers
+behind that gateway you are dealing with the first scenario: *forward_ing* .
+
+If Suricata has to protect the computer it is running on, you are dealing
+with the second scenario: *host* (see drawing 2).
+
+These two ways of using Suricata can also be combined.
+
+The easiest rule in case of the gateway-scenario to send traffic to Suricata is:
+
+::
+
+  sudo iptables -I FORWARD -j NFQUEUE
+
+In this case, all forwarded traffic goes to Suricata.
+
+In case of the host situation, these are the two most simple ``iptables`` rules;
+
+::
+
+  sudo iptables -I INPUT -j NFQUEUE
+  sudo iptables -I OUTPUT -j NFQUEUE
+
+It is possible to set a queue number. If you do not, the queue number will
+be 0 by default.
+
+Imagine you want Suricata to check for example just TCP traffic, or all
+incoming traffic on port 80, or all traffic on destination-port 80, you
+can do so like this:
+
+::
+
+  sudo iptables -I INPUT -p tcp  -j NFQUEUE
+  sudo iptables -I OUTPUT -p tcp -j NFQUEUE
+
+In this case, Suricata checks just TCP traffic.
+
+::
+
+  sudo iptables -I INPUT -p tcp --sport 80  -j NFQUEUE
+  sudo iptables -I OUTPUT -p tcp --dport 80 -j NFQUEUE
+
+In this example, Suricata checks all packets for outgoing connections to port 80.
+
+.. image:: setting-up-ipsinline-for-linux/iptables2.png
+
+.. image:: setting-up-ipsinline-for-linux/IPtables3.png
+
+To see if you have set your ``iptables`` rules correct make sure Suricata is
+running and enter:
+
+::
+
+  sudo iptables -vnL
+
+In the example you can see if packets are being logged.
+
+.. image:: setting-up-ipsinline-for-linux/iptables_vnL.png
+
+This description of the use of ``iptables`` is the way to use it with IPv4. To
+use it with IPv6 all previous mentioned commands have to start with ``ip6tables``.
+It is also possible to let Suricata check both kinds of traffic.
+
+There is also a way to use ``iptables`` with multiple networks (and interface cards). Example:
+
+.. image:: setting-up-ipsinline-for-linux/iptables4.png
+
+::
+
+  sudo iptables -I FORWARD -i eth0 -o eth1 -j NFQUEUE
+  sudo iptables -I FORWARD -i eth1 -o eth0 -j NFQUEUE
+
+The options ``-i`` (input) ``-o`` (output) can be combined with all previous mentioned
+options.
+
+If you would stop Suricata and use internet, the traffic will not come through.
+To make internet work correctly, first delete all ``iptables`` rules.
+
+To erase all ``iptables`` rules, enter:
+
+::
+
+  sudo iptables -F
+
+
+NFtables configuration
+~~~~~~~~~~~~~~~~~~~~~~
+
+The NFtables configuration is straight forward and allows mixing firewall rules
+with IPS. The concept is to create a dedicated chain for the IPS that will
+be evaluated after the firewalling rule. If your main table is named `filter`
+it can be created like so::
+
+ nft> add chain filter IPS { type filter hook forward priority 10;}
+
+To send all forwarded packets to Suricata one can use ::
+
+ nft> add rule filter IPS queue
+
+To only do it for packets exchanged between eth0 and eth1 ::
+
+ nft> add rule filter IPS iif eth0 oif eth1 queue
+ nft> add rule filter IPS iif eth1 oif eth0 queue
+
+NFQUEUE advanced options
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+The NFQUEUE mechanism supports some interesting options. The ``nftables`` configuration
+will be shown there but the features are also available in ``iptables``.
+
+The full syntax of the queuing mechanism is as follows::
+
+ nft add rule filter IPS queue num 3-5 options fanout,bypass
+
+This rule sends matching packets to 3 load-balanced queues starting at 3 and
+ending at 5. To get the packets in Suricata with this setup, you need to specify
+multiple queues on command line: ::
+
+ suricata -q 3 -q 4 -q 5
+
+`fanout` and `bypass` are the two available options:
+
+- `fanout`: When used together with load balancing, this will use the CPU ID
+  instead of connection hash as an index to map packets to the queues. The idea
+  is that you can improve performance if there’s one queue per CPU. This requires
+  total with a number of queues superior to 1 to be specified.
+- `bypass`: By default, if no userspace program is listening on an Netfilter
+  queue, then all packets that are to be queued are dropped. When this option
+  is used, the queue rule behaves like ACCEPT if there is no program listening,
+  and the packet will move on to the next table.
+
+The `bypass` option can be used to avoid downtime of link when Suricata is not
+running but this also means that the blocking feature will not be present.
+
+Setting up IPS at Layer 2
+-------------------------
+
+.. _afp-ips-l2-mode:
+
+AF_PACKET IPS mode
+~~~~~~~~~~~~~~~~~~
+
+AF_PACKET capture method is supporting a IPS/Tap mode. In this mode, you just
+need the interfaces to be up. Suricata will take care of copying the packets
+from one interface to the other. No ``iptables`` or ``nftables`` configuration is
+necessary.
+
+You need to dedicate two network interfaces for this mode. The configuration
+is made via configuration variable available in the description of an AF_PACKET
+interface.
+
+For example, the following configuration will create a Suricata acting as IPS
+between interface ``eth0`` and ``eth1``: ::
+
+ af-packet:
+   - interface: eth0
+     threads: 1
+     defrag: no
+     cluster-type: cluster_flow
+     cluster-id: 98
+     copy-mode: ips
+     copy-iface: eth1
+     buffer-size: 64535
+     use-mmap: yes
+   - interface: eth1
+     threads: 1
+     cluster-id: 97
+     defrag: no
+     cluster-type: cluster_flow
+     copy-mode: ips
+     copy-iface: eth0
+     buffer-size: 64535
+     use-mmap: yes
+
+This is a basic af-packet configuration using two interfaces. Interface
+``eth0`` will copy all received packets to ``eth1`` because of the `copy-*`
+configuration variable ::
+
+    copy-mode: ips
+    copy-iface: eth1
+
+The configuration on ``eth1`` is symmetric ::
+
+    copy-mode: ips
+    copy-iface: eth0
+
+There are some important points to consider when setting up this mode:
+
+- The implementation of this mode is dependent of the zero copy mode of
+  AF_PACKET. Thus you need to set `use-mmap` to `yes` on both interface.
+- MTU on both interfaces have to be equal: the copy from one interface to
+  the other is direct and packets bigger then the MTU will be dropped by kernel.
+- Set different values of `cluster-id` on both interfaces to avoid conflict.
+- Any network card offloading creating bigger then physical layer datagram
+  (like GRO, LRO, TSO) will result in dropped packets as the transmit path can not
+  handle them.
+- Set `stream.inline` to `auto` or `yes` so Suricata switches to
+  blocking mode.
+
+The `copy-mode` variable can take the following values:
+
+- `ips`: the drop keyword is honored and matching packets are dropped.
+- `tap`: no drop occurs, Suricata acts as a bridge
+
+Some specific care must be taken to scale the capture method on multiple
+threads. As we can't use defrag that will generate too big frames, the in
+kernel load balancing will not be correct: the IP-only fragment will not
+reach the same thread as the full featured packet of the same flow because
+the port information will not be present.
+
+A solution is to use eBPF load balancing to get an IP pair load balancing
+without fragmentation. The AF_PACKET IPS Configuration using multiple threads
+and eBPF load balancing looks like the following: ::
+
+ af-packet:
+   - interface: eth0
+     threads: 16
+     defrag: no
+     cluster-type: cluster_ebpf
+     ebpf-lb-file: /usr/libexec/suricata/ebpf/lb.bpf
+     cluster-id: 98
+     copy-mode: ips
+     copy-iface: eth1
+     buffer-size: 64535
+     use-mmap: yes
+   - interface: eth1
+     threads: 16
+     cluster-id: 97
+     defrag: no
+     cluster-type: cluster_ebpf
+     ebpf-lb-file: /usr/libexec/suricata/ebpf/lb.bpf
+     copy-mode: ips
+     copy-iface: eth0
+     buffer-size: 64535
+     use-mmap: yes
+
+The eBPF file ``/usr/libexec/suricata/ebpf/lb.bpf`` may not be present on disk.
+See :ref:`ebpf-xdp` for more information.
+
+DPDK IPS mode
+~~~~~~~~~~~~~
+
+In the same way as you would configure AF_PACKET IPS mode, you can configure the DPDK capture module.
+Prior to starting with IPS (inline) setup, it is recommended to go over :ref:`dpdk-capture-module` manual page
+to understand the setup essentials.
+
+DPDK IPS mode, similarly to AF-Packet, uses two interfaces. Packets received on the first network interface
+(``0000:3b:00.1``) are transmitted by the second network interface (``0000:3b:00.0``) and similarly,
+packets received on the second interface (``0000:3b:00.0``) are transmitted
+by the first interface (``0000:3b:00.1``). Packets are not altered in any way in this mode.
+
+The following configuration snippet configures Suricata DPDK IPS mode between two NICs: ::
+
+    dpdk:
+      eal-params:
+        proc-type: primary
+
+      interfaces:
+      - interface: 0000:3b:00.1
+        threads: 4
+        promisc: true
+        multicast: true
+        checksum-checks: true
+        checksum-checks-offload: true
+        mempool-size: 262143
+        mempool-cache-size: 511
+        rx-descriptors: 4096
+        tx-descriptors: 4096
+        copy-mode: ips
+        copy-iface: 0000:3b:00.0
+        mtu: 3000
+
+      - interface: 0000:3b:00.0
+        threads: 4
+        promisc: true
+        multicast: true
+        checksum-checks: true
+        checksum-checks-offload: true
+        mempool-size: 262143
+        mempool-cache-size: 511
+        rx-descriptors: 4096
+        tx-descriptors: 4096
+        copy-mode: ips
+        copy-iface: 0000:3b:00.1
+        mtu: 3000
+
+The previous DPDK configuration snippet outlines several things to consider:
+
+- ``copy-mode`` - see Section :ref:`afp-ips-l2-mode` for more details.
+- ``copy-iface`` - see Section :ref:`afp-ips-l2-mode` for more details.
+- ``threads`` - all interface entries must have their thread count configured
+  and paired/connected interfaces must be configured with the same amount of threads.
+- ``mtu`` - MTU must be the same on both paired interfaces.
+
+DPDK capture module also requires having CPU affinity set in the configuration file. For the best performance,
+every Suricata worker should be pinned to a separate CPU core that is not shared with any other Suricata thread
+(e.g. management threads).
+The following snippet shows a possible :ref:`suricata-yaml-threading` configuration set-up for DPDK IPS mode. ::
+
+    threading:
+      set-cpu-affinity: yes
+      cpu-affinity:
+        - management-cpu-set:
+            cpu: [ 0 ]
+        - worker-cpu-set:
+            cpu: [ 2,4,6,8,10,12,14,16 ]
+
+Netmap IPS mode
+~~~~~~~~~~~~~~~
+
+Using Netmap to support IPS requires setting up pairs of interfaces; packets are received
+on one interface within the pair, inspected by Suricata, and transmitted on the other
+paired interface. You can use native or host stack mode; host stack mode is used when the interface
+name contains the ``^`` character, e.g, ``enp6s0f0^``. host stack mode does not require
+multiple physical network interfaces.
+
+Netmap Host Stack Mode
+^^^^^^^^^^^^^^^^^^^^^^
+Netmap's host stack mode allows packets that flow through Suricata to be used with other host OS applications,
+e.g., a firewall or similar. Additionally, host stack mode allows traffic to be received and transmitted
+on one network interface card.
+
+With host stack mode, Netmap establishes a pair of host stack mode rings (one each for RX and TX). Packets
+pass through the host operating system network protocol stack. Ingress network packets flow from the network
+interface card to the network protocol stack and then into the host stack mode rings. Outbound packets
+flow from the  host stack mode rings to the network protocol stack and finally, to the network interface card.
+Suricata receives packets from the host stack mode rings and, in IPS mode, places packets to be transmitted into
+the host stack mode rings. Packets transmitted by Suricata into the host stack mode rings are available for
+other host OS applications.
+
+Paired network interfaces are specified in the ``netmap`` configuration section.
+For example, the following configuration will create a Suricata acting as IPS
+between interface ``enp6s0f0`` and ``enp6s0f1`` ::
+
+ netmap:
+   - interface: enp6s0f0
+     threads: auto
+     copy-mode: ips
+     copy-iface: enp6s0f1
+
+   - interface: enp6s0f1
+     threads: auto
+     copy-mode: ips
+     copy-iface: enp6s0f0
+
+You can specify the ``threads`` value; the default value of ``auto`` will create a
+thread for each queue supported by the NIC; restrict the thread count by specifying
+a value, e.g., ``threads: 1``
+
+This is a basic netmap configuration using two interfaces. Suricata will copy
+packets between interfaces ``enp6s0f0`` and ``en60sf1`` because of the `copy-*`
+configuration variable in interface's ``enp6s0f0`` configuration ::
+
+    copy-mode: ips
+    copy-iface: enp6s0f1
+
+The configuration on ``enp6s0f1`` is symmetric ::
+
+    copy-mode: ips
+    copy-iface: enp6s0f0
+
+
+The host stack mode feature of Netmap can be used. host stack mode doesn't require a second network
+interface.
+
+This example demonstrates host stack mode with a single physical network interface ``enp6s0f01`` ::
+
+  - interface: enp60s0f0
+    copy-mode: ips
+    copy-iface: enp6s0f0^
+
+The configuration on ``enp6s0f0^`` is symmetric ::
+
+  - interface: enp60s0f0^
+    copy-mode: ips
+    copy-iface: enp6s0f0
+
+
+Suricata will use zero-copy mode when the runmode is ``workers``.
+
+There are some important points to consider when setting up this mode:
+
+- Any network card offloading creating bigger then physical layer datagram
+  (like GRO, LRO, TSO) will result in dropped packets as the transmit path can not
+  handle them.
+- Set `stream.inline` to `auto` or `yes` so Suricata switches to
+  blocking mode. The default value is `auto`.
+
+The `copy-mode` variable can take the following values:
+
+- `ips`: the drop keyword is honored and matching packets are dropped.
+- `tap`: no drop occurs, Suricata acts as a bridge