Adding upstream version 9.1.upstream/9.1

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

8 files changed, 464 insertions, 0 deletions

diff --git a/tests/topotests/docker/README.md b/tests/topotests/docker/README.md
new file mode 100644
index 0000000..2b40994
--- /dev/null
+++ b/tests/topotests/docker/README.md
@@ -0,0 +1,72 @@
+# Topotests in Docker
+
+## Quickstart
+
+If you have Docker installed, you can run the topotests in Docker.
+The easiest way to do this, is to use the make targets from this
+repository.
+
+Your current user needs to have access to the Docker daemon. Alternatively
+you can run these commands as root.
+
+```console
+make topotests
+```
+
+This command will pull the most recent topotests image from dockerhub, compile FRR inside
+of it, and run the topotests.
+
+## Advanced Usage
+
+Internally, the topotests make target uses a shell script to pull the image and spawn the docker
+container.
+
+There are several environment variables which can be used to modify the behavior
+of the script, these can be listed by calling it with `-h`:
+
+```console
+./tests/topotests/docker/frr-topotests.sh -h
+```
+
+For example, a volume is used to cache build artifacts between multiple runs
+of the image. If you need to force a complete recompile, you can set `TOPOTEST_CLEAN`:
+
+```console
+TOPOTEST_CLEAN=1 ./tests/topotests/docker/frr-topotests.sh
+```
+
+By default, `frr-topotests.sh` will build frr and run pytest. If you append
+arguments and the first one starts with `/` or `./`, they will replace the call to
+pytest. If the appended arguments do not match this patttern, they will be provided to
+pytest as arguments.
+
+So, to run a specific test with more verbose logging:
+
+```console
+./tests/topotests/docker/frr-topotests.sh -vv -s all-protocol-startup/test_all_protocol_startup.py
+```
+
+And to compile FRR but drop into a shell instead of running pytest:
+
+```console
+./tests/topotests/docker/frr-topotests.sh /bin/bash
+```
+
+## Development
+
+The docker image just includes all the components to run the topotests, but not the topotests
+themselves. So if you just want to write tests and don't want to make changes to the environment
+provided by the docker image. You don't need to build your own docker image if you do not want to.
+
+When developing new tests, there is one caveat though: The startup script of the container will
+run a `git-clean` on its copy of the FRR tree to avoid any pollution of the container with build
+artefacts from the host. This will also result in your newly written tests being unavailable in the
+container unless at least added to the index with `git-add`.
+
+If you do want to test changes to the docker image, you can locally build the image and run the tests
+without pulling from the registry using the following commands:
+
+```console
+make topotests-build
+TOPOTEST_PULL=0 make topotests
+```
diff --git a/tests/topotests/docker/build.sh b/tests/topotests/docker/build.sh
new file mode 100755
index 0000000..aec2058
--- /dev/null
+++ b/tests/topotests/docker/build.sh
@@ -0,0 +1,11 @@
+#!/bin/bash
+# SPDX-License-Identifier: MIT
+#
+# Copyright 2018 Network Device Education Foundation, Inc. ("NetDEF")
+
+cd "$(dirname "$0")"/..
+
+exec docker build --pull \
+		  --compress \
+		  -t frrouting/topotests:latest \
+		  .
diff --git a/tests/topotests/docker/frr-topotests.sh b/tests/topotests/docker/frr-topotests.sh
new file mode 100755
index 0000000..ce373d9
--- /dev/null
+++ b/tests/topotests/docker/frr-topotests.sh
@@ -0,0 +1,155 @@
+#!/bin/bash
+# SPDX-License-Identifier: MIT
+#
+# Copyright 2018 Network Device Education Foundation, Inc. ("NetDEF")
+
+set -e
+
+if [[ "$1" = "-h" ]] || [[ "$1" = "--help" ]]; then
+	cat >&2 <<-EOF
+
+	This script runs the FRRouting topotests on the FRR tree
+	in the current working directory.
+
+	Usage: $0 [args...]
+
+	If any arguments are provided and the first argument starts with / or ./
+	the arguments are interpreted as command and will be executed instead
+	of pytest.
+
+	Behavior can be further modified by the following environment variables:
+
+	TOPOTEST_AUTOLOAD       If set to 1, the script will try to load necessary
+	                        kernel modules without asking for confirmation first.
+
+	TOPOTEST_NOLOAD         If set to 1, don't try to load necessary kernel
+	                        modules and don't even ask.
+
+	TOPOTEST_BUILDCACHE     Docker volume used for caching multiple FRR builds
+	                        over container runs. By default a
+	                        \`topotest-buildcache\` volume will be created for
+	                        that purpose.
+
+	TOPOTEST_CLEAN          Clean all previous build artifacts prior to
+	                        building. Disabled by default, set to 1 to enable.
+
+	TOPOTEST_DOC            Build the documentation associated with FRR.
+	                        Disabled by default, set to 1 to enable.
+
+	TOPOTEST_FRR            If set, don't test the FRR in the current working
+	                        directory, but the one at the given path.
+
+	TOPOTEST_LOGS           If set, don't use \`/tmp/topotest_logs\` directory
+	                        but use the provided path instead.
+
+	TOPOTEST_OPTIONS        These options are appended to the docker-run
+	                        command for starting the tests.
+
+	TOPOTEST_PULL           If set to 0, don't try to pull the most recent
+	                        version of the docker image from dockerhub.
+
+	TOPOTEST_SANITIZER      Controls whether to use the address sanitizer.
+	                        Enabled by default, set to 0 to disable.
+
+	TOPOTEST_VERBOSE        Show detailed build output.
+	                        Enabled by default, set to 0 to disable.
+
+	EOF
+	exit 1
+fi
+
+#
+# These two modules are needed to run the MPLS tests.
+# They are often not automatically loaded.
+#
+# We cannot load them from the container since we don't
+# have host kernel modules available there. If we load
+# them from the host however, they can be used just fine.
+#
+
+export PATH="$PATH:/sbin:/usr/sbin:/usr/local/sbin"
+
+if [ "$TOPOTEST_NOLOAD" != "1" ]; then
+	for module in mpls-router mpls-iptunnel; do
+		if modprobe -n $module 2> /dev/null; then
+			:
+		else
+			# If the module doesn't exist, we cannot do anything about it
+			continue
+		fi
+
+		if [ $(grep -c ${module/-/_} /proc/modules) -ne 0 ]; then
+			# If the module is loaded, we don't have to do anything
+			continue
+		fi
+
+		if [ "$TOPOTEST_AUTOLOAD" != "1" ]; then
+			echo "To run all the possible tests, we need to load $module."
+			echo -n "Do you want to proceed? [y/n] "
+			read answer
+			if [ x"$answer" != x"y" ]; then
+				echo "Not loading."
+				continue
+			fi
+		fi
+
+		if [ x"$(whoami)" = x"root" ]; then
+			modprobe $module
+		else
+			sudo modprobe $module
+		fi
+	done
+fi
+
+if [ -z "$TOPOTEST_LOGS" ]; then
+	mkdir -p /tmp/topotest_logs
+	TOPOTEST_LOGS="/tmp/topotest_logs"
+fi
+
+if [ -z "$TOPOTEST_FRR" ]; then
+	TOPOTEST_FRR="$(git rev-parse --show-toplevel || true)"
+	if [ -z "$TOPOTEST_FRR" ]; then
+		echo "Could not determine base of FRR tree." >&2
+		echo "frr-topotests only works if you have your tree in git." >&2
+		exit 1
+	fi
+	git -C "$TOPOTEST_FRR" ls-files -z > "${TOPOTEST_LOGS}/git-ls-files"
+fi
+
+if [ -z "$TOPOTEST_BUILDCACHE" ]; then
+	TOPOTEST_BUILDCACHE=topotest-buildcache
+	docker volume inspect "${TOPOTEST_BUILDCACHE}" &> /dev/null \
+		|| docker volume create "${TOPOTEST_BUILDCACHE}"
+fi
+
+if [ "${TOPOTEST_PULL:-1}" = "1" ]; then
+	docker pull frrouting/topotests:latest
+fi
+
+if [[ -n "$TMUX" ]]; then
+    TMUX_OPTIONS="-v $(dirname $TMUX):$(dirname $TMUX) -e TMUX=$TMUX -e TMUX_PANE=$TMUX_PANE"
+fi
+
+if [[ -n "$STY" ]]; then
+    SCREEN_OPTIONS="-v /run/screen:/run/screen -e STY=$STY"
+fi
+set -- --rm -i \
+        -v "$HOME:$HOME:ro" \
+	-v "$TOPOTEST_LOGS:/tmp" \
+	-v "$TOPOTEST_FRR:/root/host-frr:ro" \
+	-v "$TOPOTEST_BUILDCACHE:/root/persist" \
+	-e "TOPOTEST_CLEAN=$TOPOTEST_CLEAN" \
+	-e "TOPOTEST_VERBOSE=$TOPOTEST_VERBOSE" \
+	-e "TOPOTEST_DOC=$TOPOTEST_DOC" \
+	-e "TOPOTEST_SANITIZER=$TOPOTEST_SANITIZER" \
+	--privileged \
+        $SCREEN_OPTINS \
+        $TMUX_OPTIONS \
+	$TOPOTEST_OPTIONS \
+	frrouting/topotests:latest "$@"
+
+if [ -t 0 ]; then
+	set -- -t "$@"
+fi
+
+exec docker run "$@"
diff --git a/tests/topotests/docker/inner/compile_frr.sh b/tests/topotests/docker/inner/compile_frr.sh
new file mode 100755
index 0000000..3b296a1
--- /dev/null
+++ b/tests/topotests/docker/inner/compile_frr.sh
@@ -0,0 +1,87 @@
+#!/bin/bash
+# SPDX-License-Identifier: MIT
+#
+# Copyright 2018 Network Device Education Foundation, Inc. ("NetDEF")
+
+set -e
+
+# Load shared functions
+CDIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+. $CDIR/funcs.sh
+
+#
+# Script begin
+#
+
+if [ "${TOPOTEST_CLEAN}" != "0" ]; then
+	log_info "Cleaning FRR builddir..."
+	rm -rf $FRR_BUILD_DIR &> /dev/null
+fi
+
+log_info "Syncing FRR source with host..."
+mkdir -p $FRR_BUILD_DIR
+rsync -a --info=progress2 \
+	--from0 --files-from=/tmp/git-ls-files \
+	--chown root:root \
+	$FRR_HOST_DIR/. $FRR_BUILD_DIR/
+
+cd "$FRR_BUILD_DIR" || \
+	log_fatal "failed to find frr directory"
+
+if [ "${TOPOTEST_VERBOSE}" != "0" ]; then
+	exec 3>&1
+else
+	exec 3>/dev/null
+fi
+
+log_info "Building FRR..."
+
+if [ ! -e configure ]; then
+	bash bootstrap.sh >&3 || \
+		log_fatal "failed to bootstrap configuration"
+fi
+
+if [ "${TOPOTEST_DOC}" != "0" ]; then
+	EXTRA_CONFIGURE+=" --enable-doc "
+else
+	EXTRA_CONFIGURE+=" --disable-doc "
+fi
+
+if [ ! -e Makefile ]; then
+	if [ "${TOPOTEST_SANITIZER}" != "0" ]; then
+		export CC="gcc"
+		export CFLAGS="-O1 -g -fsanitize=address -fno-omit-frame-pointer"
+		export LDFLAGS="-g -fsanitize=address -ldl"
+		touch .address_sanitizer
+	else
+		rm -f .address_sanitizer
+	fi
+
+	bash configure >&3 \
+		--enable-static-bin \
+		--enable-static \
+		--enable-shared \
+		--enable-dev-build \
+		--with-moduledir=/usr/lib/frr/modules \
+		--prefix=/usr \
+		--localstatedir=/var/run/frr \
+		--sbindir=/usr/lib/frr \
+		--sysconfdir=/etc/frr \
+		--enable-multipath=0 \
+		--enable-fpm \
+		--enable-sharpd \
+		$EXTRA_CONFIGURE \
+		--with-pkg-extra-version=-topotests \
+		|| log_fatal "failed to configure the sources"
+fi
+
+# if '.address_sanitizer' file exists it means we are using address sanitizer.
+if [ -f .address_sanitizer ]; then
+	make -C lib CFLAGS="-g -O2" LDFLAGS="-g" clippy >&3
+fi
+
+make -j$(cpu_count) >&3 || \
+	log_fatal "failed to build the sources"
+
+make install >/dev/null || \
+	log_fatal "failed to install frr"
diff --git a/tests/topotests/docker/inner/entrypoint.sh b/tests/topotests/docker/inner/entrypoint.sh
new file mode 100755
index 0000000..44e16db
--- /dev/null
+++ b/tests/topotests/docker/inner/entrypoint.sh
@@ -0,0 +1,31 @@
+#!/bin/bash
+# SPDX-License-Identifier: MIT
+#
+# Copyright 2018 Network Device Education Foundation, Inc. ("NetDEF")
+
+# Load shared functions
+CDIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+. $CDIR/funcs.sh
+
+set -e
+
+#
+# Script begin
+#
+"${CDIR}/compile_frr.sh"
+"${CDIR}/openvswitch.sh"
+
+cd "${FRR_BUILD_DIR}/tests/topotests"
+
+log_info "Setting permissions on /tmp so we can generate logs"
+chmod 1777 /tmp
+
+if [ $# -eq 0 ] || ([[ "$1" != /* ]] && [[ "$1" != ./* ]]); then
+	export TOPOTESTS_CHECK_MEMLEAK=/tmp/memleak_
+	export TOPOTESTS_CHECK_STDERR=Yes
+	set -- pytest \
+		--junitxml /tmp/topotests.xml \
+		"$@"
+fi
+
+exec "$@"
diff --git a/tests/topotests/docker/inner/funcs.sh b/tests/topotests/docker/inner/funcs.sh
new file mode 100755
index 0000000..4ebacbb
--- /dev/null
+++ b/tests/topotests/docker/inner/funcs.sh
@@ -0,0 +1,53 @@
+#!/bin/bash
+# SPDX-License-Identifier: MIT
+#
+# Copyright 2018 Network Device Education Foundation, Inc. ("NetDEF")
+
+FRR_HOST_DIR=/root/host-frr
+FRR_BUILD_DIR=/root/persist/frr-build
+
+if [ ! -L "/root/frr" ]; then
+	ln -s $FRR_BUILD_DIR /root/frr
+fi
+
+[ -z $TOPOTEST_CLEAN ] && TOPOTEST_CLEAN=0
+[ -z $TOPOTEST_VERBOSE ] && TOPOTEST_VERBOSE=1
+[ -z $TOPOTEST_DOC ] && TOPOTEST_DOC=0
+[ -z $TOPOTEST_SANITIZER ] && TOPOTEST_SANITIZER=1
+
+log_info() {
+	local msg=$1
+
+	echo -e "=> $msg"
+}
+
+log_error() {
+	local msg=$1
+
+	echo -e "E: $msg" 2>&1
+}
+
+log_warning() {
+	local msg=$1
+
+	echo -e "W: $msg" 2>&1
+}
+
+log_fatal() {
+	local msg=$1
+
+	echo -e "F: $msg" 2>&1
+
+	exit 1
+}
+
+cpu_count() {
+	local cpu_count
+
+	cpu_count=$(cat /proc/cpuinfo	| grep -w processor | wc -l)
+	if [ $? -eq 0 ]; then
+		echo -n $cpu_count
+	else
+		echo -n 2
+	fi
+}
diff --git a/tests/topotests/docker/inner/motd.txt b/tests/topotests/docker/inner/motd.txt
new file mode 100644
index 0000000..1e2f34f
--- /dev/null
+++ b/tests/topotests/docker/inner/motd.txt
@@ -0,0 +1,15 @@
+Welcome to the topotests container.
+
+Here are some useful tips:
+* After changing the FRR/Topotests sources, you may rebuild them
+  using the command `compile_frr.sh`. The build command has the
+  following environment variables:
+  - TOPOTEST_CLEAN: whether we should distclean or not (disabled by default)
+  - TOPOTEST_VERBOSE: show build messages (enabled by default)
+  - TOPOTEST_DOC: whether we should build docs or not (disabled by default)
+  - TOPOTEST_SANITIZER: whether we should use the address sanitizer (enabled by default)
+
+  Usage example: env TOPOTEST_CLEAN=1 compile_frr.sh
+
+* The topotests log directory can be found on your host machine on
+  `/tmp/topotests_logs`.
diff --git a/tests/topotests/docker/inner/openvswitch.sh b/tests/topotests/docker/inner/openvswitch.sh
new file mode 100755
index 0000000..926a25c
--- /dev/null
+++ b/tests/topotests/docker/inner/openvswitch.sh
@@ -0,0 +1,40 @@
+#!/bin/bash
+# SPDX-License-Identifier: MIT
+#
+# Copyright 2018 Network Device Education Foundation, Inc. ("NetDEF")
+
+# Load shared functions
+CDIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+. $CDIR/funcs.sh
+
+#
+# Script begin
+#
+
+log_info "Configuring OpenvSwitch...."
+
+# Configure OpenvSwitch so we are able to run mininet
+mkdir -p /var/run/openvswitch
+ovsdb-tool create /etc/openvswitch/conf.db \
+	/usr/share/openvswitch/vswitch.ovsschema
+ovsdb-server /etc/openvswitch/conf.db \
+	--remote=punix:/var/run/openvswitch/db.sock \
+	--remote=ptcp:6640 --pidfile=ovsdb-server.pid >/dev/null 2>/dev/null & \
+	disown
+ovs-vswitchd >/dev/null 2>/dev/null & disown
+
+sleep 2
+
+ovs-vsctl --no-wait -- init
+ovs_version=$(ovs-vsctl -V | grep ovs-vsctl | awk '{print $4}')
+ovs_db_version=$(\
+	ovsdb-tool schema-version /usr/share/openvswitch/vswitch.ovsschema)
+ovs-vsctl --no-wait -- set Open_vSwitch . db-version="${ovs_db_version}"
+ovs-vsctl --no-wait -- set Open_vSwitch . ovs-version="${ovs_version}"
+ovs-vsctl --no-wait -- set Open_vSwitch . system-type="docker-ovs"
+ovs-vsctl --no-wait -- set Open_vSwitch . system-version="0.1"
+ovs-vsctl --no-wait -- \
+	set Open_vSwitch . external-ids:system-id=`cat /proc/sys/kernel/random/uuid`
+ovs-vsctl --no-wait -- set-manager ptcp:6640
+ovs-appctl -t ovsdb-server \
+	ovsdb-server/add-remote db:Open_vSwitch,Open_vSwitch,manager_options