diff options
Diffstat (limited to 'selftest/ns')
| -rw-r--r-- | selftest/ns/README | 162 | ||||
| -rwxr-xr-x | selftest/ns/add_bridge_iface.sh | 20 | ||||
| -rwxr-xr-x | selftest/ns/create_bridge.sh | 17 | ||||
| -rwxr-xr-x | selftest/ns/mk_nsenter.sh | 31 | ||||
| -rwxr-xr-x | selftest/ns/nsenter-helper.sh | 29 | ||||
| -rwxr-xr-x | selftest/ns/start_in_ns.sh | 61 |
6 files changed, 320 insertions, 0 deletions
diff --git a/selftest/ns/README b/selftest/ns/README new file mode 100644 index 0000000..896fe15 --- /dev/null +++ b/selftest/ns/README @@ -0,0 +1,162 @@ +The scripts in this directory are experimental and are used to create testenvs +in separate linux namespaces. This avoids the need for socket-wrapper. + +What are Namespaces +=================== +Namespaces allow the kernel to segregate its system resources (files, CPU, +etc), so that different processes only see the set of resources they are +allowed to use. There are several different types of namespace: network, +user, process, file, IPC, and so on. + +Key points to grasp are: +* Each type of namespace gets managed separately by the kernel, i.e. process +namespaces are managed separately to network namespaces, which are separate +to user namespaces. These scripts give each testenv its own network namespace, +but otherwise they all still share the same user/process/etc namespace. +(In future, we may want to give each testenv its own process and user +namespace, to better mimic a production DC). +* Namespaces are created using the 'unshare' utility. The new selftest +namespaces are anonymous/nameless, and so the different namespaces are +identified by the PID of the processes running within the namespace +(typically samba). +* Linux supports nesting namespaces within namespaces. In this case, each +testenv DC has its own network namespace, which is a child of the overarching +selftest namespace (which itself is a child of whatever namespace you run +'make test' from - usually this would be the root namespace). + +How does it work? +================= +Normally when 'make test' is run, every testenv uses a 10.53.57.x IP address +and socket-wrapper passes the packets between them. + +With namespaces, we also use 10.53.57.x IP addresses but have the packets pass through +the kernel's IP stack normally, as it forwards them between namespaces. + +We use veth interfaces for this. veth is a type of virtual interface supported +by the kernel. veth interfaces come in pairs, and act as a tunnel - any packets +sent on a veth interface simply end up as received packets on the pair veth +interface. + +We create a new veth interface pair for each testenv, and use them to connect +up the namespaces. One end of the veth pair is added to the main selftest +namespace, and the other end is added to a new namespace that we'll run +samba in. E.g. + +selftest.pl veth21-br ------------------------ veth21 samba (ad_dc_ntvfs) + 10.53.57.11 10.53.57.21 + Namespace 1 Namespace 2 + +However, we need to run multiple different testenvs and have them talk to +each other. So to do this, we need a bridge interface ('selftest0') to connect +up the namespaces, which essentially just acts as a hub. So connecting together +multiple testenvs looks more like this: + +selftest.pl +-- veth21-br ------------------------ veth21 samba (ad_dc_ntvfs) + | 10.53.57.21 + selftest0 --+ Namespace 2 + 10.53.57.11 | + +-- veth22-br ------------------------ veth22 samba (vampire_dc) + 10.53.57.22 + Namespace 1 Namespace 3 + +The veth interfaces are named vethX and vethX-br, where X is the +SOCKET_WRAPPER_DEFAULT_IFACE for the testenv. The vethX-br interface is always +added to the selftest0 bridge interface. + +How do I use it? +================ +To use namespaces instead of socket-wrapper, just add 'USE_NAMESPACES=1' to the +make command, e.g. + +To run the 'quick' test cases using namespaces: +USE_NAMESPACES=1 make test TESTS=quick + +To setup an ad_dc testenv using namespaces: +USE_NAMESPACES=1 SELFTEST_TESTENV=ad_dc make testenv + +You can connect secondary shells to the namespace your testenv is running in. +The command to do this is a little complicated, so a helper 'nsenter.sh' script +gets autogenerated when the testenv is created. E.g. to connect to the testenv +that the ad_dc is running in, use: +./st/ad_dc/nsenter.sh + +This script also sets up the shell with all the same $SERVER/$USERNAME/etc +variables that you normally get in xterm. + +To run the ad-dc-backup autobuild job using namespaces: +USE_NAMESPACES=1 script/autobuild.py samba-ad-dc-backup --verbose --nocleanup \ + --keeplogs --tail --testbase /tmp/samba-testbase + +Using the customdc testenv, you can basically now essentially your own +light-weight samba VM. E.g. +MY_BACKUP=/home/$USER/samba-backup-prod-domain.tar.bz2 +USE_NAMESPACES=1 BACKUP_FILE=$MY_BACKUP SELFTEST_TESTENV=customdc make testenv + +You can then talk to that DC in any other shell by using +./st/customdc/nsenter.sh which enters the DC's network namespace (with +all the $SERVER/etc env variables defined). + +How to join VMs to the testenv +---------------------------------------- +I haven't tried this (beyond basic IP connectivity), but using namespaces it +should now be possible to connect a Windows VM to a Samba testenv. + +1. Work out the main selftest.pl namespace PID manually, e.g. +SELFTEST_PID= ps waux | grep selftest.pl + +2. Create a new veth to bridge between the selftest namespace and your PC's +default namespace: +sudo ip link add dev testenv-veth0 type veth peer name testenv-veth1 + +3. Move one end of the veth tunnel into the selftest namespace: +sudo ip link set testenv-veth1 netns $SELFTEST_PID + +4. Configure the veth end in the default namespace to be in the same subnet +as the selftest network: +sudo ip link set dev testenv-veth0 up +sudo ip addr add 10.53.57.63/24 dev testenv-veth0 + +5. Enter the selftest namespace, bring that end of the pipe up, and add it to +to the main selftest0 bridge (that connects all the DCs together). We also need +to add a default route from selftest back to your PC's default namespace. +nsenter -t $SELFTEST_PID --net --user --preserve-credentials +ip link set dev testenv-veth1 up +ip link set testenv-veth1 master selftest0 +ip route add default via 10.53.57.63 +logout + +Your Windows VM and samba testenv should now be able to talk to each +other over IP! + +6. The other step is to get DNS working. You probably need to add dns_hub +(10.53.57.64) as a nameserver (at least on your Windows VM). + +This should work for using RSAT tools on samba, or joining Windows to Samba +(depending on the schema version). Joining samba to Windows is a bit more +tricky, as the namespaces are tied to the *running* samba process. + +What you'd probably want to do is run the join command to the windows VM +outside of testenv, create an offline backup-file of the resulting DB, and +then plug that backup-file into the customdc testenv. (And then follow the +above veth/bridge steps to join samba to the VM). + +Note that the namespace disappears once you stop the testenv, so you'd +need to do the above steps with creating the veth interface every time +you restarted the testenv. + +Known limitations +================= +- When running a testenv, sometimes xterm can fail to startup, due to a + permissions problem with /dev/pts. This seems to be a particular problem + with the 'none' testenv. + A short-term work-around is to use a terminal that doesn't try to access + /dev/pts, e.g. just use bash as the terminal: + TERMINAL=bash TERMINAL_ARGS='--norc' USE_NAMESPACES=1 \ + SELFTEST_TESTENV=none make testenv +- Some test cases rely on socket-wrapper, so will fail when run using + namespaces. +- Currently USE_NAMESPACES maps you (i.e. $USER) to root in the new namespace. + This means any test cases that rely on being a non-root user will fail (i.e. + anything that fails under 'sudo make test' will also fail with namespaces). +- Namespaces should work within docker, but currently the 'unshare' system + call is disallowed on the gitlab CI runners. diff --git a/selftest/ns/add_bridge_iface.sh b/selftest/ns/add_bridge_iface.sh new file mode 100755 index 0000000..4090319 --- /dev/null +++ b/selftest/ns/add_bridge_iface.sh @@ -0,0 +1,20 @@ +#!/bin/sh +# +# Configures the interfaces needed for communication between namespaces. +# This handles the bridge-end of the veth pair. +interface=$1 + +# the main bridge interface is called 'selftest0' (although in future we may +# want to segregate the different domains by using different bridges) +bridge=$2 + +# we need to wait for the child namespace to start up and add the new +# interface back to our new namespace +while ! ip link show $interface >/dev/null 2>&1; do + sleep 0.1 + echo "Waiting for $interface to be created..." +done + +# bring the bridge-end of the link up and add it to the bridge +ip link set dev $interface up +ip link set $interface master $bridge diff --git a/selftest/ns/create_bridge.sh b/selftest/ns/create_bridge.sh new file mode 100755 index 0000000..74f7eca --- /dev/null +++ b/selftest/ns/create_bridge.sh @@ -0,0 +1,17 @@ +#!/bin/sh + +# creates a bridge interface (i.e. 'selftest0') that connects together the +# veth interfaces for the various testenvs + +br_name=$1 +ip_addr=$2 +ipv6_addr=$3 + +# make sure the loopback is up (needed for pinging between namespaces, etc) +ip link set dev lo up + +# create the bridge interface and enable it +ip link add $br_name type bridge +ip addr add $ip_addr/24 dev $br_name +ip addr add $ipv6_addr/112 dev $br_name +ip link set $br_name up diff --git a/selftest/ns/mk_nsenter.sh b/selftest/ns/mk_nsenter.sh new file mode 100755 index 0000000..6ba5065 --- /dev/null +++ b/selftest/ns/mk_nsenter.sh @@ -0,0 +1,31 @@ +#!/bin/sh +# +# Helper script. If you want a 2nd shell that communicates with the testenv DC +# you can use the nsenter command to change the namespace you're in. However, +# this command is a bit unwieldy and changes depending on the testenv PID. +# We can generate a helper script on the fly that abstracts all this +# complexity, allowing you to use the same, simple command to change the +# namespace that you're in, e.g. +# st/ad_dc/nsenter.sh + +pid=$1 +exports_file=$2 + +# The basic command to enter the testenv's network namespace. +# We enter the user namespace as well (as ourself, which is really the root +# user for the namespace), otherwise we need sudo to make this work. +nsenter_cmd="nsenter -t $pid --net --user --preserve-credentials" + +# By default, the nsenter command will just start a new shell in the namespace. +# we use a wrapper helper script, which first loads all the environment +# variables that are usually defined in selftest (and prints some basic help). +helper_script="$(dirname $0)/nsenter-helper.sh $exports_file" + +# generate the dynamic script +dyn_script="$(dirname $2)/nsenter.sh" +echo "#!/bin/sh" >$dyn_script +echo "$nsenter_cmd $helper_script" >>$dyn_script +chmod 755 $dyn_script + +# return the script we created +echo "$dyn_script" diff --git a/selftest/ns/nsenter-helper.sh b/selftest/ns/nsenter-helper.sh new file mode 100755 index 0000000..4242227 --- /dev/null +++ b/selftest/ns/nsenter-helper.sh @@ -0,0 +1,29 @@ +#!/bin/sh +# +# Helper script that gets run with nsenter to manually setup a secondary shell +# session to a given namespace testenv. This basically just sets up the same +# environment variables as you normally get with selftest, for convenience. + +if [ $# -lt 1 ]; then + echo "Usage: $0 <exports-file>" + exit 1 +fi + +# we get passed a exports file with all the environment variables defined +exports_file=$1 + +# read the exports file so the new shell has appropriate variables setup +# (we export rather than sourcing here so they get inherited by the subshell) +while read -r line; do + export $line + # dump them for the user too + echo $line +done <$exports_file + +echo "" +echo "Entered $NETBIOSNAME namespace, with above variables defined." +echo "Use CTRL+D or exit to leave the namespace." +echo "" + +# start a shell session in the new namespace +$SHELL diff --git a/selftest/ns/start_in_ns.sh b/selftest/ns/start_in_ns.sh new file mode 100755 index 0000000..f16767d --- /dev/null +++ b/selftest/ns/start_in_ns.sh @@ -0,0 +1,61 @@ +#!/bin/sh +# +# Starts samba in a separate namespace. This gets passed the interface/IP +# to use, as well as the Samba command to run. The whole script gets run +# (via unshare) in a separate namespace. + +# the first 3 args are our interface-name, parent-PID, and a exports file +# containing environment variables ($SERVER, $SERVER_IP, etc) +interface=$1 +exports_file=$2 +parent_pid=$3 + +# we write the testenv environment variables to file, which makes it easier +# to work out the $SERVER, $SERVER_IP, etc +. $exports_file + +# The namespaces we use are anonymous, which means other processes would need +# to use our PID to access the new namespace +echo "-------------------------------------------------------------" +echo "Created namespace for $NETBIOSNAME ($ENVNAME) PID $$" + +# generate a helper script if the developer wants to talk to this namespace +# in another shell +mk_nsenter_script="$(dirname $0)/mk_nsenter.sh" +helper_script=$($mk_nsenter_script $$ $exports_file) + +echo "To communicate with this testenv, use: $helper_script" +echo "-------------------------------------------------------------" + +# the rest of the args are the samba command to run +shift 3 +SAMBA_CMD=$@ + +# make sure namespace loopback is up (it's needed for ping, etc) +ip link set dev lo up + +# Create the interfaces needed for communication between namespaces. +# We use a veth pair, which acts as a tunnel between the namespaces. +# One end of the veth link is added to a common bridge in the top-level (i.e. +# selftest) namespace, and the other end is added to the testenv's namespace. +# This means each testenv DC is in its own namespace, but they can talk to +# each other via the common bridge interface. +# The new veth interfaces are named "vethX" and "vethX-br", where +# X = the testenv IP (i.e. Samba::get_interface()). E.g. ad_dc = veth30, +# and veth30-br. +# The "vethX" interface will live in the new testenv's namespace. +# The "vethX-br" end is added to the bridge in the main selftest namespace. +ip link add dev $interface-br type veth peer name $interface + +# move the bridge end of the link back into the parent namespace. +ip link set $interface-br netns $parent_pid + +# configure our IP address and bring the interface up +ip addr add $SERVER_IP/24 dev $interface +# Note that samba can't bind to the IPv6 address while DAD is in progress, +# so we use 'nodad' when configuring the address +ip addr add $SERVER_IPV6/112 dev $interface nodad +ip link set dev $interface up + +# start samba +$SAMBA_CMD |