Adding upstream version 2:4.20.0+dfsg.upstream/2%4.20.0+dfsg

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 162 insertions, 0 deletions

diff --git a/selftest/ns/README b/selftest/ns/README
new file mode 100644
index 0000000..896fe15
--- /dev/null
+++ b/selftest/ns/README
@@ -0,0 +1,162 @@
+The scripts in this directory are experimental and are used to create testenvs
+in separate linux namespaces. This avoids the need for socket-wrapper.
+
+What are Namespaces
+===================
+Namespaces allow the kernel to segregate its system resources (files, CPU,
+etc), so that different processes only see the set of resources they are
+allowed to use. There are several different types of namespace: network,
+user, process, file, IPC, and so on.
+
+Key points to grasp are:
+* Each type of namespace gets managed separately by the kernel, i.e. process
+namespaces are managed separately to network namespaces, which are separate
+to user namespaces. These scripts give each testenv its own network namespace,
+but otherwise they all still share the same user/process/etc namespace.
+(In future, we may want to give each testenv its own process and user
+namespace, to better mimic a production DC).
+* Namespaces are created using the 'unshare' utility. The new selftest
+namespaces are anonymous/nameless, and so the different namespaces are
+identified by the PID of the processes running within the namespace
+(typically samba).
+* Linux supports nesting namespaces within namespaces. In this case, each
+testenv DC has its own network namespace, which is a child of the overarching
+selftest namespace (which itself is a child of whatever namespace you run
+'make test' from - usually this would be the root namespace).
+
+How does it work?
+=================
+Normally when 'make test' is run, every testenv uses a 10.53.57.x IP address
+and socket-wrapper passes the packets between them.
+
+With namespaces, we also use 10.53.57.x IP addresses but have the packets pass through
+the kernel's IP stack normally, as it forwards them between namespaces.
+
+We use veth interfaces for this. veth is a type of virtual interface supported
+by the kernel. veth interfaces come in pairs, and act as a tunnel - any packets
+sent on a veth interface simply end up as received packets on the pair veth
+interface.
+
+We create a new veth interface pair for each testenv, and use them to connect
+up the namespaces. One end of the veth pair is added to the main selftest
+namespace, and the other end is added to a new namespace that we'll run
+samba in. E.g.
+
+selftest.pl  veth21-br ------------------------ veth21 samba (ad_dc_ntvfs)
+             10.53.57.11                          10.53.57.21
+ Namespace 1                                       Namespace 2
+
+However, we need to run multiple different testenvs and have them talk to
+each other. So to do this, we need a bridge interface ('selftest0') to connect
+up the namespaces, which essentially just acts as a hub. So connecting together
+multiple testenvs looks more like this:
+
+selftest.pl     +-- veth21-br ------------------------ veth21 samba (ad_dc_ntvfs)
+                |                                      10.53.57.21
+    selftest0 --+                                        Namespace 2
+    10.53.57.11   |
+                +-- veth22-br ------------------------ veth22 samba (vampire_dc)
+                                                       10.53.57.22
+ Namespace 1                                             Namespace 3      
+
+The veth interfaces are named vethX and vethX-br, where X is the
+SOCKET_WRAPPER_DEFAULT_IFACE for the testenv. The vethX-br interface is always
+added to the selftest0 bridge interface. 
+
+How do I use it?
+================
+To use namespaces instead of socket-wrapper, just add 'USE_NAMESPACES=1' to the
+make command, e.g.
+
+To run the 'quick' test cases using namespaces:
+USE_NAMESPACES=1 make test TESTS=quick
+
+To setup an ad_dc testenv using namespaces:
+USE_NAMESPACES=1 SELFTEST_TESTENV=ad_dc make testenv
+
+You can connect secondary shells to the namespace your testenv is running in.
+The command to do this is a little complicated, so a helper 'nsenter.sh' script
+gets autogenerated when the testenv is created. E.g. to connect to the testenv
+that the ad_dc is running in, use:
+./st/ad_dc/nsenter.sh
+
+This script also sets up the shell with all the same $SERVER/$USERNAME/etc
+variables that you normally get in xterm.
+
+To run the ad-dc-backup autobuild job using namespaces:
+USE_NAMESPACES=1 script/autobuild.py samba-ad-dc-backup --verbose --nocleanup \
+ --keeplogs --tail --testbase /tmp/samba-testbase
+
+Using the customdc testenv, you can basically now essentially your own
+light-weight samba VM. E.g.
+MY_BACKUP=/home/$USER/samba-backup-prod-domain.tar.bz2
+USE_NAMESPACES=1 BACKUP_FILE=$MY_BACKUP SELFTEST_TESTENV=customdc make testenv
+
+You can then talk to that DC in any other shell by using
+./st/customdc/nsenter.sh which enters the DC's network namespace (with
+all the $SERVER/etc env variables defined).
+
+How to join VMs to the testenv
+----------------------------------------
+I haven't tried this (beyond basic IP connectivity), but using namespaces it
+should now be possible to connect a Windows VM to a Samba testenv.
+
+1. Work out the main selftest.pl namespace PID manually, e.g.
+SELFTEST_PID= ps waux | grep selftest.pl
+
+2. Create a new veth to bridge between the selftest namespace and your PC's
+default namespace:
+sudo ip link add dev testenv-veth0 type veth peer name testenv-veth1
+
+3. Move one end of the veth tunnel into the selftest namespace:
+sudo ip link set testenv-veth1 netns $SELFTEST_PID
+
+4. Configure the veth end in the default namespace to be in the same subnet
+as the selftest network:
+sudo ip link set dev testenv-veth0 up
+sudo ip addr add 10.53.57.63/24 dev testenv-veth0
+
+5. Enter the selftest namespace, bring that end of the pipe up, and add it to
+to the main selftest0 bridge (that connects all the DCs together). We also need
+to add a default route from selftest back to your PC's default namespace.
+nsenter -t $SELFTEST_PID --net --user --preserve-credentials
+ip link set dev testenv-veth1 up
+ip link set testenv-veth1 master selftest0
+ip route add default via 10.53.57.63
+logout
+
+Your Windows VM and samba testenv should now be able to talk to each
+other over IP!
+
+6. The other step is to get DNS working. You probably need to add dns_hub
+(10.53.57.64) as a nameserver (at least on your Windows VM).
+
+This should work for using RSAT tools on samba, or joining Windows to Samba
+(depending on the schema version). Joining samba to Windows is a bit more
+tricky, as the namespaces are tied to the *running* samba process.
+
+What you'd probably want to do is run the join command to the windows VM
+outside of testenv, create an offline backup-file of the resulting DB, and
+then plug that backup-file into the customdc testenv. (And then follow the
+above veth/bridge steps to join samba to the VM).
+
+Note that the namespace disappears once you stop the testenv, so you'd
+need to do the above steps with creating the veth interface every time
+you restarted the testenv.
+
+Known limitations
+=================
+- When running a testenv, sometimes xterm can fail to startup, due to a
+  permissions problem with /dev/pts. This seems to be a particular problem
+  with the 'none' testenv.
+  A short-term work-around is to use a terminal that doesn't try to access
+  /dev/pts, e.g. just use bash as the terminal:
+  TERMINAL=bash TERMINAL_ARGS='--norc' USE_NAMESPACES=1 \
+    SELFTEST_TESTENV=none make testenv
+- Some test cases rely on socket-wrapper, so will fail when run using
+  namespaces.
+- Currently USE_NAMESPACES maps you (i.e. $USER) to root in the new namespace.
+  This means any test cases that rely on being a non-root user will fail (i.e.
+  anything that fails under 'sudo make test' will also fail with namespaces).
+- Namespaces should work within docker, but currently the 'unshare' system
+  call is disallowed on the gitlab CI runners.