Merging upstream version 2.4.59.

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

2 files changed, 302 insertions, 0 deletions

diff --git a/test/README.ci b/test/README.ci
new file mode 100644
index 0000000..43600b0
--- /dev/null
+++ b/test/README.ci
@@ -0,0 +1,163 @@
+
+Variables
+---------
+
+The CI scripts use the following environment variables:
+
+* APR_VERSION - if set, APR of this version is built and installed in
+  $HOME/root/apr-$APR_VERSION - a value of "trunk" means trunk is
+  used, "*.x" means a branch, otherwise a tagged version is implied.
+
+* APR_CONFIG - arguments to pass when running APR's configure script
+  if APR_VERSION is set
+
+* APU_VERSION - if set, APR-util of this version is built and
+  installed in $HOME/root/apr-util-$APU_VERSION - a value of "*.x"
+  means a branch, otherwise a tagged version is implied.  (Since there
+  is no "trunk" for apr-util, that value cannot be used here.)
+
+* APU_CONFIG - arguments to pass when running APR-util's configure
+  script if APU_VERSION is set
+
+* CONFIG - arguments to pass to httpd's configure script.
+
+* BUILDCONFIG - arguments to pass when running httpd's ./buildconf script
+
+* MFLAGS - arguments to pass when running "make" for httpd.
+
+* SKIP_TESTING - if set, the Perl test framework is not run for the
+  build.
+
+* TEST_UBSAN - set for job using UBSan ("Undefined Behaviour Sanitizer")
+
+* TEST_MALLOC - set for job using enhanced malloc debugging.
+
+* TEST_INSTALL - set for job testing "make install"
+
+* TEST_VPATH - set for job testing srcdir!=builddir 
+
+* TEST_LDAP - set for job with slapd, running LDAP tests
+
+* TEST_SSL - set for job with SSL/TLS testing variants
+
+* TESTS - a list of Perl framework tests to run
+
+* TEST_ARGS - arguments to pass to ./t/TEST in the Perl test framework
+
+* CLEAR_CACHE - if set, the cached $HOME/root is removed before each build
+
+Caching -- NOTE, BROKEN IN GITHUB ACTIONS --
+-------
+
+Perl modules installed in $HOME/perl5 are cached.
+
+Anything installed into the $HOME/root directory is cached - notably,
+versions of APR/APR-util are installed here and cached across httpd
+build jobs without needing to be rebuilt every time.
+
+The cached installs of APR/APR-util are refreshed if the
+last-changed-revision of the build is stale.
+
+If APR_VERSION and APU_VERSION are both set to 1.x versions, then
+CLEAR_CACHE should also be set to disable APR* caching.  APR-util can
+only be rebuilt if an APR checkout is present, so a APR-util cannot be
+built from source alone.  (i.e. the scripts do not handle the case of
+cached, fresh APR plus a cached but stale APR-util)
+
+Travis to Github Actions Migration TODO
+---------------------------------------
+
+* better path filtering so e.g. CHANGES changes don't trigger CI
+* support branch conditionals again (some tests are 2.4.x only, some trunk only)
+* make caching work properly for APR + CPAN modules
+  - this is using the wrong model at the moment
+  - the cache key needs to be based off (source code, job configuration)
+  - rather than done on the fly in test/travis_before_linux.sh
+  - pebble + Rustls builds should also be cached
+* turn on failure notifications?
+* test across different Ubuntu versions again
+ - and test against OpenSSL 1.x since we're now ONLY building against 3.x
+* update the docs below for testing from PRs/feature branches
+* introduce some job ordering rather than having a flat/concurrent
+  set, if the default "./configure && make && test" works *then* start
+  jobs doing 200 different variations on ./configure --enable-XXX 
+  i.e. stop burning CPU time for a typo which breaks every job
+
+TODO list
+---------
+
+* non-x86 builds
+* MacOS build
+* Windows build
+* clang-on-Linux build
+* Use containers for non-Ubuntu-based Linux testing
+
+Known Failures
+--------------
+
+Some known failures:
+
+* prefork, and more rarely non-prefork testing sometimes catches child
+  segfaults under pool-debug from assert()s in (e.g.)
+  __pthread_tpp_change_priority where one child is destroying threads
+  which another is waiting for, or iterating through via
+  apr_pool_walk_tree().
+
+  See dev@httpd threads:
+  msg <5f4abde1b5789_13fde2ecacb40795a1@travis-tasks-5b566d48fc-drkb9.mail>
+  msg <73060f7b-df7f-ad3c-a9fa-dd666a59b31e@kippdata.de> and
+  https://bz.apache.org/bugzilla/show_bug.cgi?id=63098
+  https://bz.apache.org/bugzilla/show_bug.cgi?id=46185
+
+  Not clear if there is a real bug here which can be reproduced
+  outside of pool-debug.
+
+Testing from a Feature Branch [*** N/A FOR GITHUB ACTIONS ***]
+-----------------------------
+
+An SVN branch off trunk should be mirrored to github, and will be
+tested in the same way that trunk is in CI, so this workflow is
+available for those familiar with using Subversion and the standard
+ASF/httpd repository layout.
+
+Tested branches are listed at: https://travis-ci.com/github/apache/httpd/branches
+
+Travis will also run the tests for a PR filed against the httpd Github
+repository at https://github.com/apache/httpd or from a fork of this
+repository if enabled for the Travis user.
+
+A workflow to enable testing would be as follows, substituting
+$USERNAME for your github username:
+
+  $ git clone https://github.com/apache/httpd
+  $ cd httpd
+  $ git remote add $USERNAME git@github.com:$USERNAME/httpd.git
+  $ git checkout -b my-feature origin/trunk
+  ... do some work ...
+  $ git commit ...
+  $ git push -u $USERNAME my-feature:my-feature
+
+To enable testing for a fork, visit the settings page at
+https://travis-ci.com/$USERNAME/httpd/settings - you may need to sync
+your account via https://travis-ci.com/account/repositories for a
+freshly created fork.
+
+To create a Pull Request, go to the URL produced in the "git push"
+command output when pushing to your fork, which is something like:
+https://github.com/apache/httpd/compare/trunk...$USERNAME:trunk
+
+Once a PR has been created, travis will run the tests and link the
+results from a PR comment. All tested PRs are listed here:
+https://travis-ci.com/github/apache/httpd/pull_requests
+
+To merge from github back to SVN trunk, create a patch from e.g.:
+
+  $ git diff origin/trunk..my-feature
+
+and then apply it in SVN.  To rebase a feature once trunk has
+diverged, from a feature branch run:
+
+  $ git pull
+  $ git rebase -i origin/trunk
+
+and follow the standard rebase steps.
diff --git a/test/README.pytest b/test/README.pytest
new file mode 100644
index 0000000..474030b
--- /dev/null
+++ b/test/README.pytest
@@ -0,0 +1,139 @@
+Apache httpd pytest suite
+=========================
+Using pytest (<https://docs.pytest.org/en/6.2.x/>) and a Python >= 3.8
+for a more flexible testing of Apache httpd.
+
+Install
+-------
+If not already installed, you will need to install 'pytest' and 'OpenSSL' for
+python:
+> apt install python3-pip
+> pip install -U pytest
+> pip install -U pyopenssl
+
+And for 'h2load':
+> apt install nghttp2-client
+
+
+Usage
+-----
+In your httpd source checkout, do:
+
+> make install
+> pytest
+
+and all tests defined run on the installed executable and modules.
+> pytest test/modules/core
+runs all the core tests. You can run tests also by name selection:
+> pytest -k test_core
+runs all test cases starting with 'test_core'. Similar:
+> pytest -k test_core_001_04
+runs the test cases starting with that.
+
+Test output gets more verbose, by adding one or several '-v'. This
+also raises the error log level of the tested modules.
+> pytest -vvv -k test_h2_004_01
+run the specific test with mod_http2 at log level TRACE2.
+
+By default, test cases will configure httpd with mpm_event. You
+can change that with the invocation:
+> MPM=worker pytest test/modules/http2
+
+Some tests rely on additional tools installed in your environment
+and will 'skip' if those are not present. In a non-verbose run,
+these will appear as 's' in the output. If you run pytest more
+verbose, the skipped test cases will mention a reason for why
+they are disabled.
+
+For example, most tests in test/modules/md require an installation
+of 'pebble', an ACME test server, and look for it in $PATH.
+
+
+Workings
+--------
+All tests start httpd on their own and try hard to shut it down
+afterwards. You can abort tests with ^C and they clean up.
+
+httpd is started/stopped repeatedly in testing as configurations
+for individual test cases are changed. This is a main difference to
+the Perl test framework which starts the server with all possible
+combinations configured that are needed by tests.
+
+In test/gen/apache a server root is created with config, logs and htdocs
+directories. test/gen/apache/logs/error_log will be the log.
+Configs start in test/gen/apache/conf/httpd.conf. modules.conf is
+dynamically created for the list of modules that a test suite needs.
+
+Test cases write their specific setup in test.conf and reload/restart
+the httpd process. This makes for a small configuration in a test case.
+
+
+Development
+-----------
+
+Adding a test in an existing file is done by adding a method. Its name
+must start with 'test_' and the common practice is to have the name
+of the test suite there as well. All http2 tests start with 'test_h2_'.
+
+Following this can be any characters. If you make test cases of a
+certain feature with a common prefix, it is easier to invoke just
+them using the '-k' selector on the command line.
+
+You can also add just a new file to a test suite, if you do a new
+range of test cases that do not fit anywhere else. A very simple
+one is found in test/modules/http2/test_001_httpd_alive.py.
+
+There is a python class defined with 2 methods. One is the test
+method itself and the other one ' is
+
+    @pytest.fixture(autouse=True, scope='class')
+    def _class_scope(self, env):
+        code
+
+is marked as a pytest 'fixture'. This is some pytest magic.
+'autouse=True' means that this fixture is run, even though
+no test case uses it directly. scope='class' means that it
+is run once for all test cases in this class.
+
+As you see, this fixture gets a parameter named 'env' and
+that is the name of another pytest fixture, defined in the
+file 'conftest.py' in the same directory.
+
+    @pytest.fixture(scope="package")
+    def env(pytestconfig) -> H2TestEnv:
+        code
+
+This one runs one time per 'package', meaning for all test
+cases in this directory. And it gets the 'pytestconfig'
+as parameter which is a standard pytest fixture.
+
+So, when you run 'pytest -k test_h2_004', pytest will look
+at _all_ test cases defined and collect those with that
+prefix. For each directory with test cases found, it will
+process the 'conftest.py', boot-strapping the 'env' fixture,
+and the process the files with active test cases.
+
+As a result, if you invoke just a single test case, only
+the fixtures needed for that test case are created. This
+gives good turn around times when debugging a test case.
+
+If you want to add a new test suite, create a new directory.
+Add the files '__init__.py', 'conftest.py' and a first
+'test_suitename_something.py'. test/modules/core is the
+simplest example. 'test/modules/http2' shows how you load
+other modules. 'test/modules/md' checks and starts external
+processes (an ACME test server).
+
+
+Infrastructure
+--------------
+The test cases rely on the classes provided in 'test/pyhttpd'
+for common code in managing a httpd test instance and do
+provide some base setups:
+- a small set of virtual hosts with some files
+- access to paths and definitions (env fixture)
+- start/stop httpd and inspect the error log
+- run clients like curl and nghttp
+- create certificates for your hosts and make curl use
+  the root certs (so no --insecure calls by curl).
+