=====================
 pybuild-autopkgtest
=====================

----------------------------------------------------------------------------------------------------
invokes the test suite against requested Python versions and installed packages
----------------------------------------------------------------------------------------------------

:Manual section: 1
:Author: Antonio Terceiro, 2021

SYNOPSIS
========
  pybuild-autopkgtest

OPTIONS
=======

`pybuild-autopkgtest` takes no options or arguments. All configuration is done
via the same mechanisms you use to control `pybuild` itself, namely by having
build dependencies on the right packages, or by setting `PYBUILD_*` environment
variables in `debian/rules`.

DESCRIPTION
===========

`pybuild-autopkgtest` is an autopkgtest test runner that reuses all the
`pybuild` infrastructure to run tests against installed packages, as expected
by autopkgtest. To enable `pybuild-autopkgtest` for your package, you need to
add **Testsuite: autopkgtest-pkg-pybuild** to the source stanza in
`debian/control`. This will cause autodep8(1) to produce the correct contents
for `debian/tests/control`.

`pybuild-autopkgtest` will run the tests in exactly the same way as `pybuild`
will during the build, with the exception that the tests are not run in the
build directory. The test files are copied to a temporary directory, so that
the tests will run against the installed Python modules, and not against
anything in the source tree.

All the pybuild infrastructure is used, so for most packages you don't need to
add any extra code configure for `pybuild-autopkgtest`. For example, just
having a `python3-pytest` as a build dependency is enough to make the test
runner use `pytest` to run the tests.

The tests are executed via a temporary makefile that includes `debian/rules`
from the package, so that any environment variables defined there will also be
available during autopkgtest, including but not limited to `PYBUILD_*`
variables for configuring the behavior of `pybuild` itself.

ADOPTING PYBUILD-AUTOPKGTEST
============================

Since `pybuild-autopkgtest` reuses environment variables set in `debian/rules`,
migrating packages to use `pybuild-autopkgtest` should not require much effort.

You might have a `debian/tests/control` that duplicates what
`pybuild-autopkgtest` already does, e.g. copying the test files to a temporary
directory, changing to it, and running the tests from there. Such test entries
can usually be removed in favor of adding **Testsuite:
autopkgtest-pkg-pybuild** to `debian/control`.

In general, you want to move any test-related command line arguments to pybuild
into environment variables in `debian/rules`.

You can also have specialized, manually-written test cases, alongside the ones
autogenerated by `autodep8`. For this, both set **Testsuite:
autopkgtest-pkg-pybuild** in `debian/control` and keep your custom tests in
`debian/tests/control`.

VARYING BEHAVIOR UNDER AUTOPKGTEST
==================================

Ideally, the behavior of the tests should be the same during the build and
under autopkgtest, except for the fact that during autopkgtest the tests should
load the code from the installed package. `pybuild-autopkgtest` should do this
correctly most of the time.

There are situations, however, in which you need a slightly different behavior
during the autopkgtest run. There are a few mechanisms to support that:

- `pybuild-autopkgtest` sets the `PYBUILD_AUTOPKGTEST` environment variable to
  `1` during the test run. This way, you can add conditional behavior in
  `debian/rules`.
- Before and after running the tests, `pybuild-autopkgtest` will call the
  `debian/rules` targets `before-pybuild-autopkgtest` and
  `after-pybuild-autopkgtest`, respectively, if they exist.

SAMPLE TEST CONTROL FILE
========================

The control file produced by autodep8(1) looks like this::

    Test-Command: pybuild-autopkgtest
    Depends: @, @builddeps@,
    Restrictions: allow-stderr, skippable,
    Features: test-name=pybuild-autopkgtest

You should never need to hardcode this in `debian/tests/control`. You can add
extra items to `Restrictions` and `Depends` by providing a configuration file
for `autodep8` (`debian/tests/autopkgtest-pkg-pybuild.conf`) like this::

    extra_depends=foo, bar
    extra_restrictions=isolation-container, breaks-testbed

SEE ALSO
========
* pybuild(1)
* autopkgtest(1)
* autodep8(1)