From 75808db17caf8b960b351e3408e74142f4c85aac Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Sun, 14 Apr 2024 15:42:30 +0200 Subject: Adding upstream version 2.117.0. Signed-off-by: Daniel Baumann --- doc/tutorial/Lintian/Tutorial/TestSuite.pod | 115 +++++ doc/tutorial/Lintian/Tutorial/WritingChecks.pod | 442 +++++++++++++++++++ doc/tutorial/Lintian/Tutorial/WritingTests.pod | 539 ++++++++++++++++++++++++ 3 files changed, 1096 insertions(+) create mode 100644 doc/tutorial/Lintian/Tutorial/TestSuite.pod create mode 100644 doc/tutorial/Lintian/Tutorial/WritingChecks.pod create mode 100644 doc/tutorial/Lintian/Tutorial/WritingTests.pod (limited to 'doc/tutorial/Lintian/Tutorial') diff --git a/doc/tutorial/Lintian/Tutorial/TestSuite.pod b/doc/tutorial/Lintian/Tutorial/TestSuite.pod new file mode 100644 index 0000000..3273388 --- /dev/null +++ b/doc/tutorial/Lintian/Tutorial/TestSuite.pod @@ -0,0 +1,115 @@ +=encoding utf-8 + +=head1 NAME + +Lintian::Tutorial::TestSuite -- Quick intro to running the Lintian testsuite + +=head1 SYNOPSIS + +Warning: This document may be out of date. + +This guide will quickly introduce you to running the Lintian test +suite and some tricks. The Lintian test suite is fairly large and +accordingly it can take a substantial amount of time to run. To speed +up development, there are various options to limit the tests to run. + +If you are looking for a guide on how to write tests, please consult +L. + +=head1 DESCRIPTION + +The Lintian test suite is an extensive collection of various test +cases. The test suite is divided into 4 "sub-suites". The majority +of tests are currently located in the "tests" sub-suite. + +To run the full suite: + + $ rm -rf debian/test-out; private/build-test-packages; private/runtests + +While writing a new tag (or check) you probably only want to run a +particular (subset of the) test(s). See L for the available options. + +=head2 Running a subset of the tests + +First, you have to build the test packages with: + + $ rm -rf debian/test-out; private/build-test-packages; + +Then, the following options are available: + +=over 4 + +=item Running a single test + +To run a single test by its name, use: + + $ private/runtests --onlyrun=test:$name + +=item Running all tests for a check + +To run all tests for a given check, use: + + $ private/runtests --onlyrun=check:$name + +$check must be the name of a check (it will test for +checks/$check.desc) or "legacy". This will run all tests that start +with "$check-". + +=item Running all tests designed for a specific tag + +To run all tests that have a "Test-For" or a "Test-Against" for a given +tag, use: + + $ private/runtests --onlyrun=tag:$name + +=back + +=head2 Running tests under coverage + +This feature is currently untested. + +It is possible to run most of the tests under L. This is +done by passing I<--coverage> to I. Example: + + $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out + +Please note that L does not seem to handle multiple +threads too well. You may see spurious warnings/errors if you run the +tests with 2 or more active worker threads. + +B 1: Coverage for collections (i.e. programs in I) +does not seem to work at the moment. Therefore, they often end up with +(next to) zero coverage in the generated reports. + +B 2: L sometimes changes the output of Lintian +or tools called by Lintian. Obviously, this leads to test +failures. Therefore, you may see weird test failures (or warnings) +when running under coverage. + +=head3 Collecting the coverage you want in a reasonable time + +Collecting coverage is excruciatingly slow. This is not helped by the +fact that it becomes unreliable when run under 2 or more threads. + +Fortunately, L "appends" to its cover database. This +allows you to "slowly" build up the coverage database over multiple +runs. Example: + + $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out suite:scripts + $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out suite:debs + $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out suite:source + ... + +Or: + + $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out $check + $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out legacy + + +=head1 SEE ALSO + +L + +=cut + diff --git a/doc/tutorial/Lintian/Tutorial/WritingChecks.pod b/doc/tutorial/Lintian/Tutorial/WritingChecks.pod new file mode 100644 index 0000000..b6642e1 --- /dev/null +++ b/doc/tutorial/Lintian/Tutorial/WritingChecks.pod @@ -0,0 +1,442 @@ +=encoding utf-8 + +=head1 NAME + +Lintian::Tutorial::WritingChecks -- Writing checks for Lintian + +=head1 SYNOPSIS + +Warning: This tutorial may be outdated. + +This guide will quickly guide you through the basics of writing a +Lintian check. Most of the work is in writing the two files: + + checks/.pm + checks/.desc + +And then either adding a Lintian profile or extending an existing +one. + +=head1 DESCRIPTION + +The basics of writing a check are outlined in the Lintian User Manual +(§3.3). This tutorial will focus on the act of writing the actual +check. In this tutorial, we will assume the name of the check to be +written is "deb/pkg-check". + +The tutorial will work with a "binary" and "udeb" check. Checking +source packages works in a similar fashion. + +=head2 Create a check I<.desc> file + +As mentioned, this tutorial will focus on the writing of a check. +Please see the Lintian User Manual (§3.3) for how to do this part. + +=head2 Create the Perl check module + +Start with the template: + + # deb/pkg-check is loaded as Lintian::deb::pkg_check + # - See Lintian User Manual §3.3 for more info + package Lintian::deb::pkg_check; + + use strict; + use warnings; + + sub run { + my ($pkg, $type, $info, $proc, $group) = @_; + return; + } + +The snippet above is a simple valid check that does "nothing at all". +We will extend it in just a moment, but first let us have a look at +the arguments at the setup. + +The I sub is the entry point of our "deb/pkg-check" check; it +will be invoked once per package it should process. In our case, that +will be once per "binary" (.deb) and once per udeb package processed. + +It is given 5 arguments (in the future, possibly more), which are: + +=over 4 + +=item $pkg - The name of the package being processed. + +(Same as $proc->pkg_name) + +=item $type - The type of the package being processed. + +At the moment, $type is one of "binary" (.deb), "udeb", "source" +(.dsc) or "changes". This argument is mostly useful if certain checks +do not apply equally to all package types being processed. + +Generally it is advisable to check only binaries ("binary" and +"udeb"), sources or changes in a given check. But in rare cases, it +makes sense to lump multiple types together in the same check and this +argument helps you do that. + +(Current it is always identical to $proc->pkg_type) + +=item $info - Accessor to the data Lintian has extracted + +Basically all information you want about a given package comes from +the $info object. Sometimes referred to as either the "info object" or +(an instance of) L. + +This object (together with a properly set Needs-Info in the I<.desc> +file) will grant you access to all of the data Lintian has extracted +about this package. + +Based on the value of the $type argument, it will be one of +L, L or +L. + +(Currently it is the same as $proc->info) + +=item $proc - Basic metadata about the package + +This is an instance of L and is useful for +trivially obtaining very basic package metadata. Particularly, the +name of source package and version of source package are readily +available through this object. + +=item $group - Group of processables from the same source + +If you want to do a cross-check between different packages built from +the same source, $group helps you access those other packages +(if they are available). + +This is an instance of L. + +=back + +Now back to the coding. + +=head2 Accessing fields + +Let's do a slightly harder example. Assume we wanted to emit a tag for +all packages without a (valid) Multi-Arch field. This requires us to +A) identify if the package has a Multi-Arch field and B) identify if +the content of the field was valid. + +Starting from the top. All $info objects have a method called field, +which gives you access to a (raw) field from the control file of the +package. It returns C if said field is not present or the +content of said field otherwise. Note that field names must be given +in all lowercase letters (i.e. use "multi-arch", not "Multi-Arch"). + +This was the first half. Let's look at checking the value. Multi-arch +fields can (currently) be one of "no", "same", "foreign" or "allowed". +One way of checking this would be using the regex: + +Notice that Lintian automatically strips leading and trailing spaces +on the I line in a field. It also strips trailing spaces from +all other lines, but leading spaces and the " ."-continuation markers +are kept as is. + +=head2 Checking dependencies + +Lintian can do some checking of dependencies. For most cases it works +similar to a normal dependency check, but keep in mind that Lintian +uses I logic to determine if dependencies are satisfied (i.e. it +will not look up relations like Provides for you). + +Suppose you wanted all packages with a multi-arch "same" field to +pre-depend on the package "multiarch-support". Well, we could use the +L<< $info->relation|Lintian::Collect::Binary/relation (FIELD) >> method for +this. + +$info->relation returns an instance of L. This +object has an "implies" method that can be used to check if a package +has an explicit dependency. Note that "implies" actually checks if +one relation "implies" another (i.e. if you satisfied relationA then +you definitely also satisfied relationB). + +As with the "field"-method, field names have to be given in all +lowercase. However "relation" will never return C (not even if the +field is missing). + +=head2 Using static data files + +Currently our check mixes data and code. Namely all the valid values +for the Multi-Arch field are currently hard-coded in our check. We can +move those out of the check by using a data file. + +Lintian natively supports data files that are either "sets" or +"tables" via L (i.e. "unordered" collections). As an +added bonus, L transparently supports vendor specific +data files for us. + +First we need to make a data file containing the values. Which could be: + + # A table of all the valid values for the multi-arch field. + no + same + foreign + allowed + +This can then be stored in the data directory as +I. + +Now we can load it by using: + + use Lintian::Data; + + my $VALID_MULTI_ARCH_VALUES = + Lintian::Data->new('deb/pkg-check/multiarch-values'); + +Actually, this is not quite true. L is lazy, so it +will not load anything before we force it to do so. Most of the time +this is just an added bonus. However, if you ever have to force it to +load something immediately, you can do so by invoking its "known" +method (with an arbitrary defined string and ignore the result). + +Data files work with 3 access methods, "all", "known" and "value". + +=over 4 + +=item all + +"all" (i.e. $data->all) returns a list of all the entries in the data +file (for key/value tables, all returns the keys). The list is not +sorted in any order (not even input order). + +=item known + +"known" (i.e. $data->known('item')) returns a truth value if a given +item or key is known (present) in the data set or table. For key/pair +tables, the value associated with the key can be retrieved with +"value" (see below). + +=item value + +"value" (i.e. $data->value('key')) returns a value associated with a +key for key/value tables. For unknown keys, it returns C. If +the data file is not a key/value table but just a set, value returns +a truth value for known keys. + +=back + +While we could use both "value" and "known", we will use the latter +for readability (and to remind ourselves that this is a data set and +not a data table). + +Basically we will be replacing: + + unless exists $VALID_MULTI_ARCH_VALUES{$multiarch}; + +with + + unless $VALID_MULTI_ARCH_VALUES->known($multiarch); + +=head2 Accessing contents of the package + +Another heavily used mechanism is to check for the presence (or absence) +of a given file. Generally this is what the +L<< $info->index|Lintian::Collect::Package/index (FILE) >> and +L<< $info->sorted_index|Lintian::Collect::Package/sorted_index >> methods +are for. The "index" method returns instances of L, +which has a number of utility methods. + +If you want to loop over all files in a package, the sorted_index will +do this for you. If you are looking for a specific file (or directory), a +call to "index" will be much faster. For the contents of a specific directory, +you can use something like: + + if (my $dir = $info->index('path/to/dir/')) { + foreach my $elem ($dir->children) { + print $elem->name . " is a file" if $elem->is_file; + # ... + } + } + +Keep in mind that using the "index" or "sorted_index" method will +require that you put "unpacked" in Needs-Info. See L. + +There are also a pair of methods for accessing the control files of a +binary package. These are +L<< $info->control_index|Lintian::Collect::Package/control_index (FILE) >> and +L<< $info->sorted_control_index|Lintian::Collect::Package/sorted_control_index >>. + +=head3 Accessing contents of a file in a package + +When you actually want to see the contents of a file, you can use +L (or L) on +an object returned by e.g. +L<< $info->index|Lintian::Collect::Package/index (FILE) >>. These +methods will open the underlying file for reading (the latter +applying a gzip decompression). + +However, please do assert that the file is safe to read by calling +L first. Generally, it will +only be true for files or safely resolvable symlinks pointing to +files. Should you attempt to open a path that does not satisfy +those criteria, L will raise a trappable error at +runtime. + +Alternatively, if you access the underlying file object, you can +use the L method. Usually, you will +want to test either L or +L first to ensure you do +not follow unsafe symlinks. The "is_open_ok" check will also assert +that it is not (e.g.) a named pipe or such. + +Should you call L on a symlink that +escapes the package root, the method will throw a trappable error at +runtime. Once the path is returned, there are no more built-in +fail-safes. When you use the returned path, keep things like +"../../../../../etc/passwd"-symlink and "fifo" pipes in mind. + + +In some cases, you may even need to access the file system objects +I using L. This is, of course, discouraged +and suffers from the same issues above (all checking must be done +manually by you). Here you have to use the "unpacked", "debfiles" or +"control" methods from L or its subclasses. + + + +The following snippet may be useful for testing that a given path does +not escape the root. + + use Lintian::Util qw(is_ancestor_of); + + my $path = ...; + # The snippet applies equally well to $info->debfiles and + # $info->control (just remember to subst all occurrences of + # $info->unpacked). + my $unpacked_file = $info->unpacked($path); + if ( -f $unpacked_file && is_ancestor_of($info->unpacked, $unpacked_file)) { + # a file and contained within the package root. + } else { + # not a file or an unsafe path + } + +=head2 Keeping Needs-Info up to date + +Keeping the "Needs-Info" field of your I<.desc> file is a bit of +manual work. In the API description for the method there will +generally be a line looking something like: + + Needs-Info requirements for using methodx: Y + +Which means that the methodx requires Y to work. Here Y is a comma +separated list and each element of Y basically falls into 3 cases. + +=over 4 + +=item * The element is the word I + +In this case, the method has no "external" requirements and can be +used without any changes to your Needs-Info. The "field" method +is an example of this. + +This only makes sense if it is the only element in the list. + +=item * The element is a link to a method + +In this case, the method uses another method to do its job. An example +is the +L +method, which uses the +L +method. So using I has the same requirements as +using I. + +=item * The element is the name of a collection (e.g. "control_index"). + +In this case, the method needs the given collection to be run. So to +use (e.g.) L, +you have to put "bin-pkg-control" in your Needs-Info. + +=back + +CAVEAT: Methods can have different requirements based on the type of +package! An example of this "changelog", which requires "changelog-file" +in binary packages and "Same as debfiles" in source packages. + +=head2 Avoiding security issues + +Over the years a couple of security issues have been discovered in +Lintian. The problem is that people can in theory create some really nasty +packages. Please keep the following in mind when writing a check: + +=over 4 + +=item * Avoid 2-arg open, system/exec($shellcmd), `$shellcmd` like the +plague. + +When you get any one of those wrong you introduce "arbitrary code +execution" vulnerabilities (we learned this the hard way via +CVE-2009-4014). + +Usually 3-arg open and the non-shell variant of system/exec are +enough. When you actually need a shell pipeline, consider using +L. It also provides a I command to assist +with capturing stdout as an alternative to `$cmd` (or qx/$cmd/). + +=item * Do not trust field values. + +This is especially true if you intend to use the value as part of a +file name. Verify that the field contains what you expect before you use +it. + +=item * Use L (or, failing that, is_ancestor_of) + +You might be tempted to think that the following code is safe: + + use autodie; + + my $filename = 'some/file'; + my $ufile = $info->unpacked($filename); + if ( ! -l $ufile) { + # Looks safe, but isn't in general + open(my $fd, '<', $ufile); + ...; + } + +This is definitely unsafe if "$filename" contains at least one +directory segment. So, if in doubt, use +L to +verify that the requested file is indeed the file you think it is. A +better version of the above would be: + + use autodie, + use Lintian::Util qw(is_ancestor_of); + [...] + my $filename = 'some/file'; + my $ufile = $info->unpacked($filename); + if ( ! -l $ufile && -f $ufile && is_ancestor_of($info->unpacked, $ufile)) { + # $ufile is a file and it is contained within the package root. + open(m $fd, '<', $ufile); + ...; + } + +In some cases you can even drop the "! -l $ufile" part. + +Of course, it is much easier to use the L object +(whenever possible). + + my $filename = 'some/file'; + my $ufile = $info->index($filename); + if ( $ufile && $ufile->is_file && $ufile->is_open_ok) { + my $fd = $ufile->open; + ...; + } + +Here you can drop the " && $ufile->is_file" if you want to permit +safe symlinks. + + +For more information on the is_ancestor_of check, see +L + + +=back + +=head1 SEE ALSO + +L, L + +=cut diff --git a/doc/tutorial/Lintian/Tutorial/WritingTests.pod b/doc/tutorial/Lintian/Tutorial/WritingTests.pod new file mode 100644 index 0000000..1011235 --- /dev/null +++ b/doc/tutorial/Lintian/Tutorial/WritingTests.pod @@ -0,0 +1,539 @@ +=encoding utf-8 + +=head1 NAME + +Lintian::Tutorial::WritingTests -- Short tutorial on writing tests + +=head1 SYNOPSIS + +Warning: This document may be out of date. + +This document attempts to be a short / quick tutorial to the Lintian +test suite from a test-writer's perspective. As such, it will only +cover the standard type of tests (from the "tests" suite). + +The guide will involve writing a test for the "deb/pkg-check" check, +which was documented in the L +tutorial. + +For running tests, please see L instead. + +=head1 DESCRIPTION + +The Lintian test suite is divided into several parts. These are: + +=over 4 + +=item - + +scripts + +Small (Perl) "prove" tests. These assert that code style, data files +and/or self-contained code units (i.e. unit tests) work as intended. +They are B used for testing Lintian tags. + +=item - + +tags + +These tests all test for the presence of tags after building test +packages using skeletons. For most cases, we recommend + + Skeleton: upload-non-native + +suites are small test suites that test some particular tags for +I<.changes>, I<.deb> or I<.dsc> files. Typically, you will find the +more exotic tags here, which require some special fiddling and cannot +be built by a "standard" dh7 + dpkg build. + +=item - + +literal + +These tests look to match the literal output of Lintian. These tests +are useful as general false positives. They also catch Lintian messages +unrelated to tags. + +=back + +With this in mind, let us move on to the scope. + +=head2 Scope of the tutorial + +WARNING: THE REMAINDER OF THIS TUTORIAL IS OUT OF DATE. + +The "tests" suite alone is fairly complex on its own. To keep things +simple, the tutorial will limit itself to creating a "native" +package with no special requirements in the "tests" suite. + +In particular, note that the tags I be I for this +to work. If you followed the check writing tutorial and made the tags +pedantic, please change them into "I", "W" or "E" tags. + +Once the basics are covered, you should be better equipped to deal +with the other ("tag testing") suites or using other features of the +"tests" suite (e.g. pedantic tags). + +=head2 The design of the Lintian test suite + +The basic design of the Lintian test suite can be summed up +as I. The Debian build system is changing all the time +(albeit, slowly) and sometimes it deprecates or breaks existing +features. + +With over 400 tests all featuring the same basic parts, the test suite +features several tricks to keep up with the pace. It uses "skeletons" +(template) directories to seed the package structures and template +files to fill in the basic files (e.g. "debian/control" and +"debian/changelog"). + +This means that when a new standards-version comes along, debhelper +deprecates a feature or (more likely) Lintian adds a new tag, the +majority of the tests can quickly be adapted with only a minor effort. + +Since pedantic tags tend to require additional effort to avoid, most +Lintian tests do B run with pedantic tags enabled. + +=head2 The basics of a "native" package in the "tests" suite + +For starters, you need 2 files and 1 directory, which will be placed +in I<< t/tests/ >>. + +=head3 The desc file (mandatory) + +This is the test description file. It is a deb822 file (i.e. same +syntax as I), which contains a number of fields. + +Let's start with the following template: + + Testname: pkg-deb-check-general + Version: 1.0 + Description: General test of the pkg/deb-check check + Test-For: + missing-multi-arch-field + missing-pre-depends-on-multiarch-support + +This defines the name of the test, its sequence number (i.e. how early +it should be run), the version of the I package, a +description and the tags you intend to test for. + +In case you were wondering why "invalid-multi-arch-field" is not +listed, then it is because dpkg will not allow us to use an invalid +Multi-Arch value. Therefore, that particular tag would have to be +tested in the "debs" suite instead. + +Note that the value of the Testname field (as Source field), Version +field and Description field (as the synopsis) I be used in the +package. As such, they must obey the normal requirements for these +purposes. + +Please keep the following conventions in mind: + +=over 4 + +=item - + +The Testname should be "-" + +Note that regular Lintian checks do I have a "/", so the naming +convention works slightly better there. + +=item - + +The Version should always be "1.0" unless the test requires anything else. + +For non-native packages, the default would be "1.0-1". + +=back + +=head3 The "tags" file (mandatory, but may be empty) + +This file contains the I "expected" output of lintian. +Assuming all of the tags are "I" tags, the file should look something +like: + + I: pkg-deb-check-general-missing-ma: missing-multi-arch-field + I: pkg-deb-check-general-missing-pred: missing-pre-depends-on-multiarch-support + +=head3 The "debian/" directory (optional, but usually needed) + +The unpacked debian package in its full glory. Note that this means +that the (e.g.) I file would be I<< +t/tests//debian/debian/rules >> (note the double +"debian/"). + +The directory is seeded from I<< t/templates/tests// >>, +where I is the value of the "Skeleton" field from the "desc" +file. + +For this test, you only need a specialized control file. This file +could look something like: + + Source: {$source} + Priority: extra + Section: {$section} + Maintainer: {$author} + Standards-Version: {$standards_version} + Build-Depends: {$build_depends} + + Package: {$source}-missing-ma + Architecture: {$architecture} + Depends: $\{shlibs:Depends\}, $\{misc:Depends\} + Description: {$description} (invalid) + This is a test package designed to exercise some feature or tag of + Lintian. It is part of the Lintian test suite and may do very odd + things. It should not be installed like a regular package. It may + be an empty package. + . + Missing M-A field. + + Package: {$source}-missing-pred + Architecture: any + Depends: $\{shlibs:Depends\}, $\{misc:Depends\} + Multi-arch: same + Description: {$description} (pre-depends) + This is a test package designed to exercise some feature or tag of + Lintian. It is part of the Lintian test suite and may do very odd + things. It should not be installed like a regular package. It may + be an empty package. + . + Missing Pre-Depends. + +=head3 Running the test + +At this point, the test is in fact ready to be executed. It can be +run by using: + + $ debian/rules runtests onlyrun=pkg-deb-check-general + + OR + + $ private/runtests --dump-logs t debian/test-out pkg-deb-check-general + +However, it will not emit the correct tags unless pkg/deb-check is +part of the debian/main lintian profile. If your check is a part of a +different profile, add the "Profile: " field to the "desc" file. + +With this, the tutorial is over. Below you will find some more +resources that may be useful to your future test writing work. + +=head1 REFERENCES / APPENDIX + +=head2 A step-by-step guide of how a test case works + +Basically, the tag-testing test cases all involve building a package +and running lintian on the result. The "tests" suite does a full +build with dpkg-buildpackage, the other suites "hand-craft" only the +type of artifacts they are named after (e.g. "source" produces only +source packages). + +=head3 A test in the "tests" suite + +The basic process of a lintian test in the "tests" suite. + +=over 4 + +=item 1 + +Copy the "upstream" skeleton dir into the build dir (non-native only) + +=item 2 + +Copy the "upstream" dir from the test into the build dir (if present, non-native only) + +=item 3 + +Run the "pre_upstream" hook (if present, non-native only) + +=item 4 + +Assemble the upstream tarball (non-native only) + +=item 5 + +Copy the "debian" skeleton dir into the build dir + +=item 6 + +Copy the "debian" directory from the test into the build dir (if present) + +=item 7 + +Create debian/control and debian/changelog from "I<< >>.in" if +they do not exist. + +=item 8 + +Create an empty watch file (if missing, non-native only) + +=item 9 + +Run the "pre_build" hook (if present) + +=item 10 + +Run dpkg-buildpackage + +=item 11 + +Run lintian on the build result + +=item 12 + +Run the "post_test" hook (if present) + +=item 13 + +Run the "test_calibration" hook (if present), which may produce +a new "expected output file". + +=item 14 + +Compare the result with the expected output. + +=back + +Note that the majority of the steps are conditional on +native/non-native packages or presence of hooks. + +=head3 A test in the "debs" and the "source" suite + +The "debs" and the "source" suite share the same basic steps, which +are: + +=over 4 + +=item 1 + +Copy the skeleton dir into the build dir + +=item 2 + +Copy the test directory files into the build dir + +=item 3 + +Create changelog, control, and (debs-only) Makefile from "I<< +>>.in" if they do not exist. + +=item 4 + +Run make in the build dir + +=item 5 + +Run lintian on the produced artifact (there must be exactly one) + +=item 6 + +Compare the result with the expected output. + +=back + +=head3 A test in the "changes" suite + +The changes test is fairly simple as there is not much building. The +steps are as the following: + +=over 4 + +=item 1 + +Find or compute the test artifact as the following: + +=over 4 + +=item - + +If I<< /.changes >> exists, it is used as the +artifact. + +=item - + +Otherwise, copy I<< /.changes.in >> into the build dir +and use it as a template to create I<< /.changes +>>. The result is then used as the artifact to test. + +=back + +=item 2 + +Run lintian run on the artifact + +=item 3 + +Compare the result with the expected output + +=back + +=head2 The full layout of a test in the "tests" suite + +Each test in the "tests" suite is placed in +I<< t/tests/- >>. In these you will find some +of the following files: + +=over 4 + +=item - + +desc (mandatory) + +This is the test description file. It is a deb822 file (i.e. same +syntax as I), which contains a number of fields. + +=item - + +tags (mandatory, but may be empty) + +This file contains the "expected" output of lintian. + +This is generally sorted, though a few tests rely on the order of +the output. This can be controlled via the "Sort" field in the "desc" +file. + +=item - + +debian/ (optional, but usually what you need) + +The unpacked debian package. For "native" package tests, this is +I the "upstream" part. For "non-native" package tests, this can +be used to override files in the "upstream" part (rarely needed). + +The actual packaging files (e.g. I) would be in + + I<< t/tests//debian/debian/rules >> + +Note the double "debian". + +This part is seeded from I<< t/templates/tests// >>, +where I is the value of the "Skeleton" field from the "desc" +file. + +=item - + +upstream/ (optional, rarely needed) + +This directory is the used to create the "upstream" tarball for +"non-native" package tests. Since most tags are emitted for both +"native" and "non-native" tests, it is simpler (and slightly faster) +to use "native" packages for most tests. + +The files here should also be present with the same contents in the +debian directory unless you're intentionally creating a diff. +However, as normal with a Debian package, you can omit files entirely +from the debian directory and the deletions will be ignored by +dpkg-buildpackage. + +The directory will be seeded from I<< +t/templates/tests/.upstream/ >>, where I is the +value of the "Skeleton" field from the "desc" file. + +=item - + +post_test (optional, rarely needed) + +This script (if present) is a sed script that can be used to "massage" +the output of lintian before comparing it with the "expected output". + +The most common use for this script is to remove the architecture +name, multi-arch path, drop hardening tags or exact standards-version +number from tags output. Here are some examples files used: + + # Remove the exact standards version, so the tags file will not need + # to be updated with every new standards-version + s/\(current is ([0-9]+\.)+[0-9]\)/(current is CURRENT)/ + + # Drop all hardening tags (they can differ between architectures) + /: hardening-.*/ d + + # Massage e.g. usr/lib/i386-linux-gnu/pkgconfig into a generic path + s, usr/lib/[^/]+/pkgconfig/, usr/lib/ARCH/pkgconfig/, + +It may be useful for other cases where the output of Lintian may +change on different systems. + +=item - + +pre_build / pre_upstream (optional, special case usage) + +If present and executable, these scripts can be used to mess with the +package directory and (what will become) the upstream tarball. + +Their common use case is to create files in the tarballs that cannot +(or preferably should not) be included in the revision control system. +Common cases include "binary", "minimized" files or files with "weird" +names such as backslashes or non UTF-8 characters. + +Both scripts receive a directory as first argument, which is the +directory they should work on. For: + +=over 4 + +=item - + +pre_upstream + +The script will be run before the upstream tarball is compiled. The +first argument is the directory that will be included in the upstream +tarball. + +=item - + +pre_build + +The script will be run before dpkg-buildpackage is invoked. The first +argument is the directory of the unpacked debian source package. + +=back + +=item - + +test_calibration (optional, special case usage) + +If present and executable, this script will be invoked B +lintian I post_test (if present) have been run. The script can +then modify the expected output I the actual output. + +This is useful for those extremely rare cases where post_test is +insufficient to handle the requirements. So far, this has only been +needed for the hardening checks, where the output differs between +architectures. + +The script will be passed 3 arguments: + +=over 4 + +=item - + +Path to the "expected output" file (read-only) + +This is the "tags" file from the test suite and B be +modified. + +=item - + +Path to the "actual output" file (read-write) + +This is the file as lintian and post_test created it. + +=item - + +Path to the "calibrated expected output" (create+write) + +This file does not exist and should be created by the script, if it +wishes to change the "expected output". If this file exists when the +script terminates, this file will be used instead of the original +"expected output" file. + +=back + +=back + +=head1 SEE ALSO + +The READMEs in the suites: I, I, +I and I. + +L, L + +=cut -- cgit v1.2.3