Adding upstream version 2.117.0.upstream/2.117.0upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

4 files changed, 1135 insertions, 0 deletions

diff --git a/doc/tutorial/Lintian/Tutorial.pod b/doc/tutorial/Lintian/Tutorial.pod
new file mode 100644
index 0000000..3b088eb
--- /dev/null
+++ b/doc/tutorial/Lintian/Tutorial.pod
@@ -0,0 +1,39 @@
+=head1 NAME
+
+Lintian::Tutorial -- The newcomer's guide to Lintian
+
+=head1 SYNOPSIS
+
+Getting started with lintian in 5 minutes... maybe 10.
+
+=head1 DESCRIPTION
+
+Depending on what you want to work with there are different ways to
+approach Lintian and its code.
+
+=over 4
+
+=item * L<Writing a Lintian check|Lintian::Tutorial::WritingChecks>
+
+=item * L<Running (parts of) the Lintian test suite|Lintian::Tutorial::TestSuite>
+
+=item * L<Writing new Lintian tests|Lintian::Tutorial::WritingTests>
+
+=item * Understanding Lintian
+
+To be written.
+
+=back
+
+=begin :man
+
+=head1 SEE ALSO
+
+L<Lintian::Tutorial::WritingChecks>,
+L<Lintian::Tutorial::TestSuite>,
+L<Lintian::Tutorial::WritingTests>
+
+=end :man
+
+=cut
+
diff --git a/doc/tutorial/Lintian/Tutorial/TestSuite.pod b/doc/tutorial/Lintian/Tutorial/TestSuite.pod
new file mode 100644
index 0000000..3273388
--- /dev/null
+++ b/doc/tutorial/Lintian/Tutorial/TestSuite.pod
@@ -0,0 +1,115 @@
+=encoding utf-8
+
+=head1 NAME
+
+Lintian::Tutorial::TestSuite -- Quick intro to running the Lintian testsuite
+
+=head1 SYNOPSIS
+
+Warning: This document may be out of date.
+
+This guide will quickly introduce you to running the Lintian test
+suite and some tricks.  The Lintian test suite is fairly large and
+accordingly it can take a substantial amount of time to run.  To speed
+up development, there are various options to limit the tests to run.
+
+If you are looking for a guide on how to write tests, please consult
+L<Lintian::Tutorial::WritingTests>.
+
+=head1 DESCRIPTION
+
+The Lintian test suite is an extensive collection of various test
+cases.  The test suite is divided into 4 "sub-suites".  The majority
+of tests are currently located in the "tests" sub-suite.
+
+To run the full suite:
+
+ $ rm -rf debian/test-out; private/build-test-packages; private/runtests
+
+While writing a new tag (or check) you probably only want to run a
+particular (subset of the) test(s).  See L</Running a subset of the
+tests> for the available options.
+
+=head2 Running a subset of the tests
+
+First, you have to build the test packages with:
+
+ $ rm -rf debian/test-out; private/build-test-packages;
+
+Then, the following options are available:
+
+=over 4
+
+=item Running a single test
+
+To run a single test by its name, use:
+
+ $ private/runtests --onlyrun=test:$name
+
+=item Running all tests for a check
+
+To run all tests for a given check, use:
+
+ $ private/runtests --onlyrun=check:$name
+
+$check must be the name of a check (it will test for
+checks/$check.desc) or "legacy".  This will run all tests that start
+with "$check-".
+
+=item Running all tests designed for a specific tag
+
+To run all tests that have a "Test-For" or a "Test-Against" for a given
+tag, use:
+
+ $ private/runtests --onlyrun=tag:$name
+
+=back
+
+=head2 Running tests under coverage
+
+This feature is currently untested.
+
+It is possible to run most of the tests under L<Devel::Cover>.  This is
+done by passing I<--coverage> to I<private/runtests>.  Example:
+
+  $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out
+
+Please note that L<Devel::Cover> does not seem to handle multiple
+threads too well.  You may see spurious warnings/errors if you run the
+tests with 2 or more active worker threads.
+
+B<Caveat> 1: Coverage for collections (i.e. programs in I<collection/>)
+does not seem to work at the moment. Therefore, they often end up with
+(next to) zero coverage in the generated reports.
+
+B<Caveat> 2: L<Devel::Cover> sometimes changes the output of Lintian
+or tools called by Lintian. Obviously, this leads to test
+failures. Therefore, you may see weird test failures (or warnings)
+when running under coverage.
+
+=head3 Collecting the coverage you want in a reasonable time
+
+Collecting coverage is excruciatingly slow.  This is not helped by the
+fact that it becomes unreliable when run under 2 or more threads.
+
+Fortunately, L<Devel::Cover> "appends" to its cover database.  This
+allows you to "slowly" build up the coverage database over multiple
+runs. Example:
+
+  $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out suite:scripts
+  $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out suite:debs
+  $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out suite:source
+  ...
+
+Or:
+
+  $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out $check
+  $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out legacy
+
+
+=head1 SEE ALSO
+
+L<Lintian::Tutorial::WritingTests>
+
+=cut
+
diff --git a/doc/tutorial/Lintian/Tutorial/WritingChecks.pod b/doc/tutorial/Lintian/Tutorial/WritingChecks.pod
new file mode 100644
index 0000000..b6642e1
--- /dev/null
+++ b/doc/tutorial/Lintian/Tutorial/WritingChecks.pod
@@ -0,0 +1,442 @@
+=encoding utf-8
+
+=head1 NAME
+
+Lintian::Tutorial::WritingChecks -- Writing checks for Lintian
+
+=head1 SYNOPSIS
+
+Warning: This tutorial may be outdated.
+
+This guide will quickly guide you through the basics of writing a
+Lintian check.  Most of the work is in writing the two files:
+
+  checks/<my-check>.pm
+  checks/<my-check>.desc
+
+And then either adding a Lintian profile or extending an existing
+one.
+
+=head1 DESCRIPTION
+
+The basics of writing a check are outlined in the Lintian User Manual
+(§3.3).  This tutorial will focus on the act of writing the actual
+check.  In this tutorial, we will assume the name of the check to be
+written is "deb/pkg-check".
+
+The tutorial will work with a "binary" and "udeb" check.  Checking
+source packages works in a similar fashion.
+
+=head2 Create a check I<.desc> file
+
+As mentioned, this tutorial will focus on the writing of a check.
+Please see the Lintian User Manual (§3.3) for how to do this part.
+
+=head2 Create the Perl check module
+
+Start with the template:
+
+ # deb/pkg-check is loaded as Lintian::deb::pkg_check
+ # - See Lintian User Manual §3.3 for more info
+ package Lintian::deb::pkg_check;
+
+ use strict;
+ use warnings;
+ 
+ sub run {
+     my ($pkg, $type, $info, $proc, $group) = @_;
+     return;
+ }
+
+The snippet above is a simple valid check that does "nothing at all".
+We will extend it in just a moment, but first let us have a look at
+the arguments at the setup.
+
+The I<run> sub is the entry point of our "deb/pkg-check" check; it
+will be invoked once per package it should process.  In our case, that
+will be once per "binary" (.deb) and once per udeb package processed.
+
+It is given 5 arguments (in the future, possibly more), which are:
+
+=over 4
+
+=item $pkg - The name of the package being processed.
+
+(Same as $proc->pkg_name)
+
+=item $type - The type of the package being processed.
+
+At the moment, $type is one of "binary" (.deb), "udeb", "source"
+(.dsc) or "changes".  This argument is mostly useful if certain checks
+do not apply equally to all package types being processed.
+
+Generally it is advisable to check only binaries ("binary" and
+"udeb"), sources or changes in a given check.  But in rare cases, it
+makes sense to lump multiple types together in the same check and this
+argument helps you do that.
+
+(Current it is always identical to $proc->pkg_type)
+
+=item $info - Accessor to the data Lintian has extracted
+
+Basically all information you want about a given package comes from
+the $info object.  Sometimes referred to as either the "info object" or
+(an instance of) L<Lintian::Collect>.
+
+This object (together with a properly set Needs-Info in the I<.desc>
+file) will grant you access to all of the data Lintian has extracted
+about this package.
+
+Based on the value of the $type argument, it will be one of
+L<Lintian::Collect::Binary>, L<Lintian::Collect::Changes> or
+L<Lintian::Collect::Source>.
+
+(Currently it is the same as $proc->info)
+
+=item $proc - Basic metadata about the package
+
+This is an instance of L<Lintian::Processable> and is useful for
+trivially obtaining very basic package metadata.  Particularly, the
+name of source package and version of source package are readily
+available through this object.
+
+=item $group - Group of processables from the same source
+
+If you want to do a cross-check between different packages built from
+the same source, $group helps you access those other packages
+(if they are available).
+
+This is an instance of L<Lintian::ProcessableGroup>.
+
+=back
+
+Now back to the coding.
+
+=head2 Accessing fields
+
+Let's do a slightly harder example.  Assume we wanted to emit a tag for
+all packages without a (valid) Multi-Arch field.  This requires us to
+A) identify if the package has a Multi-Arch field and B) identify if
+the content of the field was valid.
+
+Starting from the top.  All $info objects have a method called field,
+which gives you access to a (raw) field from the control file of the
+package.  It returns C<undef> if said field is not present or the
+content of said field otherwise.  Note that field names must be given
+in all lowercase letters (i.e. use "multi-arch", not "Multi-Arch").
+
+This was the first half.  Let's look at checking the value.  Multi-arch
+fields can (currently) be one of "no", "same", "foreign" or "allowed".
+One way of checking this would be using the regex:
+
+Notice that Lintian automatically strips leading and trailing spaces
+on the I<first> line in a field.  It also strips trailing spaces from
+all other lines, but leading spaces and the " ."-continuation markers
+are kept as is.
+
+=head2 Checking dependencies
+
+Lintian can do some checking of dependencies.  For most cases it works
+similar to a normal dependency check, but keep in mind that Lintian
+uses I<pure> logic to determine if dependencies are satisfied (i.e. it
+will not look up relations like Provides for you).
+
+Suppose you wanted all packages with a multi-arch "same" field to
+pre-depend on the package "multiarch-support".  Well, we could use the
+L<< $info->relation|Lintian::Collect::Binary/relation (FIELD) >> method for
+this.
+
+$info->relation returns an instance of L<Lintian::Relation>.  This
+object has an "implies" method that can be used to check if a package
+has an explicit dependency.  Note that "implies" actually checks if
+one relation "implies" another (i.e. if you satisfied relationA then
+you definitely also satisfied relationB).
+
+As with the "field"-method, field names have to be given in all
+lowercase.  However "relation" will never return C<undef> (not even if the
+field is missing).
+
+=head2 Using static data files
+
+Currently our check mixes data and code.  Namely all the valid values
+for the Multi-Arch field are currently hard-coded in our check.  We can
+move those out of the check by using a data file.
+
+Lintian natively supports data files that are either "sets" or
+"tables" via L<Lintian::Data> (i.e. "unordered" collections).  As an
+added bonus, L<Lintian::Data> transparently supports vendor specific
+data files for us.
+
+First we need to make a data file containing the values.  Which could be:
+
+ # A table of all the valid values for the multi-arch field.
+ no
+ same
+ foreign
+ allowed
+
+This can then be stored in the data directory as
+I<data/deb/pkg-check/multiarch-values>.
+
+Now we can load it by using:
+
+ use Lintian::Data;
+ 
+ my $VALID_MULTI_ARCH_VALUES =
+     Lintian::Data->new('deb/pkg-check/multiarch-values');
+
+Actually, this is not quite true.  L<Lintian::Data> is lazy, so it
+will not load anything before we force it to do so.  Most of the time
+this is just an added bonus.  However, if you ever have to force it to
+load something immediately, you can do so by invoking its "known"
+method (with an arbitrary defined string and ignore the result).
+
+Data files work with 3 access methods, "all", "known" and "value".
+
+=over 4
+
+=item all
+
+"all" (i.e. $data->all) returns a list of all the entries in the data
+file (for key/value tables, all returns the keys).  The list is not
+sorted in any order (not even input order).
+
+=item known
+
+"known" (i.e. $data->known('item')) returns a truth value if a given
+item or key is known (present) in the data set or table.  For key/pair
+tables, the value associated with the key can be retrieved with
+"value" (see below).
+
+=item value
+
+"value" (i.e. $data->value('key')) returns a value associated with a
+key for key/value tables.  For unknown keys, it returns C<undef>.  If
+the data file is not a key/value table but just a set, value returns
+a truth value for known keys.
+
+=back
+
+While we could use both "value" and "known", we will use the latter
+for readability (and to remind ourselves that this is a data set and
+not a data table).
+
+Basically we will be replacing:
+
+  unless exists $VALID_MULTI_ARCH_VALUES{$multiarch};
+
+with
+
+  unless $VALID_MULTI_ARCH_VALUES->known($multiarch);
+
+=head2 Accessing contents of the package
+
+Another heavily used mechanism is to check for the presence (or absence)
+of a given file.  Generally this is what the
+L<< $info->index|Lintian::Collect::Package/index (FILE) >> and
+L<< $info->sorted_index|Lintian::Collect::Package/sorted_index >> methods
+are for.  The "index" method returns instances of L<Lintian::Path>,
+which has a number of utility methods.
+
+If you want to loop over all files in a package, the sorted_index will
+do this for you.  If you are looking for a specific file (or directory), a
+call to "index" will be much faster.  For the contents of a specific directory,
+you can use something like:
+
+ if (my $dir = $info->index('path/to/dir/')) {
+     foreach my $elem ($dir->children) {
+         print $elem->name . " is a file" if $elem->is_file;
+         # ...
+     }
+ }
+
+Keep in mind that using the "index" or "sorted_index" method will
+require that you put "unpacked" in Needs-Info.  See L</Keeping Needs-Info
+up to date>.
+
+There are also a pair of methods for accessing the control files of a
+binary package.  These are
+L<< $info->control_index|Lintian::Collect::Package/control_index (FILE) >> and
+L<< $info->sorted_control_index|Lintian::Collect::Package/sorted_control_index >>.
+
+=head3 Accessing contents of a file in a package
+
+When you actually want to see the contents of a file, you can use
+L<open|Lintian::Path/open> (or L<open_gz|Lintian::Path/open_gz>) on
+an object returned by e.g.
+L<< $info->index|Lintian::Collect::Package/index (FILE) >>.  These
+methods will open the underlying file for reading (the latter
+applying a gzip decompression).
+
+However, please do assert that the file is safe to read by calling
+L<is_open_ok|Lintian::Path/is_open_ok> first.  Generally, it will
+only be true for files or safely resolvable symlinks pointing to
+files.  Should you attempt to open a path that does not satisfy
+those criteria, L<Lintian::Path> will raise a trappable error at
+runtime.
+
+Alternatively, if you access the underlying file object, you can
+use the L<fs_path|Lintian::Path/fs_path> method.  Usually, you will
+want to test either L<is_open_ok|Lintian::Path/is_open_ok> or
+L<is_valid_path|Lintian::Path/is_valid_path> first to ensure you do
+not follow unsafe symlinks.  The "is_open_ok" check will also assert
+that it is not (e.g.) a named pipe or such.
+
+Should you call L<fs_path|Lintian::Path/fs_path> on a symlink that
+escapes the package root, the method will throw a trappable error at
+runtime.  Once the path is returned, there are no more built-in
+fail-safes.  When you use the returned path, keep things like
+"../../../../../etc/passwd"-symlink and "fifo" pipes in mind.
+
+
+In some cases, you may even need to access the file system objects
+I<without> using L<Lintian::Path>.  This is, of course, discouraged
+and suffers from the same issues above (all checking must be done
+manually by you).  Here you have to use the "unpacked", "debfiles" or
+"control" methods from L<Lintian::Collect> or its subclasses.
+
+
+
+The following snippet may be useful for testing that a given path does
+not escape the root.
+
+ use Lintian::Util qw(is_ancestor_of);
+ 
+ my $path = ...;
+ # The snippet applies equally well to $info->debfiles and
+ # $info->control (just remember to subst all occurrences of
+ # $info->unpacked).
+ my $unpacked_file = $info->unpacked($path);
+ if ( -f $unpacked_file && is_ancestor_of($info->unpacked, $unpacked_file)) {
+    # a file and contained within the package root.
+ } else {
+    # not a file or an unsafe path
+ }
+
+=head2 Keeping Needs-Info up to date
+
+Keeping the "Needs-Info" field of your I<.desc> file is a bit of
+manual work.  In the API description for the method there will
+generally be a line looking something like:
+
+  Needs-Info requirements for using methodx: Y
+
+Which means that the methodx requires Y to work.  Here Y is a comma
+separated list and each element of Y basically falls into 3 cases.
+
+=over 4
+
+=item * The element is the word I<none>
+
+In this case, the method has no "external" requirements and can be
+used without any changes to your Needs-Info.  The "field" method
+is an example of this.
+
+This only makes sense if it is the only element in the list.
+
+=item * The element is a link to a method
+
+In this case, the method uses another method to do its job.  An example
+is the
+L<sorted_control_index|Lintian::Collect::Binary/sorted_control_index>
+method, which uses the
+L<control_index|Lintian::Collect::Binary/control_index (FILE)>
+method.  So using I<sorted_control_index> has the same requirements as
+using I<control_index>.
+
+=item * The element is the name of a collection (e.g. "control_index").
+
+In this case, the method needs the given collection to be run.  So to
+use (e.g.) L<control_index|Lintian::Collect::Binary/control_index (FILE)>,
+you have to put "bin-pkg-control" in your Needs-Info.
+
+=back
+
+CAVEAT: Methods can have different requirements based on the type of
+package!  An example of this "changelog", which requires "changelog-file"
+in binary packages and "Same as debfiles" in source packages.
+
+=head2 Avoiding security issues
+
+Over the years a couple of security issues have been discovered in
+Lintian.  The problem is that people can in theory create some really nasty
+packages.  Please keep the following in mind when writing a check:
+
+=over 4
+
+=item * Avoid 2-arg open, system/exec($shellcmd), `$shellcmd` like the
+plague.
+
+When you get any one of those wrong you introduce "arbitrary code
+execution" vulnerabilities (we learned this the hard way via
+CVE-2009-4014).
+
+Usually 3-arg open and the non-shell variant of system/exec are
+enough.  When you actually need a shell pipeline, consider using
+L<Lintian::Command>.  It also provides a I<safe_qx> command to assist
+with capturing stdout as an alternative to `$cmd` (or qx/$cmd/).
+
+=item * Do not trust field values.
+
+This is especially true if you intend to use the value as part of a
+file name.  Verify that the field contains what you expect before you use
+it.
+
+=item * Use L<Lintian::Path> (or, failing that, is_ancestor_of)
+
+You might be tempted to think that the following code is safe:
+
+ use autodie;
+ 
+ my $filename = 'some/file';
+ my $ufile = $info->unpacked($filename);
+ if ( ! -l $ufile) {
+    # Looks safe, but isn't in general
+    open(my $fd, '<', $ufile);
+    ...;
+ }
+
+This is definitely unsafe if "$filename" contains at least one
+directory segment.  So, if in doubt, use
+L<is_ancestor_of|Lintian::Util/is_ancestor_of(PARENTDIR, PATH)> to
+verify that the requested file is indeed the file you think it is.  A
+better version of the above would be:
+
+ use autodie,
+ use Lintian::Util qw(is_ancestor_of);
+ [...]
+ my $filename = 'some/file';
+ my $ufile = $info->unpacked($filename);
+ if ( ! -l $ufile && -f $ufile && is_ancestor_of($info->unpacked, $ufile)) {
+    # $ufile is a file and it is contained within the package root.
+    open(m $fd, '<', $ufile);
+    ...;
+ }
+
+In some cases you can even drop the "! -l $ufile" part.
+
+Of course, it is much easier to use the L<Lintian::Path> object
+(whenever possible).
+ 
+ my $filename = 'some/file';
+ my $ufile = $info->index($filename);
+ if ( $ufile && $ufile->is_file && $ufile->is_open_ok) {
+     my $fd = $ufile->open;
+     ...;
+ }
+
+Here you can drop the " && $ufile->is_file" if you want to permit
+safe symlinks.
+
+
+For more information on the is_ancestor_of check, see
+L<is_ancestor_of|Lintian::Util/is_ancestor_of(PARENTDIR, PATH)>
+
+
+=back
+
+=head1 SEE ALSO
+
+L<Lintian::Tutorial::WritingTests>, L<Lintian::Tutorial::TestSuite>
+
+=cut
diff --git a/doc/tutorial/Lintian/Tutorial/WritingTests.pod b/doc/tutorial/Lintian/Tutorial/WritingTests.pod
new file mode 100644
index 0000000..1011235
--- /dev/null
+++ b/doc/tutorial/Lintian/Tutorial/WritingTests.pod
@@ -0,0 +1,539 @@
+=encoding utf-8
+
+=head1 NAME
+
+Lintian::Tutorial::WritingTests -- Short tutorial on writing tests
+
+=head1 SYNOPSIS
+
+Warning: This document may be out of date.
+
+This document attempts to be a short / quick tutorial to the Lintian
+test suite from a test-writer's perspective.  As such, it will only
+cover the standard type of tests (from the "tests" suite).
+
+The guide will involve writing a test for the "deb/pkg-check" check,
+which was documented in the L<Lintian::Tutorial::WritingChecks>
+tutorial.
+
+For running tests, please see L<Lintian::Tutorial::TestSuite> instead.
+
+=head1 DESCRIPTION
+
+The Lintian test suite is divided into several parts.  These are:
+
+=over 4
+
+=item -
+
+scripts
+
+Small (Perl) "prove" tests.  These assert that code style, data files
+and/or self-contained code units (i.e. unit tests) work as intended.
+They are B<not> used for testing Lintian tags.
+
+=item -
+
+tags
+
+These tests all test for the presence of tags after building test
+packages using skeletons. For most cases, we recommend
+
+ Skeleton: upload-non-native
+
+suites are small test suites that test some particular tags for
+I<.changes>, I<.deb> or I<.dsc> files.  Typically, you will find the
+more exotic tags here, which require some special fiddling and cannot
+be built by a "standard" dh7 + dpkg build.
+
+=item -
+
+literal
+
+These tests look to match the literal output of Lintian. These tests
+are useful as general false positives. They also catch Lintian messages
+unrelated to tags.
+
+=back
+
+With this in mind, let us move on to the scope.
+
+=head2 Scope of the tutorial
+
+WARNING: THE REMAINDER OF THIS TUTORIAL IS OUT OF DATE.
+
+The "tests" suite alone is fairly complex on its own.  To keep things
+simple, the tutorial will limit itself to creating a "native"
+package with no special requirements in the "tests" suite.
+
+In particular, note that the tags I<must not> be I<pedantic> for this
+to work.  If you followed the check writing tutorial and made the tags
+pedantic, please change them into "I", "W" or "E" tags.
+
+Once the basics are covered, you should be better equipped to deal
+with the other ("tag testing") suites or using other features of the
+"tests" suite (e.g. pedantic tags).
+
+=head2 The design of the Lintian test suite
+
+The basic design of the Lintian test suite can be summed up
+as I<less is more>.  The Debian build system is changing all the time
+(albeit, slowly) and sometimes it deprecates or breaks existing
+features.
+
+With over 400 tests all featuring the same basic parts, the test suite
+features several tricks to keep up with the pace.  It uses "skeletons"
+(template) directories to seed the package structures and template
+files to fill in the basic files (e.g. "debian/control" and
+"debian/changelog").
+
+This means that when a new standards-version comes along, debhelper
+deprecates a feature or (more likely) Lintian adds a new tag, the
+majority of the tests can quickly be adapted with only a minor effort.
+
+Since pedantic tags tend to require additional effort to avoid, most
+Lintian tests do B<not> run with pedantic tags enabled.
+
+=head2 The basics of a "native" package in the "tests" suite
+
+For starters, you need 2 files and 1 directory, which will be placed
+in I<< t/tests/<test-name> >>.
+
+=head3 The desc file (mandatory)
+
+This is the test description file.  It is a deb822 file (i.e. same
+syntax as I<debian/control>), which contains a number of fields.
+
+Let's start with the following template:
+
+ Testname: pkg-deb-check-general
+ Version: 1.0
+ Description: General test of the pkg/deb-check check
+ Test-For:
+  missing-multi-arch-field
+  missing-pre-depends-on-multiarch-support
+
+This defines the name of the test, its sequence number (i.e. how early
+it should be run), the version of the I<generated> package, a
+description and the tags you intend to test for.
+
+In case you were wondering why "invalid-multi-arch-field" is not
+listed, then it is because dpkg will not allow us to use an invalid
+Multi-Arch value.  Therefore, that particular tag would have to be
+tested in the "debs" suite instead.
+
+Note that the value of the Testname field (as Source field), Version
+field and Description field (as the synopsis) I<will> be used in the
+package.  As such, they must obey the normal requirements for these
+purposes.
+
+Please keep the following conventions in mind:
+
+=over 4
+
+=item -
+
+The Testname should be "<check-name>-<test-name>"
+
+Note that regular Lintian checks do I<not> have a "/", so the naming
+convention works slightly better there.
+
+=item -
+
+The Version should always be "1.0" unless the test requires anything else.
+
+For non-native packages, the default would be "1.0-1".
+
+=back
+
+=head3 The "tags" file (mandatory, but may be empty)
+
+This file contains the I<sorted> "expected" output of lintian.
+Assuming all of the tags are "I" tags, the file should look something
+like:
+
+ I: pkg-deb-check-general-missing-ma: missing-multi-arch-field
+ I: pkg-deb-check-general-missing-pred: missing-pre-depends-on-multiarch-support
+
+=head3 The "debian/" directory (optional, but usually needed)
+
+The unpacked debian package in its full glory.  Note that this means
+that the (e.g.) I<debian/rules> file would be I<<
+t/tests/<test-name>/debian/debian/rules >>  (note the double
+"debian/").
+
+The directory is seeded from I<< t/templates/tests/<skeleton>/ >>,
+where I<skeleton> is the value of the "Skeleton" field from the "desc"
+file.
+
+For this test, you only need a specialized control file.  This file
+could look something like:
+
+ Source: {$source}
+ Priority: extra
+ Section: {$section}
+ Maintainer: {$author}
+ Standards-Version: {$standards_version}
+ Build-Depends: {$build_depends}
+ 
+ Package: {$source}-missing-ma
+ Architecture: {$architecture}
+ Depends: $\{shlibs:Depends\}, $\{misc:Depends\}
+ Description: {$description} (invalid)
+  This is a test package designed to exercise some feature or tag of
+  Lintian.  It is part of the Lintian test suite and may do very odd
+  things.  It should not be installed like a regular package.  It may
+  be an empty package.
+  .
+  Missing M-A field.
+ 
+ Package: {$source}-missing-pred
+ Architecture: any
+ Depends: $\{shlibs:Depends\}, $\{misc:Depends\}
+ Multi-arch: same
+ Description: {$description} (pre-depends)
+  This is a test package designed to exercise some feature or tag of
+  Lintian.  It is part of the Lintian test suite and may do very odd
+  things.  It should not be installed like a regular package.  It may
+  be an empty package.
+  .
+  Missing Pre-Depends.
+
+=head3 Running the test
+
+At this point, the test is in fact ready to be executed.  It can be
+run by using:
+
+ $ debian/rules runtests onlyrun=pkg-deb-check-general
+
+ OR
+
+ $ private/runtests --dump-logs t debian/test-out pkg-deb-check-general
+
+However, it will not emit the correct tags unless pkg/deb-check is
+part of the debian/main lintian profile.  If your check is a part of a
+different profile, add the "Profile: <name>" field to the "desc" file.
+
+With this, the tutorial is over.  Below you will find some more
+resources that may be useful to your future test writing work.
+
+=head1 REFERENCES / APPENDIX
+
+=head2 A step-by-step guide of how a test case works
+
+Basically, the tag-testing test cases all involve building a package
+and running lintian on the result.  The "tests" suite does a full
+build with dpkg-buildpackage, the other suites "hand-craft" only the
+type of artifacts they are named after (e.g. "source" produces only
+source packages).
+
+=head3 A test in the "tests" suite
+
+The basic process of a lintian test in the "tests" suite.
+
+=over 4
+
+=item 1
+
+Copy the "upstream" skeleton dir into the build dir (non-native only)
+
+=item 2
+
+Copy the "upstream" dir from the test into the build dir (if present, non-native only)
+
+=item 3
+
+Run the "pre_upstream" hook (if present, non-native only)
+
+=item 4
+
+Assemble the upstream tarball (non-native only)
+
+=item 5
+
+Copy the "debian" skeleton dir into the build dir
+
+=item 6
+
+Copy the "debian" directory from the test into the build dir (if present)
+
+=item 7
+
+Create debian/control and debian/changelog from "I<< <file> >>.in" if
+they do not exist.
+
+=item 8
+
+Create an empty watch file (if missing, non-native only)
+
+=item 9
+
+Run the "pre_build" hook (if present)
+
+=item 10
+
+Run dpkg-buildpackage
+
+=item 11
+
+Run lintian on the build result
+
+=item 12
+
+Run the "post_test" hook (if present)
+
+=item 13
+
+Run the "test_calibration" hook (if present), which may produce
+a new "expected output file".
+
+=item 14
+
+Compare the result with the expected output.
+
+=back
+
+Note that the majority of the steps are conditional on
+native/non-native packages or presence of hooks.
+
+=head3 A test in the "debs" and the "source" suite
+
+The "debs" and the "source" suite share the same basic steps, which
+are:
+
+=over 4
+
+=item 1
+
+Copy the skeleton dir into the build dir
+
+=item 2
+
+Copy the test directory files into the build dir
+
+=item 3
+
+Create changelog, control, and (debs-only) Makefile from "I<< <file>
+>>.in" if they do not exist.
+
+=item 4
+
+Run make in the build dir
+
+=item 5
+
+Run lintian on the produced artifact (there must be exactly one)
+
+=item 6
+
+Compare the result with the expected output.
+
+=back
+
+=head3 A test in the "changes" suite
+
+The changes test is fairly simple as there is not much building.  The
+steps are as the following:
+
+=over 4
+
+=item 1
+
+Find or compute the test artifact as the following:
+
+=over 4
+
+=item -
+
+If I<< <test-dir>/<test-name>.changes >> exists, it is used as the
+artifact.
+
+=item -
+
+Otherwise, copy I<< <test-dir>/<test-name>.changes.in >> into the build dir
+and use it as a template to create I<< <build-dir>/<test-name>.changes
+>>.  The result is then used as the artifact to test.
+
+=back
+
+=item 2
+
+Run lintian run on the artifact
+
+=item 3
+
+Compare the result with the expected output
+
+=back
+
+=head2 The full layout of a test in the "tests" suite
+
+Each test in the "tests" suite is placed in
+I<< t/tests/<check>-<name> >>.  In these you will find some
+of the following files:
+
+=over 4
+
+=item -
+
+desc (mandatory)
+
+This is the test description file.  It is a deb822 file (i.e. same
+syntax as I<debian/control>), which contains a number of fields.
+
+=item -
+
+tags (mandatory, but may be empty)
+
+This file contains the "expected" output of lintian.
+
+This is generally sorted, though a few tests rely on the order of
+the output.  This can be controlled via the "Sort" field in the "desc"
+file.
+
+=item -
+
+debian/ (optional, but usually what you need)
+
+The unpacked debian package.  For "native" package tests, this is
+I<also> the "upstream" part.  For "non-native" package tests, this can
+be used to override files in the "upstream" part (rarely needed).
+
+The actual packaging files (e.g. I<debian/rules>) would be in
+
+ I<< t/tests/<test-name>/debian/debian/rules >>
+
+Note the double "debian".
+
+This part is seeded from I<< t/templates/tests/<skeleton>/ >>,
+where I<skeleton> is the value of the "Skeleton" field from the "desc"
+file.
+
+=item -
+
+upstream/ (optional, rarely needed)
+
+This directory is the used to create the "upstream" tarball for
+"non-native" package tests.  Since most tags are emitted for both
+"native" and "non-native" tests, it is simpler (and slightly faster)
+to use "native" packages for most tests.
+
+The files here should also be present with the same contents in the
+debian directory unless you're intentionally creating a diff.
+However, as normal with a Debian package, you can omit files entirely
+from the debian directory and the deletions will be ignored by
+dpkg-buildpackage.
+
+The directory will be seeded from I<<
+t/templates/tests/<skeleton>.upstream/ >>, where I<skeleton> is the
+value of the "Skeleton" field from the "desc" file.
+
+=item -
+
+post_test (optional, rarely needed)
+
+This script (if present) is a sed script that can be used to "massage"
+the output of lintian before comparing it with the "expected output".
+
+The most common use for this script is to remove the architecture
+name, multi-arch path, drop hardening tags or exact standards-version
+number from tags output.  Here are some examples files used:
+
+  # Remove the exact standards version, so the tags file will not need
+  # to be updated with every new standards-version
+  s/\(current is ([0-9]+\.)+[0-9]\)/(current is CURRENT)/
+
+  # Drop all hardening tags (they can differ between architectures)
+  /: hardening-.*/ d
+
+  # Massage e.g. usr/lib/i386-linux-gnu/pkgconfig into a generic path
+  s, usr/lib/[^/]+/pkgconfig/, usr/lib/ARCH/pkgconfig/,
+
+It may be useful for other cases where the output of Lintian may
+change on different systems.
+
+=item -
+
+pre_build / pre_upstream (optional, special case usage)
+
+If present and executable, these scripts can be used to mess with the
+package directory and (what will become) the upstream tarball.
+
+Their common use case is to create files in the tarballs that cannot
+(or preferably should not) be included in the revision control system.
+Common cases include "binary", "minimized" files or files with "weird"
+names such as backslashes or non UTF-8 characters.
+
+Both scripts receive a directory as first argument, which is the
+directory they should work on.  For:
+
+=over 4
+
+=item -
+
+pre_upstream
+
+The script will be run before the upstream tarball is compiled.  The
+first argument is the directory that will be included in the upstream
+tarball.
+
+=item -
+
+pre_build
+
+The script will be run before dpkg-buildpackage is invoked.  The first
+argument is the directory of the unpacked debian source package.
+
+=back
+
+=item -
+
+test_calibration (optional, special case usage)
+
+If present and executable, this script will be invoked B<after>
+lintian I<and> post_test (if present) have been run.  The script can
+then modify the expected output I<and> the actual output.
+
+This is useful for those extremely rare cases where post_test is
+insufficient to handle the requirements.  So far, this has only been
+needed for the hardening checks, where the output differs between
+architectures.
+
+The script will be passed 3 arguments:
+
+=over 4
+
+=item -
+
+Path to the "expected output" file (read-only)
+
+This is the "tags" file from the test suite and B<must not> be
+modified.
+
+=item -
+
+Path to the "actual output" file (read-write)
+
+This is the file as lintian and post_test created it.
+
+=item -
+
+Path to the "calibrated expected output" (create+write)
+
+This file does not exist and should be created by the script, if it
+wishes to change the "expected output".  If this file exists when the
+script terminates, this file will be used instead of the original
+"expected output" file.
+
+=back
+
+=back
+
+=head1 SEE ALSO
+
+The READMEs in the suites: I<t/tests/README>, I<t/changes/README>,
+I<t/debs/README> and I<t/source/README>.
+
+L<Lintian::Tutorial::WritingChecks>, L<Lintian::Tutorial::TestSuite>
+
+=cut