summaryrefslogtreecommitdiffstats
path: root/src/test/perl/README
diff options
context:
space:
mode:
Diffstat (limited to 'src/test/perl/README')
-rw-r--r--src/test/perl/README138
1 files changed, 138 insertions, 0 deletions
diff --git a/src/test/perl/README b/src/test/perl/README
new file mode 100644
index 0000000..92acf42
--- /dev/null
+++ b/src/test/perl/README
@@ -0,0 +1,138 @@
+Perl-based TAP tests
+====================
+
+src/test/perl/ contains shared infrastructure that's used by Perl-based tests
+across the source tree, particularly tests in src/bin and src/test. It's used
+to drive tests for backup and restore, replication, etc - anything that can't
+really be expressed using pg_regress or the isolation test framework.
+
+The tests are invoked via perl's 'prove' command, wrapped in PostgreSQL
+makefiles to handle instance setup etc. See the $(prove_check) and
+$(prove_installcheck) targets in Makefile.global. By default every test in the
+t/ subdirectory is run. Individual test(s) can be run instead by passing
+something like PROVE_TESTS="t/001_testname.pl t/002_othertestname.pl" to make.
+
+By default, to keep the noise low during runs, we do not set any flags via
+PROVE_FLAGS, but this can be done on the 'make' command line if desired, eg:
+
+make check-world PROVE_FLAGS='--verbose'
+
+When a test fails, the terminal output from 'prove' is usually not sufficient
+to diagnose the problem. Look into the log files that are left under
+tmp_check/log/ to get more info. Files named 'regress_log_XXX' are log
+output from the perl test scripts themselves, and should be examined first.
+Other files are postmaster logs, and may be helpful as additional data.
+
+The tests default to a timeout of 180 seconds for many individual operations.
+Slow hosts may avoid load-induced, spurious failures by setting environment
+variable PG_TEST_TIMEOUT_DEFAULT to some number of seconds greater than 180.
+Developers may see faster failures by setting that environment variable to
+some lesser number of seconds.
+
+Data directories will also be left behind for analysis when a test fails;
+they are named according to the test filename. But if the environment
+variable PG_TEST_NOCLEAN is set, the data directories will be retained
+regardless of test status. This environment variable also prevents the
+test's temporary directories from being removed.
+
+
+Writing tests
+-------------
+
+You should prefer to write tests using pg_regress in src/test/regress, or
+isolation tester specs in src/test/isolation, if possible. If not, check to
+see if your new tests make sense under an existing tree in src/test, like
+src/test/ssl, or should be added to one of the suites for an existing utility.
+
+Note that all tests and test tools should have perltidy run on them before
+patches are submitted, using perltidy --profile=src/tools/pgindent/perltidyrc
+
+Tests are written using Perl's Test::More with some PostgreSQL-specific
+infrastructure from src/test/perl providing node management, support for
+invoking 'psql' to run queries and get results, etc. You should read the
+documentation for Test::More before trying to write tests.
+
+Test scripts in the t/ subdirectory of a suite are executed in alphabetical
+order.
+
+Each test script should begin with:
+
+ use strict;
+ use warnings;
+ use PostgreSQL::Test::Cluster;
+ use PostgreSQL::Test::Utils;
+ use Test::More;
+
+then it will generally need to set up one or more nodes, run commands
+against them and evaluate the results. For example:
+
+ my $node = PostgreSQL::Test::Cluster->new('primary');
+ $node->init;
+ $node->start;
+
+ my $ret = $node->safe_psql('postgres', 'SELECT 1');
+ is($ret, '1', 'SELECT 1 returns 1');
+
+ $node->stop('fast');
+
+Each test script should end with:
+
+ done_testing();
+
+Test::More::like entails use of the qr// operator. Avoid Perl 5.8.8 bug
+#39185 by not using the "$" regular expression metacharacter in qr// when also
+using the "/m" modifier. Instead of "$", use "\n" or "(?=\n|\z)".
+
+Test::Builder::Level controls how far up in the call stack a test will look
+at when reporting a failure. This should be incremented by any subroutine
+which directly or indirectly calls test routines from Test::More, such as
+ok() or is():
+
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+Read the documentation for more on how to write tests:
+
+ perldoc Test::More
+ perldoc Test::Builder
+
+For available PostgreSQL-specific test methods and some example tests read the
+perldoc for the test modules, e.g.:
+
+ perldoc src/test/perl/PostgreSQL/Test/Cluster.pm
+
+Portability
+-----------
+
+Avoid using any bleeding-edge Perl features. We have buildfarm animals
+running Perl versions as old as 5.8.3, so your tests will be expected
+to pass on that.
+
+Also, do not use any non-core Perl modules except IPC::Run. Or, if you
+must do so for a particular test, arrange to skip the test when the needed
+module isn't present. If unsure, you can consult Module::CoreList to find
+out whether a given module is part of the Perl core, and which module
+versions shipped with which Perl releases.
+
+One way to test for compatibility with old Perl versions is to use
+perlbrew; see http://perlbrew.pl . After installing that, do
+
+ export PERLBREW_CONFIGURE_FLAGS='-de -Duseshrplib'
+ perlbrew --force install 5.8.3
+ perlbrew use 5.8.3
+ perlbrew install-cpanm
+ cpanm install Test::Simple@0.98
+ cpanm install IPC::Run@0.79
+ cpanm install ExtUtils::MakeMaker@6.50 # downgrade
+
+TIP: if Test::Simple's utf8 regression test hangs up, try setting a
+UTF8-compatible locale, e.g. "export LANG=en_US.utf8".
+
+Then re-run Postgres' configure to ensure the correct Perl is used when
+running tests. To verify that the right Perl was found:
+
+ grep ^PERL= config.log
+
+Due to limitations of cpanm, this recipe doesn't exactly duplicate the
+module list of older buildfarm animals. The discrepancies should seldom
+matter, but if you want to be sure, bypass cpanm and instead manually
+install the desired versions of Test::Simple and IPC::Run.