summaryrefslogtreecommitdiffstats
path: root/doc/postgresql-debian-packaging.md
blob: 79d4a8ab7879ac3f9acb23e8b9cb6cf74c54a3a5 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
Packaging PostgreSQL Extensions for Debian
==========================================
January 2024, Christoph Berg <myon@debian.org>

Debian ships many PostgreSQL applications and extensions as packages. I often
get asked by developers how they would get their programs packaged, and ended
up writing the same reply over and over. This is a write-up intended as a more
thorough answer.

# Anatomy of Debian packages

Debian knows two sorts of packages: "source" packages and "binary" packages.
The latter type is the `.deb` files that get installed using apt or dpkg. The
first type is what this article is mostly about. For both sorts of packages,
there is an unpacked form and a packed form.

## Binary packages

Packed binary packages are a single `.deb` file:

```
$ ls -al postgresql-16-unit_7.7-1_amd64.deb
-rw-r--r-- 1 myon myon 136740 Jan 12 14:13 postgresql-16-unit_7.7-1_amd64.deb

$ dpkg-deb -I postgresql-16-unit_7.7-1_amd64.deb
 new Debian package, version 2.0.
 size 136740 bytes: control archive=1332 bytes.
     643 bytes,    15 lines      control
    2046 bytes,    25 lines      md5sums
 Package: postgresql-16-unit
 Source: postgresql-unit
 Version: 7.7-1
 Architecture: amd64
 Maintainer: Christoph Berg <myon@debian.org>
 Installed-Size: 500
 Depends: postgresql-16, postgresql-16-jit-llvm (>= 16), libc6 (>= 2.14)
 Section: database
 Priority: optional
 Homepage: https://github.com/df7cb/postgresql-unit
 Description: SI Units for PostgreSQL
  postgresql-unit implements a PostgreSQL datatype for SI units, plus byte. The
  base units can be combined to named and unnamed derived units using operators
  defined in the PostgreSQL type system. SI prefixes are used for input and
  output, and quantities can be converted to arbitrary scale.

$ dpkg-deb -c postgresql-16-unit_7.7-1_amd64.deb
drwxr-xr-x root/root         0 2023-01-06 16:34 ./
drwxr-xr-x root/root         0 2023-01-06 16:34 ./usr/
drwxr-xr-x root/root         0 2023-01-06 16:34 ./usr/share/
drwxr-xr-x root/root         0 2023-01-06 16:34 ./usr/share/doc/
drwxr-xr-x root/root         0 2023-01-06 16:34 ./usr/share/doc/postgresql-16-unit/
-rw-r--r-- root/root       267 2023-01-06 16:34 ./usr/share/doc/postgresql-16-unit/NEWS.Debian.gz
-rw-r--r-- root/root      7202 2023-01-06 16:34 ./usr/share/doc/postgresql-16-unit/README.md.gz
-rw-r--r-- root/root       977 2023-01-06 16:34 ./usr/share/doc/postgresql-16-unit/changelog.Debian.gz
-rw-r--r-- root/root       868 2023-01-06 16:34 ./usr/share/doc/postgresql-16-unit/copyright
drwxr-xr-x root/root         0 2023-01-06 16:34 ./usr/share/postgresql/
drwxr-xr-x root/root         0 2023-01-06 16:34 ./usr/share/postgresql/16/
drwxr-xr-x root/root         0 2023-01-06 16:34 ./usr/share/postgresql/16/extension/
-rw-r--r-- root/root     19259 2023-01-06 16:34 ./usr/share/postgresql/16/extension/unit--7.sql
-rw-r--r-- root/root       244 2023-01-06 16:34 ./usr/share/postgresql/16/extension/unit.control
...
```

The unpacked binary package is of course the files being installed on the
system.

## Source packages

Unpacked source packages consist of the original upstream source code, with a `debian/` directory added.

```
postgresql-unit/
├── debian/
│   ├── changelog
│   ├── control
│   ├── control.in
│   ├── copyright
│   ├── gitlab-ci.yml
│   ├── NEWS
│   ├── pgversions
│   ├── rules*
│   ├── source/
│   │   └── format
│   ├── tests/
│   │   ├── control
│   │   └── installcheck*
│   ├── upstream/
│   │   └── metadata
│   └── watch
├── Makefile
├── NEWS.md
├── README.md
├── unit--7.sql.in
├── unit.c
└── unit.control
```

Most packages are maintained in Git, which tracks this unpacked source package
form. (Either with or without the original upstream source, more on that
later.)

Packed source packages are actually a set of files:

```
-rw-rw-r--   1 myon myon   3848 Jan 12 14:31 postgresql-unit_7.7-1.debian.tar.xz
-rw-rw-r--   1 myon myon   1100 Jan 12 14:31 postgresql-unit_7.7-1.dsc
-rw-rw-r--   1 myon myon 414114 Jan 12 14:12 postgresql-unit_7.7.orig.tar.gz
```

This transport format is used for uploading to the Debian archive and for
retrieving the source code using `apt source`. (It is not stored in Git.)

The `.orig.tar.gz` is the original source tarball as distributed by upstream,
either as download from the upstream homepage, or for projects hosted on
GitHub, often a tarball automatically generated by GitHub from an upstream Git
tag.

The `.debian.tar.xz` file contains the `debian/` directory.

The `.dsc` file is a descriptor that contains pointers to the other files in
the source package, and more meta information.

```
$ cat postgresql-unit_7.7-1.dsc
Format: 3.0 (quilt)
Source: postgresql-unit
Binary: postgresql-16-unit
Architecture: any
Version: 7.7-1
Maintainer: Christoph Berg <myon@debian.org>
Homepage: https://github.com/df7cb/postgresql-unit
Standards-Version: 4.6.1
Vcs-Browser: https://github.com/df7cb/postgresql-unit
Vcs-Git: https://github.com/df7cb/postgresql-unit.git
Testsuite: autopkgtest
Testsuite-Triggers: make
Build-Depends: bison, debhelper-compat (= 13), flex, postgresql-server-dev-all (>= 217~)
Package-List:
 postgresql-16-unit deb database optional arch=any
Checksums-Sha1:
 c2f81968bfbe83fed49b084b737e3aba423bf15a 414114 postgresql-unit_7.7.orig.tar.gz
 b8a1917ddecb99b1441218bc74a0b7cb30752235 3848 postgresql-unit_7.7-1.debian.tar.xz
Checksums-Sha256:
 411d05beeb97e5a4abf17572bfcfbb5a68d98d1018918feff995f6ee3bb03e79 414114 postgresql-unit_7.7.orig.tar.gz
 36e89c762e50ddf997b079703200c0df6967b4fe911bde8e9482d8e82dcb6a98 3848 postgresql-unit_7.7-1.debian.tar.xz
Files:
 33a22586c8b81564ba7e9c05f430ad40 414114 postgresql-unit_7.7.orig.tar.gz
 a0b31860b86c12c7173a78d6ecd525cb 3848 postgresql-unit_7.7-1.debian.tar.xz
```

## Building the source package

To get started with working with a Debian package, get the unpacked source
package. This could mean invoking `apt source`, but most often checking out the
packaging Git repository is the better option as it might contain changes that
have not been uploaded yet. It also makes contributing changes easier. For
packages, that are already part of Debian, the `debcheckout` tool can automate
that (it uses the `Vcs-Git` field in the metadata).

To build the packed source packed from the unpacked one, enter the package
directory, and invoke `dpkg-buildpackage -S --no-sign`:

```
postgresql-unit $ dpkg-buildpackage -S --no-sign
dpkg-buildpackage: info: source package postgresql-unit
dpkg-buildpackage: info: source version 7.7-1
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Christoph Berg <myon@debian.org>
 dpkg-source --before-build .
 debian/rules clean
dh clean --with pgxs
...
 dpkg-source -b .
dpkg-source: info: using source format '3.0 (quilt)'
dpkg-source: info: building postgresql-unit using existing ./postgresql-unit_7.7.orig.tar.gz
dpkg-source: info: building postgresql-unit in postgresql-unit_7.7-1.debian.tar.xz
dpkg-source: info: building postgresql-unit in postgresql-unit_7.7-1.dsc
 dpkg-genbuildinfo --build=source -O../postgresql-unit_7.7-1_source.buildinfo
 dpkg-genchanges --build=source -O../postgresql-unit_7.7-1_source.changes
dpkg-genchanges: info: including full source code in upload
 dpkg-source --after-build .
dpkg-buildpackage: info: source-only upload (original source is included)
```

Note that the artifacts produced by `dpkg-buildpackage` are always in the *parent* directory of the working directory.
To make room for that, I always create *two* levels of directory for my working copies, so a typical case looks like this:

```
postgresql-unit/
├── postgresql-unit/   <--- most commands are invoked from here
│   ├── debian/
│   │   ├── changelog
│   │   ├── control
│   │   ├── control.in
│   │   ├── copyright
│   │   ├── files
│   │   ├── gitlab-ci.yml
│   │   ├── NEWS
│   │   ├── pgversions
│   │   ├── rules*
│   │   ├── source/
│   │   │   └── format
│   │   ├── tests/
│   │   │   ├── control
│   │   │   └── installcheck*
│   │   ├── upstream/
│   │   │   └── metadata
│   │   └── watch
│   ├── Makefile
│   ├── NEWS.md
│   ├── powers.c
│   ├── powers.h
│   ├── README.md
│   ├── unit--7.sql.in
│   ├── unit.c
│   └── unit.control
├── postgresql-16-unit_7.7-1_amd64.deb
├── postgresql-16-unit-dbgsym_7.7-1_amd64.deb
├── postgresql-unit_7.7-1_amd64.changes
├── postgresql-unit_7.7-1.debian.tar.xz
├── postgresql-unit_7.7-1.dsc
└── postgresql-unit_7.7.orig.tar.gz
```

## Building binary packages

Binary packages are built using `dpkg-buildpackage --no-sign`.

```
 dpkg-buildpackage --no-sign
dpkg-buildpackage: info: source package postgresql-unit
dpkg-buildpackage: info: source version 7.7-1
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Christoph Berg <myon@debian.org>
dpkg-buildpackage: info: host architecture amd64
 dpkg-source --before-build .
 debian/rules clean
dh clean --with pgxs
...
 dpkg-source -b .
dpkg-source: info: using source format '3.0 (quilt)'
dpkg-source: info: building postgresql-unit using existing ./postgresql-unit_7.7.orig.tar.gz
dpkg-source: info: building postgresql-unit in postgresql-unit_7.7-1.debian.tar.xz
dpkg-source: info: building postgresql-unit in postgresql-unit_7.7-1.dsc
 debian/rules binary
dh binary --with pgxs
   dh_update_autotools_config
   dh_autoreconf
   dh_auto_configure
   dh_auto_build --buildsystem=pgxs
	pg_buildext build build-%v
### PostgreSQL 16 build ###
make[1]: Entering directory '/home/myon/projects/postgresql/postgresql-unit/postgresql-unit/build-16'
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wshadow=compatible-local -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -g -O2 -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -fno-omit-frame-pointer -g -O2 -ffile-prefix-map=/home/myon/projects/postgresql/postgresql-unit/postgresql-unit=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -fPIC -fvisibility=hidden -ffp-contract=off -I. -I/home/myon/postgresql/postgresql-unit/postgresql-unit -I/usr/include/postgresql/16/server -I/usr/include/postgresql/internal  -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2   -c -o unit.o /home/myon/postgresql/postgresql-unit/postgresql-unit/unit.c
...
### End 16 build ###
   create-stamp debian/debhelper-build-stamp
   dh_prep
   dh_auto_install --buildsystem=pgxs --destdir=debian/postgresql-16-unit/
	pg_buildext install build-%v postgresql-%v-unit
### PostgreSQL 16 install ###
make[1]: Entering directory '/home/myon/projects/postgresql/postgresql-unit/postgresql-unit/build-16'
/usr/bin/install -c -m 755  unit.so '/home/myon/postgresql/postgresql-unit/postgresql-unit/debian/postgresql-16-unit/usr/lib/postgresql/16/lib/unit.so'
...
### End 16 install ###
   debian/rules override_dh_installdocs
make[1]: Entering directory '/home/myon/projects/postgresql/postgresql-unit/postgresql-unit'
dh_installdocs --all README.*
make[1]: Leaving directory '/home/myon/projects/postgresql/postgresql-unit/postgresql-unit'
   dh_installchangelogs
   dh_perl
   dh_link
   debian/rules override_dh_pgxs_test
make[1]: Entering directory '/home/myon/projects/postgresql/postgresql-unit/postgresql-unit'
# defer testing to autopkgtest, the data tables are not in /usr/share/postgresql yet
make[1]: Leaving directory '/home/myon/projects/postgresql/postgresql-unit/postgresql-unit'
   dh_strip_nondeterminism
   dh_compress
   dh_fixperms
   dh_missing
   dh_dwz -a
   dh_strip -a
   dh_makeshlibs -a
   dh_shlibdeps -a
   dh_installdeb
   dh_gencontrol
   dh_md5sums
   dh_builddeb
dpkg-deb: building package 'postgresql-16-unit' in '../postgresql-16-unit_7.7-1_amd64.deb'.
dpkg-deb: building package 'postgresql-16-unit-dbgsym' in '../postgresql-16-unit-dbgsym_7.7-1_amd64.deb'.
 dpkg-genbuildinfo -O../postgresql-unit_7.7-1_amd64.buildinfo
 dpkg-genchanges -O../postgresql-unit_7.7-1_amd64.changes
dpkg-genchanges: info: including full source code in upload
 dpkg-source --after-build .
dpkg-buildpackage: info: full upload (original source is included)
```

Again, the resulting `.deb` files are placed in the parent directory.

By default, this also builds the source. If this fails (often when there are
files that differ from the tarball version, more on patches later), use `-b` to
skip building the source: `dpkg-buildpackage -b --no-sign`.

If building fails because of missing dependencies, install them using
`apt build-dep .`.

There are various front-end utilities to automate these building steps better
(git-buildpackage, sbuild, pbuilder, debuild), but dpkg-buildpackage is just
fine.

## The debian/ directory

The `debian/` directory contains metadata and build instructions for the
package. "Creating a Debian package" really means editing the files in this
directory to make the package behave as desired.

### debian/control

The control file contains one section for the source package, followed by one
or more sections for binary packages.

```
$ cat debian/control
Source: postgresql-unit
Section: database
Priority: optional
Maintainer: Christoph Berg <myon@debian.org>
Build-Depends:
 bison,
 debhelper-compat (= 13),
 flex,
 postgresql-server-dev-all (>= 217~),
Standards-Version: 4.6.1
Rules-Requires-Root: no
Vcs-Git: https://github.com/df7cb/postgresql-unit.git
Vcs-Browser: https://github.com/df7cb/postgresql-unit
Homepage: https://github.com/df7cb/postgresql-unit

Package: postgresql-16-unit
Architecture: any
Depends: ${misc:Depends}, ${shlibs:Depends}, postgresql-16
Description: SI Units for PostgreSQL
 postgresql-unit implements a PostgreSQL datatype for SI units, plus byte. The
 base units can be combined to named and unnamed derived units using operators
 defined in the PostgreSQL type system. SI prefixes are used for input and
 output, and quantities can be converted to arbitrary scale.
```

(In the case of PostgreSQL extension packages, the control file is
automatically generated from `debian/control.in`, see below. Normal packages do
not have a control.in file.)

### debian/rules

The rules file handles the actual binary package building steps, in Makefile
syntax.

Almost all packages today use a helper system called `debhelper` which consists
of various building blocks called `dh_*` that handle the build steps.
Historically, rules files would consist of long lists of `dh_*` steps (similar
to the build output above), but there is a "sequencer" command `dh` that knows
how to invoke the basic steps.

Many packages that don't need any tweaking at that level have a very short
`debian/rules` file. (Make sure the indentation is a tab, not spaces!)

```
#!/usr/bin/make -f

%:
        dh $@
```

Or in the case of PostgreSQL extensions:

```
#!/usr/bin/make -f

%:
        dh $@ --with pgxs
```

If any of the `dh_*` steps need changes, we can override them in the rules file
by adding a `override_dh_*` target:

```
#!/usr/bin/make -f

override_dh_installdocs:
        dh_installdocs --all README.*

override_dh_pgxs_test:
        # defer testing to autopkgtest, the data tables are not in /usr/share/postgresql yet

%:
        dh $@ --with pgxs
```

### Build steps

The most interesting build steps to hook into are:

#### `dh_auto_configure`

Autodetects the build system and runs `./configure`, `cmake` and the like with
a set of default options.

To add options, do:

```
override_dh_auto_configure:
        dh_auto_configure -- -DEXTRA_OPTION=foo --with-bar
```

#### `dh_auto_build`

The main build step. If the automatically run command is wrong, just override it.

```
override_dh_auto_build:
        $(MAKE) -C some_sub_dir world
```

#### `dh_auto_install`

Runs upstreams' `make install` or equivalent. The files are installed into
`DESTDIR=debian/foo` (single-binary package) or `DESTDIR=debian/tmp`
(multi-binary package).

#### `dh_install` and `debian/foo.install`

If there is more than one binary package, the files installed in `debian/tmp`
need to be distributed to the individual binary packages. This is done by
directory/file lists in `debian/*.install`.

### debian/source/format

Should be verbatim this:

```
3.0 (quilt)
```

(More on quilt and patches below.)

### debian/copyright

Debian wants copyright information on *all* files in a package. For private
package, this file can be omitted, but for anything official, this file has to
list all the copyright holders, along with any copyright terms and license
texts.

### debian/watch

To get informed about new upstream versions, Debian runs a "watch" system that
polls upstream download locations for new package versions. The `debian/watch`
file tells the `uscan` tool where to look and (optionally) how to transform
upstream's version naming scheme into a Debian-compatible one.

```
version=4
https://github.com/df7cb/postgresql-unit/tags .*/([0-9.]*).tar.gz
```

This is a simple example where `uscan` looks at some GitHub "tags" URL, parses
the HTML, and recognizes all links pointing to .tar.gz files as new versions.
The regexp part of the URL in parentheses is used as the version number.

The `uscan` tool is part of the `devscripts` package which holds a bunch of
utilities useful for packaging tasks.

## Patching the upstream source

Debian packages are based on upstream tarballs, and `dpkg-buildpackage` does
not like changed (or new) files (except in the `debian/` directory). Changes
need to be done using patch files stored in the `debian/patches/` directory,
and applied in the order listed in `debian/patches/series`.

At package build time, support is built-in in dpkg-buildpackage. Patches can
edited with whatever tool, with `quilt` being the easiest solution. (It's an
optional package that likely isn't preinstalled.) `quilt` needs some
configuration in `~/.quiltrc`:

```
QUILT_DIFF_OPTS="-p"
QUILT_REFRESH_ARGS="-p ab --no-timestamps --no-index"
QUILT_PATCHES=debian/patches
```

Some useful commands:

* Apply all patches: `quilt push -a`
* Unapply all patches: `quilt pop -a`
* Create a new patch: `quilt new foo`
* Add a file to a patch: `quilt add bar` (do this *before* changing the file!)
* Add a file and edit it: `quilt edit bar`
* Show current diff: `quilt diff`
* Save a patch after editing: `quilt refresh`

Suppose we want to fix something in the `unit.c` file:

```
$ quilt new unit-dllexport
Patch unit-dllexport is now on top

$ quilt add unit.c
File unit.c added to patch unit-dllexport

$ vi unit.c

$ quilt diff | cat
Index: postgresql-unit/unit.c
===================================================================
--- postgresql-unit.orig/unit.c	2023-11-08 20:51:04.343207806 +0100
+++ postgresql-unit/unit.c	2024-01-12 16:10:23.143509535 +0100
@@ -136,7 +136,7 @@ unit_get_definitions(void)
 
 PG_MODULE_MAGIC;
 
-void _PG_init(void);
+void PGDLLEXPORT _PG_init(void);
 
 void
 _PG_init(void)

$ quilt refresh
Refreshed patch unit-dllexport
```

This creates the `debian/patches/` directory:

```
postgresql-unit/debian/patches/
├── series
└── unit-dllexport
```

At build-time, `dpkg-buildpackage` will then automatically apply and un-apply
patches as needed.

## Tests

Package tests are run in two flavors: at build-time and later independently on
packages installed on some system.

### Build-time package tests

If an upstream project has a test suite, it is recommended to run it at build
time. In many cases, `dh_auto_test` will guess how to do that and no extra
configuration is needed. If it guesses incorrectly, use `override_dh_auto_test`
to provide a better command.

### Install-time package tests

Next to tests at build-time, we can also run tests when the binary packages are
installed on some actual system. This has the advantage that it runs when all
files are in their final location, and can also interface with other packages
and services that might not be available at build time. It can also run
periodically to spot regression, while build-time tests would usually not be
repeated.

Debian's system to run these tests is `autopkgtest`. The TL;DR version of the
documentation is this: In the `debian/tests/` directory, provide a command
(usually a shell script) that exercises some package smoke test or more complex
scenario. Register that test in `debian/tests/control`.

```
postgresql-unit/debian/tests/
├── control
└── installcheck*

$ cat debian/tests/control
Depends: @, make
Tests: installcheck
Restrictions: allow-stderr

$ cat debian/tests/installcheck
#!/bin/sh
pg_buildext -i '--locale=C.UTF-8' installcheck
```

In `Depends` the `@` is a shorthand for all binary packages built from this
source. Other packages that the tests needs (and that the binaries don't depend
on) can be listed here. `Restrictions` declare tests properties. Examples are
`allow-stderr` (don't consider stderr output to be a test failure) and
`root-needed` (run test as root instead of an unprivileged user).

In Debian, these tests are run automatically for QA, see
https://ci.debian.net/. More details are in the autopkgtest package
documentation in /usr/share/doc/autopkgtest/.

# PostgreSQL extension packages

Packages for PostgreSQL extensions work as described above, but since
extensions have to be compiled for each PostgreSQL major version separately,
things are a bit more complex.

For Debian, the process is changed to still have one source package per
upstream project, but to build separate binary packages for each PostgreSQL
major version. The naming scheme is `postgresql-NN-foo`. (In case the upstream
project is called `pg_foo`, make a judgment call if `postgresql-NN-pg-foo` or
`postgresql-NN-foo` is better.)

In Debian, only one PostgreSQL major version is supported at a time, but in the
https://apt.postgresql.org/ repository, many major versions are supported in
parallel (currently PostgreSQL 10 everything newer, even when 10 is already
EOL).

## `debian/control.in`

In order not to have to edit the list of binary packages built when a new
PostgreSQL major version comes out, or when a source package is built both for
Debian and for apt.postgresql.org, the `debian/control` file is generated from
a template in `debian/control.in`.

```
$ cat debian/control.in
Source: postgresql-unit
Section: database
Priority: optional
Maintainer: Christoph Berg <myon@debian.org>
Build-Depends:
 bison,
 debhelper-compat (= 13),
 flex,
 postgresql-server-dev-all (>= 217~),
Standards-Version: 4.6.2
Rules-Requires-Root: no
Vcs-Git: https://github.com/df7cb/postgresql-unit.git
Vcs-Browser: https://github.com/df7cb/postgresql-unit
Homepage: https://github.com/df7cb/postgresql-unit

Package: postgresql-PGVERSION-unit
Architecture: any
Depends: ${misc:Depends}, ${shlibs:Depends}, ${postgresql:Depends}
Description: SI Units for PostgreSQL
 postgresql-unit implements a PostgreSQL datatype for SI units, plus byte. The
 base units can be combined to named and unnamed derived units using operators
 defined in the PostgreSQL type system. SI prefixes are used for input and
 output, and quantities can be converted to arbitrary scale.
```

The section with the `PGVERSION` token is duplicated for each major version
supported, with the version number filled in.

## `debian/pgversions`

Not every package supports all PostgreSQL major versions. The
`debian/pgversions` file is used to mark which versions are actually supported,
so apt.postgresql.org can skip building the other version.

Unless we know the exact versions supported, we should use `all`:

```
$ cat debian/pgversions
all
```

If 14 or newer is supported:

```
$ cat debian/pgversions
14+
```

### Supported versions

`debian/pgversions` lists the versions supported *by the package*. The other
half of that system is the set of versions supported *by the system*. This list
is configured in `/etc/postgresql-common/supported_versions` or by setting the
`PG_SUPPORTED_VERSIONS` environment variable. (Set this variable to test
building for other major versions.) The actual build process will use the
intersection of these two lists.

## `pg_buildext`

The process of building PostgreSQL extensions for several major versions is
automated by the `pg_buildext` utility. It provides commands for the most
common tasks.

Package building:

* `pg_buildext supported-versions` - print list of supported versions.
  Use this to loop over versions in `debian/rules`.
* `pg_buildext build build-%v` - build in `build-%v` directory
* `pg_buildext install build-%v postgresql-%v-unit` - invoke `make install`
* `pg_buildext installcheck build-%v postgresql-%v-unit` - invoke
  `make installcheck` for build-time testing
* `pg_buildext loop postgresql-%v-unit` - use instead of
  `build/install/installcheck` if the package doesn't support out-of-tree
  builds in subdirectories
* `pg_buildext updatecontrol` - rebuild `debian/control` from
  `debian/control.in`. Run this manually when the set of supported versions has
  changed. This is not run automatically because the Debian packaging policy
  forbids changing the set of binary packages at build time. (In environments
  where this is not an issue, set `PG_UPDATECONTROL=yes`.)

Package testing:

* `pg_buildext installed-versions` - print list of installed versions.
  Use this to loop over versions in `debian/tests/*`.
* `pg_buildext installcheck` - invoke `make installcheck` on installed packages

## `dh --with pgxs`

A debhelper extension `pgxs` is provided that adds builds steps to the `dh`
build sequence.

```
$ cat debian/rules
#!/usr/bin/make -f

%:
        dh $@ --with pgxs
```

If the package doesn't support out-of-tree builds, use
`dh $@ --with pgxs_loop`.

```
   dh_auto_build --buildsystem=pgxs
        pg_buildext build build-%v
   dh_auto_install --buildsystem=pgxs
        pg_buildext install build-%v postgresql-%v-unit
   dh_pgxs_test
        pg_buildext installcheck . build-%v postgresql-%v-unit
```

To override any of these steps, use `override_dh_auto_*` in `debian/rules`.

## Package template

To get started with a new package, the `dh_make_pgxs` tool can generate a
skeleton `debian/` directory:

```
$ dh_make_pgxs
```

If the auto-detected values are wrong, hit `^C` and add more command line
parameters.