summaryrefslogtreecommitdiffstats
path: root/MANIFEST-FORMAT.md
blob: 37773c21705fbc4cb0d520934714aa7086eaed34 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
# The debputy manifest format

_This is [reference guide] and is primarily useful if you have an idea of what you are looking for._
_If you are new to `debputy`, maybe you want to read [GETTING-STARTED-WITH-dh-debputy.md](GETTING-STARTED-WITH-dh-debputy.md) first._

<!-- To writers and reviewers: Check the documentation against https://documentation.divio.com/ -->


## Prerequisites

This guide assumes familiarity with Debian packaging in general.  Notably, you should understand
the different between a (Debian) source package and a (Debian) binary package (e.g., `.deb`) plus
how these concepts relates to `debian/control` (the source control file).

Additionally, the reader is expected to have an understanding of globs and substitution variables.

It is probably very useful to have an understanding on how a binary package is
assembled.  While `debputy` handles the assembly for you, this document will not go in details
with this. Prior experience with `debhelper` (notably the `dh`-style `debian/rules`) may be
useful but should not be a strict requirement.


# The basic manifest

The manifest is a YAML document with a dictionary at the top level layer.  As usual with YAML
versions, you can choose to leave it implicit.  All manifests must include a manifest version, which
will enable the format to change over time.  For now, there is only one version (`0.1`) and you have
to include the line:

    manifest-version: "0.1"


On its own, the manifest containing only `manifest-version: "..."` will not do anything.  So if you
end up only having the `manifest-version` key in the manifest, you can just remove the manifest and
rely entirely on the built-in rules.


# Path matching rules

Most of the manifest is about declaring rules for a given path such as "foo must be a symlink"
or "bar must be owned by root:tty and have mode 02755".  

The manifest contains the following types of matches:

 1) Exact path matches.  These specify the path inside the debian package exactly without any
    form of wildcards (e.g., `*` or `?`).  However, they can use substitution variables.
    Examples include:
    * `usr/bin/sudo`
    * `usr/lib/{{DEB_HOST_MULTIARCH}}/libfoo.so`

    Having a leading `/` is optional.  I.e. `/usr/bin/sudo` and `usr/bin/sudo` are considered the
    same path.

 2) Glob based path matches.  These specify a rule that match any path that matches a given
    glob.  These rules must contain a glob character (e.g., `*`) _and_ a `/`. Examples include:

    * `foo/*`
    * `foo/*.txt`
    * `usr/lib/{{DEB_HOST_MULTIARCH}}/lib*.so*`

    Note that if the glob does not contain a `/`, then it falls into the Basename glob rule
    below.

 3) Basename glob matches.  These specify a rule that match any path where the basename matches
    a given glob.  They must contain a glob character (e.g., `*`) but either must not have a
    `/` in them at all or start with `**/` and not have any `/` from there.
    Examples include:

    * `"*.la"`
    * `"**/*.md"`
    * `"**/LICENSE"`

    The examples use explicit quoting because YAML often interprets `*` as an anchor rule in the
    places where you are likely to use this kind of glob.  The use of the `**/`-prefix is
    optional when the basename is a glob.  If you wanted to match all paths with the basename
    of exactly `LICENSE`, then you have to use the prefix (that is, use `**/LICENSE`) as `LICENSE`
    would be interpreted as an exact match for a top-level path.

However, there are also cases where these matching rules can cause conflicts.  This is covered
in the [Conflict resolution](#conflict-resolution) section below.


## Limitations on debputy path rules

Path rules:

 1) Must match relatively to the package root.
 2) Never resolves a symlink.
    * If `bin/ls` is a symlink, then `bin/ls` matches the symlink (and never the target)
    * If `bin` is a symlink, then `bin/ls` (or `bin/*`) would fail to match, because
      matching would require resolving the symlink.

These limitations are in place because of implementation details in `debputy`.


## Conflict resolution

The manifest can contain seemly mutually exclusive rules.  As an example, if you ask for
`foo/symlink` to be a symlink but also state that you want to remove `foo` entirely
from the debian package then the manifest now has two mutually exclusive requests.

To resolve these problems, `debputy` relies on the following rules for conflict resolutions:

  1. Requests are loosely-speaking ranked and applied from top to bottom:
     1. `installations` (ordered; "first match wins")
     2. `transformations` (ordered; "all matching rules applies")

     The "first match wins" rule is exactly what it says.

     The "all matching rules applies" rule means that each rule is applied in order.  Often
     this behaves like a simple case of either "first match wins" or "last match wins" (depending
     on the context).

     Note for transformation rules, an early rule can change the file system layout, which will
     affect whether a later rule matches.  This is similar to how shell globs commands work:

          $ rm usr/lib/libfoo.la
          $ chmod 0644 usr/lib/*

     Here the glob used with `chmod` will not match `usr/lib/libfoo.la` because it was removed.
     As noted, a similar logic applies to transformation rules.

  2. The ordered rules (such as every transformation inside `transformations`) are applied in the
     listed order (top to bottom). Due to the explicit ordering, such rules generally do not trigger
     other conflict resolution rules.

     Note keep in mind that all rules must in general apply to at least one path.

Note that `debputy` will in some cases enforce that rules are not redundant.  This feature is currently
only fully implemented for `installations`.

## All definitions must be used

Whenever you define a request or rule in the manifest, `debputy` will insist on it being used
at least once.  The exception to this rule being conditional rules where the condition
evaluates to `false` (in which case the rule never used and does not trigger an error).

This is useful for several reasons:

 1. The definition may have been shadowed by another rule and would have been ignored otherwise
 2. The definition may no longer be useful, but its present might confuse a future reader of
    the manifest.

In all cases, `debputy` will tell you if a definition was unused and where you can find that
definition.


## debputy globs

In general, the following rules applies to globs in `debputy`.

 * The `*` match 0 or more times any characters except `/`.
 * The `?` match exactly one character except `/`.
 * The glob `foo/*` matches _everything_ inside `foo/` including hidden files (i.e., paths starting 
   with `.`) unlike `bash`/`sh` globs. However, `foo/*` does _not_ match `foo/` itself (this latter
   part matches `bash`/`sh` behaviour and should be unsurprising).
 * For the special-cases where `**` is supported, then `**` matches zero or more levels of directories.
   This means that `foo/**/*` match any path beneath `foo` (but still not `foo`).  This is mostly relevant
   for built-in path matches as it is currently not possible to define `foo/**/...` patterns in the manifest.

Note that individual rules (such as `clean-after-removal`) may impose special cases to how globs
work. The rules will explicitly list if they divert from the above listed glob rules.

# Rules for substituting manifest variables

The `debputy` tool supports substitution in various places (usually paths) via the following
rules.  That means:

 1) All substitutions must start with `{{` and end with `}}`.  The part inbetween is
    the `MANIFEST_VARIABLE` and must match the regular expression `[A-Za-z0-9][-_:0-9A-Za-z]*`.
    Note that you can use space around the variable name if you feel that increases readability.
    (That is, `{{ FOO }}` can be used as an alternative to `{{FOO}}`).
 2) The `MANIFEST_VARIABLE` will be result from a set of built-in variables and the variables from
    `dpkg-architecture`.
 3) You can use `{{token:DOUBLE_OPEN_CURLY_BRACE}}` and `{{token:DOUBLE_CLOSE_CURLY_BRACE}}` (built-in
    variables)  if you want a literal `{{` or `}}`  would otherwise have triggered an undesired expansion.
 4) All `{{MANIFEST_VARIABLE}}` must refer to a defined variable.
    - You can see the full list of `debputy` and plugin provided manifest variables via:
      `debputy plugin list manifest-variables`. The manifest itself can declare its own variables
      beyond that list. Please refer to the [Manifest Variables](#manifest-variables-variables)
      section manifest variables declared inside the manifest.
 5) There are no expression syntax inside the `{{ ... }}` (unlike jinja2 and other template languages).
    This rule may be changed in the future (with a new manifest version).

Keep in mind that substitution _cannot_ be used everywhere.  There are specific places where
it can be used.  Also, substitution _cannot be used_ to introduce globs into paths.  When a
substitution occurs inside a path all characters inserted are treated as literal characters.

Note: While manifest variables can be substituted into various places in the `debputy` manifest, they
are distinct from `dpkg`'s "substvars" (`man 5 deb-substvars`) that are used in the `debian/control`
file.

## Built-in or common substitution variables

 * `{{token:NEWLINE}}` or `{{token:NL}}` expands to a literal newline (LF) `\n`.
 * `{{token:TAB}}` expands to a literal tab `\t`.
 * `{{token:OPEN_CURLY_BRACE}}`  / `{{token:CLOSE_CURLY_BRACE}}` expand to `{` / `}`
 * `{{token:DOUBLE_OPEN_CURLY_BRACE}}` / `{{token:DOUBLE_CLOSE_CURLY_BRACE}}` expands to `{{` / `}}`.
 * `{{PACKAGE}}` expands to the binary package name of the current binary package. This substitution
   only works/applies when the substitution occurs in the context of a concrete binary package.
 * Plus any of the variables produced by `dpkg-architecture`, such as `{{DEB_HOST_MULTIARCH}}`.

The following variables from `/usr/share/dpkg/pkg-info.mk` (`dpkg`) are also available:

 * DEB_SOURCE (as `{{DEB_SOURCE}}`)
 * DEB_VERSION (as `{{DEB_VERSION}}`)
 * DEB_VERSION_EPOCH_UPSTREAM (as `{{DEB_VERSION_EPOCH_UPSTREAM}}`)
 * DEB_VERSION_UPSTREAM_REVISION (as `{{DEB_VERSION_UPSTREAM_REVISION}}`)
 * DEB_VERSION_UPSTREAM (as `{{DEB_VERSION_UPSTREAM}}`)
 * SOURCE_DATE_EPOCH (as `{{SOURCE_DATE_EPOCH}}`)

These have the same definition as those from the `dpkg` provided makefile.


# Restrictions on defining ownership of paths

In some parts of the manifest, you can specify which user or group should have ownership of
a given path.  As an example, you can define a directory to be owned by `root` with group `tty`
(think `chown root:tty <some/path>`).

Ownership is generally defined via the keys `owner` and `group`.  For each of them, you can use
one of the following formats:

 1) A name (e.g., `owner: root`).
 2) An id (e.g., `owner: 0`). Please avoid using quotes around the ID in YAML as that can
    cause `debputy` to read the number as a name.
 3) A name and an id with a colon inbetween (e.g., `owner: "root:0"`).  The name must always
    come first here.  You may have to quote the value to prevent the YAML parser from being
    confused.

All three forms are valid and provide the same result.  Unless you have a compelling reason to
pick a particular form, the name-only is recommended for simplicity.  Notably, it does not
require your co-maintainer or future you to remember what the IDs mean.

Regardless of which form you pick:

 1) The provided owner must be defined by Debian `base-passwd` file, which are the only users guaranteed
    to be present on every Debian system.
    * Concretely, `debputy` verifies the input against `/usr/share/base-passwd/passwd.master` and
      `/usr/share/base-passwd/group.master` (except for `root` / `0` as an optimization).

 2) If the `name:id` form is used, then the name and the id values must match.  I.e., `root:2` would
    be invalid as the id for `root` is defined to be `0` in the `base-passwd` data files.

 3) The `debputy` tool maintains a `deny`-list of owners that it refuses even though `base-passwd`
    defines them. As a notable non-exhaustive example, `debputy` considers `nobody` or id `65534` 
    (the ID of `nobody` / `nogroup`) to be invalid owners.


# Conditional rules

There are cases, where a given rule should only apply in certain cases - such as only when a given
build profile is active (`DEB_BUILD_PROFILES` / `dpkg-buildpackage -P`).  For rules that
*support being conditional*, the condition is generally defined via the `when:` key and the condition
is then described beneath the `when:`.

As an example:

    packages:
        util-linux:
            transformations:
            - create-symlink
                  path: sbin/agetty
                  target: /sbin/getty
                  when:
                      # On Hurd, the package "hurd" ships "sbin/getty".
                      arch-matches: '!hurd-any'


When the condition under `when:` resolves to `true`, the rule will and must be used.  When the
condition resolves to `false`, the rule will not be applied even if it could have been.  However,
the rule may still be "partially" evaluated.  As an example, for installation rules, the source
patterns will still be evaluated to reserve what it would have matched, so that following rules
behave deterministically regardless of how the condition evaluates.

Note that conditions are *not* used as a conflict resolution and as such two conditional rules
can still cause conflicts even though their conditions are mutually exclusive.  This may be
changed in a later version of `debputy` provided `debputy` can assert the two conditions
are mutually exclusive.

The `when:` key has either a mapping, a list or a string as value depending on the condition.
Each supported condition is described in the following subsections.

## Architecture match condition `arch-matches` (mapping)

Sometimes, a rule needs to be conditional on the architecture.  This can be done by using the
`arch-matches` rule. In 99.99% of the cases, `arch-matches` will be form you are looking for
and practically behaves like a comparison against `dpkg-architecture -qDEB_HOST_ARCH`.

As an example:

    packages:
        util-linux:
            transformations:
            - create-symlink
                  path: sbin/agetty
                  target: /sbin/getty
                  when:
                      # On Hurd, the package "hurd" ships "sbin/getty".
                      arch-matches: '!hurd-any'

The `arch-matches` must be defined as a part of a mapping, where `arch-matches` is the key. The
value must be a string in the form of a space separated list architecture names or architecture
wildcards (same syntax as the  architecture restriction in Build-Depends in debian/control except
there is no enclosing `[]` brackets). The names/wildcards can optionally be prefixed by `!` to
negate them.  However, either *all* names / wildcards must have negation or *none* of them may
have it.

For the cross-compiling specialists or curious people: The `arch-matches` rule behaves like a
`package-context-arch-matches` in the context of a binary package and like
`source-context-arch-matches` otherwise. The details of those are covered in their own section.

## Explicit source or binary package context architecture match condition `source-context-arch-matches`, `package-context-arch-matches` (mapping)

**These are special-case conditions**. Unless you know that you have a very special-case,
you should probably use `arch-matches` instead. These conditions are aimed at people with
corner-case special architecture needs. It also assumes the reader is familiar with the
`arch-matches` condition.

To understand these rules, here is a quick primer on `debputy`'s concept of "source context"
vs "(binary) package context" architecture.  For a native build, these two contexts are the
same except that in the package context an `Architecture: all` package always resolve to
`all` rather than `DEB_HOST_ARCH`. As a consequence, `debputy` forbids `arch-matches` and
`package-context-arch-matches` in the context of an `Architecture: all` package as a warning
to the packager that condition does not make sense.

In the very rare case that you need an architecture condition for an `Architecture: all` package,
you can use `source-context-arch-matches`. However, this means your `Architecture: all` package
is not reproducible between different build hosts (which has known to be relevant for some
very special cases).

Additionally, for the 0.0001% case you are building a cross-compiling compiler (that is,
`DEB_HOST_ARCH != DEB_TARGET_ARCH` and you are working with `gcc` or similar) `debputy` can be
instructed (opt-in) to use `DEB_TARGET_ARCH` rather than `DEB_HOST_ARCH` for certain packages when
evaluating an architecture condition in context of a binary package. This can be useful if the
compiler produces supporting libraries that need to be built for the `DEB_TARGET_ARCH` rather than
the `DEB_HOST_ARCH`.  This is where `arch-matches` or `package-context-arch-matches` can differ
subtly from `source-context-arch-matches` in how they evaluate the condition.  This opt-in currently
relies on setting `X-DH-Build-For-Type: target` for each of the relevant packages in
`debian/control`.  However, unless you are a cross-compiling specialist, you will probably never
need to care about nor use any of this.

Accordingly, the possible conditions are:

 * `arch-matches`: This is the form recommended to laymen and as the default use-case. This
   conditional acts `package-context-arch-matches` if the condition is used in the context
   of a binary package. Otherwise, it acts as `source-context-arch-matches`.

 * `source-context-arch-matches`: With this conditional, the provided architecture constraint is compared
   against the build time provided host architecture (`dpkg-architecture -qDEB_HOST_ARCH`). This can
   be useful when an `Architecture: all` package needs an architecture condition for some reason.

 * `package-context-arch-matches`: With this conditional, the provided architecture constraint is compared
   against the package's resolved architecture. This condition can only be used in the context of a binary
   package (usually, under `packages.<name>.`).  If the package is an `Architecture: all` package, the
   condition will fail with an error as the condition always have the same outcome. For all other
   packages, the package's resolved architecture is the same as the build time provided host architecture
   (`dpkg-architecture -qDEB_HOST_ARCH`).

   - However, as noted above there is a special case for when compiling a cross-compiling compiler, where
     this behaves subtly different from `source-context-arch-matches`.

All conditions are used the same way as `arch-matches`. Simply replace `arch-matches` with the other
condition. See the `arch-matches` description for an example.

## Active build profile match condition `build-profiles-matches` (mapping)

The `build-profiles-matches` condition is used to assert whether the active build profiles
(`DEB_BUILD_PROFILES` / `dpkg-buildpackage -P`) matches a given build profile restriction.

As an example:

    # TODO: Not the best example (`create-symlink` is an unlikely use-case for this condition)
    packages:
        foo:
            transformations:
            - create-symlink
                  path: sbin/agetty
                  target: /sbin/getty
                  when:
                      build-profiles-matches: '<!pkg.foo.mycustomprofile>'

The `build-profiles-matches` must be defined as a part of a mapping, where `build-profiles-matches`
is the key.  The value is a string using the same syntax as the `Build-Profiles` field from `debian/control`
(i.e., a space separated list of `<[!]profile ...>` groups).

## Can run produced binaries `can-execute-compiled-binaries` (string)

The `can-execute-compiled-binaries` condition is used to assert the build can assume
that all compiled binaries can be run as-if they were native binaries. For native
builds, this condition always evaluates to `true`.  For cross builds, the condition
is generally evaluates to `false`.  However, there are special-cases where binaries
can be run during cross-building. Accordingly, this condition is subtly different
from the `cross-compiling` condition.

Note this condition should *not* be used when you know the binary has been built
for the build architecture (`DEB_BUILD_ARCH`) or for determining whether build-time tests
should be run (for build-time tests, please use the `run-build-time-tests` condition instead).
Some upstream build systems are advanced enough to distinguish building a final product vs.
building a helper tool that needs to run during build.  The latter will often be compiled by
a separate compiler (often using `$(CC_FOR_BUILD)`, `cc_for_build` or similar variable names
in upstream build systems for that compiler).

As an example:

    # TODO: Not the best example (`create-symlink` is an unlikely use-case for this condition)
    packages:
        foo:
            transformations:
            - create-symlink
                  path: sbin/agetty
                  target: /sbin/getty
                  # Only for native builds or when we can transparently run a compiled
                  when: can-execute-compiled-binaries

The `can-execute-compiled-binaries` condition is specified as a string.

## Cross-Compiling condition `cross-compiling` (string)

The `cross-compiling` condition is used to determine if the current build is performing a cross
build (i.e., `DEB_BUILD_GNU_TYPE` != `DEB_HOST_GNU_TYPE`). Often this has consequences for what
is possible to do.

Note if you specifically want to know:

 * whether build-time tests should be run, then please use the `run-build-time-tests` condition.
 * whether compiled binaries can be run as if it was a native binary, please use the
   `can-execute-compiled-binaries` condition instead.  That condition accounts for cross-building
   in its evaluation.

As an example:

    # TODO: Not the best example (`create-symlink` is an unlikely use-case for this condition)
    packages:
        foo:
            transformations:
            - create-symlink
                  path: sbin/agetty
                  target: /sbin/getty
                  when: cross-compiling

The `cross-compiling` condition is specified as a string.

## Whether build time tests should be run `run-build-time-tests` (string)

The `run-build-time-tests` condition is used to determine whether (build time) tests should
be run for this build.  This condition roughly translates into whether `nocheck` is present
in `DEB_BUILD_OPTIONS`.

In general, the manifest *should not* prevent build time tests from being run during cross-builds.

As an example:

    # TODO: Not the best example (`create-symlink` is an unlikely use-case for this condition)
    packages:
        foo:
            transformations:
            - create-symlink
                  path: sbin/agetty
                  target: /sbin/getty
                  when: run-build-time-tests

The `run-build-time-tests` condition is specified as a string.

## Negated condition `not` (mapping)

It is possible to negate a condition via the `not` condition.

As an example:

    packages:
        util-linux:
            transformations:
            - create-symlink
                  path: sbin/getty
                  target: /sbin/agetty
                  when:
                      # On Hurd, the package "hurd" ships "sbin/getty".
                      # This example happens to also be an alternative to `arch-marches: '!hurd-any`
                      not:
                          arch-matches: 'hurd-any'

The `not` condition is specified as a mapping, where the key is `not` and the
value is a nested condition.

## All or any of a list of conditions `all-of`/`any-of` (list)

It is possible to aggregate conditions using the `all-of` or `any-of` condition. This provide
`X and Y` and `X or Y` semantics (respectively).

As an example:

    packages:
        util-linux:
            transformations:
                - create-symlink
                  path: sbin/getty
                  target: /sbin/agetty
                  when:
                      # Only ship getty on linux except for s390(x)
                      all-of:
                          - arch-matches: 'linux-any'
                          - arch-matches: '!s390 !s390x'

The `all-of` and `any-of` conditions are specified as lists, where each entry is a nested condition.
The lists need at least 2 entries as with fewer entries the `all-of` and `any-of` conditions are
redundant.

# Packager provided definitions

For more complex manifests or packages, it is possible define some common attributes for reuse.

## Manifest Variables (`variables`)

It is possible to provide custom manifest variables via the `variables` attribute.  An example:

    manifest-version: '0.1'
    definitions:
      variables:
        LIBPATH: "/usr/lib/{{DEB_HOST_MULTIARCH}}"
        SONAME: "1"
    installations:
      - install:
           source: build/libfoo.so.{{SONAME}}*
           # The quotes here is for the YAML parser's sake.
           dest-dir: "{{LIBPATH}}"
           into: libfoo{{SONAME}}

The value of the `variables` key must be a mapping, where each key is a new variable name and
the related value is the value of said key. The keys must be valid variable name and not shadow
existing variables (that is, variables such as `PACKAGE` and `DEB_HOST_MULTIARCH` *cannot* be
redefined). The value for each variable *can* refer to *existing* variables as seen in the
example above.

As usual, `debputy` will insist that all declared variables must be used.

Limitations:
 * When declaring variables that depends on another variable declared in the manifest, the
   order is important. The variables are resolved from top to bottom.
 * When a manifest variable depends on another manifest variable, the existing variable is
   currently always resolved in source context. As a consequence, some variables such as
  `{{PACKAGE}}` cannot be used when defining a variable. This restriction may be
   lifted in the future.

# Installations

For source packages building a single binary, the `dh_auto_install` from debhelper will default to
providing everything from upstream's install in the binary package.  The `debputy` tool matches this
behaviour and accordingly, the `installations` feature is only relevant in this case when you need to
manually specify something upstream's install did not cover.

For sources, that build multiple binaries, where `dh_auto_install` does not detect anything to install,
or when `dh_auto_install --destdir debian/tmp` is used, the `installations` section of the manifest is
used to declare what goes into which binary package. An example:

    installations:
      - install:
            sources: "usr/bin/foo"
            into: foo
      - install:
            sources: "usr/*"
            into: foo-extra


All installation rules are processed in order (top to bottom).  Once a path has been matched, it can
no longer be matched by future rules.  In the above example, then `usr/bin/foo` would be in the `foo`
package while everything in `usr` *except* `usr/bin/foo` would be in `foo-extra`.  If these had been
ordered in reverse, the `usr/bin/foo` rule would not have matched anything and caused `debputy`
to reject the input as an error on that basis.  This behaviour is similar to "DEP-5" copyright files,
except the order is reversed ("DEP-5" uses "last match wins", where here we are doing "first match wins")

In the rare case that some path need to be installed into two packages at the same time, then this is
generally done by changing `into` into a list of packages.

All installations are currently run in *source* package context.  This implies that:

  1) No package specific substitutions are available. Notably `{{PACKAGE}}` cannot be resolved.
  2) All conditions are evaluated in source context.  For 99.9% of users, this makes no difference,
     but there is a cross-build feature that changes the "per package" architecture which is affected.

This is a limitation that should be fixed in `debputy`.

**Attention debhelper users**: Note the difference between `dh_install` (etc.) vs. `debputy` on
overlapping matches for installation.

## Install rule search directories

Most install rules apply their patterns against search directories such as `debian/tmp` by default.

The default search directory order (highest priority first) is:

 1) The upstream install directory (usually, `debian/tmp`)
 2) The source package root directory (`.`)

Each search directory is tried in order.  When a pattern matches an entry in a search directory (even
if that entry is reserved by another package), further search directories will *not* be tried. As an example,
consider the pattern `usr/bin/foo*` and the files:

  `SOURCE_ROOT/debian/tmp/usr/bin/foo.sh`
  `SOURCE_ROOT/usr/bin/foo.pl`

Here the pattern will only match `SOURCE_ROOT/debian/tmp/usr/bin/foo.sh` and not `SOURCE_ROOT/usr/bin/foo.pl`.

## Automatic discard rules

The `debputy` framework provides some built-in discard rules that are applied by default during installation
time.  These are always active and implicit, but can be overwritten by exact path matches for install rules.

The `debputy` tool itself provides the following discard rules:

 * Discard of `.la` files. Their use is rare but not unheard of. You may need to overwrite this.
 * Discard of python byte code (such as `__pycache__` directories).
 * Discard of editor backup files (such as `*~`, `*.bak`, etc.).
 * Discard of Version control files (such as `.gitignore`, etc.).
 * Discard of GNU info's `dir` (`usr/share/info/dir`) as it causes file conflicts with other packages.
 * Discard of `DEBIAN` directory.

Note: Third-party plugins may provide additional automatic discard rules. Please use
`debputy plugin list automatic-discard-rules` to see all known automatic discard rules.

If you find yourself needing a particular path installed that has been discarded by default, you can overrule
the default discard by spelling out the path. As an example, if you needed to install a `libfoo.la` file,
you could do:

    installations:
      - install:
            sources:
            # By-pass automatic discard of `libfoo.la` - no globs *cannot* be used!
             - "usr/lib/libfoo.la"
             - "usr/lib/libfoo*.so*"
            into: libfoo1

## Generic install (`install`)

The generic `install` rule can be used to install arbitrary paths into packages and is *similar* to how
`dh_install` from debhelper works.  It is a two "primary" uses.

  1) The classic "install into directory" similar to the standard `dh_install`
  2) The "install as" similar to `dh-exec`'s `foo => bar` feature.

Examples:

    installations:
      - install:
            source: "usr/bin/tool-a"
            dest-dir: "usr/bin"
            into: foo
      - install:
            source: "usr/bin/tool-b"
            # Implicit dest-dir: "usr/bin
            into: foo-extra
      - install:
            source: "usr/bin/tool-c.sh"
            # Rename as a part of installing.
            as: "usr/bin/tool-c"
            into: foo-util


The value for `install` is a string, a list of strings or mapping. When it is a mapping, the
mapping has the following key/value pairs:

 * `source` or `sources` (required): A path match (`source`) or a list of path matches (`sources`) defining
   the source path(s) to be installed. The path match(es) can use globs.  Each match is tried against default
   search directories.
    - When a symlink is matched, then the symlink (not its target) is installed as-is.  When a directory is
     matched, then the directory is installed along with all the contents that have not already been installed
     somewhere.

 * `into` (conditional): Either a package name or a list of package names for which these paths should be
   installed. This key is conditional on whether there are multiple binary packages listed in
   `debian/control`.  When there is only one binary package, then that binary is the default for `into`.
   Otherwise, the key is required.

 * `dest-dir` (optional): A path defining the destination *directory*.  The value *cannot* use globs, but can
   use substitution.  If neither `as` nor `dest-dir` is given, then `dest-dir` defaults to the directory name
   of the `source`.

 * `as` (optional): A path defining the path to install the source as. This is a full path.  This option is
   mutually exclusive with `dest-dir` and `sources` (but not `source`).  When `as` is given, then `source` must
   match exactly one "not yet matched" path.

 * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules).

When the input is a string or a list of string, then that value is used as shorthand for `source`
or `sources` (respectively).  This form can only be used when `into` is not required.


## Install documentation (`install-docs`)

This install rule resemble that of `dh_installdocs`.  It is a shorthand over the generic
`install` rule with the following key features:

 1) The default `dest-dir` is to use the package's documentation directory (usually something
    like `/usr/share/doc/{{PACKAGE}}`, though it respects the "main documentation package"
    recommendation from Debian Policy). The `dest-dir` or `as` can be set in case the
    documentation in question goes into another directory or with a concrete path.  In this
    case, it is still "better" than `install` due to the remaining benefits.
 2) The rule comes with pre-defined conditional logic for skipping the rule under
    `DEB_BUILD_OPTIONS=nodoc`, so you do not have to write that conditional yourself.
 3) The `into` parameter can be omitted as long as there is a exactly one non-`udeb` package
    listed in `debian/control`.

With these two things in mind, it behaves just like the `install` rule.

Note: It is often worth considering to use a more specialized version of the `install-docs`
rule when one such is available. If you are looking to install an example or a manpage,
consider whether `install-examples` or `install-man` might be a better fit for your
use-case.

Examples:

    installations:
      - install-docs:
            sources:
              - "docs/README.some-topic.md"
              - "docs/README.another-topic.md"
            into: foo


The value for `install-docs` is a string, a list of strings or mapping. When it is
a mapping, the mapping has the following key/value pairs:

 * `source` or `sources` (required): A path match (`source`) or a list of path matches (`sources`) defining
   the source path(s) to be installed. The path match(es) can use globs.  Each match is tried against default
   search directories.
     - When a symlink is matched, then the symlink (not its target) is installed as-is.  When a directory is
       matched, then the directory is installed along with all the contents that have not already been installed
       somewhere.

     - **CAVEAT**: Specifying `source: docs` where `docs` resolves to a directory for `install-docs`
       will give you an `docs/docs` directory in the package, which is rarely what you want. Often, you
       can solve this by using `docs/*` instead.

 * `dest-dir` (optional): A path defining the destination *directory*.  The value *cannot* use globs, but can
   use substitution.  If neither `as` nor `dest-dir` is given, then `dest-dir` defaults to the relevant package
   documentation directory (a la `/usr/share/doc/{{PACKAGE}}`).

 * `into` (conditional): Either a package name or a list of package names for which these paths should be
   installed as docs.  This key is conditional on whether there are multiple (non-`udeb`) binary
   packages  listed in `debian/control`.  When there is only one (non-`udeb`) binary package, then that binary
   is the default for `into`. Otherwise, the key is required.

 * `as` (optional): A path defining the path to install the source as. This is a full path.  This option is
   mutually exclusive with `dest-dir` and `sources` (but not `source`).  When `as` is given, then `source` must
   match exactly one "not yet matched" path.

 * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules).  This condition will
   be combined with the built-in condition provided by these rules (rather than replacing it).

When the input is a string or a list of string, then that value is used as shorthand for `source`
or `sources` (respectively).  This form can only be used when `into` is not required.

Note: While the canonical name for this rule use plural, the `install-doc` variant is accepted as
alternative name.

## Install examples (`install-examples`)

This install rule resemble that of `dh_installexamples`.  It is a shorthand over the generic `
install` rule with the following key features:

 1) It pre-defines the `dest-dir` that respects the "main documentation package" recommendation from
    Debian Policy. The `install-examples` will use the `examples` subdir for the package documentation
    dir.
 2) The rule comes with pre-defined conditional logic for skipping the rule under
    `DEB_BUILD_OPTIONS=nodoc`, so you do not have to write that conditional yourself.
 3) The `into` parameter can be omitted as long as there is a exactly one non-`udeb` package
    listed in `debian/control`.

With these two things in mind, it behaves just like the `install` rule.

Examples:

    installations:
      - install-examples:
            source: "examples/*"
            into: foo


The value for `install-examples` is a string, a list of strings or mapping. When it is
a mapping, the mapping has the following key/value pairs:

* `source` or `sources` (required): A path match (`source`) or a list of path matches (`sources`) defining
  the source path(s) to be installed. The path match(es) can use globs.  Each match is tried against default
  search directories.
    - When a symlink is matched, then the symlink (not its target) is installed as-is.  When a directory is
      matched, then the directory is installed along with all the contents that have not already been installed
      somewhere.

    - **CAVEAT**: Specifying `source: examples` where `examples` resolves to a directory for `install-examples`
      will give you an `examples/examples` directory in the package, which is rarely what you want. Often, you
      can solve this by using `examples/*` instead.

* `into` (conditional): Either a package name or a list of package names for which these paths should be
  installed as examples.  This key is conditional on whether there are multiple (non-`udeb`) binary
  packages  listed in `debian/control`.  When there is only one (non-`udeb`) binary package, then that binary
  is the default for `into`. Otherwise, the key is required.


* `when` (optional): A condition as defined in [Conditional rules](#conditional-rules).  This condition will
  be combined with the built-in condition provided by these rules (rather than replacing it).

When the input is a string or a list of string, then that value is used as shorthand for `source`
or `sources` (respectively).  This form can only be used when `into` is not required.

Note: While the canonical name for this rule use plural, the `install-example` variant is accepted as
alternative name.

## Install manpages (`install-man`)

Install rule for installing manpages similar to `dh_installman`. It is a shorthand over the generic
`install` rule with the following key features:

 1) The rule can only match files (notably, symlinks cannot be matched by this rule).
 2) The `dest-dir` is computed per source file based on the manpage's section and language.
 3) The `into` parameter can be omitted as long as there is a exactly one non-`udeb` package
    listed in `debian/control`.
 4) The rule comes with manpage specific attributes such as `language` and `section` for when the
    auto-detection is insufficient.
 5) The rule comes with pre-defined conditional logic for skipping the rule under `DEB_BUILD_OPTIONS=nodoc`,
    so you do not have to write that conditional yourself.

With these things in mind, the rule behaves similar to the `install` rule.

Examples:

    installations:
      - install-man:
            source: "man/foo.1"
            into: foo
      - install-man:
            source: "man/foo.de.1"
            language: derive-from-basename
            into: foo


The value for `install-man` is a string, a list of strings or mapping. When it is a mapping, the mapping
has the following key/value pairs:

 * `source` or `sources` (required): A path match (`source`) or a list of path matches (`sources`) defining
   the source path(s) to be installed. The path match(es) can use globs.  Each match is tried against default
   search directories. Only files can be matched.

 * `into` (conditional): Either a package name or a list of package names for which these paths should be
   installed as manpages.  This key is conditional on whether there are multiple (non-`udeb`) binary
   packages  listed in `debian/control`.  When there is only one (non-`udeb`) binary package, then that binary
   is the default for `into`. Otherwise, the key is required.

 * `section` (optional): If provided, it must be an integer between 1 and 9 (both inclusive), defining the
    section the manpages belong overriding any auto-detection that `debputy` would have performed.

 * `language` (optional): If provided, it must be either a 2 letter language code (such as `de`), a 5 letter
   language + dialect code (such as `pt_BR`), or one of the special keywords `C`, `derive-from-path`, or
   `derive-from-basename`.  The default is `derive-from-path`.
   - When `language` is `C`, then the manpages are assumed to be "untranslated".
   - When `language` is a language code (with or without dialect), then all manpages matched will be assumed
     to be translated to that concrete language / dialect.
   - When `language` is `derive-from-path`, then `debputy` attempts to derive the language from the path
     (`man/<language>/man<section>`).  This matches the default of `dh_installman`. When no language can
     be found for a given source, `debputy` behaves like language was `C`.
   - When `language` is `derive-from-basename`, then `debputy` attempts to derive the language from the
     basename (`foo.<language>.1`) similar to `dh_installman` previous default.  When no language can
     be found for a given source, `debputy` behaves like language was `C`.  Note this is prone to
     false positives where `.pl`, `.so` or similar two-letter extensions gets mistaken for a language
     code (`.pl` can both be "Polish" or "Perl Script", `.so` can both be "Somali" and "Shared Object"
     documentation).  In this configuration, such extensions are always assumed to be a language.

 * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules).  This condition will
   be combined with the built-in condition provided by these rules (rather than replacing it).


When the input is a string or a list of string, then that value is used as shorthand for `source`
or `sources` (respectively).  This form can only be used when `into` is not required.

Comparison with debhelper: The `dh_installman` uses `derive-from-path` as default and then falls back
to `derive-from-basename`.  The `debputy` tool does *not* feature the same fallback logic.  If you want
the `derive-from-basename` with all of its false-positives, you have to explicitly request it.


## Discard (or exclude) upstream provided paths (`discard`)

When installing paths from `debian/tmp` into packages, it might be useful to ignore some paths that you never
need installed.  This can be done with the `discard` rule.


The value for `discard` is a string, a list of strings or mapping. When it is a mapping, the mapping
has the following key/value pairs:

 * `path` or `paths` (required): A path match (`path`) or a list of path matches (`paths`) defining the
   source path(s) that should not be installed anywhere. The path match(es) can use globs.
    - When a symlink is matched, then the symlink (not its target) is discarded as-is.  When a directory is
     matched, then the directory is discarded along with all the contents that have not already been installed
     somewhere.

 * `search-dir` or `search-dirs` (optional): A path (`search-dir`) or a list to paths (`search-dirs`) that
    defines which search directories apply to. This attribute is primarily useful for source packages that
    uses [per package search dirs](#custom-installation-time-search-directories-installation-search-dirs),
    and you want to restrict a discard rule to a subset of the relevant search dirs. Note all listed
    search directories must be either an explicit search requested by the packager or a search directory
    that `debputy` provided automatically (such as `debian/tmp`). Listing other paths will make `debputy`
    report an error.

    - Note that the `path` or `paths` must match at least one entry in any of the search directories unless
      *none* of the search directories exist (or the condition in `required-when` evaluates to false). When
      none of the search directories exist, the discard rule is silently skipped. This special-case enables
      you to have discard rules only applicable to certain builds that are only performed conditionally.

 * `required-when` (optional): A condition as defined in [Conditional rules](#conditional-rules). The discard
   rule is always applied. When the conditional is present and evaluates to false, the discard rule can
   silently match nothing. When the condition is absent, *or* it evaluates to true, then each pattern
   provided must match at least one path.

When the input is a string or a list of string, then that value is used as shorthand for `path`
or `paths` (respectively).

Once a path is discarded, it cannot be matched by any other install rules.  A path that is discarded, is
considered handled when `debputy` checks for paths you might have forgotten to install.  The `discard`
feature is therefore *also* replaces the `debian/not-installed` file used by `debhelper` and `cdbs`.

Note: A discard rule applies to *all* relevant search directories at the same time (including the source
root directory) unlike other install rules that only applies to the first search directory with a *match*.
This is to match the intuition that if you discard something, you want it gone no matter which search
directory it happened to be in.

## Multi destination install (`multi-dest-install`)

Special use install rule for installing the same source multiple times into the same package.

It works similar to the `install` rule except:

 1) `dest-dir` is renamed to `dest-dirs` and is conditionally mandatory (either `as` or `dest-dirs`
    must be provided).
 2) Both `as` and `dest-dirs` are now list of paths must have at least two paths when provided.

Please see `debputy plugin show pluggable-manifest-rules multi-dest-install` for the full documentation.

# Binary package rules

Inside the manifest, the `packages` mapping can be used to define requests for the binary packages
you want `debputy` to produce.  Each key inside `packages` must be the name of a binary package
defined in `debian/control`.  The value is a dictionary defining which features that `debputy`
should apply to that binary package.  An example could be:


    packages:
        foo:
            transformations:
                - create-symlink:
                      path: usr/share/foo/my-first-symlink
                      target: /usr/share/bar/symlink-target
                - create-symlink:
                      path: usr/lib/{{DEB_HOST_MULTIARCH}}/my-second-symlink
                      target: /usr/lib/{{DEB_HOST_MULTIARCH}}/baz/symlink-target
        bar:
            transformations:
            - create-directories:
               - some/empty/directory.d
               - another/empty/integration-point.d
            - create-directories:
                 path: a/third-empty/directory.d
                 owner: www-data
                 group: www-data

In this case, `debputy` will create some symlinks inside the `foo` package and some directories for
the `bar` package.  The following subsections define the keys you can use under each binary package.


## Transformations (`packages.{{PACKAGE}}.transformations`)

You can define a `transformations` under the package definition, which is a list a transformation
rules.  An example:

    packages:
        foo:
            transformations:
              - remove: 'usr/share/doc/{{PACKAGE}}/INSTALL.md'
              - move:
                    source: bar/*
                    target: foo/


Transformations are ordered and are applied in the listed order.  A path can be matched by multiple
transformations; how that plays out depends on which transformations are applied and in which order.
A quick summary:

 - Transformations that modify the file system layout affect how path matches in later transformations.
   As an example, `move` and `remove` transformations affects what globs and path matches expand to in
   later transformation rules.

 - For other transformations generally the latter transformation overrules the earlier one, when they
   overlap or conflict.

### Remove transformation rule (`remove`)

The remove transformation rule is mostly only useful for single binary source packages, where
everything from upstream's build system is installed automatically into the package.  In those case,
you might find yourself with some files that are _not_ relevant for the Debian package (but would be
relevant for other distros or for non-distro local builds).  Common examples include `INSTALL` files
or `LICENSE` files (when they are just a subset of `debian/copyright`).

In the manifest, you can ask `debputy` to remove paths from the debian package by using the `remove`
transformation rule.  An example being:

    packages:
        foo:
            transformations:
              - remove: 'usr/share/doc/{{PACKAGE}}/INSTALL.md'


The value for `remove` is a string, a list of strings or mapping. When it is a mapping, the mapping
has the following key/value pairs:

 * `path` or `paths` (required): A path match (`path`) or a list of path matches (`paths`) defining the
   path(s) inside the package that should be removed. The path match(es) can use globs.
    - When a symlink is matched, then the symlink (not its target) is removed as-is.  When a directory is
      matched, then the directory is removed along with all the contents.

 * `keep-empty-parent-dirs` (optional): A boolean determining whether to prune parent directories that become
   empty as a consequence of this rule.  When provided and `true`, this rule will leave empty directories
   behind. Otherwise, if this rule causes a directory to become empty that directory will be removed.

 * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules).  This condition will
   be combined with the built-in condition provided by these rules (rather than replacing it).

When the input is a string or a list of string, then that value is used as shorthand for `path`
or `paths` (respectively).

Note that `remove` removes paths from future glob matches and transformation rules.

This rule behaves roughly like the following shell snippet when applied:

```shell
set -e
for p in ${paths}; do
  rm -fr "${p}"
  if [ "${keep_empty_parent_dirs}" != "true" ]; then
    rmdir --ignore-fail-on-non-empty --parents "$(dirname "${p}")"
  fi
done
```

### Move transformation rule (`move`)

The move transformation rule is mostly only useful for single binary source packages, where
everything from upstream's build system is installed automatically into the package.  In those case,
you might find yourself with some files that need to be renamed to match Debian specific requirements.

This can be done with the `move` transformation rule, which is a rough emulation of the `mv` command
line tool.  An example being:

    packages:
        foo:
            transformations:
              - move:
                    source: bar/*
                    target: foo/
              # Note this example leaves bar/ as an empty directory. Real world usage would probably
              # follow this with an remove rule for bar/ to avoid an empty directory

The value for `move` is a mapping with the following key/value pairs:

 * `source` (required): A path match defining the source path(s) to be renamed.  The value can use globs and
   substitutions.

 * `target` (required): A path defining the target path.  The value *cannot* use globs, but can use substitution.
   If the target ends with a literal `/` (prior to substitution), the target will *always* be a directory.

 * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules)

There are two basic cases:
 1. If `source` match exactly one path and `target` is not a directory (including it does not exist),
    then `target` is removed (if it existed) and `source` is renamed to `target`.
    - If the `target` is a directory, and you intentionally want to replace it with a non-directory, please
      add an explicit `remove` transformation for the directory prior to this transformation.

 2. If `source` match more than one path *or* `target` is a directory (or is specified with a trailing slash),
    then all matching sources a moved into `target` retaining their basename.  If `target` already contains
    overlapping basenames with `source`, then the transformation rule with abort with an error if the overlap
    contains a directory. Otherwise, any overlapping paths in `target` will be implicitly removed first.

    - If the replacement of a directory is intentional, please add an explicit `remove` rule for it first.
    - In the case that the `source` glob is exotic enough to match two distinct paths with the same basename,
      then `debputy` will reject the transformation on account of it being ambiguous.

In either case, parent directories of `target` are created as necessary as long as they do not trigger a conflict
(or require traversing symlinks or non-directories).  Additionally, the paths matched by `source` will no longer
match anything (since they are now renamed/relocated), which may affect what path future path matches will apply
to.

Note that like the `mv`-command, while the `source` paths are no longer present, their parent directories
will remain.

### Create symlinks transformation rule (`create-symlink`)

Often, the upstream build system will provide the symlinks for you.  However, in some cases, it is useful for
the packager to define distribution specific symlinks. This can be done via the `create-symlink` transformation
rule.  An example of how to do this is:

    packages:
        foo:
            transformations:
            - create-symlink:
                  path: usr/share/foo/my-first-symlink
                  target: /usr/share/bar/symlink-target
            - create-symlink:
                  path: usr/lib/{{DEB_HOST_MULTIARCH}}/my-second-symlink
                  target: /usr/lib/{{DEB_HOST_MULTIARCH}}/baz/symlink-target

The value for the `create-symlink` key is a mapping, which contains the following keys:

 * `path` (required): The path that should be a symlink.  The path may contain substitution variables
   such as `{{DEB_HOST_MULTIARCH}}` but _cannot_ use globs.  Parent directories are implicitly created
   as necessary.
   * Note that if `path` already exists, the behaviour of this transformation depends on the value of
     `replacement-rule`.

 * `target` (required): Where the symlink should point to. The target may contain substitution variables
   such as `{{DEB_HOST_MULTIARCH}}` but _cannot_ use globs.  The link target is _not_ required to exist inside
   the package.
   * The `debputy` tool will normalize the target according to the rules of the Debian Policy.  Use absolute
     or relative target at your own preference.

 * `replacement-rule` (optional): This attribute defines how to handle if `path` already exists. It can be
   set to one of the following values:
   - `error-if-exists`: When `path` already exists, `debputy` will stop with an error.  This is similar to
     `ln -s` semantics.
   - `error-if-directory`: When `path` already exists, **and** it is a directory, `debputy` will stop with an
     error. Otherwise, remove the `path` first and then create the symlink.  This is similar to `ln -sf`
     semantics.
   - `abort-on-non-empty-directory` (default): When `path` already exists, then it will be removed provided
     it is a non-directory **or** an *empty* directory and the symlink will then be created.  If the path is
     a *non-empty* directory, `debputy` will stop with an error.
   - `discard-existing`: When `path` already exists, it will be removed. If the `path` is a directory, all
     its contents will be removed recursively along with the directory. Finally, the symlink is created.
     This is similar to having an explicit `remove` rule just prior to the `create-symlink` that is conditional
     on `path` existing (plus the condition defined in `when` if any).

   Keep in mind, that `replacement-rule` only applies if `path` exists.  If the symlink cannot be created,
   because a part of `path` exist and is *not* a directory, then `create-symlink` will fail regardless of the
   value in `replacement-rule`.

 * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules)

This rule behaves roughly like the following shell snippet when applied:

```shell
set -e
case "${replacement_rule}" in
  error-if-directory)
    F_FLAG="-f"
  ;;
  abort-on-non-empty-directory)
    if [ -d "${path}" ]; then
      rmdir "${path}"
    fi
    F_FLAG="-f"
  ;;
  discard-existing)
    rm -fr "${path}"
  ;;
esac
install -o "root" -g "root" -m "755" -d "$(dirname "${path}")"
ln -s ${F_FLAG} "${target}" "${path}"
```

### Create directories transformation rule (`create-directories`)

NOTE: This transformation is only really needed if you need to create an empty directory somewhere
in your package as an integration point.  All `debputy` transformations will create directories
as required.

In most cases, upstream build systems and `debputy` will create all the relevant directories.  However, in some
rare cases you may want to explicitly define a path to be a directory.  Maybe to silence a linter that is
warning you about a directory being empty, or maybe you need an empty directory that nothing else is creating
for you. This can be done via the `create-directories` transformation rule. An example being:

    packages:
        bar:
            create-directories:
            - some/empty/directory.d
            - another/empty/integration-point.d
            - path: a/third-empty/directory.d
              owner: www-data
              group: www-data

The value for the `create-directories` key is either a string, a list of string or a mapping. When it is a
mapping, the mapping has the following key/value pairs:

 * `path` or `paths` (required): A path (`path`) or a list of path (`paths`) defining the
   path(s) inside the package that should be created as directories. The path(es) _cannot_ use globs
   but can use substitution variables.  Parent directories are implicitly created (with owner `root:root`
   and mode `0755` - only explicitly listed directories are affected by the owner/mode options)

 * `owner` (optional, default `root`): Denotes the owner of the directory (but _not_ what is inside the directory).

 * `group` (optional, default `root`): Denotes the group of the directory (but _not_ what is inside the directory).

 * `mode` (optional, default `"0755"`): Denotes the mode of the directory (but _not_ what is inside the directory).
   Note that numeric mode must always be given as a string (i.e., with quotes).  Symbolic mode can be used as well.
   If symbolic mode uses a relative definition (e.g., `o-rx`), then it is relative to the directory's current mode
   (if it already exists) or `0755` if the directory is created by this transformation.

 * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules)

When the input is a string or a list of string, then that value is used as shorthand for `path`
or `paths` (respectively).

Unless you have a specific need for the mapping form, you are recommended to use the shorthand form of
just listing the directories you want created.


Note that implicitly created directories (by this or other transformations) always have owner `root:root`
and mode `"0755"`.  If you need a directory tree with special ownership/mode, you will have to list all the
directories in that tree explicitly with the relevant attributes OR use `path-metadata` transformation rule
to change their metadata after creation.


This rule behaves roughly like the following shell snippet when applied:

```shell
set -e
for p in ${paths}; do
  install -o "root" -g "root" -m "755" -d "${p}"
  chown "${owner}:${group}" "${p}"
  chmod "${mode}" "${p}"
done
```

### Change path owner/group or mode (`path-metadata`)

The `debputy` normalizes the path metadata (such as ownership and mode) similar to `dh_fixperms`.
For most packages, the default is what you want.  However, in some cases, the package has a special
case or two that `debputy` does not cover.  In that case, you can tell `debputy` to use the metadata
you want by using the `path-metadata` transformation.  An example being:

    packages:
        foo:
            transformations:
              - path-metadata:
                  path: usr/bin/sudo
                  mode: "0755"
              - path-metadata:
                  path: usr/bin/write:
                  group: tty
             - path-metadata:
                  path: /usr/sbin/spine
                  capabilities: cap_net_raw+ep


The value for the `path-metadata` key is a mapping. The mapping has the following key/value pairs:
Each key defines a path or a glob that may


* `path` or `paths` (required): A path match (`path`) or a list of path matches (`paths`) defining the
  path(s) inside the package that should be affected. The path match(es) can use globs and substitution
  variables. Special-rules for matches:
    - Symlinks are never followed and will never be matched by this rule.
    - Directory handling depends on the `recursive` attribute.

 * `owner` (conditional): Denotes the owner of the paths matched by `path` or `paths`. When omitted, no change of
   owner is done.

 * `group` (conditional): Denotes the group of the paths matched by `path` or `paths`. When omitted, no change of
   group is done.

 * `mode` (conditional): Denotes the mode of the paths matched by `path` or `paths`. When omitted, no change in
   mode is done. Note that numeric mode must always be given as a string (i.e., with quotes).  Symbolic mode can
   be used as well. If symbolic mode uses a relative definition (e.g., `o-rx`), then it is relative to the
   directory's current mode.

 * `capabilities` (conditional): Denotes a Linux capability that should be applied to the path. When provided,
   `debputy` will cause the capability to be applied to all *files* denoted by the `path`/`paths` attribute
   on install (via `postinst configure`) provided that `setcap` is installed on the system when the
   `postinst configure` is run.
   - If any non-file paths are matched, the `capabilities` will *not* be applied to those paths.

 * `capability-mode` (optional): Denotes the mode to apply to the path *if* the Linux capability denoted in
   `capabilities` was successfully applied. If omitted, it defaults to `a-s` as generally capabilities are
   used to avoid "setuid"/"setgid" binaries. The `capability-mode` is relative to the *final* path mode
   (the mode of the path in the produced `.deb`). The `capability-mode` attribute cannot be used if
   `capabilities` is omitted.

 * `recursive` (optional, default `false`): When a directory is matched, then the metadata changes are applied
    to the directory itself. When `recursive` is `true`, then the transformation is *also* applied to all paths
    beneath the directory.

 * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules)

At least one of `owner`, `group`, `mode`, or `capabilities` must be provided.

This rule behaves roughly like the following shell snippet when applied:

```shell
MAYBE_R_FLAG=$(${recursive} && printf "%s" "-R")

for p in ${path}; do
  if [ -n "${owner}" ] || [ -n "${group}" ]; then
    chown $MAYBE_R_FLAG "${owner}:${group}" "${p}"
  fi
  if [ -n "${mode}" ]; then
    chmod $MAYBE_R_FLAG "${mode}" "${p}"
  fi
done
```

(except all symlinks will be ignored)

## Service management (`services`)

If you have non-standard requirements for certain services in the package, you can define those via
the `services` attribute.

    packages:
        foo:
            services:
              - service: "foo"
                enable-on-install: false
              - service: "bar"
                on-upgrade: stop-then-start


The `services` attribute must contain a non-empty list, where each element in that list is a mapping.
Each mapping has the following key/value pairs:

 * `service` (required): Name of the service to match. The name is usually the basename of the service file.
   However, aliases can also be used for relevant system managers. When aliases **and** multiple service
   managers are involved, then the rule will apply to all matches. See alias handling below.

   - Note: For systemd, the `.service` suffix can be omitted from name, but other suffixes such as `.timer`
     cannot.

 * `type-of-service` (optional, defaults to `service`): The type of service this rule applies to. To act on a
   `systemd` timer, you would set this to `timer` (etc.). Each service manager defines its own set of types
   of services.

 * `service-scope` (optional, defaults to `system`): The scope of the service. It must be either `system` and
   `user`.
   - Note: The keyword is defined to support `user`, but `debputy` does not support `user` services at the moment
     (the detection logic is missing).

 * `service-manager` or `service-managers` (optional): Which service managers this rule is for. When omitted, all
   service managers with this service will be affected. This can be used to specify separate rules for the same
   service under different service managers.
   - When this attribute is explicitly given, then all the listed service managers must provide at least one
     service matching the definition. In contract, when it is omitted, then all service manager integrations
     are consulted but as long as at least one service is match from any service manager, the rule is accepted.

 * `enable-on-install` (optional): Whether to automatically enable the service on installation. Note: This does
   **not** affect whether the service will be started nor how restarts during upgrades will happen.
   - If omitted, the plugin detecting the service decides the default.

 * `start-on-install` (optional): Whether to automatically start the service on installation. Whether it is
   enabled or how upgrades are handled have separate attributes.
   - If omitted, the plugin detecting the service decides the default.

 * `on-upgrade` (optional): How `debputy` should handle the service during upgrades. The default depends on the
   plugin detecting the service. Valid values are:

   - `do-nothing`: During an upgrade, the package should not attempt to stop, reload or restart the service.
   - `reload`: During an upgrade, prefer reloading the service rather than restarting if possible. Note that
     the result may become `restart` instead if the service manager integration determines that `reload` is
     not supported.
   - `restart`: During an upgrade, `restart` the service post upgrade. The service will be left running during
     the upgrade process.
   - `stop-then-start`: Stop the service before the upgrade, preform the upgrade and then start the service.

### Service managers and aliases

When defining a service rule, you can use any name that any of the relevant service managers would call the
service. As an example, consider a package that has the following services:

 * A `sysvinit` service called `foo`
 * A `systemd` service called `bar.service` with `Alias=foo.service` in its definition.

Here, depending on which service managers are relevant to the rule, you can use different names to match.
When the rule applies to the `systemd` service manager, then either of the following names can be used:

 * `bar.service` (the "canonical" name in the systemd world)
 * `foo.service` (the defined alias)
 * `bar` + `foo` (automatic aliases based on the above)

Now, if rule *also* applies to the `sysvinit` service manager, then any of those 4 names would cause the
rule to apply to both the `systemd` and the `sysvinit` services.

To show concrete examples:

    ...:
            services:
              # Only applies to systemd. Either of the 4 names would have work.
              - service: "foo.service"
                on-upgrade: stop-then-start
                service-manager: systemd

    ...:
            services:
              # Only applies to sysvinit. Must use `foo` since the 3 other names only applies when systemd
              # is involved.
              - service: "foo"
                on-upgrade: stop-then-start
                service-manager: sysvinit

    ...:
            services:
              # Applies to both systemd and sysvinit; this works because the `systemd` service provides an
              # alias for `foo`. If the systemd service did not have that alias, only the `systemd` service
              # would have been matched.
              - service: bar
                enable-on-install: false

## Custom binary version (`binary-version`)

In the *rare* case that you need a binary package to have a custom version, you can use the `binary-version:`
key to describe the desired package version.  An example being:

    packages:
        foo:
            # The foo package needs a different epoch because we took it over from a different
            # source package with higher epoch version
            binary-version: '1:{{DEB_VERSION_UPSTREAM_REVISION}}'

Use this feature sparingly as it is generally not possible to undo as each version must be monotonously
higher than the previous one.  This feature translates into `-v` option for `dpkg-gencontrol`.

The value for the `binary-version` key is a string that defines the binary version.  Generally, you will
want it to contain one of the versioned related substitution variables such as 
`{{DEB_VERSION_UPSTREAM_REVISION}}`.  Otherwise, you will have to remember to bump the version manually
with each upload as versions cannot be reused and the package would not support binNMUs either.


## Remove runtime created paths on purge or post removal (`clean-after-removal`)

For some packages, it is necessary to clean up some run-time created paths. Typical use cases are
deleting log files, cache files, or persistent state. This can be done via the `clean-after-removal`.
An example being:

    packages:
        foo:
            clean-after-removal:
            - /var/log/foo/*.log
            - /var/log/foo/*.log.gz
            - path: /var/log/foo/
              ignore-non-empty-dir: true
            - /etc/non-conffile-configuration.conf
            - path: /var/cache/foo
              recursive: true


The `clean-after-removal` key accepts a list, where each element is either a mapping, a string or a list
of strings. When an element is a mapping, then the following key/value pairs are applicable:

 * `path` or `paths` (required): A path match (`path`) or a list of path matches (`paths`) defining the
   path(s) that should be removed after clean. The path match(es) can use globs and manifest variables.
   Every path matched will by default be removed via `rm -f` or `rmdir` depending on whether the path
   provided ends with a *literal* `/`. Special-rules for matches:
    - Glob is interpreted by the shell, so shell (`/bin/sh`) rules apply to globs rather than
      `debputy`'s glob rules.  As an example, `foo/*` will **not** match `foo/.hidden-file`.
    - `debputy` cannot evaluate whether these paths/globs will match the desired paths (or anything at
      all). Be sure to test the resulting package.
    - When a symlink is matched, it is not followed.
    - Directory handling depends on the `recursive` attribute and whether the pattern ends with a literal
      "/".
    - `debputy` has restrictions on the globs being used to prevent rules that could cause massive damage
      to the system.

 * `recursive` (optional): When `true`, the removal rule will use `rm -fr` rather than `rm -f` or `rmdir`
    meaning any directory matched will be deleted along with all of its contents.

 * `ignore-non-empty-dir` (optional): When `true`, each path must be or match a directory (and as a
   consequence each path must with a literal `/`). The affected directories will be deleted only if they
   are empty. Non-empty directories will be skipped. This option is mutually exclusive with `recursive`.

 * `delete-on` (optional, defaults to `purge`): This attribute defines when the removal happens. It can
   be set to one of the following values:
   - `purge`: The removal happens with the package is being purged. This is the default. At a technical
     level, the removal occurs at `postrm purge`.
   - `removal`: The removal happens immediately after the package has been removed. At a technical level,
     the removal occurs at `postrm remove`.


This feature resembles the concept of `rpm`'s `%ghost` files.

## Custom installation time search directories (`installation-search-dirs`)

For source packages that does multiple build, it can be an advantage to provide a custom list of
installation-time search directories. This can be done via the `installation-search-dirs` key. A common
example is building  the source twice with different optimization and feature settings where the second
build is for the `debian-installer` (in the form of a `udeb` package). A sample manifest snippet could
look something like:

    installations:
    - install:
        # Because of the search order (see below), `foo` installs `debian/tmp/usr/bin/tool`,
        # while `foo-udeb` installs `debian/tmp-udeb/usr/bin/tool` (assuming both paths are
        # available). Note the rule can be split into two with the same effect if that aids
        # readability or understanding.
        source: usr/bin/tool
        into:
          - foo
          - foo-udeb
    packages:
        foo-udeb:
            installation-search-dirs:
            - debian/tmp-udeb


The `installation-search-dirs` key accepts a list, where each element is a path (str) relative from the
source root to the directory that should be used as a search directory (absolute paths are still interpreted
as relative to the source root).  This list should contain all search directories that should be applicable
for this package (except the source root itself, which is always appended after the provided list). If the
key is omitted, then `debputy` will provide a default  search order (In the `dh` integration, the default
is the directory `debian/tmp`).

If a non-existing or non-directory path is listed, then it will be skipped (info-level note). If the path
exists and is a directory, it will also be checked for "not-installed" paths.


[reference documentation]: https://documentation.divio.com/reference/