diff options
Diffstat (limited to 'MANIFEST-FORMAT.md')
| -rw-r--r-- | MANIFEST-FORMAT.md | 1414 |
1 files changed, 1414 insertions, 0 deletions
diff --git a/MANIFEST-FORMAT.md b/MANIFEST-FORMAT.md new file mode 100644 index 0000000..13bedf7 --- /dev/null +++ b/MANIFEST-FORMAT.md @@ -0,0 +1,1414 @@ +# The debputy manifest format + +_This is [reference guide] and is primarily useful if you have an idea of what you are looking for._ +_If you are new to `debputy`, maybe you want to read [GETTING-STARTED-WITH-dh-debputy.md](GETTING-STARTED-WITH-dh-debputy.md) first._ + +<!-- To writers and reviewers: Check the documentation against https://documentation.divio.com/ --> + + +## Prerequisites + +This guide assumes familiarity with Debian packaging in general. Notably, you should understand +the different between a (Debian) source package and a (Debian) binary package (e.g., `.deb`) plus +how these concepts relates to `debian/control` (the source control file). + +Additionally, the reader is expected to have an understanding of globs and substitution variables. + +It is probably very useful to have an understanding on how a binary package is +assembled. While `debputy` handles the assembly for you, this document will not go in details +with this. Prior experience with `debhelper` (notably the `dh`-style `debian/rules`) may be +useful but should not be a strict requirement. + + +# The basic manifest + +The manifest is a YAML document with a dictionary at the top level layer. As usual with YAML +versions, you can choose to leave it implicit. All manifests must include a manifest version, which +will enable the format to change over time. For now, there is only one version (`0.1`) and you have +to include the line: + + manifest-version: "0.1" + + +On its own, the manifest containing only `manifest-version: "..."` will not do anything. So if you +end up only having the `manifest-version` key in the manifest, you can just remove the manifest and +rely entirely on the built-in rules. + + +# Path matching rules + +Most of the manifest is about declaring rules for a given path such as "foo must be a symlink" +or "bar must be owned by root:tty and have mode 02755". + +The manifest contains the following types of matches: + + 1) Exact path matches. These specify the path inside the debian package exactly without any + form of wildcards (e.g., `*` or `?`). However, they can use substitution variables. + Examples include: + * `usr/bin/sudo` + * `usr/lib/{{DEB_HOST_MULTIARCH}}/libfoo.so` + + Having a leading `/` is optional. I.e. `/usr/bin/sudo` and `usr/bin/sudo` are considered the + same path. + + 2) Glob based path matches. These specify a rule that match any path that matches a given + glob. These rules must contain a glob character (e.g., `*`) _and_ a `/`. Examples include: + + * `foo/*` + * `foo/*.txt` + * `usr/lib/{{DEB_HOST_MULTIARCH}}/lib*.so*` + + Note that if the glob does not contain a `/`, then it falls into the Basename glob rule + below. + + 3) Basename glob matches. These specify a rule that match any path where the basename matches + a given glob. They must contain a glob character (e.g., `*`) but either must not have a + `/` in them at all or start with `**/` and not have any `/` from there. + Examples include: + + * `"*.la"` + * `"**/*.md"` + * `"**/LICENSE"` + + The examples use explicit quoting because YAML often interprets `*` as an anchor rule in the + places where you are likely to use this kind of glob. The use of the `**/`-prefix is + optional when the basename is a glob. If you wanted to match all paths with the basename + of exactly `LICENSE`, then you have to use the prefix (that is, use `**/LICENSE`) as `LICENSE` + would be interpreted as an exact match for a top-level path. + +However, there are also cases where these matching rules can cause conflicts. This is covered +in the [Conflict resolution](#conflict-resolution) section below. + + +## Limitations on debputy path rules + +Path rules: + + 1) Must match relatively to the package root. + 2) Never resolves a symlink. + * If `bin/ls` is a symlink, then `bin/ls` matches the symlink (and never the target) + * If `bin` is a symlink, then `bin/ls` (or `bin/*`) would fail to match, because + matching would require resolving the symlink. + +These limitations are in place because of implementation details in `debputy`. + + +## Conflict resolution + +The manifest can contain seemly mutually exclusive rules. As an example, if you ask for +`foo/symlink` to be a symlink but also state that you want to remove `foo` entirely +from the debian package then the manifest now has two mutually exclusive requests. + +To resolve these problems, `debputy` relies on the following rules for conflict resolutions: + + 1. Requests are loosely-speaking ranked and applied from top to bottom: + 1. `installations` (ordered; "first match wins") + 2. `transformations` (ordered; "all matching rules applies") + + The "first match wins" rule is exactly what it says. + + The "all matching rules applies" rule means that each rule is applied in order. Often + this behaves like a simple case of either "first match wins" or "last match wins" (depending + on the context). + + Note for transformation rules, an early rule can change the file system layout, which will + affect whether a later rule matches. This is similar to how shell globs commands work: + + $ rm usr/lib/libfoo.la + $ chmod 0644 usr/lib/* + + Here the glob used with `chmod` will not match `usr/lib/libfoo.la` because it was removed. + As noted, a similar logic applies to transformation rules. + + 2. The ordered rules (such as every transformation inside `transformations`) are applied in the + listed order (top to bottom). Due to the explicit ordering, such rules generally do not trigger + other conflict resolution rules. + + Note keep in mind that all rules must in general apply to at least one path. + +Note that `debputy` will in some cases enforce that rules are not redundant. This feature is currently +only fully implemented for `installations`. + +## All definitions must be used + +Whenever you define a request or rule in the manifest, `debputy` will insist on it being used +at least once. The exception to this rule being conditional rules where the condition +evaluates to `false` (in which case the rule never used and does not trigger an error). + +This is useful for several reasons: + + 1. The definition may have been shadowed by another rule and would have been ignored otherwise + 2. The definition may no longer be useful, but its present might confuse a future reader of + the manifest. + +In all cases, `debputy` will tell you if a definition was unused and where you can find that +definition. + + +## debputy globs + +In general, the following rules applies to globs in `debputy`. + + * The `*` match 0 or more times any characters except `/`. + * The `?` match exactly one character except `/`. + * The glob `foo/*` matches _everything_ inside `foo/` including hidden files (i.e., paths starting + with `.`) unlike `bash`/`sh` globs. However, `foo/*` does _not_ match `foo/` itself (this latter + part matches `bash`/`sh` behaviour and should be unsurprising). + * For the special-cases where `**` is supported, then `**` matches zero or more levels of directories. + This means that `foo/**/*` match any path beneath `foo` (but still not `foo`). This is mostly relevant + for built-in path matches as it is currently not possible to define `foo/**/...` patterns in the manifest. + +Note that individual rules (such as `clean-after-removal`) may impose special cases to how globs +work. The rules will explicitly list if they divert from the above listed glob rules. + +# Rules for substituting manifest variables + +The `debputy` tool supports substitution in various places (usually paths) via the following +rules. That means: + + 1) All substitutions must start with `{{` and end with `}}`. The part inbetween is + the `MANIFEST_VARIABLE` and must match the regular expression `[A-Za-z0-9][-_:0-9A-Za-z]*`. + Note that you can use space around the variable name if you feel that increases readability. + (That is, `{{ FOO }}` can be used as an alternative to `{{FOO}}`). + 2) The `MANIFEST_VARIABLE` will be result from a set of built-in variables and the variables from + `dpkg-architecture`. + 3) You can use `{{token:DOUBLE_OPEN_CURLY_BRACE}}` and `{{token:DOUBLE_CLOSE_CURLY_BRACE}}` (built-in + variables) if you want a literal `{{` or `}}` would otherwise have triggered an undesired expansion. + 4) All `{{MANIFEST_VARIABLE}}` must refer to a defined variable. + - You can see the full list of `debputy` and plugin provided manifest variables via: + `debputy plugin list manifest-variables`. The manifest itself can declare its own variables + beyond that list. Please refer to the [Manifest Variables](#manifest-variables-variables) + section manifest variables declared inside the manifest. + 5) There are no expression syntax inside the `{{ ... }}` (unlike jinja2 and other template languages). + This rule may be changed in the future (with a new manifest version). + +Keep in mind that substitution _cannot_ be used everywhere. There are specific places where +it can be used. Also, substitution _cannot be used_ to introduce globs into paths. When a +substitution occurs inside a path all characters inserted are treated as literal characters. + +Note: While manifest variables can be substituted into various places in the `debputy` manifest, they +are distinct from `dpkg`'s "substvars" (`man 5 deb-substvars`) that are used in the `debian/control` +file. + +## Built-in or common substitution variables + + * `{{token:NEWLINE}}` or `{{token:NL}}` expands to a literal newline (LF) `\n`. + * `{{token:TAB}}` expands to a literal tab `\t`. + * `{{token:OPEN_CURLY_BRACE}}` / `{{token:CLOSE_CURLY_BRACE}}` expand to `{` / `}` + * `{{token:DOUBLE_OPEN_CURLY_BRACE}}` / `{{token:DOUBLE_CLOSE_CURLY_BRACE}}` expands to `{{` / `}}`. + * `{{PACKAGE}}` expands to the binary package name of the current binary package. This substitution + only works/applies when the substitution occurs in the context of a concrete binary package. + * Plus any of the variables produced by `dpkg-architecture`, such as `{{DEB_HOST_MULTIARCH}}`. + +The following variables from `/usr/share/dpkg/pkg-info.mk` (`dpkg`) are also available: + + * DEB_SOURCE (as `{{DEB_SOURCE}}`) + * DEB_VERSION (as `{{DEB_VERSION}}`) + * DEB_VERSION_EPOCH_UPSTREAM (as `{{DEB_VERSION_EPOCH_UPSTREAM}}`) + * DEB_VERSION_UPSTREAM_REVISION (as `{{DEB_VERSION_UPSTREAM_REVISION}}`) + * DEB_VERSION_UPSTREAM (as `{{DEB_VERSION_UPSTREAM}}`) + * SOURCE_DATE_EPOCH (as `{{SOURCE_DATE_EPOCH}}`) + +These have the same definition as those from the `dpkg` provided makefile. + + +# Restrictions on defining ownership of paths + +In some parts of the manifest, you can specify which user or group should have ownership of +a given path. As an example, you can define a directory to be owned by `root` with group `tty` +(think `chown root:tty <some/path>`). + +Ownership is generally defined via the keys `owner` and `group`. For each of them, you can use +one of the following formats: + + 1) A name (e.g., `owner: root`). + 2) An id (e.g., `owner: 0`). Please avoid using quotes around the ID in YAML as that can + cause `debputy` to read the number as a name. + 3) A name and an id with a colon inbetween (e.g., `owner: "root:0"`). The name must always + come first here. You may have to quote the value to prevent the YAML parser from being + confused. + +All three forms are valid and provide the same result. Unless you have a compelling reason to +pick a particular form, the name-only is recommended for simplicity. Notably, it does not +require your co-maintainer or future you to remember what the IDs mean. + +Regardless of which form you pick: + + 1) The provided owner must be defined by Debian `base-passwd` file, which are the only users guaranteed + to be present on every Debian system. + * Concretely, `debputy` verifies the input against `/usr/share/base-passwd/passwd.master` and + `/usr/share/base-passwd/group.master` (except for `root` / `0` as an optimization). + + 2) If the `name:id` form is used, then the name and the id values must match. I.e., `root:2` would + be invalid as the id for `root` is defined to be `0` in the `base-passwd` data files. + + 3) The `debputy` tool maintains a `deny`-list of owners that it refuses even though `base-passwd` + defines them. As a notable non-exhaustive example, `debputy` considers `nobody` or id `65534` + (the ID of `nobody` / `nogroup`) to be invalid owners. + + +# Conditional rules + +There are cases, where a given rule should only apply in certain cases - such as only when a given +build profile is active (`DEB_BUILD_PROFILES` / `dpkg-buildpackage -P`). For rules that +*support being conditional*, the condition is generally defined via the `when:` key and the condition +is then described beneath the `when:`. + +As an example: + + packages: + util-linux: + transformations: + - create-symlink + path: sbin/agetty + target: /sbin/getty + when: + # On Hurd, the package "hurd" ships "sbin/getty". + arch-matches: '!hurd-any' + + +When the condition under `when:` resolves to `true`, the rule will and must be used. When the +condition resolves to `false`, the rule will not be applied even if it could have been. However, +the rule may still be "partially" evaluated. As an example, for installation rules, the source +patterns will still be evaluated to reserve what it would have matched, so that following rules +behave deterministically regardless of how the condition evaluates. + +Note that conditions are *not* used as a conflict resolution and as such two conditional rules +can still cause conflicts even though their conditions are mutually exclusive. This may be +changed in a later version of `debputy` provided `debputy` can assert the two conditions +are mutually exclusive. + +The `when:` key has either a mapping, a list or a string as value depending on the condition. +Each supported condition is described in the following subsections. + +## Architecture match condition `arch-matches` (mapping) + +Sometimes, a rule needs to be conditional on the architecture. This can be done by using the +`arch-matches` rule. In 99.99% of the cases, `arch-matches` will be form you are looking for +and practically behaves like a comparison against `dpkg-architecture -qDEB_HOST_ARCH`. + +As an example: + + packages: + util-linux: + transformations: + - create-symlink + path: sbin/agetty + target: /sbin/getty + when: + # On Hurd, the package "hurd" ships "sbin/getty". + arch-matches: '!hurd-any' + +The `arch-matches` must be defined as a part of a mapping, where `arch-matches` is the key. The +value must be a string in the form of a space separated list architecture names or architecture +wildcards (same syntax as the architecture restriction in Build-Depends in debian/control except +there is no enclosing `[]` brackets). The names/wildcards can optionally be prefixed by `!` to +negate them. However, either *all* names / wildcards must have negation or *none* of them may +have it. + +For the cross-compiling specialists or curious people: The `arch-matches` rule behaves like a +`package-context-arch-matches` in the context of a binary package and like +`source-context-arch-matches` otherwise. The details of those are covered in their own section. + +## Explicit source or binary package context architecture match condition `source-context-arch-matches`, `package-context-arch-matches` (mapping) + +**These are special-case conditions**. Unless you know that you have a very special-case, +you should probably use `arch-matches` instead. These conditions are aimed at people with +corner-case special architecture needs. It also assumes the reader is familiar with the +`arch-matches` condition. + +To understand these rules, here is a quick primer on `debputy`'s concept of "source context" +vs "(binary) package context" architecture. For a native build, these two contexts are the +same except that in the package context an `Architecture: all` package always resolve to +`all` rather than `DEB_HOST_ARCH`. As a consequence, `debputy` forbids `arch-matches` and +`package-context-arch-matches` in the context of an `Architecture: all` package as a warning +to the packager that condition does not make sense. + +In the very rare case that you need an architecture condition for an `Architecture: all` package, +you can use `source-context-arch-matches`. However, this means your `Architecture: all` package +is not reproducible between different build hosts (which has known to be relevant for some +very special cases). + +Additionally, for the 0.0001% case you are building a cross-compiling compiler (that is, +`DEB_HOST_ARCH != DEB_TARGET_ARCH` and you are working with `gcc` or similar) `debputy` can be +instructed (opt-in) to use `DEB_TARGET_ARCH` rather than `DEB_HOST_ARCH` for certain packages when +evaluating an architecture condition in context of a binary package. This can be useful if the +compiler produces supporting libraries that need to be built for the `DEB_TARGET_ARCH` rather than +the `DEB_HOST_ARCH`. This is where `arch-matches` or `package-context-arch-matches` can differ +subtly from `source-context-arch-matches` in how they evaluate the condition. This opt-in currently +relies on setting `X-DH-Build-For-Type: target` for each of the relevant packages in +`debian/control`. However, unless you are a cross-compiling specialist, you will probably never +need to care about nor use any of this. + +Accordingly, the possible conditions are: + + * `arch-matches`: This is the form recommended to laymen and as the default use-case. This + conditional acts `package-context-arch-matches` if the condition is used in the context + of a binary package. Otherwise, it acts as `source-context-arch-matches`. + + * `source-context-arch-matches`: With this conditional, the provided architecture constraint is compared + against the build time provided host architecture (`dpkg-architecture -qDEB_HOST_ARCH`). This can + be useful when an `Architecture: all` package needs an architecture condition for some reason. + + * `package-context-arch-matches`: With this conditional, the provided architecture constraint is compared + against the package's resolved architecture. This condition can only be used in the context of a binary + package (usually, under `packages.<name>.`). If the package is an `Architecture: all` package, the + condition will fail with an error as the condition always have the same outcome. For all other + packages, the package's resolved architecture is the same as the build time provided host architecture + (`dpkg-architecture -qDEB_HOST_ARCH`). + + - However, as noted above there is a special case for when compiling a cross-compiling compiler, where + this behaves subtly different from `source-context-arch-matches`. + +All conditions are used the same way as `arch-matches`. Simply replace `arch-matches` with the other +condition. See the `arch-matches` description for an example. + +## Active build profile match condition `build-profiles-matches` (mapping) + +The `build-profiles-matches` condition is used to assert whether the active build profiles +(`DEB_BUILD_PROFILES` / `dpkg-buildpackage -P`) matches a given build profile restriction. + +As an example: + + # TODO: Not the best example (`create-symlink` is an unlikely use-case for this condition) + packages: + foo: + transformations: + - create-symlink + path: sbin/agetty + target: /sbin/getty + when: + build-profiles-matches: '<!pkg.foo.mycustomprofile>' + +The `build-profiles-matches` must be defined as a part of a mapping, where `build-profiles-matches` +is the key. The value is a string using the same syntax as the `Build-Profiles` field from `debian/control` +(i.e., a space separated list of `<[!]profile ...>` groups). + +## Can run produced binaries `can-execute-compiled-binaries` (string) + +The `can-execute-compiled-binaries` condition is used to assert the build can assume +that all compiled binaries can be run as-if they were native binaries. For native +builds, this condition always evaluates to `true`. For cross builds, the condition +is generally evaluates to `false`. However, there are special-cases where binaries +can be run during cross-building. Accordingly, this condition is subtly different +from the `cross-compiling` condition. + +Note this condition should *not* be used when you know the binary has been built +for the build architecture (`DEB_BUILD_ARCH`) or for determining whether build-time tests +should be run (for build-time tests, please use the `run-build-time-tests` condition instead). +Some upstream build systems are advanced enough to distinguish building a final product vs. +building a helper tool that needs to run during build. The latter will often be compiled by +a separate compiler (often using `$(CC_FOR_BUILD)`, `cc_for_build` or similar variable names +in upstream build systems for that compiler). + +As an example: + + # TODO: Not the best example (`create-symlink` is an unlikely use-case for this condition) + packages: + foo: + transformations: + - create-symlink + path: sbin/agetty + target: /sbin/getty + # Only for native builds or when we can transparently run a compiled + when: can-execute-compiled-binaries + +The `can-execute-compiled-binaries` condition is specified as a string. + +## Cross-Compiling condition `cross-compiling` (string) + +The `cross-compiling` condition is used to determine if the current build is performing a cross +build (i.e., `DEB_BUILD_GNU_TYPE` != `DEB_HOST_GNU_TYPE`). Often this has consequences for what +is possible to do. + +Note if you specifically want to know: + + * whether build-time tests should be run, then please use the `run-build-time-tests` condition. + * whether compiled binaries can be run as if it was a native binary, please use the + `can-execute-compiled-binaries` condition instead. That condition accounts for cross-building + in its evaluation. + +As an example: + + # TODO: Not the best example (`create-symlink` is an unlikely use-case for this condition) + packages: + foo: + transformations: + - create-symlink + path: sbin/agetty + target: /sbin/getty + when: cross-compiling + +The `cross-compiling` condition is specified as a string. + +## Whether build time tests should be run `run-build-time-tests` (string) + +The `run-build-time-tests` condition is used to determine whether (build time) tests should +be run for this build. This condition roughly translates into whether `nocheck` is present +in `DEB_BUILD_OPTIONS`. + +In general, the manifest *should not* prevent build time tests from being run during cross-builds. + +As an example: + + # TODO: Not the best example (`create-symlink` is an unlikely use-case for this condition) + packages: + foo: + transformations: + - create-symlink + path: sbin/agetty + target: /sbin/getty + when: run-build-time-tests + +The `run-build-time-tests` condition is specified as a string. + +## Negated condition `not` (mapping) + +It is possible to negate a condition via the `not` condition. + +As an example: + + packages: + util-linux: + transformations: + - create-symlink + path: sbin/getty + target: /sbin/agetty + when: + # On Hurd, the package "hurd" ships "sbin/getty". + # This example happens to also be an alternative to `arch-marches: '!hurd-any` + not: + arch-matches: 'hurd-any' + +The `not` condition is specified as a mapping, where the key is `not` and the +value is a nested condition. + +## All or any of a list of conditions `all-of`/`any-of` (list) + +It is possible to aggregate conditions using the `all-of` or `any-of` condition. This provide +`X and Y` and `X or Y` semantics (respectively). + +As an example: + + packages: + util-linux: + transformations: + - create-symlink + path: sbin/getty + target: /sbin/agetty + when: + # Only ship getty on linux except for s390(x) + all-of: + - arch-matches: 'linux-any' + - arch-matches: '!s390 !s390x' + +The `all-of` and `any-of` conditions are specified as lists, where each entry is a nested condition. +The lists need at least 2 entries as with fewer entries the `all-of` and `any-of` conditions are +redundant. + +# Packager provided definitions + +For more complex manifests or packages, it is possible define some common attributes for reuse. + +## Manifest Variables (`variables`) + +It is possible to provide custom manifest variables via the `variables` attribute. An example: + + manifest-version: '0.1' + definitions: + variables: + LIBPATH: "/usr/lib/{{DEB_HOST_MULTIARCH}}" + SONAME: "1" + installations: + - install: + source: build/libfoo.so.{{SONAME}}* + # The quotes here is for the YAML parser's sake. + dest-dir: "{{LIBPATH}}" + into: libfoo{{SONAME}} + +The value of the `variables` key must be a mapping, where each key is a new variable name and +the related value is the value of said key. The keys must be valid variable name and not shadow +existing variables (that is, variables such as `PACKAGE` and `DEB_HOST_MULTIARCH` *cannot* be +redefined). The value for each variable *can* refer to *existing* variables as seen in the +example above. + +As usual, `debputy` will insist that all declared variables must be used. + +Limitations: + * When declaring variables that depends on another variable declared in the manifest, the + order is important. The variables are resolved from top to bottom. + * When a manifest variable depends on another manifest variable, the existing variable is + currently always resolved in source context. As a consequence, some variables such as + `{{PACKAGE}}` cannot be used when defining a variable. This restriction may be + lifted in the future. + +# Installations + +For source packages building a single binary, the `dh_auto_install` from debhelper will default to +providing everything from upstream's install in the binary package. The `debputy` tool matches this +behaviour and accordingly, the `installations` feature is only relevant in this case when you need to +manually specify something upstream's install did not cover. + +For sources, that build multiple binaries, where `dh_auto_install` does not detect anything to install, +or when `dh_auto_install --destdir debian/tmp` is used, the `installations` section of the manifest is +used to declare what goes into which binary package. An example: + + installations: + - install: + sources: "usr/bin/foo" + into: foo + - install: + sources: "usr/*" + into: foo-extra + + +All installation rules are processed in order (top to bottom). Once a path has been matched, it can +no longer be matched by future rules. In the above example, then `usr/bin/foo` would be in the `foo` +package while everything in `usr` *except* `usr/bin/foo` would be in `foo-extra`. If these had been +ordered in reverse, the `usr/bin/foo` rule would not have matched anything and caused `debputy` +to reject the input as an error on that basis. This behaviour is similar to "DEP-5" copyright files, +except the order is reversed ("DEP-5" uses "last match wins", where here we are doing "first match wins") + +In the rare case that some path need to be installed into two packages at the same time, then this is +generally done by changing `into` into a list of packages. + +All installations are currently run in *source* package context. This implies that: + + 1) No package specific substitutions are available. Notably `{{PACKAGE}}` cannot be resolved. + 2) All conditions are evaluated in source context. For 99.9% of users, this makes no difference, + but there is a cross-build feature that changes the "per package" architecture which is affected. + +This is a limitation that should be fixed in `debputy`. + +**Attention debhelper users**: Note the difference between `dh_install` (etc.) vs. `debputy` on +overlapping matches for installation. + +## Install rule search directories + +Most install rules apply their patterns against search directories such as `debian/tmp` by default. + +The default search directory order (highest priority first) is: + + 1) The upstream install directory (usually, `debian/tmp`) + 2) The source package root directory (`.`) + +Each search directory is tried in order. When a pattern matches an entry in a search directory (even +if that entry is reserved by another package), further search directories will *not* be tried. As an example, +consider the pattern `usr/bin/foo*` and the files: + + `SOURCE_ROOT/debian/tmp/usr/bin/foo.sh` + `SOURCE_ROOT/usr/bin/foo.pl` + +Here the pattern will only match `SOURCE_ROOT/debian/tmp/usr/bin/foo.sh` and not `SOURCE_ROOT/usr/bin/foo.pl`. + +## Automatic discard rules + +The `debputy` framework provides some built-in discard rules that are applied by default during installation +time. These are always active and implicit, but can be overwritten by exact path matches for install rules. + +The `debputy` tool itself provides the following discard rules: + + * Discard of `.la` files. Their use is rare but not unheard of. You may need to overwrite this. + * Discard of python byte code (such as `__pycache__` directories). + * Discard of editor backup files (such as `*~`, `*.bak`, etc.). + * Discard of Version control files (such as `.gitignore`, etc.). + * Discard of GNU info's `dir` (`usr/share/info/dir`) as it causes file conflicts with other packages. + * Discard of `DEBIAN` directory. + +Note: Third-party plugins may provide additional automatic discard rules. Please use +`debputy plugin list automatic-discard-rules` to see all known automatic discard rules. + +If you find yourself needing a particular path installed that has been discarded by default, you can overrule +the default discard by spelling out the path. As an example, if you needed to install a `libfoo.la` file, +you could do: + + installations: + - install: + sources: + # By-pass automatic discard of `libfoo.la` - no globs *cannot* be used! + - "usr/lib/libfoo.la" + - "usr/lib/libfoo*.so*" + into: libfoo1 + +## Generic install (`install`) + +The generic `install` rule can be used to install arbitrary paths into packages and is *similar* to how +`dh_install` from debhelper works. It is a two "primary" uses. + + 1) The classic "install into directory" similar to the standard `dh_install` + 2) The "install as" similar to `dh-exec`'s `foo => bar` feature. + +Examples: + + installations: + - install: + source: "usr/bin/tool-a" + dest-dir: "usr/bin" + into: foo + - install: + source: "usr/bin/tool-b" + # Implicit dest-dir: "usr/bin + into: foo-extra + - install: + source: "usr/bin/tool-c.sh" + # Rename as a part of installing. + as: "usr/bin/tool-c" + into: foo-util + + +The value for `install` is a string, a list of strings or mapping. When it is a mapping, the +mapping has the following key/value pairs: + + * `source` or `sources` (required): A path match (`source`) or a list of path matches (`sources`) defining + the source path(s) to be installed. The path match(es) can use globs. Each match is tried against default + search directories. + - When a symlink is matched, then the symlink (not its target) is installed as-is. When a directory is + matched, then the directory is installed along with all the contents that have not already been installed + somewhere. + + * `into` (conditional): Either a package name or a list of package names for which these paths should be + installed. This key is conditional on whether there are multiple binary packages listed in + `debian/control`. When there is only one binary package, then that binary is the default for `into`. + Otherwise, the key is required. + + * `dest-dir` (optional): A path defining the destination *directory*. The value *cannot* use globs, but can + use substitution. If neither `as` nor `dest-dir` is given, then `dest-dir` defaults to the directory name + of the `source`. + + * `as` (optional): A path defining the path to install the source as. This is a full path. This option is + mutually exclusive with `dest-dir` and `sources` (but not `source`). When `as` is given, then `source` must + match exactly one "not yet matched" path. + + * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules). + +When the input is a string or a list of string, then that value is used as shorthand for `source` +or `sources` (respectively). This form can only be used when `into` is not required. + + +## Install documentation (`install-docs`) + +This install rule resemble that of `dh_installdocs`. It is a shorthand over the generic +`install` rule with the following key features: + + 1) The default `dest-dir` is to use the package's documentation directory (usually something + like `/usr/share/doc/{{PACKAGE}}`, though it respects the "main documentation package" + recommendation from Debian Policy). The `dest-dir` or `as` can be set in case the + documentation in question goes into another directory or with a concrete path. In this + case, it is still "better" than `install` due to the remaining benefits. + 2) The rule comes with pre-defined conditional logic for skipping the rule under + `DEB_BUILD_OPTIONS=nodoc`, so you do not have to write that conditional yourself. + 3) The `into` parameter can be omitted as long as there is a exactly one non-`udeb` package + listed in `debian/control`. + +With these two things in mind, it behaves just like the `install` rule. + +Note: It is often worth considering to use a more specialized version of the `install-docs` +rule when one such is available. If you are looking to install an example or a manpage, +consider whether `install-examples` or `install-man` might be a better fit for your +use-case. + +Examples: + + installations: + - install-docs: + sources: + - "docs/README.some-topic.md" + - "docs/README.another-topic.md" + into: foo + + +The value for `install-docs` is a string, a list of strings or mapping. When it is +a mapping, the mapping has the following key/value pairs: + + * `source` or `sources` (required): A path match (`source`) or a list of path matches (`sources`) defining + the source path(s) to be installed. The path match(es) can use globs. Each match is tried against default + search directories. + - When a symlink is matched, then the symlink (not its target) is installed as-is. When a directory is + matched, then the directory is installed along with all the contents that have not already been installed + somewhere. + + - **CAVEAT**: Specifying `source: docs` where `docs` resolves to a directory for `install-docs` + will give you an `docs/docs` directory in the package, which is rarely what you want. Often, you + can solve this by using `docs/*` instead. + + * `dest-dir` (optional): A path defining the destination *directory*. The value *cannot* use globs, but can + use substitution. If neither `as` nor `dest-dir` is given, then `dest-dir` defaults to the relevant package + documentation directory (a la `/usr/share/doc/{{PACKAGE}}`). + + * `into` (conditional): Either a package name or a list of package names for which these paths should be + installed as docs. This key is conditional on whether there are multiple (non-`udeb`) binary + packages listed in `debian/control`. When there is only one (non-`udeb`) binary package, then that binary + is the default for `into`. Otherwise, the key is required. + + * `as` (optional): A path defining the path to install the source as. This is a full path. This option is + mutually exclusive with `dest-dir` and `sources` (but not `source`). When `as` is given, then `source` must + match exactly one "not yet matched" path. + + * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules). This condition will + be combined with the built-in condition provided by these rules (rather than replacing it). + +When the input is a string or a list of string, then that value is used as shorthand for `source` +or `sources` (respectively). This form can only be used when `into` is not required. + +Note: While the canonical name for this rule use plural, the `install-doc` variant is accepted as +alternative name. + +## Install examples (`install-examples`) + +This install rule resemble that of `dh_installexamples`. It is a shorthand over the generic ` +install` rule with the following key features: + + 1) It pre-defines the `dest-dir` that respects the "main documentation package" recommendation from + Debian Policy. The `install-examples` will use the `examples` subdir for the package documentation + dir. + 2) The rule comes with pre-defined conditional logic for skipping the rule under + `DEB_BUILD_OPTIONS=nodoc`, so you do not have to write that conditional yourself. + 3) The `into` parameter can be omitted as long as there is a exactly one non-`udeb` package + listed in `debian/control`. + +With these two things in mind, it behaves just like the `install` rule. + +Examples: + + installations: + - install-examples: + source: "examples/*" + into: foo + + +The value for `install-examples` is a string, a list of strings or mapping. When it is +a mapping, the mapping has the following key/value pairs: + +* `source` or `sources` (required): A path match (`source`) or a list of path matches (`sources`) defining + the source path(s) to be installed. The path match(es) can use globs. Each match is tried against default + search directories. + - When a symlink is matched, then the symlink (not its target) is installed as-is. When a directory is + matched, then the directory is installed along with all the contents that have not already been installed + somewhere. + + - **CAVEAT**: Specifying `source: examples` where `examples` resolves to a directory for `install-examples` + will give you an `examples/examples` directory in the package, which is rarely what you want. Often, you + can solve this by using `examples/*` instead. + +* `into` (conditional): Either a package name or a list of package names for which these paths should be + installed as examples. This key is conditional on whether there are multiple (non-`udeb`) binary + packages listed in `debian/control`. When there is only one (non-`udeb`) binary package, then that binary + is the default for `into`. Otherwise, the key is required. + + +* `when` (optional): A condition as defined in [Conditional rules](#conditional-rules). This condition will + be combined with the built-in condition provided by these rules (rather than replacing it). + +When the input is a string or a list of string, then that value is used as shorthand for `source` +or `sources` (respectively). This form can only be used when `into` is not required. + +Note: While the canonical name for this rule use plural, the `install-example` variant is accepted as +alternative name. + +## Install manpages (`install-man`) + +Install rule for installing manpages similar to `dh_installman`. It is a shorthand over the generic +`install` rule with the following key features: + + 1) The rule can only match files (notably, symlinks cannot be matched by this rule). + 2) The `dest-dir` is computed per source file based on the manpage's section and language. + 3) The `into` parameter can be omitted as long as there is a exactly one non-`udeb` package + listed in `debian/control`. + 4) The rule comes with manpage specific attributes such as `language` and `section` for when the + auto-detection is insufficient. + 5) The rule comes with pre-defined conditional logic for skipping the rule under `DEB_BUILD_OPTIONS=nodoc`, + so you do not have to write that conditional yourself. + +With these things in mind, the rule behaves similar to the `install` rule. + +Examples: + + installations: + - install-man: + source: "man/foo.1" + into: foo + - install-man: + source: "man/foo.de.1" + language: derive-from-basename + into: foo + + +The value for `install-man` is a string, a list of strings or mapping. When it is a mapping, the mapping +has the following key/value pairs: + + * `source` or `sources` (required): A path match (`source`) or a list of path matches (`sources`) defining + the source path(s) to be installed. The path match(es) can use globs. Each match is tried against default + search directories. Only files can be matched. + + * `into` (conditional): Either a package name or a list of package names for which these paths should be + installed as manpages. This key is conditional on whether there are multiple (non-`udeb`) binary + packages listed in `debian/control`. When there is only one (non-`udeb`) binary package, then that binary + is the default for `into`. Otherwise, the key is required. + + * `section` (optional): If provided, it must be an integer between 1 and 9 (both inclusive), defining the + section the manpages belong overriding any auto-detection that `debputy` would have performed. + + * `language` (optional): If provided, it must be either a 2 letter language code (such as `de`), a 5 letter + language + dialect code (such as `pt_BR`), or one of the special keywords `C`, `derive-from-path`, or + `derive-from-basename`. The default is `derive-from-path`. + - When `language` is `C`, then the manpages are assumed to be "untranslated". + - When `language` is a language code (with or without dialect), then all manpages matched will be assumed + to be translated to that concrete language / dialect. + - When `language` is `derive-from-path`, then `debputy` attempts to derive the language from the path + (`man/<language>/man<section>`). This matches the default of `dh_installman`. When no language can + be found for a given source, `debputy` behaves like language was `C`. + - When `language` is `derive-from-basename`, then `debputy` attempts to derive the language from the + basename (`foo.<language>.1`) similar to `dh_installman` previous default. When no language can + be found for a given source, `debputy` behaves like language was `C`. Note this is prone to + false positives where `.pl`, `.so` or similar two-letter extensions gets mistaken for a language + code (`.pl` can both be "Polish" or "Perl Script", `.so` can both be "Somali" and "Shared Object" + documentation). In this configuration, such extensions are always assumed to be a language. + + * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules). This condition will + be combined with the built-in condition provided by these rules (rather than replacing it). + + +When the input is a string or a list of string, then that value is used as shorthand for `source` +or `sources` (respectively). This form can only be used when `into` is not required. + +Comparison with debhelper: The `dh_installman` uses `derive-from-path` as default and then falls back +to `derive-from-basename`. The `debputy` tool does *not* feature the same fallback logic. If you want +the `derive-from-basename` with all of its false-positives, you have to explicitly request it. + + +## Discard (or exclude) upstream provided paths (`discard`) + +When installing paths from `debian/tmp` into packages, it might be useful to ignore some paths that you never +need installed. This can be done with the `discard` rule. + + +The value for `discard` is a string, a list of strings or mapping. When it is a mapping, the mapping +has the following key/value pairs: + + * `path` or `paths` (required): A path match (`path`) or a list of path matches (`paths`) defining the + source path(s) that should not be installed anywhere. The path match(es) can use globs. + - When a symlink is matched, then the symlink (not its target) is discarded as-is. When a directory is + matched, then the directory is discarded along with all the contents that have not already been installed + somewhere. + + * `search-dir` or `search-dirs` (optional): A path (`search-dir`) or a list to paths (`search-dirs`) that + defines which search directories apply to. This attribute is primarily useful for source packages that + uses [per package search dirs](#custom-installation-time-search-directories-installation-search-dirs), + and you want to restrict a discard rule to a subset of the relevant search dirs. Note all listed + search directories must be either an explicit search requested by the packager or a search directory + that `debputy` provided automatically (such as `debian/tmp`). Listing other paths will make `debputy` + report an error. + + - Note that the `path` or `paths` must match at least one entry in any of the search directories unless + *none* of the search directories exist (or the condition in `required-when` evaluates to false). When + none of the search directories exist, the discard rule is silently skipped. This special-case enables + you to have discard rules only applicable to certain builds that are only performed conditionally. + + * `required-when` (optional): A condition as defined in [Conditional rules](#conditional-rules). The discard + rule is always applied. When the conditional is present and evaluates to false, the discard rule can + silently match nothing. When the condition is absent, *or* it evaluates to true, then each pattern + provided must match at least one path. + +When the input is a string or a list of string, then that value is used as shorthand for `path` +or `paths` (respectively). + +Once a path is discarded, it cannot be matched by any other install rules. A path that is discarded, is +considered handled when `debputy` checks for paths you might have forgotten to install. The `discard` +feature is therefore *also* replaces the `debian/not-installed` file used by `debhelper` and `cdbs`. + +Note: A discard rule applies to *all* relevant search directories at the same time (including the source +root directory) unlike other install rules that only applies to the first search directory with a *match*. +This is to match the intuition that if you discard something, you want it gone no matter which search +directory it happened to be in. + +## Multi destination install (`multi-dest-install`) + +Special use install rule for installing the same source multiple times into the same package. + +It works similar to the `install` rule except: + + 1) `dest-dir` is renamed to `dest-dirs` and is conditionally mandatory (either `as` or `dest-dirs` + must be provided). + 2) Both `as` and `dest-dirs` are now list of paths must have at least two paths when provided. + +Please see `debputy plugin show plugable-manifest-rules multi-dest-install` for the full documentation. + +# Binary package rules + +Inside the manifest, the `packages` mapping can be used to define requests for the binary packages +you want `debputy` to produce. Each key inside `packages` must be the name of a binary package +defined in `debian/control`. The value is a dictionary defining which features that `debputy` +should apply to that binary package. An example could be: + + + packages: + foo: + transformations: + - create-symlink: + path: usr/share/foo/my-first-symlink + target: /usr/share/bar/symlink-target + - create-symlink: + path: usr/lib/{{DEB_HOST_MULTIARCH}}/my-second-symlink + target: /usr/lib/{{DEB_HOST_MULTIARCH}}/baz/symlink-target + bar: + transformations: + - create-directories: + - some/empty/directory.d + - another/empty/integration-point.d + - create-directories: + path: a/third-empty/directory.d + owner: www-data + group: www-data + +In this case, `debputy` will create some symlinks inside the `foo` package and some directories for +the `bar` package. The following subsections define the keys you can use under each binary package. + + +## Transformations (`packages.{{PACKAGE}}.transformations`) + +You can define a `transformations` under the package definition, which is a list a transformation +rules. An example: + + packages: + foo: + transformations: + - remove: 'usr/share/doc/{{PACKAGE}}/INSTALL.md' + - move: + source: bar/* + target: foo/ + + +Transformations are ordered and are applied in the listed order. A path can be matched by multiple +transformations; how that plays out depends on which transformations are applied and in which order. +A quick summary: + + - Transformations that modify the file system layout affect how path matches in later transformations. + As an example, `move` and `remove` transformations affects what globs and path matches expand to in + later transformation rules. + + - For other transformations generally the latter transformation overrules the earlier one, when they + overlap or conflict. + +### Remove transformation rule (`remove`) + +The remove transformation rule is mostly only useful for single binary source packages, where +everything from upstream's build system is installed automatically into the package. In those case, +you might find yourself with some files that are _not_ relevant for the Debian package (but would be +relevant for other distros or for non-distro local builds). Common examples include `INSTALL` files +or `LICENSE` files (when they are just a subset of `debian/copyright`). + +In the manifest, you can ask `debputy` to remove paths from the debian package by using the `remove` +transformation rule. An example being: + + packages: + foo: + transformations: + - remove: 'usr/share/doc/{{PACKAGE}}/INSTALL.md' + + +The value for `remove` is a string, a list of strings or mapping. When it is a mapping, the mapping +has the following key/value pairs: + + * `path` or `paths` (required): A path match (`path`) or a list of path matches (`paths`) defining the + path(s) inside the package that should be removed. The path match(es) can use globs. + - When a symlink is matched, then the symlink (not its target) is removed as-is. When a directory is + matched, then the directory is removed along with all the contents. + + * `keep-empty-parent-dirs` (optional): A boolean determining whether to prune parent directories that become + empty as a consequence of this rule. When provided and `true`, this rule will leave empty directories + behind. Otherwise, if this rule causes a directory to become empty that directory will be removed. + + * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules). This condition will + be combined with the built-in condition provided by these rules (rather than replacing it). + +When the input is a string or a list of string, then that value is used as shorthand for `path` +or `paths` (respectively). + +Note that `remove` removes paths from future glob matches and transformation rules. + +This rule behaves roughly like the following shell snippet when applied: + +```shell +set -e +for p in ${paths}; do + rm -fr "${p}" + if [ "${keep_empty_parent_dirs}" != "true" ]; then + rmdir --ignore-fail-on-non-empty --parents "$(dirname "${p}")" + fi +done +``` + +### Move transformation rule (`move`) + +The move transformation rule is mostly only useful for single binary source packages, where +everything from upstream's build system is installed automatically into the package. In those case, +you might find yourself with some files that need to be renamed to match Debian specific requirements. + +This can be done with the `move` transformation rule, which is a rough emulation of the `mv` command +line tool. An example being: + + packages: + foo: + transformations: + - move: + source: bar/* + target: foo/ + # Note this example leaves bar/ as an empty directory. Real world usage would probably + # follow this with an remove rule for bar/ to avoid an empty directory + +The value for `move` is a mapping with the following key/value pairs: + + * `source` (required): A path match defining the source path(s) to be renamed. The value can use globs and + substitutions. + + * `target` (required): A path defining the target path. The value *cannot* use globs, but can use substitution. + If the target ends with a literal `/` (prior to substitution), the target will *always* be a directory. + + * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules) + +There are two basic cases: + 1. If `source` match exactly one path and `target` is not a directory (including it does not exist), + then `target` is removed (if it existed) and `source` is renamed to `target`. + - If the `target` is a directory, and you intentionally want to replace it with a non-directory, please + add an explicit `remove` transformation for the directory prior to this transformation. + + 2. If `source` match more than one path *or* `target` is a directory (or is specified with a trailing slash), + then all matching sources a moved into `target` retaining their basename. If `target` already contains + overlapping basenames with `source`, then the transformation rule with abort with an error if the overlap + contains a directory. Otherwise, any overlapping paths in `target` will be implicitly removed first. + + - If the replacement of a directory is intentional, please add an explicit `remove` rule for it first. + - In the case that the `source` glob is exotic enough to match two distinct paths with the same basename, + then `debputy` will reject the transformation on account of it being ambiguous. + +In either case, parent directories of `target` are created as necessary as long as they do not trigger a conflict +(or require traversing symlinks or non-directories). Additionally, the paths matched by `source` will no longer +match anything (since they are now renamed/relocated), which may affect what path future path matches will apply +to. + +Note that like the `mv`-command, while the `source` paths are no longer present, their parent directories +will remain. + +### Create symlinks transformation rule (`create-symlink`) + +Often, the upstream build system will provide the symlinks for you. However, in some cases, it is useful for +the packager to define distribution specific symlinks. This can be done via the `create-symlink` transformation +rule. An example of how to do this is: + + packages: + foo: + transformations: + - create-symlink: + path: usr/share/foo/my-first-symlink + target: /usr/share/bar/symlink-target + - create-symlink: + path: usr/lib/{{DEB_HOST_MULTIARCH}}/my-second-symlink + target: /usr/lib/{{DEB_HOST_MULTIARCH}}/baz/symlink-target + +The value for the `create-symlink` key is a mapping, which contains the following keys: + + * `path` (required): The path that should be a symlink. The path may contain substitution variables + such as `{{DEB_HOST_MULTIARCH}}` but _cannot_ use globs. Parent directories are implicitly created + as necessary. + * Note that if `path` already exists, the behaviour of this transformation depends on the value of + `replacement-rule`. + + * `target` (required): Where the symlink should point to. The target may contain substitution variables + such as `{{DEB_HOST_MULTIARCH}}` but _cannot_ use globs. The link target is _not_ required to exist inside + the package. + * The `debputy` tool will normalize the target according to the rules of the Debian Policy. Use absolute + or relative target at your own preference. + + * `replacement-rule` (optional): This attribute defines how to handle if `path` already exists. It can be + set to one of the following values: + - `error-if-exists`: When `path` already exists, `debputy` will stop with an error. This is similar to + `ln -s` semantics. + - `error-if-directory`: When `path` already exists, **and** it is a directory, `debputy` will stop with an + error. Otherwise, remove the `path` first and then create the symlink. This is similar to `ln -sf` + semantics. + - `abort-on-non-empty-directory` (default): When `path` already exists, then it will be removed provided + it is a non-directory **or** an *empty* directory and the symlink will then be created. If the path is + a *non-empty* directory, `debputy` will stop with an error. + - `discard-existing`: When `path` already exists, it will be removed. If the `path` is a directory, all + its contents will be removed recursively along with the directory. Finally, the symlink is created. + This is similar to having an explicit `remove` rule just prior to the `create-symlink` that is conditional + on `path` existing (plus the condition defined in `when` if any). + + Keep in mind, that `replacement-rule` only applies if `path` exists. If the symlink cannot be created, + because a part of `path` exist and is *not* a directory, then `create-symlink` will fail regardless of the + value in `replacement-rule`. + + * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules) + +This rule behaves roughly like the following shell snippet when applied: + +```shell +set -e +case "${replacement_rule}" in + error-if-directory) + F_FLAG="-f" + ;; + abort-on-non-empty-directory) + if [ -d "${path}" ]; then + rmdir "${path}" + fi + F_FLAG="-f" + ;; + discard-existing) + rm -fr "${path}" + ;; +esac +install -o "root" -g "root" -m "755" -d "$(dirname "${path}")" +ln -s ${F_FLAG} "${target}" "${path}" +``` + +### Create directories transformation rule (`create-directories`) + +NOTE: This transformation is only really needed if you need to create an empty directory somewhere +in your package as an integration point. All `debputy` transformations will create directories +as required. + +In most cases, upstream build systems and `debputy` will create all the relevant directories. However, in some +rare cases you may want to explicitly define a path to be a directory. Maybe to silence a linter that is +warning you about a directory being empty, or maybe you need an empty directory that nothing else is creating +for you. This can be done via the `create-directories` transformation rule. An example being: + + packages: + bar: + create-directories: + - some/empty/directory.d + - another/empty/integration-point.d + - path: a/third-empty/directory.d + owner: www-data + group: www-data + +The value for the `create-directories` key is either a string, a list of string or a mapping. When it is a +mapping, the mapping has the following key/value pairs: + + * `path` or `paths` (required): A path (`path`) or a list of path (`paths`) defining the + path(s) inside the package that should be created as directories. The path(es) _cannot_ use globs + but can use substitution variables. Parent directories are implicitly created (with owner `root:root` + and mode `0755` - only explicitly listed directories are affected by the owner/mode options) + + * `owner` (optional, default `root`): Denotes the owner of the directory (but _not_ what is inside the directory). + + * `group` (optional, default `root`): Denotes the group of the directory (but _not_ what is inside the directory). + + * `mode` (optional, default `"0755"`): Denotes the mode of the directory (but _not_ what is inside the directory). + Note that numeric mode must always be given as a string (i.e., with quotes). Symbolic mode can be used as well. + If symbolic mode uses a relative definition (e.g., `o-rx`), then it is relative to the directory's current mode + (if it already exists) or `0755` if the directory is created by this transformation. + + * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules) + +When the input is a string or a list of string, then that value is used as shorthand for `path` +or `paths` (respectively). + +Unless you have a specific need for the mapping form, you are recommended to use the shorthand form of +just listing the directories you want created. + + +Note that implicitly created directories (by this or other transformations) always have owner `root:root` +and mode `"0755"`. If you need a directory tree with special ownership/mode, you will have to list all the +directories in that tree explicitly with the relevant attributes OR use `path-metadata` transformation rule +to change their metadata after creation. + + +This rule behaves roughly like the following shell snippet when applied: + +```shell +set -e +for p in ${paths}; do + install -o "root" -g "root" -m "755" -d "${p}" + chown "${owner}:${group}" "${p}" + chmod "${mode}" "${p}" +done +``` + +### Change path owner/group or mode (`path-metadata`) + +The `debputy` normalizes the path metadata (such as ownership and mode) similar to `dh_fixperms`. +For most packages, the default is what you want. However, in some cases, the package has a special +case or two that `debputy` does not cover. In that case, you can tell `debputy` to use the metadata +you want by using the `path-metadata` transformation. An example being: + + packages: + foo: + transformations: + - path-metadata: + path: usr/bin/sudo + mode: "0755" + - path-metadata: + path: usr/bin/write: + group: tty + - path-metadata: + path: /usr/sbin/spine + capabilities: cap_net_raw+ep + + +The value for the `path-metadata` key is a mapping. The mapping has the following key/value pairs: +Each key defines a path or a glob that may + + +* `path` or `paths` (required): A path match (`path`) or a list of path matches (`paths`) defining the + path(s) inside the package that should be affected. The path match(es) can use globs and substitution + variables. Special-rules for matches: + - Symlinks are never followed and will never be matched by this rule. + - Directory handling depends on the `recursive` attribute. + + * `owner` (conditional): Denotes the owner of the paths matched by `path` or `paths`. When omitted, no change of + owner is done. + + * `group` (conditional): Denotes the group of the paths matched by `path` or `paths`. When omitted, no change of + group is done. + + * `mode` (conditional): Denotes the mode of the paths matched by `path` or `paths`. When omitted, no change in + mode is done. Note that numeric mode must always be given as a string (i.e., with quotes). Symbolic mode can + be used as well. If symbolic mode uses a relative definition (e.g., `o-rx`), then it is relative to the + directory's current mode. + + * `capabilities` (conditional): Denotes a Linux capability that should be applied to the path. When provided, + `debputy` will cause the capability to be applied to all *files* denoted by the `path`/`paths` attribute + on install (via `postinst configure`) provided that `setcap` is installed on the system when the + `postinst configure` is run. + - If any non-file paths are matched, the `capabilities` will *not* be applied to those paths. + + * `capability-mode` (optional): Denotes the mode to apply to the path *if* the Linux capability denoted in + `capabilities` was successfully applied. If omitted, it defaults to `a-s` as generally capabilities are + used to avoid "setuid"/"setgid" binaries. The `capability-mode` is relative to the *final* path mode + (the mode of the path in the produced `.deb`). The `capability-mode` attribute cannot be used if + `capabilities` is omitted. + + * `recursive` (optional, default `false`): When a directory is matched, then the metadata changes are applied + to the directory itself. When `recursive` is `true`, then the transformation is *also* applied to all paths + beneath the directory. + + * `when` (optional): A condition as defined in [Conditional rules](#conditional-rules) + +At least one of `owner`, `group`, `mode`, or `capabilities` must be provided. + +This rule behaves roughly like the following shell snippet when applied: + +```shell +MAYBE_R_FLAG=$(${recursive} && printf "%s" "-R") + +for p in ${path}; do + if [ -n "${owner}" ] || [ -n "${group}" ]; then + chown $MAYBE_R_FLAG "${owner}:${group}" "${p}" + fi + if [ -n "${mode}" ]; then + chmod $MAYBE_R_FLAG "${mode}" "${p}" + fi +done +``` + +(except all symlinks will be ignored) + +## Custom binary version (`binary-version`) + +In the *rare* case that you need a binary package to have a custom version, you can use the `binary-version:` +key to describe the desired package version. An example being: + + packages: + foo: + # The foo package needs a different epoch because we took it over from a different + # source package with higher epoch version + binary-version: '1:{{DEB_VERSION_UPSTREAM_REVISION}}' + +Use this feature sparingly as it is generally not possible to undo as each version must be monotonously +higher than the previous one. This feature translates into `-v` option for `dpkg-gencontrol`. + +The value for the `binary-version` key is a string that defines the binary version. Generally, you will +want it to contain one of the versioned related substitution variables such as +`{{DEB_VERSION_UPSTREAM_REVISION}}`. Otherwise, you will have to remember to bump the version manually +with each upload as versions cannot be reused and the package would not support binNMUs either. + + +## Remove runtime created paths on purge or post removal (`clean-after-removal`) + +For some packages, it is necessary to clean up some run-time created paths. Typical use cases are +deleting log files, cache files, or persistent state. This can be done via the `clean-after-removal`. +An example being: + + packages: + foo: + clean-after-removal: + - /var/log/foo/*.log + - /var/log/foo/*.log.gz + - path: /var/log/foo/ + ignore-non-empty-dir: true + - /etc/non-conffile-configuration.conf + - path: /var/cache/foo + recursive: true + + +The `clean-after-removal` key accepts a list, where each element is either a mapping, a string or a list +of strings. When an element is a mapping, then the following key/value pairs are applicable: + + * `path` or `paths` (required): A path match (`path`) or a list of path matches (`paths`) defining the + path(s) that should be removed after clean. The path match(es) can use globs and manifest variables. + Every path matched will by default be removed via `rm -f` or `rmdir` depending on whether the path + provided ends with a *literal* `/`. Special-rules for matches: + - Glob is interpreted by the shell, so shell (`/bin/sh`) rules apply to globs rather than + `debputy`'s glob rules. As an example, `foo/*` will **not** match `foo/.hidden-file`. + - `debputy` cannot evaluate whether these paths/globs will match the desired paths (or anything at + all). Be sure to test the resulting package. + - When a symlink is matched, it is not followed. + - Directory handling depends on the `recursive` attribute and whether the pattern ends with a literal + "/". + - `debputy` has restrictions on the globs being used to prevent rules that could cause massive damage + to the system. + + * `recursive` (optional): When `true`, the removal rule will use `rm -fr` rather than `rm -f` or `rmdir` + meaning any directory matched will be deleted along with all of its contents. + + * `ignore-non-empty-dir` (optional): When `true`, each path must be or match a directory (and as a + consequence each path must with a literal `/`). The affected directories will be deleted only if they + are empty. Non-empty directories will be skipped. This option is mutually exclusive with `recursive`. + + * `delete-on` (optional, defaults to `purge`): This attribute defines when the removal happens. It can + be set to one of the following values: + - `purge`: The removal happens with the package is being purged. This is the default. At a technical + level, the removal occurs at `postrm purge`. + - `removal`: The removal happens immediately after the package has been removed. At a technical level, + the removal occurs at `postrm remove`. + + +This feature resembles the concept of `rpm`'s `%ghost` files. + +## Custom installation time search directories (`installation-search-dirs`) + +For source packages that does multiple build, it can be an advantage to provide a custom list of +installation-time search directories. This can be done via the `installation-search-dirs` key. A common +example is building the source twice with different optimization and feature settings where the second +build is for the `debian-installer` (in the form of a `udeb` package). A sample manifest snippet could +look something like: + + installations: + - install: + # Because of the search order (see below), `foo` installs `debian/tmp/usr/bin/tool`, + # while `foo-udeb` installs `debian/tmp-udeb/usr/bin/tool` (assuming both paths are + # available). Note the rule can be split into two with the same effect if that aids + # readability or understanding. + source: usr/bin/tool + into: + - foo + - foo-udeb + packages: + foo-udeb: + installation-search-dirs: + - debian/tmp-udeb + + +The `installation-search-dirs` key accepts a list, where each element is a path (str) relative from the +source root to the directory that should be used as a search directory (absolute paths are still interpreted +as relative to the source root). This list should contain all search directories that should be applicable +for this package (except the source root itself, which is always appended after the provided list). If the +key is omitted, then `debputy` will provide a default search order (In the `dh` integration, the default +is the directory `debian/tmp`). + +If a non-existing or non-directory path is listed, then it will be skipped (info-level note). If the path +exists and is a directory, it will also be checked for "not-installed" paths. + + +[reference documentation]: https://documentation.divio.com/reference/ |