diff options
Diffstat (limited to 'GETTING-STARTED-WITH-dh-debputy.md')
| -rw-r--r-- | GETTING-STARTED-WITH-dh-debputy.md | 316 |
1 files changed, 316 insertions, 0 deletions
diff --git a/GETTING-STARTED-WITH-dh-debputy.md b/GETTING-STARTED-WITH-dh-debputy.md new file mode 100644 index 0000000..5d5d253 --- /dev/null +++ b/GETTING-STARTED-WITH-dh-debputy.md @@ -0,0 +1,316 @@ +# Getting started with `dh-debputy` + +_This is [how-to guide] and is primarily aimed at getting a task done._ + +<!-- To writers and reviewers: Check the documentation against https://documentation.divio.com/ --> + +This document will help you convert a Debian source package using the `dh` sequencer from debhelper to +use `dh-debputy`. Prerequisites for this how-to guide: + + * You have a Debian source package using the `dh` sequencer. Ideally a simple one as not all packages + can be converted at this time. Note that `debputy` does *not* interact well with most third-party + `dh` addons. You are recommended to start with source packages without third-party `dh` addons. + * It is strongly recommended that your package is bit-for-bit reproducible before starting the conversion + as that makes it easier to spot bugs introduced by the conversion! The output of `debputy` will *not* + be bit-for-bit reproducible with `debhelper` in all cases. However, the differences should be easy to + review with `diffoscope` if the package was already bit-for-bit reproducible. + - Note that `debputy` does not use `strip-nondeterminism`. The bit-for-bit reproducible property should + ideally not rely on `strip-nondeterminism` for now. + +Note that during the conversion (particularly Step 1 and Step 2), you may find that `debputy` cannot +support the requirements for your package for now. Feel free to file an issue for what is holding you +back in the [debputy issue tracker]. + +Prerequisites +------------- + +This guide assumes familiarity with Debian packaging in general. Notably, you should understand +the different between a (Debian) source package and a (Debian) binary package (e.g., `.deb`) plus +how these concepts relates to `debian/control` (the source control file). + +Additionally, since this is about `debputy` integration with debhelper, the reader is expected to +be familiar with `debhelper` (notably the `dh` style `debian/rules`). + +## Step 1: Have `debputy` convert relevant debhelper files + +The `debputy` integration with debhelper removes (replaces) some debhelper tools, but does **not** +read their relevant config files. These should instead be converted in to the new format. + +You can have `debputy` convert these for you by using the following command: + + # Dry-run conversion, creates `debian/debputy-manifest.new` if anything is migrated and prints a summary + # of what is done. Remove the --no-act when you are ready to commit the conversion. + $ debputy migrate-from-dh --no-act + +If relevant, `debputy` may inform you that: + + 1) the package is using an unsupported feature. Unless that functionality can easily be removed (e.g., it is now + obsolete) or easily replaced, then you probably do not want to convert the package to `debputy` at this time. + * One common source of unsupported features are dh hook targets (such as override targets), which will be covered + in slightly more detail in the next section. If `debputy` detected any hook targets, it is probably worth it to + check if these can be migrated before continuing. + * It is worth noting that the migration tool will update an existing manifest when re-run. You can safely "jump" + around in the order of the steps, when you try to migrate, if that better suits you. + 2) the migration would trigger a conflict. This can occur for two reasons: + * The debhelper configuration has the conflict (example [#934499]), where debhelper is being lenient and ignores + the problem. In this case, you need to resolve the conflict in the debhelper config and re-run `debputy`. + * The package has a manifest already with a similar (but not identical) definition of what the migration would + generate. In this case, you need to reconcile the definitions manually (and removing one of them). After that + you can re-run `debputy`. + + +As an example, if you had a `debian/foo.links` (with `foo` being defined in `debian/control`) containing the following: + + usr/share/foo/my-first-symlink usr/share/bar/symlink-target + usr/lib/{{DEB_HOST_MULTIARCH}}/my-second-symlink usr/lib/{{DEB_HOST_MULTIARCH}}/baz/symlink-target + +The `debputy migrate-from-dh` tool would generate a manifest looking something like this: + + manifest-version: "0.1" + packages: + foo: + transformations: + - create-symlink: + path: usr/share/foo/my-first-symlink + target: /usr/share/bar/symlink-target + - create-symlink: + path: usr/lib/{{DEB_HOST_MULTIARCH}}/my-second-symlink + target: /usr/lib/{{DEB_HOST_MULTIARCH}}/baz/symlink-target + +## Step 2: Migrate override/hook-targets in debian/rules + +Once you activate the `debputy` debhelper integration (see Step 3), the following debhelper tools will be +removed from the `dh` sequence. + + * `dh_install` + * `dh_installdocs` + * `dh_installchangelogs` + * `dh_installexamples` + * `dh_installman` + * `dh_installcatalogs` **(!)** + * `dh_installcron` + * `dh_installifupdown` + * `dh_installdebconf` **(!)** + * `dh_installemacsen` **(!)** + * `dh_installinfo` + * `dh_installinit` **(!)** + * `dh_installsysusers` + * `dh_installtmpfiles` + * `dh_installsystemd` **(!)** + * `dh_installsystemduser` **(!)** + * `dh_installmenu` **(!)** + * `dh_installmime` + * `dh_installmodules` + * `dh_installlogcheck` + * `dh_installlogrotate` + * `dh_installpam` + * `dh_installppp` + * `dh_installudev` **(!)** + * `dh_installgsettings` + * `dh_installinitramfs` + * `dh_installalternatives` + * `dh_bugfiles` + * `dh_ucf` **(!)** + * `dh_lintian` + * `dh_icons` + * `dh_perl` + * `dh_usrlocal` **(!)** + * `dh_links` + * `dh_installwm` **(!)** + * `dh_installxfonts` + * `dh_strip_nondeterminism` + * `dh_compress` + * `dh_fixperms` + * `dh_dwz` + * `dh_strip` + * `dh_makeshlibs` + * `dh_shlibdeps` + * `dh_missing` + * `dh_installdeb` + * `dh_gencontrol` + * `dh_md5sums` + * `dh_builddeb` + +Have a look at `debian/rules` and migrate any overrides you have for them to `debputy` features or other +hook targets. See also the subsections below for concrete advice on how to deal with override or hook targets +for some of these tools. However, since `debhelper` hooks are arbitrary code execution hooks, there will be +many cases that the guide will not be able to cover or where `debputy` may not have the feature to support your +hook target. + +While you could manually force these tools to be run via a hook target, they are very likely to feature +interact with `debputy`. Either causing `debputy` to replace their output completely or by having the tool +overwrite what `debputy` did (depending on the exact order). If you find, you *really* need to run one of these +tools, because `debputy` is not supporting a particular feature they have, then you are probably better off waiting +with migrating the package in question. + +Tools marked with **(!)** have no debputy support at all. If you rely on these tools, migration is unlikely +to succeed. Other tools will have some form of support (often at least a commonly used flow/feature set). +Where possible, `debputy migrate-from-dh --no-act` will detect these completely unsupported tools via existence +of their config files or indirectly debhelper hook targets for these tools. However, some tools may only be +detected late into the build (which is the case with `dh_usrlocal`). + +### Review and migrate any installation code from `dh_install`, `dh_installdocs`, etc. (if any) + +All code in `debian/rules` that involves copying or moving files into packages or around in packages must +be moved to the manifest's `installations` section. The migration tool will attempt to auto-migrate +any rules from `debian/install` (etc.). However, be aware of: + + 1) The migration tool assumes none of install rules overlap. Most packages do not have overlapping + install rules as it tends to lead to file conflicts. If the install rules overlap, `debputy` will + detect it at runtime and stop with an error. In that case, you will have to tweak the migrated rules + manually. + 2) Any overrule target that copies or moves files around in packages must be moved to `installations` + (per source) or `transformations` (per package) depending on the case. + - For source packages installing content via `debian/tmp`, you can use `install` to rename paths as + you install them and `discard` (under `installations`) to ignore paths that should not be installed. + - For source packages installing content via `debian/<pkg>`, then everything in there is "auto-installed". + If you need to tweak that content, you can use `remove` or `move` transformations (under `transformations`) + for manipulation the content. + +Note that the migrator "blindly" appends new rules to the bottom of `installations` if you have any existing +rules (per "none of the install rules overlap"-logic mentioned above). If you cannot convert all debhelper config +files in one go, or you manually created some installation rules before running the migrator, you may need to +manually re-order the generated installation rules to avoid conflicts due to inadequate ordering. You probably +want to do so any way for readability/grouping purposes. + + +#### Double-check the `language` settings on all `install-man` rules in `installations` + +The `dh_installman` tool auto-detects the language for manpages via two different methods: + + 1) By path name (Does the installation path look like `man/<language>/man<section>/...`?) + 2) By basename (Does the basename look like `foo.<language>.<section>`?) + +Both methods are used in order with first match being the method of choice. Unfortunately, the second +method is prune to false-positives. Does `foo.pl.1` mean a Polish translation of `foo.1` or is it the +manpage for a Perl script called `foo.pl` (similar happens for languages/file extensions). + +To avoid this issue, `debputy` uses 1) by default and only that. Option 2) can be chosen by setting +`language: derive-from-basename` on the concrete install rule. The problem is that the migration tool +has to guess, and it is hard to tell if rules like `man/*.1` would need option 2) - especially when the +migration tool does not attempt to resolve the glob (which it currently does not). + +Therefore, take a critical look at the generated `install-man` rules and the generated `language` property +(or lack thereof). + + +### Convert your overrides or excludes for `dh_fixperms` (if any) + +The `debputy` tool will normalize permissions like `dh_fixperms` during package build. If you have +any special requirements that `dh_fixperms` did not solve for you, you will have to tell `debputy` +about them. + +If you had something like: + + override_dh_fixperms: + dh_fixperms -X bin/sudo + +and the goal was to have `dh_fixperms` not touch the mode but the ownership (root:root) was fine, you +would have the manifest `debian/debputy.manifest`: + + manifest-version: "0.1" + packages: + foo: + transformations: + - path-metadata: + path: usr/bin/sudo + mode: "04755" + +Note you have to spell out the desired mode for this file. + +On the other hand, if your `debian/rules` had something like: + + execute_after_dh_fixperms: + chown www-data:www-data debian/foo/var/lib/something-owned-by-www-data + +Then the manifest would look something like: + + manifest-version: "0.1" + packages: + foo: + transformations: + - path-metadata: + path: var/lib/something-owned-by-www-data + owner: www-data + group: www-data + +This can be combined with an explicit `mode: "02755"` if you also need a non-default mode. + +The paths provided here support substitution variables (`usr/lib/{{DEB_HOST_MULTIARCH}}/...`) and +some _limited_ glob support (`usr/bin/sudo*`). + +_Remember to merge your manifest with previous steps rather than replacing it!_ Note that +`debputy migrate-from-dh` will merge its changes into existing manifests and can safely be re-run +after adding/writing this base manifest. + +### Convert your overrides for `dh_gencontrol` (if any) + +If the package uses an override to choose a custom version for a binary package, then it is possible in `debputy` +by using the `binary-version` key under the package. Here is an example to force the package `foo` to have +epoch `1`: + + manifest-version: "0.1" + packages: + foo: + # The foo package needs a different epoch because we took it over from a different + # source package with higher epoch version + binary-version: '1:{{DEB_VERSION_UPSTREAM_REVISION}}' + +Useful if the source took over a binary package from a different source and that binary had a higher +epoch version. + +Note that only limited manipulation of the version is supported, since your options are generally +limited to expanding one of the following version related variables: + + * `DEB_VERSION` - same definition as the one from `/usr/share/dpkg/pkg-info.mk` (from `dpkg`) + * `DEB_VERSION_EPOCH_UPSTREAM` - ditto + * `DEB_VERSION_UPSTREAM_REVISION` - ditto + * `DEB_VERSION_UPSTREAM` - ditto + +If the override is needed for dynamic substitution variables or binary versions that cannot be done with +the above substitutions, then it might be better to hold off on the conversion. + +_Remember to merge your manifest with previous steps rather than replacing it!_ Note that +`debputy migrate-from-dh` will merge its changes into existing manifests and can safely be re-run +after adding/writing this base manifest. + +## Step 3: Adding the `dh-sequence-zz-debputy` sequence to Build-Depends + +The recommended way to do so is to add `dh-sequence-zz-debputy` to the `Build-Depends:` field. + +The `zz-` in `zz-debputy` is there to ensure the `debputy` add-on is loaded last by debhelper for +the rare case there any other debhelper addons still active as the `debputy` sequence does not really +play well with other `dh` addons. When there are other add-ons, it is generally better for `debputy` to +be loaded last (as add-on order matters per [#885580]). + +## Step 4: Replace third-party dh add-ons to debputy plugins + +Packages using third-party `dh` add-ons may need to replace these with `debputy` plugins assuming the +add-on has a `debputy` plugin to replace it in the first place. To request a `debputy` plugin, you +will have to add `debputy-plugin-X` into your `Build-Depends`, where `X` is the name of the plugin. + +Note that `debputy` does not support the same features as `debhelper` at the moment for conditional +plugin loading. Therefore, the plugins must be in `Build-Depends`. + +## Step 5: Verify the build + +At this stage, if there are no errors in your previous steps, you should be able to build your +changed package with `debputy`. We recommend that you take time to verify this. For some packages, +there was no conversion to do in the previous steps, and you would not even need a manifest at all +in this case. However, we still recommend that you verify the build is successful here and now. + +The `debputy` supports bit-for-bit reproducibility in its output. However, the output is not bit-for-bit +reproducible with `debhelper`. You are recommended to use `diffoscope` to compare the `debhelper` +built-version with the `debputy` built version to confirm that all the changes are benign. + +However, `debputy` is bit-for-bit reproducible in its output with `(fakeroot) dpkg-deb -b`. Should you +spot a difference where `debputy` does not produce bit-for-bit identical results with `dpkg-deb` (tar +format, file ordering, etc.), then please file a bug against `debputy` with a reproducing test case. + +Once you have verified your built, the conversion is done. :) At this point, you can consider +looking at other features that `debputy` supports that might be useful to you. + +[how-to guide]: https://documentation.divio.com/how-to-guides/ +[#885580]: https://bugs.debian.org/885580 +[#934499]: https://bugs.debian.org/934499 +[debputy issue tracker]: https://salsa.debian.org/debian/debputy/-/issues |