Adding upstream version 0.1.21.upstream/0.1.21

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 316 insertions, 0 deletions

diff --git a/GETTING-STARTED-WITH-dh-debputy.md b/GETTING-STARTED-WITH-dh-debputy.md
new file mode 100644
index 0000000..5d5d253
--- /dev/null
+++ b/GETTING-STARTED-WITH-dh-debputy.md
@@ -0,0 +1,316 @@
+# Getting started with `dh-debputy`
+
+_This is [how-to guide] and is primarily aimed at getting a task done._
+
+<!-- To writers and reviewers: Check the documentation against https://documentation.divio.com/ -->
+
+This document will help you convert a Debian source package using the `dh` sequencer from debhelper to
+use `dh-debputy`.  Prerequisites for this how-to guide:
+
+ * You have a Debian source package using the `dh` sequencer.  Ideally a simple one as not all packages
+   can be converted at this time. Note that `debputy` does *not* interact well with most third-party
+   `dh` addons. You are recommended to start with source packages without third-party `dh` addons.
+ * It is strongly recommended that your package is bit-for-bit reproducible before starting the conversion
+   as that makes it easier to spot bugs introduced by the conversion!  The output of `debputy` will *not*
+   be bit-for-bit reproducible with `debhelper` in all cases. However, the differences should be easy to
+   review with `diffoscope` if the package was already bit-for-bit reproducible.
+   - Note that `debputy` does not use `strip-nondeterminism`. The bit-for-bit reproducible property should
+     ideally not rely on `strip-nondeterminism` for now.
+
+Note that during the conversion (particularly Step 1 and Step 2), you may find that `debputy` cannot
+support the requirements for your package for now.  Feel free to file an issue for what is holding you
+back in the [debputy issue tracker].
+
+Prerequisites
+-------------
+
+This guide assumes familiarity with Debian packaging in general.  Notably, you should understand
+the different between a (Debian) source package and a (Debian) binary package (e.g., `.deb`) plus
+how these concepts relates to `debian/control` (the source control file).
+
+Additionally, since this is about `debputy` integration with debhelper, the reader is expected to
+be familiar with `debhelper` (notably the `dh` style `debian/rules`).
+
+## Step 1: Have `debputy` convert relevant debhelper files
+
+The `debputy` integration with debhelper removes (replaces) some debhelper tools, but does **not**
+read their relevant config files.  These should instead be converted in to the new format. 
+
+You can have `debputy` convert these for you by using the following command:
+
+    # Dry-run conversion, creates `debian/debputy-manifest.new` if anything is migrated and prints a summary
+    # of what is done.  Remove the --no-act when you are ready to commit the conversion.
+    $ debputy migrate-from-dh --no-act
+
+If relevant, `debputy` may inform you that:
+
+ 1) the package is using an unsupported feature.  Unless that functionality can easily be removed (e.g., it is now
+    obsolete) or easily replaced, then you probably do not want to convert the package to `debputy` at this time.
+    * One common source of unsupported features are dh hook targets (such as override targets), which will be covered
+      in slightly more detail in the next section.  If `debputy` detected any hook targets, it is probably worth it to
+      check if these can be migrated before continuing.
+    * It is worth noting that the migration tool will update an existing manifest when re-run. You can safely "jump"
+      around in the order of the steps, when you try to migrate, if that better suits you.
+ 2) the migration would trigger a conflict.  This can occur for two reasons:
+    * The debhelper configuration has the conflict (example [#934499]), where debhelper is being lenient and ignores
+       the problem.  In this case, you need to resolve the conflict in the debhelper config and re-run `debputy`.
+    * The package has a manifest already with a similar (but not identical) definition of what the migration would
+       generate.  In this case, you need to reconcile the definitions manually (and removing one of them).  After that
+       you can re-run `debputy`.
+
+
+As an example, if you had a `debian/foo.links` (with `foo` being defined in `debian/control`) containing the following:
+
+    usr/share/foo/my-first-symlink usr/share/bar/symlink-target
+    usr/lib/{{DEB_HOST_MULTIARCH}}/my-second-symlink usr/lib/{{DEB_HOST_MULTIARCH}}/baz/symlink-target
+
+The `debputy migrate-from-dh` tool would generate a manifest looking something like this:
+
+    manifest-version: "0.1"
+    packages:
+        foo:
+            transformations:
+             - create-symlink:
+                  path: usr/share/foo/my-first-symlink
+                  target: /usr/share/bar/symlink-target
+             - create-symlink:
+                  path: usr/lib/{{DEB_HOST_MULTIARCH}}/my-second-symlink
+                  target: /usr/lib/{{DEB_HOST_MULTIARCH}}/baz/symlink-target
+
+## Step 2: Migrate override/hook-targets in debian/rules
+
+Once you activate the `debputy` debhelper integration (see Step 3), the following debhelper tools will be
+removed from the `dh` sequence.
+
+ * `dh_install`
+ * `dh_installdocs`
+ * `dh_installchangelogs`
+ * `dh_installexamples`
+ * `dh_installman`
+ * `dh_installcatalogs` **(!)**
+ * `dh_installcron`
+ * `dh_installifupdown`
+ * `dh_installdebconf` **(!)**
+ * `dh_installemacsen` **(!)**
+ * `dh_installinfo`
+ * `dh_installinit` **(!)**
+ * `dh_installsysusers`
+ * `dh_installtmpfiles`
+ * `dh_installsystemd` **(!)**
+ * `dh_installsystemduser` **(!)**
+ * `dh_installmenu` **(!)**
+ * `dh_installmime`
+ * `dh_installmodules`
+ * `dh_installlogcheck`
+ * `dh_installlogrotate`
+ * `dh_installpam`
+ * `dh_installppp`
+ * `dh_installudev`  **(!)**
+ * `dh_installgsettings`
+ * `dh_installinitramfs`
+ * `dh_installalternatives`
+ * `dh_bugfiles`
+ * `dh_ucf` **(!)**
+ * `dh_lintian`
+ * `dh_icons`
+ * `dh_perl`
+ * `dh_usrlocal` **(!)**
+ * `dh_links`
+ * `dh_installwm` **(!)**
+ * `dh_installxfonts`
+ * `dh_strip_nondeterminism`
+ * `dh_compress`
+ * `dh_fixperms`
+ * `dh_dwz`
+ * `dh_strip`
+ * `dh_makeshlibs`
+ * `dh_shlibdeps`
+ * `dh_missing`
+ * `dh_installdeb`
+ * `dh_gencontrol`
+ * `dh_md5sums`
+ * `dh_builddeb`
+
+Have a look at `debian/rules` and migrate any overrides you have for them to `debputy` features or other
+hook targets.  See also the subsections below for concrete advice on how to deal with override or hook targets
+for some of these tools. However, since `debhelper` hooks are arbitrary code execution hooks, there will be
+many cases that the guide will not be able to cover or where `debputy` may not have the feature to support your
+hook target.
+
+While you could manually force these tools to be run via a hook target, they are very likely to feature
+interact with `debputy`. Either causing `debputy` to replace their output completely or by having the tool
+overwrite what `debputy` did (depending on the exact order).  If you find, you *really* need to run one of these
+tools, because `debputy` is not supporting a particular feature they have, then you are probably better off waiting
+with migrating the package in question.
+
+Tools marked with **(!)** have no debputy support at all.  If you rely on these tools, migration is unlikely
+to succeed.  Other tools will have some form of support (often at least a commonly used flow/feature set).
+Where possible, `debputy migrate-from-dh --no-act` will detect these completely unsupported tools via existence
+of their config files or indirectly debhelper hook targets for these tools.  However, some tools may only be
+detected late into the build (which is the case with `dh_usrlocal`).
+
+### Review and migrate any installation code from `dh_install`, `dh_installdocs`, etc. (if any)
+
+All code in `debian/rules` that involves copying or moving files into packages or around in packages must
+be moved to the manifest's `installations` section.  The migration tool will attempt to auto-migrate
+any rules from `debian/install` (etc.). However, be aware of:
+
+ 1) The migration tool assumes none of install rules overlap.  Most packages do not have overlapping
+    install rules as it tends to lead to file conflicts.  If the install rules overlap, `debputy` will
+    detect it at runtime and stop with an error. In that case, you will have to tweak the migrated rules
+    manually.
+ 2) Any overrule target that copies or moves files around in packages must be moved to `installations`
+    (per source) or `transformations` (per package) depending on the case.
+    - For source packages installing content via `debian/tmp`, you can use `install` to rename paths as
+      you install them and `discard` (under `installations`) to ignore paths that should not be installed.
+    - For source packages installing content via `debian/<pkg>`, then everything in there is "auto-installed".
+      If you need to tweak that content, you can use `remove` or `move` transformations (under `transformations`)
+      for manipulation the content.
+
+Note that the migrator "blindly" appends new rules to the bottom of `installations` if you have any existing
+rules (per "none of the install rules overlap"-logic mentioned above).  If you cannot convert all debhelper config
+files in one go, or you manually created some installation rules before running the migrator, you may need to
+manually re-order the generated installation rules to avoid conflicts due to inadequate ordering.  You probably
+want to do so any way for readability/grouping purposes.
+
+
+#### Double-check the `language` settings on all `install-man` rules in `installations`
+
+The `dh_installman` tool auto-detects the language for manpages via two different methods:
+
+ 1) By path name (Does the installation path look like `man/<language>/man<section>/...`?)
+ 2) By basename (Does the basename look like `foo.<language>.<section>`?)
+
+Both methods are used in order with first match being the method of choice.  Unfortunately, the second
+method is prune to false-positives.  Does `foo.pl.1` mean a Polish translation of `foo.1` or is it the
+manpage for a Perl script called `foo.pl` (similar happens for languages/file extensions).
+
+To avoid this issue, `debputy` uses 1) by default and only that.  Option 2) can be chosen by setting
+`language: derive-from-basename` on the concrete install rule.  The problem is that the migration tool
+has to guess, and it is hard to tell if rules like `man/*.1` would need option 2) - especially when the
+migration tool does not attempt to resolve the glob (which it currently does not).
+
+Therefore, take a critical look at the generated `install-man` rules and the generated `language` property
+(or lack thereof).
+
+
+### Convert your overrides or excludes for `dh_fixperms` (if any)
+
+The `debputy` tool will normalize permissions like `dh_fixperms` during package build.  If you have
+any special requirements that `dh_fixperms` did not solve for you, you will have to tell `debputy`
+about them.
+
+If you had something like:
+
+    override_dh_fixperms:
+        dh_fixperms -X bin/sudo
+
+and the goal was to have `dh_fixperms` not touch the mode but the ownership (root:root) was fine, you
+would have the manifest `debian/debputy.manifest`:
+
+    manifest-version: "0.1"
+    packages:
+        foo:
+            transformations:
+             - path-metadata:
+                  path: usr/bin/sudo
+                  mode: "04755"
+
+Note you have to spell out the desired mode for this file.
+
+On the other hand, if your `debian/rules` had something like:
+
+    execute_after_dh_fixperms:
+        chown www-data:www-data debian/foo/var/lib/something-owned-by-www-data
+
+Then the manifest would look something like:
+
+    manifest-version: "0.1"
+    packages:
+        foo:
+            transformations:
+             - path-metadata:
+                  path: var/lib/something-owned-by-www-data
+                  owner: www-data
+                  group: www-data
+
+This can be combined with an explicit `mode: "02755"` if you also need a non-default mode.
+
+The paths provided here support substitution variables (`usr/lib/{{DEB_HOST_MULTIARCH}}/...`) and
+some _limited_ glob support (`usr/bin/sudo*`).
+
+_Remember to merge your manifest with previous steps rather than replacing it!_  Note that
+`debputy migrate-from-dh` will merge its changes into existing manifests and can safely be re-run
+after adding/writing this base manifest.
+
+### Convert your overrides for `dh_gencontrol` (if any)
+
+If the package uses an override to choose a custom version for a binary package, then it is possible in `debputy`
+by  using the `binary-version` key under the package.  Here is an example to force the package `foo` to have
+epoch `1`:
+
+    manifest-version: "0.1"
+    packages:
+        foo:
+            # The foo package needs a different epoch because we took it over from a different
+            # source package with higher epoch version
+            binary-version: '1:{{DEB_VERSION_UPSTREAM_REVISION}}'
+
+Useful if the source took over a binary package from a different source and that binary had a higher
+epoch version.
+
+Note that only limited manipulation of the version is supported, since your options are generally
+limited to expanding one of the following version related variables:
+
+ * `DEB_VERSION` - same definition as the one from `/usr/share/dpkg/pkg-info.mk` (from `dpkg`)
+ * `DEB_VERSION_EPOCH_UPSTREAM` - ditto
+ * `DEB_VERSION_UPSTREAM_REVISION` - ditto
+ * `DEB_VERSION_UPSTREAM` - ditto
+
+If the override is needed for dynamic substitution variables or binary versions that cannot be done with
+the above substitutions, then it might be better to hold off on the conversion.
+
+_Remember to merge your manifest with previous steps rather than replacing it!_  Note that
+`debputy migrate-from-dh` will merge its changes into existing manifests and can safely be re-run
+after adding/writing this base manifest.
+
+## Step 3: Adding the `dh-sequence-zz-debputy` sequence to Build-Depends
+
+The recommended way to do so is to add `dh-sequence-zz-debputy` to the `Build-Depends:` field.
+
+The `zz-` in `zz-debputy` is there to ensure the `debputy` add-on is loaded last by debhelper for
+the rare case there any other debhelper addons still active as the `debputy` sequence does not really
+play well with other `dh` addons. When there are other add-ons, it is generally better for `debputy` to
+be loaded last (as add-on order matters per [#885580]).
+
+## Step 4: Replace third-party dh add-ons to debputy plugins
+
+Packages using third-party `dh` add-ons may need to replace these with `debputy` plugins assuming the
+add-on has a `debputy` plugin to replace it in the first place.  To request a `debputy` plugin, you
+will have to add `debputy-plugin-X` into your `Build-Depends`, where `X` is the name of the plugin.
+
+Note that `debputy` does not support the same features as `debhelper` at the moment for conditional
+plugin loading.  Therefore, the plugins must be in `Build-Depends`.
+
+## Step 5: Verify the build
+
+At this stage, if there are no errors in your previous steps, you should be able to build your
+changed package with `debputy`.  We recommend that you take time to verify this.  For some packages,
+there was no conversion to do in the previous steps, and you would not even need a manifest at all
+in this case.  However, we still recommend that you verify the build is successful here and now.
+
+The `debputy` supports bit-for-bit reproducibility in its output. However, the output is not bit-for-bit
+reproducible with `debhelper`. You are recommended to use `diffoscope` to compare the `debhelper`
+built-version with the `debputy` built version to confirm that all the changes are benign.
+
+However, `debputy` is bit-for-bit reproducible in its output with `(fakeroot) dpkg-deb -b`. Should you
+spot a difference where `debputy` does not produce bit-for-bit identical results with `dpkg-deb` (tar
+format, file ordering, etc.), then please file a bug against `debputy` with a reproducing test case.
+
+Once you have verified your built, the conversion is done. :)  At this point, you can consider
+looking at other features that `debputy` supports that might be useful to you.
+
+[how-to guide]: https://documentation.divio.com/how-to-guides/
+[#885580]: https://bugs.debian.org/885580
+[#934499]: https://bugs.debian.org/934499
+[debputy issue tracker]: https://salsa.debian.org/debian/debputy/-/issues