Merging upstream version 0.1.22.

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 120 insertions, 68 deletions

diff --git a/GETTING-STARTED-WITH-dh-debputy.md b/GETTING-STARTED-WITH-dh-debputy.md
index 5d5d253..d14e6e4 100644
--- a/GETTING-STARTED-WITH-dh-debputy.md
+++ b/GETTING-STARTED-WITH-dh-debputy.md
@@ -4,20 +4,16 @@ _This is [how-to guide] and is primarily aimed at getting a task done._
 
 <!-- To writers and reviewers: Check the documentation against https://documentation.divio.com/ -->
 
-This document will help you convert a Debian source package using the `dh` sequencer from debhelper to
-use `dh-debputy`.  Prerequisites for this how-to guide:
+This document will help you convert a Debian source package using the `dh` sequencer from `debhelper` to
+use `dh-debputy`, where `debputy` is integrated with `debhelper`.  Prerequisites for this how-to guide:
 
- * You have a Debian source package using the `dh` sequencer.  Ideally a simple one as not all packages
-   can be converted at this time. Note that `debputy` does *not* interact well with most third-party
-   `dh` addons. You are recommended to start with source packages without third-party `dh` addons.
+ * You have a Debian source package using the `dh` sequencer on `debhelper` compat level 12 or later.
  * It is strongly recommended that your package is bit-for-bit reproducible before starting the conversion
    as that makes it easier to spot bugs introduced by the conversion!  The output of `debputy` will *not*
    be bit-for-bit reproducible with `debhelper` in all cases. However, the differences should be easy to
    review with `diffoscope` if the package was already bit-for-bit reproducible.
-   - Note that `debputy` does not use `strip-nondeterminism`. The bit-for-bit reproducible property should
-     ideally not rely on `strip-nondeterminism` for now.
 
-Note that during the conversion (particularly Step 1 and Step 2), you may find that `debputy` cannot
+Note that during the conversion (particularly Step 2 and Step 3), you may find that `debputy` cannot
 support the requirements for your package for now.  Feel free to file an issue for what is holding you
 back in the [debputy issue tracker].
 
@@ -31,40 +27,79 @@ how these concepts relates to `debian/control` (the source control file).
 Additionally, since this is about `debputy` integration with debhelper, the reader is expected to
 be familiar with `debhelper` (notably the `dh` style `debian/rules`).
 
-## Step 1: Have `debputy` convert relevant debhelper files
+## Step 1: Choose the level of migration target
 
-The `debputy` integration with debhelper removes (replaces) some debhelper tools, but does **not**
+When migrating a package to `debputy`, you have to choose at which level you want `debputy` to manage
+your packaging. At the time of writing, your options are:
+
+ 1) `dh-sequence-zz-debputy-rrr`: Minimal integration necessary to get support for `Rules-Requires-Root: no`
+    in all packages.  It is compatible with most `dh` based packages as it only replaces very few helpers
+    and therefore usually also requires very little migration.
+
+ 2) `dh-sequence-zz-debputy`: In this integration mode, `debhelper` manages the upstream build system
+    (that is, anything up to and including `dh_auto_install`). Everything else is managed by `debputy`.
+    This mode is less likely to be compatible with complex packaging at the time of writing. Notably,
+    most `debhelper` add-ons will not be compatible with `debputy` in this integration mode.
+    - For this integration mode, you are recommended to pick a simple package as many packages cannot be
+      converted at this time. Note that this mode does *not* interact well with most third-party
+      `dh` addons. You are recommended to start with source packages without third-party `dh` addons.
+
+Since you can always migrate from "less integrated" to "more integrated", you are recommended to start
+with `dh-sequence-zz-debputy-rrr` first. If that works, you can re-run the migration with
+`dh-sequence-zz-debputy` as the target to see if further integration seems feasible / desirable.
+
+Note: More options may appear in the future.
+
+## Step 2: Have `debputy` convert relevant `debhelper` files
+
+The `debputy` integration with `debhelper` removes (replaces) some debhelper tools, but does **not**
 read their relevant config files.  These should instead be converted in to the new format. 
 
 You can have `debputy` convert these for you by using the following command:
 
     # Dry-run conversion, creates `debian/debputy-manifest.new` if anything is migrated and prints a summary
-    # of what is done.  Remove the --no-act when you are ready to commit the conversion.
-    $ debputy migrate-from-dh --no-act
+    # of what is done and what you manually need to fix.  Replace `--no-apply-changes` with `--apply-changes`
+    # when you are ready to commit the conversion.
+    #  - Replace MIGRATION_TARGET in the command with the concrete target from step 1
+    $ debputy migrate-from-dh --migration-target MIGRATION_TARGET --no-apply-changes
+
+Note: Running `debputy migrate-from-dh ...` multiple times is supported and often recommended as it can
+help you assert that all detectable issues have been fixed.
 
 If relevant, `debputy` may inform you that:
 
- 1) the package is using an unsupported feature.  Unless that functionality can easily be removed (e.g., it is now
+ 1) the package needs to change build dependencies or/and activate `dh` add-ons. Concretely, it will ask you to
+    ensure that the relevant `dh` add-on is added for `debputy` sequence matching the migration target. Additionally,
+    it may ask you to add build-dependencies for activating `debputy` plugins to match certain `dh` add-ons.
+    - Note that if `debputy` asks you to add a `debputy` plugin without removing the `dh` add-on it supposedly
+      replaces then keep the `dh` active. Sometimes the add-on is only partly converted and sometimes the `dh`
+      sequence is used to pull relevant build-dependencies.
+
+ 2) the package is using an unsupported feature.  Unless that functionality can easily be removed (e.g., it is now
     obsolete) or easily replaced, then you probably do not want to convert the package to `debputy` at this time.
     * One common source of unsupported features are dh hook targets (such as override targets), which will be covered
       in slightly more detail in the next section.  If `debputy` detected any hook targets, it is probably worth it to
-      check if these can be migrated before continuing.
+      check if these can be migrated before continuing or run earlier in a hook target that is not removed.
+    * Other cases can be that the package uses a feature, where `debputy` behaves differently than `debhelper` would
+      at the given compat level. In this case, `debputy` will recommend that you perform a compat upgrade in
+      `debhelper` before migrating to `debputy`.
     * It is worth noting that the migration tool will update an existing manifest when re-run. You can safely "jump"
       around in the order of the steps, when you try to migrate, if that better suits you.
- 2) the migration would trigger a conflict.  This can occur for two reasons:
+
+ 3) the migration would trigger a conflict.  This can occur for two reasons:
     * The debhelper configuration has the conflict (example [#934499]), where debhelper is being lenient and ignores
        the problem.  In this case, you need to resolve the conflict in the debhelper config and re-run `debputy`.
     * The package has a manifest already with a similar (but not identical) definition of what the migration would
        generate.  In this case, you need to reconcile the definitions manually (and removing one of them).  After that
        you can re-run `debputy`.
 
-
 As an example, if you had a `debian/foo.links` (with `foo` being defined in `debian/control`) containing the following:
 
     usr/share/foo/my-first-symlink usr/share/bar/symlink-target
     usr/lib/{{DEB_HOST_MULTIARCH}}/my-second-symlink usr/lib/{{DEB_HOST_MULTIARCH}}/baz/symlink-target
 
-The `debputy migrate-from-dh` tool would generate a manifest looking something like this:
+The `debputy migrate-from-dh --migration-target dh-sequence-zz-debputy` tool would generate a manifest looking
+something like this:
 
     manifest-version: "0.1"
     packages:
@@ -77,10 +112,47 @@ The `debputy migrate-from-dh` tool would generate a manifest looking something l
                   path: usr/lib/{{DEB_HOST_MULTIARCH}}/my-second-symlink
                   target: /usr/lib/{{DEB_HOST_MULTIARCH}}/baz/symlink-target
 
-## Step 2: Migrate override/hook-targets in debian/rules
 
-Once you activate the `debputy` debhelper integration (see Step 3), the following debhelper tools will be
-removed from the `dh` sequence.
+## Step 3: Migrate override/hook-targets in debian/rules
+
+Have a look at the hook targets that `debputy migrate-from-dh` flags as unsupported and migrate them to
+`debputy` features or move them to earlier hook targets that are supported.  See also the subsections
+below for concrete advice on how to deal with override or hook targets for some of these tools. However,
+since `debhelper` hooks are arbitrary code execution hooks, there will be many cases that the guide will
+not be able to cover or where `debputy` may not have the feature to support your hook target.
+
+While you could manually force any of the removed `debhelper` tools to be run via a hook target, they are
+very likely to feature  interact with `debputy`. Either causing `debputy` to replace their output completely
+or by having the tool overwrite what `debputy` did (depending on the exact order).  If you find, you
+*really* need to run one of these tools, because `debputy` is not supporting a particular feature they have,
+then you are probably better off not migrate to this level of `debputy` integration at this time.
+
+
+### Affected debhelper command list for `dh-sequence-zz-debputy-rrr` integration mode
+
+The following `debhelper` commands are replaced in the `dh-sequence-zz-debputy-rrr` integration mode. Generally,
+`debputy migrate-from-dh` will warn you if there is anything to worry about in relation to these commands.
+
+ * `dh_fixperms`
+ * `dh_gencontrol`
+ * `dh_md5sums`
+ * `dh_builddeb`
+
+In case something is flagged, it is most likely a hook target, which either have to be converted to `debputy`'s
+features or moved earlier. The most common cases are hook targets for `dh_fixperms` and `dh_gencontrol`, which
+have sections below advising how to approach those.
+
+### Affected debhelper command list for `dh-sequence-zz-debputy` integration mode
+
+The following `debhelper` commands are replaced in the `dh-sequence-zz-debputy` integration mode. The
+`debputy migrate-from-dh` command will warn you where it can when some of these are used. However, some
+usage may only become apparent during package build. Therefore, `debputy migrate-from-dh` can be silent
+even though `debputy` during the build will flag an unsupported feature.
+
+You are recommended to skim through this list for commands where your package might be using non-trivial
+or non-default features, as those are likely to prevent migration at this time. Pay extra attention to
+commands marked with **(!)** as `debputy` has zero or almost zero support for features from these
+commands. Other tools will have some form of support (often at least a commonly used flow/feature set).
 
  * `dh_install`
  * `dh_installdocs`
@@ -93,10 +165,14 @@ removed from the `dh` sequence.
  * `dh_installdebconf` **(!)**
  * `dh_installemacsen` **(!)**
  * `dh_installinfo`
- * `dh_installinit` **(!)**
+ * `dh_installinit`
+    - Only default service mode (enable, standard restart) are supported. Any use of `--no-start`, etc.
+      is unsupported
  * `dh_installsysusers`
  * `dh_installtmpfiles`
- * `dh_installsystemd` **(!)**
+ * `dh_installsystemd`
+    - Only default service mode (enable, standard restart) are supported. Any use of `--no-start`, etc.
+      is unsupported
  * `dh_installsystemduser` **(!)**
  * `dh_installmenu` **(!)**
  * `dh_installmime`
@@ -129,53 +205,49 @@ removed from the `dh` sequence.
  * `dh_installdeb`
  * `dh_gencontrol`
  * `dh_md5sums`
- * `dh_builddeb`
 
-Have a look at `debian/rules` and migrate any overrides you have for them to `debputy` features or other
-hook targets.  See also the subsections below for concrete advice on how to deal with override or hook targets
-for some of these tools. However, since `debhelper` hooks are arbitrary code execution hooks, there will be
-many cases that the guide will not be able to cover or where `debputy` may not have the feature to support your
-hook target.
-
-While you could manually force these tools to be run via a hook target, they are very likely to feature
-interact with `debputy`. Either causing `debputy` to replace their output completely or by having the tool
-overwrite what `debputy` did (depending on the exact order).  If you find, you *really* need to run one of these
-tools, because `debputy` is not supporting a particular feature they have, then you are probably better off waiting
-with migrating the package in question.
-
-Tools marked with **(!)** have no debputy support at all.  If you rely on these tools, migration is unlikely
-to succeed.  Other tools will have some form of support (often at least a commonly used flow/feature set).
-Where possible, `debputy migrate-from-dh --no-act` will detect these completely unsupported tools via existence
-of their config files or indirectly debhelper hook targets for these tools.  However, some tools may only be
-detected late into the build (which is the case with `dh_usrlocal`).
+As mentioned, `debputy migrate-from-dh --no-act` will detect these completely unsupported tools via existence
+of their config files or indirectly debhelper hook targets for these tools where possible.  However, some
+tools may only be detected late into the build (which is the case with `dh_usrlocal` as a concrete
+example).
 
 ### Review and migrate any installation code from `dh_install`, `dh_installdocs`, etc. (if any)
 
+_This is section does **not** apply to the `dh-sequence-zz-debputy-rrr` integration mode._
+
 All code in `debian/rules` that involves copying or moving files into packages or around in packages must
 be moved to the manifest's `installations` section.  The migration tool will attempt to auto-migrate
 any rules from `debian/install` (etc.). However, be aware of:
 
  1) The migration tool assumes none of install rules overlap.  Most packages do not have overlapping
     install rules as it tends to lead to file conflicts.  If the install rules overlap, `debputy` will
-    detect it at runtime and stop with an error. In that case, you will have to tweak the migrated rules
-    manually.
- 2) Any overrule target that copies or moves files around in packages must be moved to `installations`
+    detect it at *runtime* (not migration time) and stop with an error. In that case, you will have to
+    tweak the migrated rules manually.
+
+ 2) Any hook target that copies or moves files around in packages must be moved to `installations`
     (per source) or `transformations` (per package) depending on the case.
+
     - For source packages installing content via `debian/tmp`, you can use `install` to rename paths as
       you install them and `discard` (under `installations`) to ignore paths that should not be installed.
+
     - For source packages installing content via `debian/<pkg>`, then everything in there is "auto-installed".
       If you need to tweak that content, you can use `remove` or `move` transformations (under `transformations`)
       for manipulation the content.
 
-Note that the migrator "blindly" appends new rules to the bottom of `installations` if you have any existing
+Keep in mind that the migrator "blindly" appends new rules to the bottom of `installations` if you have any existing
 rules (per "none of the install rules overlap"-logic mentioned above).  If you cannot convert all debhelper config
 files in one go, or you manually created some installation rules before running the migrator, you may need to
 manually re-order the generated installation rules to avoid conflicts due to inadequate ordering.  You probably
 want to do so any way for readability/grouping purposes.
 
+Note: For very complex hook targets that manipulate context in packages, it is possible to keep the logic in
+`debian/rules` by moving the logic to `execute_after_dh_auto_install`. This will mainly be useful if you are
+using complex rules for pruning or moving around files in a way that are not easily representable via globs.
 
 #### Double-check the `language` settings on all `install-man` rules in `installations`
 
+_This is section does **not** apply to the `dh-sequence-zz-debputy-rrr` integration mode._
+
 The `dh_installman` tool auto-detects the language for manpages via two different methods:
 
  1) By path name (Does the installation path look like `man/<language>/man<section>/...`?)
@@ -183,17 +255,15 @@ The `dh_installman` tool auto-detects the language for manpages via two differen
 
 Both methods are used in order with first match being the method of choice.  Unfortunately, the second
 method is prune to false-positives.  Does `foo.pl.1` mean a Polish translation of `foo.1` or is it the
-manpage for a Perl script called `foo.pl` (similar happens for languages/file extensions).
+manpage for a Perl script called `foo.pl` (similar happens for other languages/file extensions).
 
 To avoid this issue, `debputy` uses 1) by default and only that.  Option 2) can be chosen by setting
-`language: derive-from-basename` on the concrete install rule.  The problem is that the migration tool
-has to guess, and it is hard to tell if rules like `man/*.1` would need option 2) - especially when the
-migration tool does not attempt to resolve the glob (which it currently does not).
+`language: derive-from-basename` on the concrete installation rule.  The problem is that the migration tool
+has to guess, and it is hard to tell if rules like `man/*.1` would need option 2).
 
 Therefore, take a critical look at the generated `install-man` rules and the generated `language` property
 (or lack thereof).
 
-
 ### Convert your overrides or excludes for `dh_fixperms` (if any)
 
 The `debputy` tool will normalize permissions like `dh_fixperms` during package build.  If you have
@@ -274,25 +344,7 @@ _Remember to merge your manifest with previous steps rather than replacing it!_
 `debputy migrate-from-dh` will merge its changes into existing manifests and can safely be re-run
 after adding/writing this base manifest.
 
-## Step 3: Adding the `dh-sequence-zz-debputy` sequence to Build-Depends
-
-The recommended way to do so is to add `dh-sequence-zz-debputy` to the `Build-Depends:` field.
-
-The `zz-` in `zz-debputy` is there to ensure the `debputy` add-on is loaded last by debhelper for
-the rare case there any other debhelper addons still active as the `debputy` sequence does not really
-play well with other `dh` addons. When there are other add-ons, it is generally better for `debputy` to
-be loaded last (as add-on order matters per [#885580]).
-
-## Step 4: Replace third-party dh add-ons to debputy plugins
-
-Packages using third-party `dh` add-ons may need to replace these with `debputy` plugins assuming the
-add-on has a `debputy` plugin to replace it in the first place.  To request a `debputy` plugin, you
-will have to add `debputy-plugin-X` into your `Build-Depends`, where `X` is the name of the plugin.
-
-Note that `debputy` does not support the same features as `debhelper` at the moment for conditional
-plugin loading.  Therefore, the plugins must be in `Build-Depends`.
-
-## Step 5: Verify the build
+## Step 4: Verify the build
 
 At this stage, if there are no errors in your previous steps, you should be able to build your
 changed package with `debputy`.  We recommend that you take time to verify this.  For some packages,