summaryrefslogtreecommitdiffstats
path: root/debian/README.Contributor
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-13 12:24:44 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-13 12:24:44 +0000
commitbf772a107fc4511b6e3e5aafddb762ca3fa74f0b (patch)
tree8d994c2f3d8e9f01da4d334f90a20798760d9318 /debian/README.Contributor
parentAdding upstream version 1:10.11.6. (diff)
downloadmariadb-bf772a107fc4511b6e3e5aafddb762ca3fa74f0b.tar.xz
mariadb-bf772a107fc4511b6e3e5aafddb762ca3fa74f0b.zip
Adding debian version 1:10.11.6-2.debian/1%10.11.6-2
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'debian/README.Contributor')
-rw-r--r--debian/README.Contributor420
1 files changed, 420 insertions, 0 deletions
diff --git a/debian/README.Contributor b/debian/README.Contributor
new file mode 100644
index 00000000..45708b52
--- /dev/null
+++ b/debian/README.Contributor
@@ -0,0 +1,420 @@
+# README for Debian packaging contributors
+
+This documentation describes how to contribute to the official Debian packages
+of MariaDB. The packaging in Debian repositories is not identical to the packaging
+in mariadb.org repositories, but whatever is in Debian repositories will eventually
+be upstreamed.
+
+
+## Development environment and tools
+
+Use a recent version of Debian or Ubuntu as the environment for Debian packaging
+testing and development. Preferred environment is Debian Sid (unstable).
+
+Install the tool used to manage and build the source:
+
+ sudo apt-get install git-buildpackage
+
+
+## Getting the source
+
+The official Debian package source is hosted on the Debian Gitlab server under
+the MariaDB/MySQL packaging team at https://salsa.debian.org/mariadb-team/. You
+are welcome to fork it and make merge requests.
+
+To get the latest official Debian packaging source of `mariadb`, clone the
+source repository with all relevant branches (main branch `debian/latest`) to
+your local environment using _git-buildpackage_:
+
+ gbp clone https://salsa.debian.org/mariadb-team/mariadb-server.git
+
+If you have your own fork and SSH keys set up on Salsa, you can run:
+
+ gbp clone git@salsa.debian.org:<username>/mariadb-server.git
+
+
+The clone needs to be run only once. On later runs you can refresh your clone with
+relevant branches using:
+
+ gbp pull --force
+
+
+## Building the packages
+
+Build binaries, run testsuite and build Debian packages with:
+
+ gbp buildpackage
+
+On the first run git-buildpackage will complain if some of the build dependencies
+defined in debian/control are missing. Simply install those packages and run the
+build again.
+
+A quick command to install all dependencies:
+
+ sudo mk-build-deps -r -i debian/control -t "apt-get -y -o Debug::pkgProblemResolver=yes --no-install-recommends"
+
+If the build fails, the easiest way to clean up before a new run is
+
+ git clean -fdx && git reset --hard
+
+
+### Build options
+
+If you want to skip the mysql-test-run step (which takes a lot of time) set
+the following environment variable:
+
+ export DEB_BUILD_OPTIONS="nocheck"
+
+If you want to run the build in parallel on 2 CPUs and have verbose output:
+
+ export DEB_BUILD_OPTIONS="parallel=2 verbose"
+
+The options above can also be combined freely to get desired behavior.
+
+
+### Using special build environments
+
+If you want to ensure all build dependencies are clean, you can build inside a
+Docker or sbuild (Debian tool) environment.
+
+#### Build in Docker
+
+First make a working directory for the build artifacts. Inside that directory
+clone the repository. Then start a Docker session using whatever Debian/Ubuntu
+image you want with the command:
+
+ docker run -it -v ${PWD}:/build -w /build debian:sid bash
+
+This will start a session, where you are as the root user in the path /build
+inside the Docker container. Here you can `cd` into the source directory,
+install dependencies and start the build. Note that when you exit the session,
+everything will be lost apart from the files you had inside the mounted volume
+in `/build`.
+
+#### Build using sbuild
+
+If you prefer sbuild, you can build with something like:
+
+ gbp buildpackage --git-builder=sbuild -A -v -d unstable
+
+## Creating a feature or bugfix branch
+
+The repository layout follows the DEP-14 standard:
+https://dep-team.pages.debian.net/deps/dep14/
+
+All new features and also bug fixes are done only in the `debian/latest` branch.
+The release branches for Debian and Ubuntu are only used for security updates.
+
+To prepare the Salsa pull request, create a bugfix branch from master with:
+
+ git checkout -b bugfix/NNNNNN-example-name
+
+After this you can develop with all the usual git commit and push commands
+until you have in your fork at Salsa the desired change and you are ready
+to open the merge request.
+
+
+### Notes about how to make changes in the proper way
+
+First consider submitting your patch upstream. Upstream MariaDB makes frequent
+maintenance releases and any fix done upstream will therefore be included in
+Debian relatively quickly. You can send email to the developers mailing list
+or open a pull request at https://github.com/MariaDB/server.
+
+Follow these instructions if your fix is about packaging in Debian specifically.
+Start by using `gitk --all` or similar tool to browse the previous changes. Try
+to follow similar pattern in your new changes.
+
+Keep in mind that all changes must done only for files residing in the `debian/`
+sub-directory. If you need to create changes outside the `debian/` directory,
+then you need to create a patch file using the same pattern as the patches
+found in `debian/patches` and activated by a line in `debian/patches/series`.
+
+Do not bundle in your commit any changes to `debian/changelog`. The correct
+changelog entries will be created later by the maintainer using `git-dch` in an
+automated fashion.
+
+For an example of a patch adding commit see
+https://salsa.debian.org/mariadb-team/mariadb-server/-/commit/7972a38e
+
+
+# Quality assurance tips
+
+Ensure most packaging files are formatted correctly:
+
+ wrap-and-sort -av
+
+Check man pages for syntax errors:
+
+ LC_ALL=en_US.UTF-8 MANROFFSEQ='' MANWIDTH=80 man --warnings -E UTF-8 -l -Tutf8 -Z mariadb.1 >/dev/null
+
+Find spelling errors:
+
+ find * -type f | xargs spellintian
+
+
+# Debugging tips
+
+## Debug mariadb-test-run failures
+
+If the test suite is failing on Launchpad or some other CI system where you
+don't have access to the build artifacts, you can extend the debian/rules file
+to print out valuable information with the commands:
+
+ cd $(BUILDDIR)/mysql-test && find var/log/ -ls || true
+ cd $(BUILDDIR)/mysql-test && cat var/log/*.err || true
+ cd $(BUILDDIR)/mysql-test && tail -n 1000 var/log/*.log || true
+
+The `cd` is required on every line since it is a Makefile and the actual command
+needs to run in the correct directory. Also, the `|| true` at the end ensures
+the build will complete and not abort if any of those debug steps fail.
+
+## Debugging with gdb
+
+If the `mariadb-test-run` fails on a `mariadbd` crash it should produce a core
+dump file, from which a full stack trace can be produced with:
+
+ cd $(BUILDDIR)/mysql-test && gdb --batch --ex 'thr a a bt' var/log/*/mysqld.1/core || true
+
+To attach `gdb` on a running process and get a stack trace run:
+
+ gdb -p $(pgrep -x mariadbd) /usr/sbin/mariadbd
+ set height 0
+ set logging file /tmp/mysqld.log
+ set logging on
+ thread apply all backtrace full
+
+The readability of the stack traces depends on if symbols are available on the
+system. In Debian and Ubuntu all (C/C++) software is automatically built with
+debug symbols, but to save disk space they are distributed in separate packages
+(usually with `-dbg` or `-dbgsym` suffix) which users need to install in the
+rare case stack traces are needed. See the Debian and Ubuntu documentation on
+how to enable the repository that has the debug symbol packages.
+
+* https://wiki.ubuntu.com/Debug%20Symbol%20Packages
+* https://wiki.debian.org/HowToGetABacktrace
+
+## Debug build
+
+A debug build can be created using the following build flags:
+
+ -DCMAKE_BUILD_TYPE=Debug \
+ -DMYSQL_MAINTAINER_MODE=OFF \
+
+The latter flag ensures a build does not stop on warnings but continues to the
+end.
+
+A 'mariadbd' binary from a debug build can be started with argument '--debug' to
+be verbose about what is going on in the server. Debug binaries should not be
+used in production as they are slower than normal binaries.
+
+Core dumps and stack traces can be produced on any build running with
+`--core-file --stack-trace` and *debug builds are not needed to run `gdb`*.
+
+## Debugging a running server
+
+Linux distros come standard with tools like `strace` and `lsof` which can also
+be used to inspect what processes are doing (no need for debug build). For
+example to see what `mariadbd` is writing to the database files can be viewed
+with:
+
+ strace -ffp $(pgrep -x mariadbd) -e pwrite,write,fsync,fdatasync,sync,send,sendto,sendmsg
+ lsof -a -p $(pgrep -x mariadbd) | grep "/var/lib/mysql"
+
+## Compare changes between builds
+
+Diffoscope can be used to investigate small changes between recent builds:
+
+ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd) registry.salsa.debian.org/reproducible-builds/diffoscope --html-dir report mariadb-server-1.deb mariadb-server-2.deb
+ firefox report/index.html
+
+## Test autopkgtest locally
+
+If Debian CI fails (or Ubuntu CI) one might need to debug the autopkgtests
+manually. The easiest way to do it is to start a Docker container that has
+access to the packaging source directory via a local mount:
+
+ laptop$ docker run -it -v ${PWD}:/build -w /build debian:sid bash
+ container$ apt update && apt install -y autopkgtest
+ container$ autopkgtest --no-built-binaries --shell-fail -- null
+
+Edit the files in `debian/tests` in your favorite code editor and re-run the
+`autopkgtest -- null` until the tests are passing. When the autopkgtests work
+the container can be shut down and the valid `debian/tests` committed in git.
+
+If you want to iterate on a single test, use `--test-name`, e.g.
+`autopkgtest --no-built-binaries --test-name=configuration-tracing -- null`.
+
+If you don't want to use the MariaDB binaries from Debian Sid but instead build
+them from the source tree to be used in autopkgtest directly, simply omit
+`--no-built-binaries` from the `autopkgtest` command.
+
+For more information please read:
+* https://manpages.debian.org/unstable/autopkgtest/autopkgtest.1.en.html
+* https://salsa.debian.org/ci-team/autopkgtest/-/tree/master/doc
+
+## Debug installation/upgrade
+
+To see what exactly the Debian maintainer scripts run, they can be made verbose with:
+
+ export DEBIAN_SCRIPT_DEBUG=1
+ apt install ...
+
+The source files of the Debian maintainer scripts are not the final ones, as the
+package building stage may make changes and additions to them. To view a
+maintainer script in the final form on an installed system run:
+
+ cat /var/lib/dpkg/info/mariadb-server.postinst
+
+To review the my.cnf status run:
+
+ find /etc/mysql/ -ls
+ update-alternatives --display my.cnf
+
+## Debug apt Depends/Conflicts/Breaks
+
+It can be quite frustrating to debug situations where `apt` (or `apt-get`) fails
+on an install or upgrade with an error message like:
+
+ The following packages have unmet dependencies:
+ mariadb-client : Depends: mariadb-client-10.5 but 1:10.5.12 is to be installed
+ mariadb-server : Depends: mariadb-server-10.5 but 1:10.5.12 is to be installed
+ mariadb-test : Depends: mariadb-client-10.5 but 1:10.5.12 is to be installed
+ Depends: mariadb-server-10.5 but 1:10.5.12 is to be installed
+ E: Unable to correct problems, you have held broken packages.
+
+To make apt show debug information on what it tried to resolve and how it failed
+enable debug features by addin a file in `/etc/apt/apt.conf.d/` with:
+
+ Debug::pkgProblemResolver 1;
+ Debug::pkgDepCache::AutoInstall 1;
+ Debug::pkgDepCache::Marker 1;
+
+>lternatively append options directly to `apt` commands:
+
+ apt dist-upgrade -o Debug::pkgProblemResolver=1
+
+It can be also quite annoying to rebuild the entire package to debug small
+changes in the `debian/control` file. To have a much faster change->test->change
+cycle one can simply instruct `apt` to use a custom `Packages` file to read the
+`control` data.
+
+First ensure `apt` forgets all repositories:
+
+ rm /etc/apt/sources.list
+ apt clean
+ apt update
+
+Download a Packages file for so it can be edited:
+
+ curl -O http://ftp.debian.org/debian/dists/sid/main/binary-amd64/Packages.xz
+ unxz Packages.xz
+ cp Packages Packages.orig
+
+Open the file in an editor, scroll down to the MariadB packages and make any
+changes you want and then test them:
+
+ nano Packages
+ apt install --with-source ./Packages -s mariadb-server -o Debug::pkgDepCache::Marker=1 -o Debug::pkgDepCache::AutoInstall=1 -o Debug::pkgProblemResolver=1
+
+The example uses maximum verbosity but it is naturally not mandatory. When the
+solution has been found, compare to the original and transfer the changes into
+the actual debian/control in the MariaDB packaging:
+
+ diff -u Packages.orig Packages
+
+## Test install/upgrade with local package repository
+
+Normally the fastest way to test that the built *.deb files install and upgrade
+properly is simply to run `apt` directly on them inside a container that has
+access to the .deb files via a local mount:
+
+ laptop$ docker run -it -v ${PWD}:/build -w /build debian:sid bash
+ container$ apt update && apt install ./*.deb
+
+Some bugs however occur only when apt does various dependency resolving and can
+only be tested with an installation from an actual apt repository. The fastest
+way to get a directory with deb files served via a local repository is to run:
+
+ apt install apt-utils
+ apt-ftparchive packages . > Packages
+ apt-ftparchive release . > Release
+ echo 'deb [trusted=yes] file:/build/mariadb-bionic ./' >> /etc/apt/sources.list
+ apt update
+ apt install mariadb-server
+
+The example above assumes that the .deb files are in path `/build`.
+
+## Check Breaks/Replaces
+
+MariaDB is not only a massive package by itself, it also has several parallel
+major releases at any given time and also other variants (MySQL, Percona Server)
+the packaging might interact with.
+
+The standard Salsa-CI pipeline checks Breaks/Replaces for what is currently in
+the Debian repositories, but to check Breaks/Replaces across all known
+repositories one needs to run:
+
+ docker run -it -v ${PWD}:/build -w /build debian:sid bash
+ apt update
+ apt install --yes python3-junit.xml python3-debian apt-file
+ curl -O https://salsa.debian.org/salsa-ci-team/pipeline/-/raw/master/images/scripts/check_for_missing_breaks_replaces.py
+ chmod +x check_for_missing_breaks_replaces.py
+ apt install --no-install-recommends --yes gpg gpg-agent dirmngr ca-certificates curl debian-archive-keyring
+ curl -sS https://mariadb.org/mariadb_release_signing_key.asc -o /etc/apt/trusted.gpg.d/mariadb.asc
+ gpg --list-keys # Initialize default keyring
+ gpg --no-default-keyring --keyring gnupg-ring:/etc/apt/trusted.gpg.d/mariadb.gpg --keyserver hkps://keyserver.ubuntu.com:443 --recv-keys 871920D1991BC93C 3B4FE6ACC0B21F32 CBF8D6FD518E17E1 7638D0442B90D010 8C718D3B5072E1F5 9334A25F8507EFA5 CBCB082A1BB943DB 467B942D3A79BD29
+ chmod 644 /etc/apt/trusted.gpg.d/mariadb.gpg
+ cat > /etc/apt/sources.list.d/mariadb.list <<EOF
+ deb http://deb.debian.org/debian bullseye main
+ deb http://deb.debian.org/debian buster main
+ deb http://deb.debian.org/debian stretch main
+ deb [trusted=yes] http://deb.debian.org/debian jessie main
+ deb http://archive.ubuntu.com/ubuntu/ jammy main restricted universe multiverse
+ deb http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse
+ deb http://archive.ubuntu.com/ubuntu/ bionic main restricted universe multiverse
+ deb http://archive.ubuntu.com/ubuntu/ xenial main restricted universe multiverse
+ deb http://archive.ubuntu.com/ubuntu/ trusty main restricted universe multiverse
+ deb https://archive.mariadb.org/mariadb-10.11/repo/debian bullseye main
+ deb https://archive.mariadb.org/mariadb-10.10/repo/debian bullseye main
+ deb https://archive.mariadb.org/mariadb-10.9/repo/debian bullseye main
+ deb https://archive.mariadb.org/mariadb-10.8/repo/debian bullseye main
+ deb https://archive.mariadb.org/mariadb-10.7/repo/debian bullseye main
+ deb https://archive.mariadb.org/mariadb-10.6/repo/debian buster main
+ deb https://archive.mariadb.org/mariadb-10.5/repo/debian buster main
+ deb https://archive.mariadb.org/mariadb-10.4/repo/debian buster main
+ deb https://archive.mariadb.org/mariadb-10.3/repo/debian buster main
+ deb https://archive.mariadb.org/mariadb-10.2/repo/debian buster main
+ deb https://archive.mariadb.org/mariadb-10.1/repo/debian stretch main
+ deb [trusted=yes] https://archive.mariadb.org/mariadb-10.0/repo/debian jessie main
+ deb [trusted=yes] https://archive.mariadb.org/mariadb-5.5/repo/debian wheezy main
+ deb https://repo.mysql.com/apt/ubuntu/ jammy mysql-8.0
+ deb https://repo.mysql.com/apt/ubuntu/ focal mysql-8.0
+ deb https://repo.mysql.com/apt/debian/ buster mysql-8.0
+ deb https://repo.mysql.com/apt/debian/ buster mysql-5.7
+ deb https://repo.mysql.com/apt/debian/ buster mysql-5.6
+ deb https://repo.mysql.com/apt/debian/ buster mysql-cluster-8.0
+ deb https://repo.mysql.com/apt/debian/ buster mysql-cluster-7.6
+ deb https://repo.mysql.com/apt/debian/ buster mysql-cluster-7.5
+ deb https://repo.mysql.com/apt/debian/ buster mysql-tools
+ deb https://repo.percona.com/apt/ bullseye main
+ deb https://repo.percona.com/apt/ buster main
+ deb https://repo.percona.com/apt/ stretch main
+ deb https://repo.percona.com/apt/ jessie main
+ deb https://repo.percona.com/apt/ wheezy main
+ EOF
+ apt-file update
+ ./check_for_missing_breaks_replaces.py --changes-file mariadb-*.changes --debug
+
+## Check reverse dependencies
+
+When making changes to the MariaDB packaging in Debian and Ubuntu, keep in mind
+that there are hundreds of packages that depend on MariaDB. Most of them can be
+found by running:
+
+ apt rdepends 'default-mysql*' 'default-libmysql*' 'mariadb*' 'libmariadb*'
+
+The separate command/package 'apt-rdepends' can also check for reverse
+build-dependencies.
+
+Please be diligent in all changes to not wreak havoc in Debian.