1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
|
# README for Debian packaging contributors
This documentation describes how to contribute to the official Debian packages
of MariaDB. The packaging in Debian repositories is not identical to the packaging
in mariadb.org repositories, but whatever is in Debian repositories will eventually
be upstreamed.
## Development environment and tools
Use a recent version of Debian or Ubuntu as the environment for Debian packaging
testing and development. Preferred environment is Debian Sid (unstable).
Install the tool used to manage and build the source:
sudo apt-get install git-buildpackage
## Getting the source
The official Debian package source is hosted on the Debian Gitlab server under
the MariaDB/MySQL packaging team at https://salsa.debian.org/mariadb-team/. You
are welcome to fork it and make merge requests.
To get the latest official Debian packaging source of `mariadb`, clone the
source repository with all relevant branches (main branch `debian/latest`) to
your local environment using _git-buildpackage_:
gbp clone https://salsa.debian.org/mariadb-team/mariadb-server.git
If you have your own fork and SSH keys set up on Salsa, you can run:
gbp clone git@salsa.debian.org:<username>/mariadb-server.git
The clone needs to be run only once. On later runs you can refresh your clone with
relevant branches using:
gbp pull --force
## Building the packages
Build binaries, run testsuite and build Debian packages with:
gbp buildpackage
On the first run git-buildpackage will complain if some of the build dependencies
defined in debian/control are missing. Simply install those packages and run the
build again.
A quick command to install all dependencies:
sudo mk-build-deps -r -i debian/control -t "apt-get -y -o Debug::pkgProblemResolver=yes --no-install-recommends"
If the build fails, the easiest way to clean up before a new run is
git clean -fdx && git reset --hard
### Build options
If you want to skip the mysql-test-run step (which takes a lot of time) set
the following environment variable:
export DEB_BUILD_OPTIONS="nocheck"
If you want to run the build in parallel on 2 CPUs and have verbose output:
export DEB_BUILD_OPTIONS="parallel=2 verbose"
The options above can also be combined freely to get desired behavior.
### Using special build environments
If you want to ensure all build dependencies are clean, you can build inside a
Docker or sbuild (Debian tool) environment.
#### Build in Docker
First make a working directory for the build artifacts. Inside that directory
clone the repository. Then start a Docker session using whatever Debian/Ubuntu
image you want with the command:
docker run -it -v ${PWD}:/build -w /build debian:sid bash
This will start a session, where you are as the root user in the path /build
inside the Docker container. Here you can `cd` into the source directory,
install dependencies and start the build. Note that when you exit the session,
everything will be lost apart from the files you had inside the mounted volume
in `/build`.
#### Build using sbuild
If you prefer sbuild, you can build with something like:
gbp buildpackage --git-builder=sbuild -A -v -d unstable
## Creating a feature or bugfix branch
The repository layout follows the DEP-14 standard:
https://dep-team.pages.debian.net/deps/dep14/
All new features and also bug fixes are done only in the `debian/latest` branch.
The release branches for Debian and Ubuntu are only used for security updates.
To prepare the Salsa pull request, create a bugfix branch from master with:
git checkout -b bugfix/NNNNNN-example-name
After this you can develop with all the usual git commit and push commands
until you have in your fork at Salsa the desired change and you are ready
to open the merge request.
### Notes about how to make changes in the proper way
First consider submitting your patch upstream. Upstream MariaDB makes frequent
maintenance releases and any fix done upstream will therefore be included in
Debian relatively quickly. You can send email to the developers mailing list
or open a pull request at https://github.com/MariaDB/server.
Follow these instructions if your fix is about packaging in Debian specifically.
Start by using `gitk --all` or similar tool to browse the previous changes. Try
to follow similar pattern in your new changes.
Keep in mind that all changes must done only for files residing in the `debian/`
sub-directory. If you need to create changes outside the `debian/` directory,
then you need to create a patch file using the same pattern as the patches
found in `debian/patches` and activated by a line in `debian/patches/series`.
Do not bundle in your commit any changes to `debian/changelog`. The correct
changelog entries will be created later by the maintainer using `git-dch` in an
automated fashion.
For an example of a patch adding commit see
https://salsa.debian.org/mariadb-team/mariadb-server/-/commit/7972a38e
# Quality assurance tips
Ensure most packaging files are formatted correctly:
wrap-and-sort -av
Check man pages for syntax errors:
LC_ALL=en_US.UTF-8 MANROFFSEQ='' MANWIDTH=80 man --warnings -E UTF-8 -l -Tutf8 -Z mariadb.1 >/dev/null
Find spelling errors:
find * -type f | xargs spellintian
# Debugging tips
## Debug mariadb-test-run failures
If the test suite is failing on Launchpad or some other CI system where you
don't have access to the build artifacts, you can extend the debian/rules file
to print out valuable information with the commands:
cd $(BUILDDIR)/mysql-test && find var/log/ -ls || true
cd $(BUILDDIR)/mysql-test && cat var/log/*.err || true
cd $(BUILDDIR)/mysql-test && tail -n 1000 var/log/*.log || true
The `cd` is required on every line since it is a Makefile and the actual command
needs to run in the correct directory. Also, the `|| true` at the end ensures
the build will complete and not abort if any of those debug steps fail.
## Debugging with gdb
If the `mariadb-test-run` fails on a `mariadbd` crash it should produce a core
dump file, from which a full stack trace can be produced with:
cd $(BUILDDIR)/mysql-test && gdb --batch --ex 'thr a a bt' var/log/*/mysqld.1/core || true
To attach `gdb` on a running process and get a stack trace run:
gdb -p $(pgrep -x mariadbd) /usr/sbin/mariadbd
set height 0
set logging file /tmp/mysqld.log
set logging on
thread apply all backtrace full
The readability of the stack traces depends on if symbols are available on the
system. In Debian and Ubuntu all (C/C++) software is automatically built with
debug symbols, but to save disk space they are distributed in separate packages
(usually with `-dbg` or `-dbgsym` suffix) which users need to install in the
rare case stack traces are needed. See the Debian and Ubuntu documentation on
how to enable the repository that has the debug symbol packages.
* https://wiki.ubuntu.com/Debug%20Symbol%20Packages
* https://wiki.debian.org/HowToGetABacktrace
## Debug build
A debug build can be created using the following build flags:
-DCMAKE_BUILD_TYPE=Debug \
-DMYSQL_MAINTAINER_MODE=OFF \
The latter flag ensures a build does not stop on warnings but continues to the
end.
A 'mariadbd' binary from a debug build can be started with argument '--debug' to
be verbose about what is going on in the server. Debug binaries should not be
used in production as they are slower than normal binaries.
Core dumps and stack traces can be produced on any build running with
`--core-file --stack-trace` and *debug builds are not needed to run `gdb`*.
## Debugging a running server
Linux distros come standard with tools like `strace` and `lsof` which can also
be used to inspect what processes are doing (no need for debug build). For
example to see what `mariadbd` is writing to the database files can be viewed
with:
strace -ffp $(pgrep -x mariadbd) -e pwrite,write,fsync,fdatasync,sync,send,sendto,sendmsg
lsof -a -p $(pgrep -x mariadbd) | grep "/var/lib/mysql"
## Compare changes between builds
Diffoscope can be used to investigate small changes between recent builds:
docker run --rm -t -w $(pwd) -v $(pwd):$(pwd) registry.salsa.debian.org/reproducible-builds/diffoscope --html-dir report mariadb-server-1.deb mariadb-server-2.deb
firefox report/index.html
## Test autopkgtest locally
If Debian CI fails (or Ubuntu CI) one might need to debug the autopkgtests
manually. The easiest way to do it is to start a Docker container that has
access to the packaging source directory via a local mount:
laptop$ docker run -it -v ${PWD}:/build -w /build debian:sid bash
container$ apt update && apt install -y autopkgtest
container$ autopkgtest --no-built-binaries --shell-fail -- null
Edit the files in `debian/tests` in your favorite code editor and re-run the
`autopkgtest -- null` until the tests are passing. When the autopkgtests work
the container can be shut down and the valid `debian/tests` committed in git.
If you want to iterate on a single test, use `--test-name`, e.g.
`autopkgtest --no-built-binaries --test-name=configuration-tracing -- null`.
If you don't want to use the MariaDB binaries from Debian Sid but instead build
them from the source tree to be used in autopkgtest directly, simply omit
`--no-built-binaries` from the `autopkgtest` command.
For more information please read:
* https://manpages.debian.org/unstable/autopkgtest/autopkgtest.1.en.html
* https://salsa.debian.org/ci-team/autopkgtest/-/tree/master/doc
## Debug installation/upgrade
To see what exactly the Debian maintainer scripts run, they can be made verbose with:
export DEBIAN_SCRIPT_DEBUG=1
apt install ...
The source files of the Debian maintainer scripts are not the final ones, as the
package building stage may make changes and additions to them. To view a
maintainer script in the final form on an installed system run:
cat /var/lib/dpkg/info/mariadb-server.postinst
To review the my.cnf status run:
find /etc/mysql/ -ls
update-alternatives --display my.cnf
## Debug apt Depends/Conflicts/Breaks
It can be quite frustrating to debug situations where `apt` (or `apt-get`) fails
on an install or upgrade with an error message like:
The following packages have unmet dependencies:
mariadb-client : Depends: mariadb-client-10.5 but 1:10.5.12 is to be installed
mariadb-server : Depends: mariadb-server-10.5 but 1:10.5.12 is to be installed
mariadb-test : Depends: mariadb-client-10.5 but 1:10.5.12 is to be installed
Depends: mariadb-server-10.5 but 1:10.5.12 is to be installed
E: Unable to correct problems, you have held broken packages.
To make apt show debug information on what it tried to resolve and how it failed
enable debug features by addin a file in `/etc/apt/apt.conf.d/` with:
Debug::pkgProblemResolver 1;
Debug::pkgDepCache::AutoInstall 1;
Debug::pkgDepCache::Marker 1;
>lternatively append options directly to `apt` commands:
apt dist-upgrade -o Debug::pkgProblemResolver=1
It can be also quite annoying to rebuild the entire package to debug small
changes in the `debian/control` file. To have a much faster change->test->change
cycle one can simply instruct `apt` to use a custom `Packages` file to read the
`control` data.
First ensure `apt` forgets all repositories:
rm /etc/apt/sources.list
apt clean
apt update
Download a Packages file for so it can be edited:
curl -O http://ftp.debian.org/debian/dists/sid/main/binary-amd64/Packages.xz
unxz Packages.xz
cp Packages Packages.orig
Open the file in an editor, scroll down to the MariadB packages and make any
changes you want and then test them:
nano Packages
apt install --with-source ./Packages -s mariadb-server -o Debug::pkgDepCache::Marker=1 -o Debug::pkgDepCache::AutoInstall=1 -o Debug::pkgProblemResolver=1
The example uses maximum verbosity but it is naturally not mandatory. When the
solution has been found, compare to the original and transfer the changes into
the actual debian/control in the MariaDB packaging:
diff -u Packages.orig Packages
## Test install/upgrade with local package repository
Normally the fastest way to test that the built *.deb files install and upgrade
properly is simply to run `apt` directly on them inside a container that has
access to the .deb files via a local mount:
laptop$ docker run -it -v ${PWD}:/build -w /build debian:sid bash
container$ apt update && apt install ./*.deb
Some bugs however occur only when apt does various dependency resolving and can
only be tested with an installation from an actual apt repository. The fastest
way to get a directory with deb files served via a local repository is to run:
apt install apt-utils
apt-ftparchive packages . > Packages
apt-ftparchive release . > Release
echo 'deb [trusted=yes] file:/build/mariadb-bionic ./' >> /etc/apt/sources.list
apt update
apt install mariadb-server
The example above assumes that the .deb files are in path `/build`.
## Check Breaks/Replaces
MariaDB is not only a massive package by itself, it also has several parallel
major releases at any given time and also other variants (MySQL, Percona Server)
the packaging might interact with.
The standard Salsa-CI pipeline checks Breaks/Replaces for what is currently in
the Debian repositories, but to check Breaks/Replaces across all known
repositories one needs to run:
docker run -it -v ${PWD}:/build -w /build debian:sid bash
apt update
apt install --yes python3-junit.xml python3-debian apt-file
curl -O https://salsa.debian.org/salsa-ci-team/pipeline/-/raw/master/images/scripts/check_for_missing_breaks_replaces.py
chmod +x check_for_missing_breaks_replaces.py
apt install --no-install-recommends --yes gpg gpg-agent dirmngr ca-certificates curl debian-archive-keyring
curl -sS https://mariadb.org/mariadb_release_signing_key.asc -o /etc/apt/trusted.gpg.d/mariadb.asc
gpg --list-keys # Initialize default keyring
gpg --no-default-keyring --keyring gnupg-ring:/etc/apt/trusted.gpg.d/mariadb.gpg --keyserver hkps://keyserver.ubuntu.com:443 --recv-keys 871920D1991BC93C 3B4FE6ACC0B21F32 CBF8D6FD518E17E1 7638D0442B90D010 8C718D3B5072E1F5 9334A25F8507EFA5 CBCB082A1BB943DB 467B942D3A79BD29
chmod 644 /etc/apt/trusted.gpg.d/mariadb.gpg
cat > /etc/apt/sources.list.d/mariadb.list <<EOF
deb http://deb.debian.org/debian bullseye main
deb http://deb.debian.org/debian buster main
deb http://deb.debian.org/debian stretch main
deb [trusted=yes] http://deb.debian.org/debian jessie main
deb http://archive.ubuntu.com/ubuntu/ jammy main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ bionic main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ xenial main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ trusty main restricted universe multiverse
deb https://archive.mariadb.org/mariadb-10.11/repo/debian bullseye main
deb https://archive.mariadb.org/mariadb-10.10/repo/debian bullseye main
deb https://archive.mariadb.org/mariadb-10.9/repo/debian bullseye main
deb https://archive.mariadb.org/mariadb-10.8/repo/debian bullseye main
deb https://archive.mariadb.org/mariadb-10.7/repo/debian bullseye main
deb https://archive.mariadb.org/mariadb-10.6/repo/debian buster main
deb https://archive.mariadb.org/mariadb-10.5/repo/debian buster main
deb https://archive.mariadb.org/mariadb-10.4/repo/debian buster main
deb https://archive.mariadb.org/mariadb-10.3/repo/debian buster main
deb https://archive.mariadb.org/mariadb-10.2/repo/debian buster main
deb https://archive.mariadb.org/mariadb-10.1/repo/debian stretch main
deb [trusted=yes] https://archive.mariadb.org/mariadb-10.0/repo/debian jessie main
deb [trusted=yes] https://archive.mariadb.org/mariadb-5.5/repo/debian wheezy main
deb https://repo.mysql.com/apt/ubuntu/ jammy mysql-8.0
deb https://repo.mysql.com/apt/ubuntu/ focal mysql-8.0
deb https://repo.mysql.com/apt/debian/ buster mysql-8.0
deb https://repo.mysql.com/apt/debian/ buster mysql-5.7
deb https://repo.mysql.com/apt/debian/ buster mysql-5.6
deb https://repo.mysql.com/apt/debian/ buster mysql-cluster-8.0
deb https://repo.mysql.com/apt/debian/ buster mysql-cluster-7.6
deb https://repo.mysql.com/apt/debian/ buster mysql-cluster-7.5
deb https://repo.mysql.com/apt/debian/ buster mysql-tools
deb https://repo.percona.com/apt/ bullseye main
deb https://repo.percona.com/apt/ buster main
deb https://repo.percona.com/apt/ stretch main
deb https://repo.percona.com/apt/ jessie main
deb https://repo.percona.com/apt/ wheezy main
EOF
apt-file update
./check_for_missing_breaks_replaces.py --changes-file mariadb-*.changes --debug
## Check reverse dependencies
When making changes to the MariaDB packaging in Debian and Ubuntu, keep in mind
that there are hundreds of packages that depend on MariaDB. Most of them can be
found by running:
apt rdepends 'default-mysql*' 'default-libmysql*' 'mariadb*' 'libmariadb*'
The separate command/package 'apt-rdepends' can also check for reverse
build-dependencies.
Please be diligent in all changes to not wreak havoc in Debian.
|
