diff options
Diffstat (limited to '')
| -rw-r--r-- | docs/HACKING.md | 296 |
1 files changed, 152 insertions, 144 deletions
diff --git a/docs/HACKING.md b/docs/HACKING.md index 45334d8..51499d7 100644 --- a/docs/HACKING.md +++ b/docs/HACKING.md @@ -7,41 +7,33 @@ SPDX-License-Identifier: LGPL-2.1-or-later # Hacking on systemd -We welcome all contributions to systemd. If you notice a bug or a missing -feature, please feel invited to fix it, and submit your work as a +We welcome all contributions to systemd. +If you notice a bug or a missing feature, please feel invited to fix it, and submit your work as a [GitHub Pull Request (PR)](https://github.com/systemd/systemd/pull/new). -Please make sure to follow our [Coding Style](/CODING_STYLE) when submitting -patches. Also have a look at our [Contribution Guidelines](/CONTRIBUTING). - -When adding new functionality, tests should be added. For shared functionality -(in `src/basic/` and `src/shared/`) unit tests should be sufficient. The general -policy is to keep tests in matching files underneath `src/test/`, -e.g. `src/test/test-path-util.c` contains tests for any functions in -`src/basic/path-util.c`. If adding a new source file, consider adding a matching -test executable. For features at a higher level, tests in `src/test/` are very -strongly recommended. If that is not possible, integration tests in `test/` are -encouraged. - -Please also have a look at our list of [code quality tools](/CODE_QUALITY) we -have setup for systemd, to ensure our codebase stays in good shape. - -Please always test your work before submitting a PR. For many of the components -of systemd testing is straightforward as you can simply compile systemd and -run the relevant tool from the build directory. - -For some components (most importantly, systemd/PID 1 itself) this is not -possible, however. In order to simplify testing for cases like this we provide -a set of `mkosi` build files directly in the source tree. -[mkosi](https://github.com/systemd/mkosi) is a tool for building clean OS images -from an upstream distribution in combination with a fresh build of the project -in the local working directory. To make use of this, please install `mkosi` v19 -or newer using your distribution's package manager or from the -[GitHub repository](https://github.com/systemd/mkosi). `mkosi` will build an -image for the host distro by default. First, run `mkosi genkey` to generate a key -and certificate to be used for secure boot and verity signing. After that is done, -it is sufficient to type `mkosi` in the systemd project directory to generate a disk -image you can boot either in `systemd-nspawn` or in a UEFI-capable VM: +Please make sure to follow our [Coding Style](/CODING_STYLE) when submitting patches. +Also have a look at our [Contribution Guidelines](/CONTRIBUTING). + +When adding new functionality, tests should be added. +For shared functionality (in `src/basic/` and `src/shared/`) unit tests should be sufficient. +The general policy is to keep tests in matching files underneath `src/test/`, +e.g. `src/test/test-path-util.c` contains tests for any functions in `src/basic/path-util.c`. +If adding a new source file, consider adding a matching test executable. +For features at a higher level, tests in `src/test/` are very strongly recommended. +If that is not possible, integration tests in `test/` are encouraged. + +Please always test your work before submitting a PR. +For many of the components of systemd testing is straightforward as you can simply compile systemd and run the relevant tool from the build directory. + +For some components (most importantly, systemd/PID 1 itself) this is not possible, however. +In order to simplify testing for cases like this we provide a set of `mkosi` config files directly in the source tree. +[mkosi](https://mkosi.systemd.io/) +is a tool for building clean OS images from an upstream distribution in combination with a fresh build of the project in the local working directory. +To make use of this, please install `mkosi` v19 or newer using your distribution's package manager or from the +[GitHub repository](https://github.com/systemd/mkosi). +`mkosi` will build an image for the host distro by default. +First, run `mkosi genkey` to generate a key and certificate to be used for secure boot and verity signing. +After that is done, it is sufficient to type `mkosi` in the systemd project directory to generate a disk image you can boot either in `systemd-nspawn` or in a UEFI-capable VM: ```sh $ sudo mkosi boot # nspawn still needs sudo for now @@ -53,11 +45,62 @@ or: $ mkosi qemu ``` -Every time you rerun the `mkosi` command a fresh image is built, incorporating -all current changes you made to the project tree. +Every time you rerun the `mkosi` command a fresh image is built, +incorporating all current changes you made to the project tree. -Putting this all together, here's a series of commands for preparing a patch -for systemd: +By default a directory image is built. +This requires `virtiofsd` to be installed on the host. +To build a disk image instead which does not require `virtiofsd`, add the following to `mkosi.local.conf`: + +```conf +[Output] +Format=disk +``` + +To boot in UEFI mode instead of using QEMU's direct kernel boot, add the following to `mkosi.local.conf`: + +```conf +[Host] +QemuFirmware=uefi +``` + +To avoid having to build a new image all the time when iterating on a patch, +add the following to `mkosi.local.conf`: + +```conf +[Host] +RuntimeBuildSources=yes +``` + +After enabling this setting, the source and build directories will be mounted to +`/work/src` and `/work/build` respectively when booting the image as a container +or virtual machine. To build the latest changes and re-install, run +`meson install -C /work/build --only-changed` in the container or virtual machine +and optionally restart the daemon(s) you're working on using +`systemctl restart <units>` or `systemctl daemon-reexec` if you're working on pid1 +or `systemctl soft-reboot` to restart everything. + +Aside from the image, the `mkosi.output` directory will also be populated with a +set of distribution packages. Assuming you're running the same distribution and +release as the mkosi image, you can install these rpms on your host or test +system as well for any testing or debugging that cannot easily be performed in a +VM or container. + +By default, no debuginfo packages are produced. To produce debuginfo packages, +run mkosi with the `WITH_DEBUG` environment variable set to `1`: + +```sh +$ mkosi -E WITH_DEBUG=1 -f +``` + +or configure it in `mkosi.local.conf`: + +```conf +[Content] +Environment=WITH_DEBUG=1 +``` + +Putting this all together, here's a series of commands for preparing a patch for systemd: ```sh $ git clone https://github.com/systemd/mkosi.git # If mkosi v19 or newer is not packaged by your distribution @@ -74,9 +117,8 @@ $ git push -u <REMOTE> # where REMOTE is your "fork" on GitHub And after that, head over to your repo on GitHub and click "Compare & pull request" -If you want to do a local build without mkosi, most distributions also provide -very simple and convenient ways to install most development packages necessary -to build systemd: +If you want to do a local build without mkosi, +most distributions also provide very simple and convenient ways to install most development packages necessary to build systemd: ```sh # Fedora @@ -105,85 +147,72 @@ Happy hacking! Some source files are generated during build. We use two templating engines: * meson's `configure_file()` directive uses syntax with `@VARIABLE@`. - See the - [Meson docs for `configure_file()`](https://mesonbuild.com/Reference-manual.html#configure_file) - for details. +See the [Meson docs for `configure_file()`](https://mesonbuild.com/Reference-manual.html#configure_file) for details. {% raw %} * most files are rendered using jinja2, with `{{VARIABLE}}` and `{% if … %}`, - `{% elif … %}`, `{% else … %}`, `{% endif … %}` blocks. `{# … #}` is a - jinja2 comment, i.e. that block will not be visible in the rendered - output. `{% raw %} … `{% endraw %}`{{ '{' }}{{ '% endraw %' }}}` creates a block - where jinja2 syntax is not interpreted. +`{% elif … %}`, `{% else … %}`, `{% endif … %}` blocks. `{# … #}` is a jinja2 comment, +i.e. that block will not be visible in the rendered output. +`{% raw %} … `{% endraw %}`{{ '{' }}{{ '% endraw %' }}}` creates a block where jinja2 syntax is not interpreted. - See the - [Jinja Template Designer Documentation](https://jinja.palletsprojects.com/en/3.1.x/templates/#synopsis) - for details. +See the [Jinja Template Designer Documentation](https://jinja.palletsprojects.com/en/3.1.x/templates/#synopsis) for details. Please note that files for both template engines use the `.in` extension. ## Developer and release modes -In the default meson configuration (`-Dmode=developer`), certain checks are -enabled that are suitable when hacking on systemd (such as internal -documentation consistency checks). Those are not useful when compiling for -distribution and can be disabled by setting `-Dmode=release`. +In the default meson configuration (`-Dmode=developer`), +certain checks are enabled that are suitable when hacking on systemd (such as internal documentation consistency checks). +Those are not useful when compiling for distribution and can be disabled by setting `-Dmode=release`. ## Sanitizers in mkosi -See [Testing systemd using sanitizers](/TESTING_WITH_SANITIZERS) for more information -on how to build with sanitizers enabled in mkosi. +See [Testing systemd using sanitizers](/TESTING_WITH_SANITIZERS) for more information on how to build with sanitizers enabled in mkosi. ## Fuzzers -systemd includes fuzzers in `src/fuzz/` that use libFuzzer and are automatically -run by [OSS-Fuzz](https://github.com/google/oss-fuzz) with sanitizers. -To add a fuzz target, create a new `src/fuzz/fuzz-foo.c` file with a `LLVMFuzzerTestOneInput` -function and add it to the list in `src/fuzz/meson.build`. +systemd includes fuzzers in `src/fuzz/` that use libFuzzer and are automatically run by [OSS-Fuzz](https://github.com/google/oss-fuzz) with sanitizers. +To add a fuzz target, create a new `src/fuzz/fuzz-foo.c` file with a `LLVMFuzzerTestOneInput` function and add it to the list in `src/fuzz/meson.build`. -Whenever possible, a seed corpus and a dictionary should also be added with new -fuzz targets. The dictionary should be named `src/fuzz/fuzz-foo.dict` and the seed -corpus should be built and exported as `$OUT/fuzz-foo_seed_corpus.zip` in -`tools/oss-fuzz.sh`. +Whenever possible, a seed corpus and a dictionary should also be added with new fuzz targets. +The dictionary should be named `src/fuzz/fuzz-foo.dict` and the seed corpus should be built and exported as `$OUT/fuzz-foo_seed_corpus.zip` in `tools/oss-fuzz.sh`. -The fuzzers can be built locally if you have libFuzzer installed by running -`tools/oss-fuzz.sh`, or by running: +The fuzzers can be built locally if you have libFuzzer installed by running `tools/oss-fuzz.sh`, or by running: -``` +```sh CC=clang CXX=clang++ \ meson setup build-libfuzz -Dllvm-fuzz=true -Db_sanitize=address,undefined -Db_lundef=false \ - -Dc_args='-fno-omit-frame-pointer -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION' +-Dc_args='-fno-omit-frame-pointer -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION' ninja -C build-libfuzz fuzzers ``` -Each fuzzer then can be then run manually together with a directory containing -the initial corpus: +Each fuzzer then can be then run manually together with a directory containing the initial corpus: ``` export UBSAN_OPTIONS=print_stacktrace=1:print_summary=1:halt_on_error=1 build-libfuzz/fuzz-varlink-idl test/fuzz/fuzz-varlink-idl/ ``` -Note: the `halt_on_error=1` UBSan option is especially important, otherwise -the fuzzer won't crash when undefined behavior is triggered. +Note: the `halt_on_error=1` UBSan option is especially important, +otherwise the fuzzer won't crash when undefined behavior is triggered. You should also confirm that the fuzzers can be built and run using [the OSS-Fuzz toolchain](https://google.github.io/oss-fuzz/advanced-topics/reproducing/#building-using-docker): -``` +```sh path_to_systemd=... git clone --depth=1 https://github.com/google/oss-fuzz cd oss-fuzz for sanitizer in address undefined memory; do - for engine in libfuzzer afl honggfuzz; do - ./infra/helper.py build_fuzzers --sanitizer "$sanitizer" --engine "$engine" \ - --clean systemd "$path_to_systemd" +for engine in libfuzzer afl honggfuzz; do +./infra/helper.py build_fuzzers --sanitizer "$sanitizer" --engine "$engine" \ +--clean systemd "$path_to_systemd" - ./infra/helper.py check_build --sanitizer "$sanitizer" --engine "$engine" \ - -e ALLOWED_BROKEN_TARGETS_PERCENTAGE=0 systemd - done +./infra/helper.py check_build --sanitizer "$sanitizer" --engine "$engine" \ +-e ALLOWED_BROKEN_TARGETS_PERCENTAGE=0 systemd +done done ./infra/helper.py build_fuzzers --clean --architecture i386 systemd "$path_to_systemd" @@ -193,8 +222,8 @@ done ./infra/helper.py coverage --no-corpus-download systemd ``` -If you find a bug that impacts the security of systemd, please follow the -guidance in [CONTRIBUTING.md](/CONTRIBUTING) on how to report a security vulnerability. +If you find a bug that impacts the security of systemd, +please follow the guidance in [CONTRIBUTING.md](/CONTRIBUTING) on how to report a security vulnerability. For more details on building fuzzers and integrating with OSS-Fuzz, visit: @@ -203,55 +232,39 @@ For more details on building fuzzers and integrating with OSS-Fuzz, visit: ## Debugging binaries that need to run as root in vscode -When trying to debug binaries that need to run as root, we need to do some custom configuration in vscode to -have it try to run the applications as root and to ask the user for the root password when trying to start -the binary. To achieve this, we'll use a custom debugger path which points to a script that starts `gdb` as -root using `pkexec`. pkexec will prompt the user for their root password via a graphical interface. This -guide assumes the C/C++ extension is used for debugging. +When trying to debug binaries that need to run as root, +we need to do some custom configuration in vscode to have it try to run the applications as root and to ask the user for the root password when trying to start the binary. +To achieve this, we'll use a custom debugger path which points to a script that starts `gdb` as root using `pkexec`. +pkexec will prompt the user for their root password via a graphical interface. +This guide assumes the C/C++ extension is used for debugging. -First, create a file `sgdb` in the root of the systemd repository with the following contents and make it -executable: +First, create a file `sgdb` in the root of the systemd repository with the following contents and make it executable: -``` +```sh #!/bin/sh exec pkexec gdb "$@" ``` -Then, open launch.json in vscode, and set `miDebuggerPath` to `${workspaceFolder}/sgdb` for the corresponding -debug configuration. Now, whenever you try to debug the application, vscode will try to start gdb as root via -pkexec which will prompt you for your password via a graphical interface. After entering your password, -vscode should be able to start debugging the application. +Then, open launch.json in vscode, and set `miDebuggerPath` to `${workspaceFolder}/sgdb` for the corresponding debug configuration. +Now, whenever you try to debug the application, vscode will try to start gdb as root via pkexec which will prompt you for your password via a graphical interface. +After entering your password, vscode should be able to start debugging the application. -For more information on how to set up a debug configuration for C binaries, please refer to the official -vscode documentation [here](https://code.visualstudio.com/docs/cpp/launch-json-reference) +For more information on how to set up a debug configuration for C binaries, +please refer to the official vscode documentation [here](https://code.visualstudio.com/docs/cpp/launch-json-reference) ## Debugging systemd with mkosi + vscode -To simplify debugging systemd when testing changes using mkosi, we're going to show how to attach -[VSCode](https://code.visualstudio.com/)'s debugger to an instance of systemd running in a mkosi image using -QEMU. - -To allow VSCode's debugger to attach to systemd running in a mkosi image, we have to make sure it can access -the virtual machine spawned by mkosi where systemd is running. mkosi makes this possible via a handy SSH -option that makes the generated image accessible via SSH when booted. Thus you must build the image with -`mkosi --ssh`. The easiest way to set the option is to create a file `mkosi.local.conf` in the root of the -repository and add the following contents: - -``` -[Host] -Ssh=yes -RuntimeTrees=. -``` - -Also make sure that the SSH agent is running on your system and that you've added your SSH key to it with -`ssh-add`. Also make sure that `virtiofsd` is installed. +To simplify debugging systemd when testing changes using mkosi, we're going to show how to attach [VSCode](https://code.visualstudio.com/)'s debugger to an instance of systemd running in a mkosi image using QEMU. -After rebuilding the image and booting it with `mkosi qemu`, you should now be able to connect to it by -running `mkosi ssh` from the same directory in another terminal window. +To allow VSCode's debugger to attach to systemd running in a mkosi image, +we have to make sure it can access the virtual machine spawned by mkosi where systemd is running. +After booting the image with `mkosi qemu`, +you should now be able to connect to it by running `mkosi ssh` from the same directory in another terminal window. -Now we need to configure VSCode. First, make sure the C/C++ extension is installed. If you're already using -a different extension for code completion and other IDE features for C in VSCode, make sure to disable the -corresponding parts of the C/C++ extension in your VSCode user settings by adding the following entries: +Now we need to configure VSCode. +First, make sure the C/C++ extension is installed. +If you're already using a different extension for code completion and other IDE features for C in VSCode, +make sure to disable the corresponding parts of the C/C++ extension in your VSCode user settings by adding the following entries: ```json "C_Cpp.formatting": "Disabled", @@ -260,9 +273,9 @@ corresponding parts of the C/C++ extension in your VSCode user settings by addin "C_Cpp.suggestSnippets": false, ``` -With the extension set up, we can create the launch.json file in the .vscode/ directory to tell the VSCode -debugger how to attach to the systemd instance running in our mkosi container/VM. Create the file, and possibly -the directory, and add the following contents: +With the extension set up, +we can create the launch.json file in the .vscode/ directory to tell the VSCode debugger how to attach to the systemd instance running in our mkosi container/VM. +Create the file, and possibly the directory, and add the following contents: ```json { @@ -276,16 +289,12 @@ the directory, and add the following contents: "name": "systemd", "pipeTransport": { "pipeProgram": "mkosi", - "pipeArgs": [ - "-C", - "/path/to/systemd/repo/directory/on/host/system/", - "ssh" - ], + "pipeArgs": ["-C", "${workspaceFolder}", "ssh"], "debuggerPath": "/usr/bin/gdb" }, "MIMode": "gdb", "sourceFileMap": { - "/root/src/systemd": { + "/work/src": { "editorPath": "${workspaceFolder}", "useForBreakpoints": false }, @@ -295,29 +304,28 @@ the directory, and add the following contents: } ``` -Now that the debugger knows how to connect to our process in the container/VM and we've set up the necessary -source mappings, go to the "Run and Debug" window and run the "systemd" debug configuration. If everything -goes well, the debugger should now be attached to the systemd instance running in the container/VM. You can -attach breakpoints from the editor and enjoy all the other features of VSCode's debugger. +Now that the debugger knows how to connect to our process in the container/VM and we've set up the necessary source mappings, +go to the "Run and Debug" window and run the "systemd" debug configuration. +If everything goes well, the debugger should now be attached to the systemd instance running in the container/VM. +You can attach breakpoints from the editor and enjoy all the other features of VSCode's debugger. -To debug systemd components other than PID 1, set "program" to the full path of the component you want to -debug and set "processId" to "${command:pickProcess}". Now, when starting the debugger, VSCode will ask you -the PID of the process you want to debug. Run `systemctl show --property MainPID --value <component>` in the -container to figure out the PID and enter it when asked and VSCode will attach to that process instead. +To debug systemd components other than PID 1, +set "program" to the full path of the component you want to debug and set "processId" to "${command:pickProcess}". +Now, when starting the debugger, VSCode will ask you the PID of the process you want to debug. +Run `systemctl show --property MainPID --value <component>` +in the container to figure out the PID and enter it when asked and VSCode will attach to that process instead. ## Debugging systemd-boot -During boot, systemd-boot and the stub loader will output messages like -`systemd-boot@0x0A` and `systemd-stub@0x0B`, providing the base of the loaded -code. This location can then be used to attach to a QEMU session (provided it -was run with `-s`). See `debug-sd-boot.sh` script in the tools folder which -automates this processes. +During boot, systemd-boot and the stub loader will output messages like `systemd-boot@0x0A` and `systemd-stub@0x0B`, +providing the base of the loaded code. +This location can then be used to attach to a QEMU session (provided it was run with `-s`). +See `debug-sd-boot.sh` script in the tools folder which automates this processes. If the debugger is too slow to attach to examine an early boot code passage, -the call to `DEFINE_EFI_MAIN_FUNCTION()` can be modified to enable waiting. As -soon as the debugger has control, we can then run `set variable wait = 0` or -`return` to continue. Once the debugger has attached, setting breakpoints will -work like usual. +the call to `DEFINE_EFI_MAIN_FUNCTION()` can be modified to enable waiting. +As soon as the debugger has control, we can then run `set variable wait = 0` or `return` to continue. +Once the debugger has attached, setting breakpoints will work like usual. To debug systemd-boot in an IDE such as VSCode we can use a launch configuration like this: ```json |