summaryrefslogtreecommitdiffstats
path: root/docs/JOURNAL_FILE_FORMAT.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/JOURNAL_FILE_FORMAT.md')
-rw-r--r--docs/JOURNAL_FILE_FORMAT.md90
1 files changed, 44 insertions, 46 deletions
diff --git a/docs/JOURNAL_FILE_FORMAT.md b/docs/JOURNAL_FILE_FORMAT.md
index e0737c5..7d3b039 100644
--- a/docs/JOURNAL_FILE_FORMAT.md
+++ b/docs/JOURNAL_FILE_FORMAT.md
@@ -8,8 +8,9 @@ SPDX-License-Identifier: LGPL-2.1-or-later
# Journal File Format
_Note that this document describes the binary on-disk format of journals only.
-For interfacing with web technologies there's the [Journal JSON Format](JOURNAL_EXPORT_FORMATS.md#journal-json-format).
-For transfer of journal data across the network there's the [Journal Export Format](JOURNAL_EXPORT_FORMATS.md#journal-export-format)._
+For interfacing with web technologies there's the [Journal JSON Format](JOURNAL_EXPORT_FORMATS#journal-json-format).
+For transfer of journal data across the network there's the
+[Journal Export Format](JOURNAL_EXPORT_FORMATS#journal-export-format)._
The systemd journal stores log data in a binary format with several features:
@@ -20,65 +21,62 @@ The systemd journal stores log data in a binary format with several features:
* Support for in-line compression
* Support for in-line Forward Secure Sealing
-This document explains the basic structure of the file format on disk. We are
-making this available primarily to allow review and provide documentation. Note
-that the actual implementation in the [systemd
-codebase](https://github.com/systemd/systemd/blob/main/src/libsystemd/sd-journal/) is the
-only ultimately authoritative description of the format, so if this document
-and the code disagree, the code is right. That said we'll of course try hard to
-keep this document up-to-date and accurate.
-
-Instead of implementing your own reader or writer for journal files we ask you
-to use the [Journal's native C
-API](https://www.freedesktop.org/software/systemd/man/sd-journal.html) to access
-these files. It provides you with full access to the files, and will not
-withhold any data. If you find a limitation, please ping us and we might add
-some additional interfaces for you.
-
-If you need access to the raw journal data in serialized stream form without C
-API our recommendation is to make use of the [Journal Export
-Format](https://systemd.io/JOURNAL_EXPORT_FORMATS#journal-export-format), which you can
-get via `journalctl -o export` or via `systemd-journal-gatewayd`. The export
-format is much simpler to parse, but complete and accurate. Due to its
-stream-based nature it is not indexed.
-
-_Or, to put this in other words: this low-level document is probably not what
-you want to use as base of your project. You want our [C
-API](https://www.freedesktop.org/software/systemd/man/sd-journal.html) instead!
+This document explains the basic structure of the file format on disk.
+We are making this available primarily to allow review and provide documentation.
+Note that the actual implementation in the
+[systemd codebase](https://github.com/systemd/systemd/blob/main/src/libsystemd/sd-journal/)
+is the only ultimately authoritative description of the format,
+so if this document and the code disagree, the code is right.
+That said we'll of course try hard to keep this document up-to-date and accurate.
+
+Instead of implementing your own reader or writer for journal files we ask you to use the
+[Journal's native CAPI](https://www.freedesktop.org/software/systemd/man/sd-journal.html)
+to access these files.
+It provides you with full access to the files, and will not withhold any data.
+If you find a limitation, please ping us and we might add some additional interfaces for you.
+
+If you need access to the raw journal data in serialized stream form without C API our recommendation is to make use of the
+[Journal Export Format](JOURNAL_EXPORT_FORMATS#journal-export-format),
+which you can get via `journalctl -o export` or via `systemd-journal-gatewayd`.
+The export format is much simpler to parse, but complete and accurate.
+Due to its stream-based nature it is not indexed.
+
+_Or, to put this in other words: this low-level document is probably not what you want to use as base of your project.
+You want our [C API](https://www.freedesktop.org/software/systemd/man/sd-journal.html) instead!
And if you really don't want the C API, then you want the
-[Journal Export Format or Journal JSON Format](JOURNAL_EXPORT_FORMATS)
-instead! This document is primarily for your entertainment and education.
+[Journal Export Format or Journal JSON Format](/JOURNAL_EXPORT_FORMATS) instead!
+This document is primarily for your entertainment and education.
Thank you!_
-This document assumes you have a basic understanding of the journal concepts,
-the properties of a journal entry and so on. If not, please go and read up,
-then come back! This is a good opportunity to read about the [basic properties
-of journal
-entries](https://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html),
-in particular realize that they may include binary non-text data (though
-usually don't), and the same field might have multiple values assigned within
-the same entry.
-
-This document describes the current format of systemd 246. The documented
-format is compatible with the format used in the first versions of the journal,
+This document assumes you have a basic understanding of the journal concepts, the properties of a journal entry and so on.
+If not, please go and read up, then come back!
+This is a good opportunity to read about the
+[basic properties of journal entries](https://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html),
+in particular realize that they may include binary non-text data (though usually don't),
+and the same field might have multiple values assigned within the same entry.
+
+This document describes the current format of systemd 246.
+The documented format is compatible with the format used in the first versions of the journal,
but received various compatible and incompatible additions since.
-If you are wondering why the journal file format has been created in the first
-place instead of adopting an existing database implementation, please have a
-look [at this
-thread](https://lists.freedesktop.org/archives/systemd-devel/2012-October/007054.html).
+If you are wondering why the journal file format has been created in the first place instead of adopting an existing database implementation,
+please have a look [at this thread](https://lists.freedesktop.org/archives/systemd-devel/2012-October/007054.html).
## Basics
* All offsets, sizes, time values, hashes (and most other numeric values) are 32-bit/64-bit unsigned integers in LE format.
* Offsets are always relative to the beginning of the file.
-* The 64-bit hash function siphash24 is used for newer journal files. For older files [Jenkins lookup3](https://en.wikipedia.org/wiki/Jenkins_hash_function) is used, more specifically `jenkins_hashlittle2()` with the first 32-bit integer it returns as higher 32-bit part of the 64-bit value, and the second one uses as lower 32-bit part.
+* The 64-bit hash function siphash24 is used for newer journal files.
+ For older files [Jenkins lookup3](https://en.wikipedia.org/wiki/Jenkins_hash_function) is used,
+ more specifically `jenkins_hashlittle2()` with the first 32-bit integer it returns as higher 32-bit part of the 64-bit value,
+ and the second one uses as lower 32-bit part.
* All structures are aligned to 64-bit boundaries and padded to multiples of 64-bit
* The format is designed to be read and written via memory mapping using multiple mapped windows.
* All time values are stored in usec since the respective epoch.
* Wall clock time values are relative to the Unix time epoch, i.e. January 1st, 1970. (`CLOCK_REALTIME`)
-* Monotonic time values are always stored jointly with the kernel boot ID value (i.e. `/proc/sys/kernel/random/boot_id`) they belong to. They tend to be relative to the start of the boot, but aren't for containers. (`CLOCK_MONOTONIC`)
+* Monotonic time values are always stored jointly with the kernel boot ID value (i.e. `/proc/sys/kernel/random/boot_id`) they belong to.
+ They tend to be relative to the start of the boot, but aren't for containers. (`CLOCK_MONOTONIC`)
* Randomized, unique 128-bit IDs are used in various locations. These are generally UUID v4 compatible, but this is not a requirement.
## General Rules