diff options
Diffstat (limited to 'upstream/mageia-cauldron/man5/btrfs.5')
| -rw-r--r-- | upstream/mageia-cauldron/man5/btrfs.5 | 2902 |
1 files changed, 2902 insertions, 0 deletions
diff --git a/upstream/mageia-cauldron/man5/btrfs.5 b/upstream/mageia-cauldron/man5/btrfs.5 new file mode 100644 index 00000000..254a8894 --- /dev/null +++ b/upstream/mageia-cauldron/man5/btrfs.5 @@ -0,0 +1,2902 @@ +.\" Man page generated from reStructuredText. +. +. +.nr rst2man-indent-level 0 +. +.de1 rstReportMargin +\\$1 \\n[an-margin] +level \\n[rst2man-indent-level] +level margin: \\n[rst2man-indent\\n[rst2man-indent-level]] +- +\\n[rst2man-indent0] +\\n[rst2man-indent1] +\\n[rst2man-indent2] +.. +.de1 INDENT +.\" .rstReportMargin pre: +. RS \\$1 +. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin] +. nr rst2man-indent-level +1 +.\" .rstReportMargin post: +.. +.de UNINDENT +. RE +.\" indent \\n[an-margin] +.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]] +.nr rst2man-indent-level -1 +.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]] +.in \\n[rst2man-indent\\n[rst2man-indent-level]]u +.. +.TH "BTRFS" "5" "Jan 09, 2024" "6.6.3" "BTRFS" +.SH NAME +btrfs \- topics about the BTRFS filesystem (mount options, supported file attributes and other) +.SH DESCRIPTION +.sp +This document describes topics related to BTRFS that are not specific to the +tools. Currently covers: +.INDENT 0.0 +.IP 1. 4 +mount options +.IP 2. 4 +filesystem features +.IP 3. 4 +checksum algorithms +.IP 4. 4 +compression +.IP 5. 4 +sysfs interface +.IP 6. 4 +filesystem exclusive operations +.IP 7. 4 +filesystem limits +.IP 8. 4 +bootloader support +.IP 9. 4 +file attributes +.IP 10. 4 +zoned mode +.IP 11. 4 +control device +.IP 12. 4 +filesystems with multiple block group profiles +.IP 13. 4 +seeding device +.IP 14. 4 +RAID56 status and recommended practices +.IP 15. 4 +storage model, hardware considerations +.UNINDENT +.SH MOUNT OPTIONS +.SS BTRFS SPECIFIC MOUNT OPTIONS +.sp +This section describes mount options specific to BTRFS. For the generic mount +options please refer to \fBmount(8)\fP manual page. The options are sorted alphabetically +(discarding the \fIno\fP prefix). +.sp +\fBNOTE:\fP +.INDENT 0.0 +.INDENT 3.5 +Most mount options apply to the whole filesystem and only options in the +first mounted subvolume will take effect. This is due to lack of implementation +and may change in the future. This means that (for example) you can\(aqt set +per\-subvolume \fInodatacow\fP, \fInodatasum\fP, or \fIcompress\fP using mount options. This +should eventually be fixed, but it has proved to be difficult to implement +correctly within the Linux VFS framework. +.UNINDENT +.UNINDENT +.sp +Mount options are processed in order, only the last occurrence of an option +takes effect and may disable other options due to constraints (see e.g. +\fInodatacow\fP and \fIcompress\fP). The output of \fBmount\fP command shows which options +have been applied. +.INDENT 0.0 +.TP +.B acl, noacl +(default: on) +.sp +Enable/disable support for POSIX Access Control Lists (ACLs). See the +\fBacl(5)\fP manual page for more information about ACLs. +.sp +The support for ACL is build\-time configurable (BTRFS_FS_POSIX_ACL) and +mount fails if \fIacl\fP is requested but the feature is not compiled in. +.UNINDENT +.INDENT 0.0 +.TP +.B autodefrag, noautodefrag +(since: 3.0, default: off) +.sp +Enable automatic file defragmentation. +When enabled, small random writes into files (in a range of tens of kilobytes, +currently it\(aqs 64KiB) are detected and queued up for the defragmentation process. +May not be well suited for large database workloads. +.sp +The read latency may increase due to reading the adjacent blocks that make up the +range for defragmentation, successive write will merge the blocks in the new +location. +.sp +\fBWARNING:\fP +.INDENT 7.0 +.INDENT 3.5 +Defragmenting with Linux kernel versions < 3.9 or ≥ 3.14\-rc2 as +well as with Linux stable kernel versions ≥ 3.10.31, ≥ 3.12.12 or +≥ 3.13.4 will break up the reflinks of COW data (for example files +copied with \fBcp \-\-reflink\fP, snapshots or de\-duplicated data). +This may cause considerable increase of space usage depending on the +broken up reflinks. +.UNINDENT +.UNINDENT +.TP +.B barrier, nobarrier +(default: on) +.sp +Ensure that all IO write operations make it through the device cache and are stored +permanently when the filesystem is at its consistency checkpoint. This +typically means that a flush command is sent to the device that will +synchronize all pending data and ordinary metadata blocks, then writes the +superblock and issues another flush. +.sp +The write flushes incur a slight hit and also prevent the IO block +scheduler to reorder requests in a more effective way. Disabling barriers gets +rid of that penalty but will most certainly lead to a corrupted filesystem in +case of a crash or power loss. The ordinary metadata blocks could be yet +unwritten at the time the new superblock is stored permanently, expecting that +the block pointers to metadata were stored permanently before. +.sp +On a device with a volatile battery\-backed write\-back cache, the \fInobarrier\fP +option will not lead to filesystem corruption as the pending blocks are +supposed to make it to the permanent storage. +.TP +.B check_int, check_int_data, check_int_print_mask=<value> +(since: 3.0, default: off) +.sp +These debugging options control the behavior of the integrity checking +module (the BTRFS_FS_CHECK_INTEGRITY config option required). The main goal is +to verify that all blocks from a given transaction period are properly linked. +.sp +\fIcheck_int\fP enables the integrity checker module, which examines all +block write requests to ensure on\-disk consistency, at a large +memory and CPU cost. +.sp +\fIcheck_int_data\fP includes extent data in the integrity checks, and +implies the \fIcheck_int\fP option. +.sp +\fIcheck_int_print_mask\fP takes a bitmask of BTRFSIC_PRINT_MASK_* values +as defined in \fIfs/btrfs/check\-integrity.c\fP, to control the integrity +checker module behavior. +.sp +See comments at the top of \fIfs/btrfs/check\-integrity.c\fP +for more information. +.TP +.B clear_cache +Force clearing and rebuilding of the free space cache if something +has gone wrong. +.sp +For free space cache \fIv1\fP, this only clears (and, unless \fInospace_cache\fP is +used, rebuilds) the free space cache for block groups that are modified while +the filesystem is mounted with that option. To actually clear an entire free +space cache \fIv1\fP, see \fBbtrfs check \-\-clear\-space\-cache v1\fP\&. +.sp +For free space cache \fIv2\fP, this clears the entire free space cache. +To do so without requiring to mounting the filesystem, see +\fBbtrfs check \-\-clear\-space\-cache v2\fP\&. +.sp +See also: \fIspace_cache\fP\&. +.TP +.B commit=<seconds> +(since: 3.12, default: 30) +.sp +Set the interval of periodic transaction commit when data are synchronized +to permanent storage. Higher interval values lead to larger amount of unwritten +data, which has obvious consequences when the system crashes. +The upper bound is not forced, but a warning is printed if it\(aqs more than 300 +seconds (5 minutes). Use with care. +.TP +.B compress, compress=<type[:level]>, compress\-force, compress\-force=<type[:level]> +(default: off, level support since: 5.1) +.sp +Control BTRFS file data compression. Type may be specified as \fIzlib\fP, +\fIlzo\fP, \fIzstd\fP or \fIno\fP (for no compression, used for remounting). If no type +is specified, \fIzlib\fP is used. If \fIcompress\-force\fP is specified, +then compression will always be attempted, but the data may end up uncompressed +if the compression would make them larger. +.sp +Both \fIzlib\fP and \fIzstd\fP (since version 5.1) expose the compression level as a +tunable knob with higher levels trading speed and memory (\fIzstd\fP) for higher +compression ratios. This can be set by appending a colon and the desired level. +ZLIB accepts the range [1, 9] and ZSTD accepts [1, 15]. If no level is set, +both currently use a default level of 3. The value 0 is an alias for the +default level. +.sp +Otherwise some simple heuristics are applied to detect an incompressible file. +If the first blocks written to a file are not compressible, the whole file is +permanently marked to skip compression. As this is too simple, the +\fIcompress\-force\fP is a workaround that will compress most of the files at the +cost of some wasted CPU cycles on failed attempts. +Since kernel 4.15, a set of heuristic algorithms have been improved by using +frequency sampling, repeated pattern detection and Shannon entropy calculation +to avoid that. +.sp +\fBNOTE:\fP +.INDENT 7.0 +.INDENT 3.5 +If compression is enabled, \fInodatacow\fP and \fInodatasum\fP are disabled. +.UNINDENT +.UNINDENT +.TP +.B datacow, nodatacow +(default: on) +.sp +Enable data copy\-on\-write for newly created files. +\fINodatacow\fP implies \fInodatasum\fP, and disables \fIcompression\fP\&. All files created +under \fInodatacow\fP are also set the NOCOW file attribute (see \fBchattr(1)\fP). +.sp +\fBNOTE:\fP +.INDENT 7.0 +.INDENT 3.5 +If \fInodatacow\fP or \fInodatasum\fP are enabled, compression is disabled. +.UNINDENT +.UNINDENT +.sp +Updates in\-place improve performance for workloads that do frequent overwrites, +at the cost of potential partial writes, in case the write is interrupted +(system crash, device failure). +.TP +.B datasum, nodatasum +(default: on) +.sp +Enable data checksumming for newly created files. +\fIDatasum\fP implies \fIdatacow\fP, i.e. the normal mode of operation. All files created +under \fInodatasum\fP inherit the \(dqno checksums\(dq property, however there\(aqs no +corresponding file attribute (see \fBchattr(1)\fP). +.sp +\fBNOTE:\fP +.INDENT 7.0 +.INDENT 3.5 +If \fInodatacow\fP or \fInodatasum\fP are enabled, compression is disabled. +.UNINDENT +.UNINDENT +.sp +There is a slight performance gain when checksums are turned off, the +corresponding metadata blocks holding the checksums do not need to updated. +The cost of checksumming of the blocks in memory is much lower than the IO, +modern CPUs feature hardware support of the checksumming algorithm. +.UNINDENT +.INDENT 0.0 +.TP +.B degraded +(default: off) +.sp +Allow mounts with fewer devices than the RAID profile constraints +require. A read\-write mount (or remount) may fail when there are too many devices +missing, for example if a stripe member is completely missing from RAID0. +.sp +Since 4.14, the constraint checks have been improved and are verified on the +chunk level, not at the device level. This allows degraded mounts of +filesystems with mixed RAID profiles for data and metadata, even if the +device number constraints would not be satisfied for some of the profiles. +.sp +Example: metadata \-\- raid1, data \-\- single, devices \-\- \fB/dev/sda\fP, \fB/dev/sdb\fP +.sp +Suppose the data are completely stored on \fIsda\fP, then missing \fIsdb\fP will not +prevent the mount, even if 1 missing device would normally prevent (any) +\fIsingle\fP profile to mount. In case some of the data chunks are stored on \fIsdb\fP, +then the constraint of single/data is not satisfied and the filesystem +cannot be mounted. +.UNINDENT +.INDENT 0.0 +.TP +.B device=<devicepath> +Specify a path to a device that will be scanned for BTRFS filesystem during +mount. This is usually done automatically by a device manager (like udev) or +using the \fBbtrfs device scan\fP command (e.g. run from the initial ramdisk). In +cases where this is not possible the \fIdevice\fP mount option can help. +.sp +\fBNOTE:\fP +.INDENT 7.0 +.INDENT 3.5 +Booting e.g. a RAID1 system may fail even if all filesystem\(aqs \fIdevice\fP +paths are provided as the actual device nodes may not be discovered by the +system at that point. +.UNINDENT +.UNINDENT +.TP +.B discard, discard=sync, discard=async, nodiscard +(default: async when devices support it since 6.2, async support since: 5.6) +.sp +Enable discarding of freed file blocks. This is useful for SSD devices, thinly +provisioned LUNs, or virtual machine images; however, every storage layer must +support discard for it to work. +.sp +In the synchronous mode (\fIsync\fP or without option value), lack of asynchronous +queued TRIM on the backing device TRIM can severely degrade performance, +because a synchronous TRIM operation will be attempted instead. Queued TRIM +requires newer than SATA revision 3.1 chipsets and devices. +.sp +The asynchronous mode (\fIasync\fP) gathers extents in larger chunks before sending +them to the devices for TRIM. The overhead and performance impact should be +negligible compared to the previous mode and it\(aqs supposed to be the preferred +mode if needed. +.sp +If it is not necessary to immediately discard freed blocks, then the \fBfstrim\fP +tool can be used to discard all free blocks in a batch. Scheduling a TRIM +during a period of low system activity will prevent latent interference with +the performance of other operations. Also, a device may ignore the TRIM command +if the range is too small, so running a batch discard has a greater probability +of actually discarding the blocks. +.TP +.B enospc_debug, noenospc_debug +(default: off) +.sp +Enable verbose output for some ENOSPC conditions. It\(aqs safe to use but can +be noisy if the system reaches near\-full state. +.TP +.B fatal_errors=<action> +(since: 3.4, default: bug) +.sp +Action to take when encountering a fatal error. +.INDENT 7.0 +.TP +.B bug +\fIBUG()\fP on a fatal error, the system will stay in the crashed state and may be +still partially usable, but reboot is required for full operation +.TP +.B panic +\fIpanic()\fP on a fatal error, depending on other system configuration, this may +be followed by a reboot. Please refer to the documentation of kernel boot +parameters, e.g. \fIpanic\fP, \fIoops\fP or \fIcrashkernel\fP\&. +.UNINDENT +.TP +.B flushoncommit, noflushoncommit +(default: off) +.sp +This option forces any data dirtied by a write in a prior transaction to commit +as part of the current commit, effectively a full filesystem sync. +.sp +This makes the committed state a fully consistent view of the file system from +the application\(aqs perspective (i.e. it includes all completed file system +operations). This was previously the behavior only when a snapshot was +created. +.sp +When off, the filesystem is consistent but buffered writes may last more than +one transaction commit. +.TP +.B fragment=<type> +(depends on compile\-time option CONFIG_BTRFS_DEBUG, since: 4.4, default: off) +.sp +A debugging helper to intentionally fragment given \fItype\fP of block groups. The +type can be \fIdata\fP, \fImetadata\fP or \fIall\fP\&. This mount option should not be used +outside of debugging environments and is not recognized if the kernel config +option \fICONFIG_BTRFS_DEBUG\fP is not enabled. +.TP +.B nologreplay +(default: off, even read\-only) +.sp +The tree\-log contains pending updates to the filesystem until the full commit. +The log is replayed on next mount, this can be disabled by this option. See +also \fItreelog\fP\&. Note that \fInologreplay\fP is the same as \fInorecovery\fP\&. +.sp +\fBWARNING:\fP +.INDENT 7.0 +.INDENT 3.5 +Currently, the tree log is replayed even with a read\-only mount! To +disable that behaviour, mount also with \fInologreplay\fP\&. +.UNINDENT +.UNINDENT +.TP +.B max_inline=<bytes> +(default: min(2048, page size) ) +.sp +Specify the maximum amount of space, that can be inlined in +a metadata b\-tree leaf. The value is specified in bytes, optionally +with a K suffix (case insensitive). In practice, this value +is limited by the filesystem block size (named \fIsectorsize\fP at mkfs time), +and memory page size of the system. In case of sectorsize limit, there\(aqs +some space unavailable due to b\-tree leaf headers. For example, a 4KiB +sectorsize, maximum size of inline data is about 3900 bytes. +.sp +Inlining can be completely turned off by specifying 0. This will increase data +block slack if file sizes are much smaller than block size but will reduce +metadata consumption in return. +.sp +\fBNOTE:\fP +.INDENT 7.0 +.INDENT 3.5 +The default value has changed to 2048 in kernel 4.6. +.UNINDENT +.UNINDENT +.TP +.B metadata_ratio=<value> +(default: 0, internal logic) +.sp +Specifies that 1 metadata chunk should be allocated after every \fIvalue\fP data +chunks. Default behaviour depends on internal logic, some percent of unused +metadata space is attempted to be maintained but is not always possible if +there\(aqs not enough space left for chunk allocation. The option could be useful to +override the internal logic in favor of the metadata allocation if the expected +workload is supposed to be metadata intense (snapshots, reflinks, xattrs, +inlined files). +.TP +.B norecovery +(since: 4.5, default: off) +.sp +Do not attempt any data recovery at mount time. This will disable \fIlogreplay\fP +and avoids other write operations. Note that this option is the same as +\fInologreplay\fP\&. +.sp +\fBNOTE:\fP +.INDENT 7.0 +.INDENT 3.5 +The opposite option \fIrecovery\fP used to have different meaning but was +changed for consistency with other filesystems, where \fInorecovery\fP is used for +skipping log replay. BTRFS does the same and in general will try to avoid any +write operations. +.UNINDENT +.UNINDENT +.TP +.B rescan_uuid_tree +(since: 3.12, default: off) +.sp +Force check and rebuild procedure of the UUID tree. This should not +normally be needed. +.TP +.B rescue +(since: 5.9) +.sp +Modes allowing mount with damaged filesystem structures. +.INDENT 7.0 +.IP \(bu 2 +\fIusebackuproot\fP (since: 5.9, replaces standalone option \fIusebackuproot\fP) +.IP \(bu 2 +\fInologreplay\fP (since: 5.9, replaces standalone option \fInologreplay\fP) +.IP \(bu 2 +\fIignorebadroots\fP, \fIibadroots\fP (since: 5.11) +.IP \(bu 2 +\fIignoredatacsums\fP, \fIidatacsums\fP (since: 5.11) +.IP \(bu 2 +\fIall\fP (since: 5.9) +.UNINDENT +.TP +.B skip_balance +(since: 3.3, default: off) +.sp +Skip automatic resume of an interrupted balance operation. The operation can +later be resumed with \fBbtrfs balance resume\fP, or the paused state can be +removed with \fBbtrfs balance cancel\fP\&. The default behaviour is to resume an +interrupted balance immediately after a volume is mounted. +.TP +.B space_cache, space_cache=<version>, nospace_cache +(\fInospace_cache\fP since: 3.2, \fIspace_cache=v1\fP and \fIspace_cache=v2\fP since 4.5, default: \fIspace_cache=v2\fP) +.sp +Options to control the free space cache. The free space cache greatly improves +performance when reading block group free space into memory. However, managing +the space cache consumes some resources, including a small amount of disk +space. +.sp +There are two implementations of the free space cache. The original +one, referred to as \fIv1\fP, used to be a safe default but has been +superseded by \fIv2\fP\&. The \fIv1\fP space cache can be disabled at mount time +with \fInospace_cache\fP without clearing. +.sp +On very large filesystems (many terabytes) and certain workloads, the +performance of the \fIv1\fP space cache may degrade drastically. The \fIv2\fP +implementation, which adds a new b\-tree called the free space tree, addresses +this issue. Once enabled, the \fIv2\fP space cache will always be used and cannot +be disabled unless it is cleared. Use \fIclear_cache,space_cache=v1\fP or +\fIclear_cache,nospace_cache\fP to do so. If \fIv2\fP is enabled, and \fIv1\fP space +cache will be cleared (at the first mount) and kernels without \fIv2\fP +support will only be able to mount the filesystem in read\-only mode. +On an unmounted filesystem the caches (both versions) can be cleared by +\(dqbtrfs check \-\-clear\-space\-cache\(dq. +.sp +The \fI\%btrfs\-check(8)\fP and \fI:doc:\(gamkfs.btrfs\fP commands have full \fIv2\fP free space +cache support since v4.19. +.sp +If a version is not explicitly specified, the default implementation will be +chosen, which is \fIv2\fP\&. +.TP +.B ssd, ssd_spread, nossd, nossd_spread +(default: SSD autodetected) +.sp +Options to control SSD allocation schemes. By default, BTRFS will +enable or disable SSD optimizations depending on status of a device with +respect to rotational or non\-rotational type. This is determined by the +contents of \fI/sys/block/DEV/queue/rotational\fP). If it is 0, the \fIssd\fP option is +turned on. The option \fInossd\fP will disable the autodetection. +.sp +The optimizations make use of the absence of the seek penalty that\(aqs inherent +for the rotational devices. The blocks can be typically written faster and +are not offloaded to separate threads. +.sp +\fBNOTE:\fP +.INDENT 7.0 +.INDENT 3.5 +Since 4.14, the block layout optimizations have been dropped. This used +to help with first generations of SSD devices. Their FTL (flash translation +layer) was not effective and the optimization was supposed to improve the wear +by better aligning blocks. This is no longer true with modern SSD devices and +the optimization had no real benefit. Furthermore it caused increased +fragmentation. The layout tuning has been kept intact for the option +\fIssd_spread\fP\&. +.UNINDENT +.UNINDENT +.sp +The \fIssd_spread\fP mount option attempts to allocate into bigger and aligned +chunks of unused space, and may perform better on low\-end SSDs. \fIssd_spread\fP +implies \fIssd\fP, enabling all other SSD heuristics as well. The option \fInossd\fP +will disable all SSD options while \fInossd_spread\fP only disables \fIssd_spread\fP\&. +.TP +.B subvol=<path> +Mount subvolume from \fIpath\fP rather than the toplevel subvolume. The +\fIpath\fP is always treated as relative to the toplevel subvolume. +This mount option overrides the default subvolume set for the given filesystem. +.TP +.B subvolid=<subvolid> +Mount subvolume specified by a \fIsubvolid\fP number rather than the toplevel +subvolume. You can use \fBbtrfs subvolume list\fP of \fBbtrfs subvolume show\fP to see +subvolume ID numbers. +This mount option overrides the default subvolume set for the given filesystem. +.sp +\fBNOTE:\fP +.INDENT 7.0 +.INDENT 3.5 +If both \fIsubvolid\fP and \fIsubvol\fP are specified, they must point at the +same subvolume, otherwise the mount will fail. +.UNINDENT +.UNINDENT +.TP +.B thread_pool=<number> +(default: min(NRCPUS + 2, 8) ) +.sp +The number of worker threads to start. NRCPUS is number of on\-line CPUs +detected at the time of mount. Small number leads to less parallelism in +processing data and metadata, higher numbers could lead to a performance hit +due to increased locking contention, process scheduling, cache\-line bouncing or +costly data transfers between local CPU memories. +.TP +.B treelog, notreelog +(default: on) +.sp +Enable the tree logging used for \fIfsync\fP and \fIO_SYNC\fP writes. The tree log +stores changes without the need of a full filesystem sync. The log operations +are flushed at sync and transaction commit. If the system crashes between two +such syncs, the pending tree log operations are replayed during mount. +.sp +\fBWARNING:\fP +.INDENT 7.0 +.INDENT 3.5 +Currently, the tree log is replayed even with a read\-only mount! To +disable that behaviour, also mount with \fInologreplay\fP\&. +.UNINDENT +.UNINDENT +.sp +The tree log could contain new files/directories, these would not exist on +a mounted filesystem if the log is not replayed. +.TP +.B usebackuproot +(since: 4.6, default: off) +.sp +Enable autorecovery attempts if a bad tree root is found at mount time. +Currently this scans a backup list of several previous tree roots and tries to +use the first readable. This can be used with read\-only mounts as well. +.sp +\fBNOTE:\fP +.INDENT 7.0 +.INDENT 3.5 +This option has replaced \fIrecovery\fP\&. +.UNINDENT +.UNINDENT +.TP +.B user_subvol_rm_allowed +(default: off) +.sp +Allow subvolumes to be deleted by their respective owner. Otherwise, only the +root user can do that. +.sp +\fBNOTE:\fP +.INDENT 7.0 +.INDENT 3.5 +Historically, any user could create a snapshot even if he was not owner +of the source subvolume, the subvolume deletion has been restricted for that +reason. The subvolume creation has been restricted but this mount option is +still required. This is a usability issue. +Since 4.18, the \fBrmdir(2)\fP syscall can delete an empty subvolume just like an +ordinary directory. Whether this is possible can be detected at runtime, see +\fIrmdir_subvol\fP feature in \fIFILESYSTEM FEATURES\fP\&. +.UNINDENT +.UNINDENT +.UNINDENT +.SS DEPRECATED MOUNT OPTIONS +.sp +List of mount options that have been removed, kept for backward compatibility. +.INDENT 0.0 +.TP +.B recovery +(since: 3.2, default: off, deprecated since: 4.5) +.sp +\fBNOTE:\fP +.INDENT 7.0 +.INDENT 3.5 +This option has been replaced by \fIusebackuproot\fP and should not be used +but will work on 4.5+ kernels. +.UNINDENT +.UNINDENT +.TP +.B inode_cache, noinode_cache +(removed in: 5.11, since: 3.0, default: off) +.sp +\fBNOTE:\fP +.INDENT 7.0 +.INDENT 3.5 +The functionality has been removed in 5.11, any stale data created by +previous use of the \fIinode_cache\fP option can be removed by +\fI\%btrfs rescue clear\-ino\-cache\fP\&. +.UNINDENT +.UNINDENT +.UNINDENT +.SS NOTES ON GENERIC MOUNT OPTIONS +.sp +Some of the general mount options from \fBmount(8)\fP that affect BTRFS and are +worth mentioning. +.INDENT 0.0 +.TP +.B noatime +under read intensive work\-loads, specifying \fInoatime\fP significantly improves +performance because no new access time information needs to be written. Without +this option, the default is \fIrelatime\fP, which only reduces the number of +inode atime updates in comparison to the traditional \fIstrictatime\fP\&. The worst +case for atime updates under \fIrelatime\fP occurs when many files are read whose +atime is older than 24 h and which are freshly snapshotted. In that case the +atime is updated and COW happens \- for each file \- in bulk. See also +\fI\%https://lwn.net/Articles/499293/\fP \- \fIAtime and btrfs: a bad combination? (LWN, 2012\-05\-31)\fP\&. +.sp +Note that \fInoatime\fP may break applications that rely on atime uptimes like +the venerable Mutt (unless you use maildir mailboxes). +.UNINDENT +.SH FILESYSTEM FEATURES +.sp +The basic set of filesystem features gets extended over time. The backward +compatibility is maintained and the features are optional, need to be +explicitly asked for so accidental use will not create incompatibilities. +.sp +There are several classes and the respective tools to manage the features: +.INDENT 0.0 +.TP +.B at mkfs time only +This is namely for core structures, like the b\-tree nodesize or checksum +algorithm, see \fI\%mkfs.btrfs(8)\fP for more details. +.TP +.B after mkfs, on an unmounted filesystem +Features that may optimize internal structures or add new structures to support +new functionality, see \fI\%btrfstune(8)\fP\&. The command +\fBbtrfs inspect\-internal dump\-super /dev/sdx\fP +will dump a superblock, you can map the value of +\fIincompat_flags\fP to the features listed below +.TP +.B after mkfs, on a mounted filesystem +The features of a filesystem (with a given UUID) are listed in +\fB/sys/fs/btrfs/UUID/features/\fP, one file per feature. The status is stored +inside the file. The value \fI1\fP is for enabled and active, while \fI0\fP means the +feature was enabled at mount time but turned off afterwards. +.sp +Whether a particular feature can be turned on a mounted filesystem can be found +in the directory \fB/sys/fs/btrfs/features/\fP, one file per feature. The value \fI1\fP +means the feature can be enabled. +.UNINDENT +.sp +List of features (see also \fI\%mkfs.btrfs(8)\fP section +\fI\%FILESYSTEM FEATURES\fP): +.INDENT 0.0 +.TP +.B big_metadata +(since: 3.4) +.sp +the filesystem uses \fInodesize\fP for metadata blocks, this can be bigger than the +page size +.TP +.B block_group_tree +(since: 6.1) +.sp +block group item representation using a dedicated b\-tree, this can greatly +reduce mount time for large filesystems +.TP +.B compress_lzo +(since: 2.6.38) +.sp +the \fIlzo\fP compression has been used on the filesystem, either as a mount option +or via \fBbtrfs filesystem defrag\fP\&. +.TP +.B compress_zstd +(since: 4.14) +.sp +the \fIzstd\fP compression has been used on the filesystem, either as a mount option +or via \fBbtrfs filesystem defrag\fP\&. +.TP +.B default_subvol +(since: 2.6.34) +.sp +the default subvolume has been set on the filesystem +.TP +.B extended_iref +(since: 3.7) +.sp +increased hardlink limit per file in a directory to 65536, older kernels +supported a varying number of hardlinks depending on the sum of all file name +sizes that can be stored into one metadata block +.TP +.B free_space_tree +(since: 4.5) +.sp +free space representation using a dedicated b\-tree, successor of v1 space cache +.TP +.B metadata_uuid +(since: 5.0) +.sp +the main filesystem UUID is the metadata_uuid, which stores the new UUID only +in the superblock while all metadata blocks still have the UUID set at mkfs +time, see \fI\%btrfstune(8)\fP for more +.TP +.B mixed_backref +(since: 2.6.31) +.sp +the last major disk format change, improved backreferences, now default +.TP +.B mixed_groups +(since: 2.6.37) +.sp +mixed data and metadata block groups, i.e. the data and metadata are not +separated and occupy the same block groups, this mode is suitable for small +volumes as there are no constraints how the remaining space should be used +(compared to the split mode, where empty metadata space cannot be used for data +and vice versa) +.sp +on the other hand, the final layout is quite unpredictable and possibly highly +fragmented, which means worse performance +.TP +.B no_holes +(since: 3.14) +.sp +improved representation of file extents where holes are not explicitly +stored as an extent, saves a few percent of metadata if sparse files are used +.TP +.B raid1c34 +(since: 5.5) +.sp +extended RAID1 mode with copies on 3 or 4 devices respectively +.TP +.B RAID56 +(since: 3.9) +.sp +the filesystem contains or contained a RAID56 profile of block groups +.TP +.B rmdir_subvol +(since: 4.18) +.sp +indicate that \fBrmdir(2)\fP syscall can delete an empty subvolume just like an +ordinary directory. Note that this feature only depends on the kernel version. +.TP +.B skinny_metadata +(since: 3.10) +.sp +reduced\-size metadata for extent references, saves a few percent of metadata +.TP +.B send_stream_version +(since: 5.10) +.sp +number of the highest supported send stream version +.TP +.B supported_checksums +(since: 5.5) +.sp +list of checksum algorithms supported by the kernel module, the respective +modules or built\-in implementing the algorithms need to be present to mount +the filesystem, see section \fI\%CHECKSUM ALGORITHMS\fP\&. +.TP +.B supported_sectorsizes +(since: 5.13) +.sp +list of values that are accepted as sector sizes (\fBmkfs.btrfs \-\-sectorsize\fP) by +the running kernel +.TP +.B supported_rescue_options +(since: 5.11) +.sp +list of values for the mount option \fIrescue\fP that are supported by the running +kernel, see \fI\%btrfs(5)\fP +.TP +.B zoned +(since: 5.12) +.sp +zoned mode is allocation/write friendly to host\-managed zoned devices, +allocation space is partitioned into fixed\-size zones that must be updated +sequentially, see section \fI\%ZONED MODE\fP +.UNINDENT +.SH SWAPFILE SUPPORT +.sp +A swapfile, when active, is a file\-backed swap area. It is supported since kernel 5.0. +Use \fBswapon(8)\fP to activate it, until then (respectively again after deactivating it +with \fBswapoff(8)\fP) it\(aqs just a normal file (with NODATACOW set), for which the special +restrictions for active swapfiles don\(aqt apply. +.sp +There are some limitations of the implementation in BTRFS and Linux swap +subsystem: +.INDENT 0.0 +.IP \(bu 2 +filesystem \- must be only single device +.IP \(bu 2 +filesystem \- must have only \fIsingle\fP data profile +.IP \(bu 2 +subvolume \- cannot be snapshotted if it contains any active swapfiles +.IP \(bu 2 +swapfile \- must be preallocated (i.e. no holes) +.IP \(bu 2 +swapfile \- must be NODATACOW (i.e. also NODATASUM, no compression) +.UNINDENT +.sp +The limitations come namely from the COW\-based design and mapping layer of +blocks that allows the advanced features like relocation and multi\-device +filesystems. However, the swap subsystem expects simpler mapping and no +background changes of the file block location once they\(aqve been assigned to +swap. +.sp +With active swapfiles, the following whole\-filesystem operations will skip +swapfile extents or may fail: +.INDENT 0.0 +.IP \(bu 2 +balance \- block groups with extents of any active swapfiles are skipped and +reported, the rest will be processed normally +.IP \(bu 2 +resize grow \- unaffected +.IP \(bu 2 +resize shrink \- works as long as the extents of any active swapfiles are +outside of the shrunk range +.IP \(bu 2 +device add \- if the new devices do not interfere with any already active swapfiles +this operation will work, though no new swapfile can be activated +afterwards +.IP \(bu 2 +device delete \- if the device has been added as above, it can be also deleted +.IP \(bu 2 +device replace \- ditto +.UNINDENT +.sp +When there are no active swapfiles and a whole\-filesystem exclusive operation +is running (e.g. balance, device delete, shrink), the swapfiles cannot be +temporarily activated. The operation must finish first. +.sp +To create and activate a swapfile run the following commands: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +# truncate \-s 0 swapfile +# chattr +C swapfile +# fallocate \-l 2G swapfile +# chmod 0600 swapfile +# mkswap swapfile +# swapon swapfile +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +Since version 6.1 it\(aqs possible to create the swapfile in a single command +(except the activation): +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +# btrfs filesystem mkswapfile \-\-size 2G swapfile +# swapon swapfile +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +Please note that the UUID returned by the \fImkswap\fP utility identifies the swap +\(dqfilesystem\(dq and because it\(aqs stored in a file, it\(aqs not generally visible and +usable as an identifier unlike if it was on a block device. +.sp +Once activated the file will appear in \fB/proc/swaps\fP: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +# cat /proc/swaps +Filename Type Size Used Priority +/path/swapfile file 2097152 0 \-2 +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +The swapfile can be created as one\-time operation or, once properly created, +activated on each boot by the \fBswapon \-a\fP command (usually started by the +service manager). Add the following entry to \fI/etc/fstab\fP, assuming the +filesystem that provides the \fI/path\fP has been already mounted at this point. +Additional mount options relevant for the swapfile can be set too (like +priority, not the BTRFS mount options). +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +/path/swapfile none swap defaults 0 0 +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +From now on the subvolume with the active swapfile cannot be snapshotted until +the swapfile is deactivated again by \fBswapoff\fP\&. Then the swapfile is a +regular file and the subvolume can be snapshotted again, though this would prevent +another activation any swapfile that has been snapshotted. New swapfiles (not +snapshotted) can be created and activated. +.sp +Otherwise, an inactive swapfile does not affect the containing subvolume. Activation +creates a temporary in\-memory status and prevents some file operations, but is +not stored permanently. +.SH HIBERNATION +.sp +A swapfile can be used for hibernation but it\(aqs not straightforward. Before +hibernation a resume offset must be written to file \fI/sys/power/resume_offset\fP +or the kernel command line parameter \fIresume_offset\fP must be set. +.sp +The value is the physical offset on the device. Note that \fBthis is not the same +value that\fP \fBfilefrag\fP \fBprints as physical offset!\fP +.sp +Btrfs filesystem uses mapping between logical and physical addresses but here +the physical can still map to one or more device\-specific physical block +addresses. It\(aqs the device\-specific physical offset that is suitable as resume +offset. +.sp +Since version 6.1 there\(aqs a command \fI\%btrfs inspect\-internal map\-swapfile\fP +that will print the device physical offset and the adjusted value for +\fB/sys/power/resume_offset\fP\&. Note that the value is divided by page size, i.e. +it\(aqs not the offset itself. +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +# btrfs filesystem mkswapfile swapfile +# btrfs inspect\-internal map\-swapfile swapfile +Physical start: 811511726080 +Resume offset: 198122980 +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +For scripting and convenience the option \fI\-r\fP will print just the offset: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +# btrfs inspect\-internal map\-swapfile \-r swapfile +198122980 +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +The command \fBmap\-swapfile\fP also verifies all the requirements, i.e. no holes, +single device, etc. +.SH TROUBLESHOOTING +.sp +If the swapfile activation fails please verify that you followed all the steps +above or check the system log (e.g. \fBdmesg\fP or \fBjournalctl\fP) for more +information. +.sp +Notably, the \fBswapon\fP utility exits with a message that does not say what +failed: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +# swapon /path/swapfile +swapon: /path/swapfile: swapon failed: Invalid argument +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +The specific reason is likely to be printed to the system log by the btrfs +module: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +# journalctl \-t kernel | grep swapfile +kernel: BTRFS warning (device sda): swapfile must have single data profile +.ft P +.fi +.UNINDENT +.UNINDENT +.SH CHECKSUM ALGORITHMS +.sp +Data and metadata are checksummed by default, the checksum is calculated before +write and verified after reading the blocks from devices. The whole metadata +block has a checksum stored inline in the b\-tree node header, each data block +has a detached checksum stored in the checksum tree. +.sp +There are several checksum algorithms supported. The default and backward +compatible is \fIcrc32c\fP\&. Since kernel 5.5 there are three more with different +characteristics and trade\-offs regarding speed and strength. The following list +may help you to decide which one to select. +.INDENT 0.0 +.TP +.B CRC32C (32bit digest) +default, best backward compatibility, very fast, modern CPUs have +instruction\-level support, not collision\-resistant but still good error +detection capabilities +.TP +.B XXHASH (64bit digest) +can be used as CRC32C successor, very fast, optimized for modern CPUs utilizing +instruction pipelining, good collision resistance and error detection +.TP +.B SHA256 (256bit digest) +a cryptographic\-strength hash, relatively slow but with possible CPU +instruction acceleration or specialized hardware cards, FIPS certified and +in wide use +.TP +.B BLAKE2b (256bit digest) +a cryptographic\-strength hash, relatively fast with possible CPU acceleration +using SIMD extensions, not standardized but based on BLAKE which was a SHA3 +finalist, in wide use, the algorithm used is BLAKE2b\-256 that\(aqs optimized for +64bit platforms +.UNINDENT +.sp +The \fIdigest size\fP affects overall size of data block checksums stored in the +filesystem. The metadata blocks have a fixed area up to 256 bits (32 bytes), so +there\(aqs no increase. Each data block has a separate checksum stored, with +additional overhead of the b\-tree leaves. +.sp +Approximate relative performance of the algorithms, measured against CRC32C +using reference software implementations on a 3.5GHz intel CPU: +.TS +center; +|l|l|l|l|. +_ +T{ +Digest +T} T{ +Cycles/4KiB +T} T{ +Ratio +T} T{ +Implementation +T} +_ +T{ +CRC32C +T} T{ +1700 +T} T{ +1.00 +T} T{ +CPU instruction +T} +_ +T{ +XXHASH +T} T{ +2500 +T} T{ +1.44 +T} T{ +reference impl. +T} +_ +T{ +SHA256 +T} T{ +105000 +T} T{ +61 +T} T{ +reference impl. +T} +_ +T{ +SHA256 +T} T{ +36000 +T} T{ +21 +T} T{ +libgcrypt/AVX2 +T} +_ +T{ +SHA256 +T} T{ +63000 +T} T{ +37 +T} T{ +libsodium/AVX2 +T} +_ +T{ +BLAKE2b +T} T{ +22000 +T} T{ +13 +T} T{ +reference impl. +T} +_ +T{ +BLAKE2b +T} T{ +19000 +T} T{ +11 +T} T{ +libgcrypt/AVX2 +T} +_ +T{ +BLAKE2b +T} T{ +19000 +T} T{ +11 +T} T{ +libsodium/AVX2 +T} +_ +.TE +.sp +Many kernels are configured with SHA256 as built\-in and not as a module. +The accelerated versions are however provided by the modules and must be loaded +explicitly (\fBmodprobe sha256\fP) before mounting the filesystem to make use of +them. You can check in \fB/sys/fs/btrfs/FSID/checksum\fP which one is used. If you +see \fIsha256\-generic\fP, then you may want to unmount and mount the filesystem +again, changing that on a mounted filesystem is not possible. +Check the file \fB/proc/crypto\fP, when the implementation is built\-in, you\(aqd find +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +name : sha256 +driver : sha256\-generic +module : kernel +priority : 100 +\&... +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +while accelerated implementation is e.g. +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +name : sha256 +driver : sha256\-avx2 +module : sha256_ssse3 +priority : 170 +\&... +.ft P +.fi +.UNINDENT +.UNINDENT +.SH COMPRESSION +.sp +Btrfs supports transparent file compression. There are three algorithms +available: ZLIB, LZO and ZSTD (since v4.14), with various levels. +The compression happens on the level of file extents and the algorithm is +selected by file property, mount option or by a defrag command. +You can have a single btrfs mount point that has some files that are +uncompressed, some that are compressed with LZO, some with ZLIB, for instance +(though you may not want it that way, it is supported). +.sp +Once the compression is set, all newly written data will be compressed, i.e. +existing data are untouched. Data are split into smaller chunks (128KiB) before +compression to make random rewrites possible without a high performance hit. Due +to the increased number of extents the metadata consumption is higher. The +chunks are compressed in parallel. +.sp +The algorithms can be characterized as follows regarding the speed/ratio +trade\-offs: +.INDENT 0.0 +.TP +.B ZLIB +.INDENT 7.0 +.IP \(bu 2 +slower, higher compression ratio +.IP \(bu 2 +levels: 1 to 9, mapped directly, default level is 3 +.IP \(bu 2 +good backward compatibility +.UNINDENT +.TP +.B LZO +.INDENT 7.0 +.IP \(bu 2 +faster compression and decompression than ZLIB, worse compression ratio, designed to be fast +.IP \(bu 2 +no levels +.IP \(bu 2 +good backward compatibility +.UNINDENT +.TP +.B ZSTD +.INDENT 7.0 +.IP \(bu 2 +compression comparable to ZLIB with higher compression/decompression speeds and different ratio +.IP \(bu 2 +levels: 1 to 15, mapped directly (higher levels are not available) +.IP \(bu 2 +since 4.14, levels since 5.1 +.UNINDENT +.UNINDENT +.sp +The differences depend on the actual data set and cannot be expressed by a +single number or recommendation. Higher levels consume more CPU time and may +not bring a significant improvement, lower levels are close to real time. +.SH HOW TO ENABLE COMPRESSION +.sp +Typically the compression can be enabled on the whole filesystem, specified for +the mount point. Note that the compression mount options are shared among all +mounts of the same filesystem, either bind mounts or subvolume mounts. +Please refer to \fI\%btrfs(5)\fP section +\fI\%MOUNT OPTIONS\fP\&. +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +$ mount \-o compress=zstd /dev/sdx /mnt +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +This will enable the \fBzstd\fP algorithm on the default level (which is 3). +The level can be specified manually too like \fBzstd:3\fP\&. Higher levels compress +better at the cost of time. This in turn may cause increased write latency, low +levels are suitable for real\-time compression and on reasonably fast CPU don\(aqt +cause noticeable performance drops. +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +$ btrfs filesystem defrag \-czstd file +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +The command above will start defragmentation of the whole \fIfile\fP and apply +the compression, regardless of the mount option. (Note: specifying level is not +yet implemented). The compression algorithm is not persistent and applies only +to the defragmentation command, for any other writes other compression settings +apply. +.sp +Persistent settings on a per\-file basis can be set in two ways: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +$ chattr +c file +$ btrfs property set file compression zstd +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +The first command is using legacy interface of file attributes inherited from +ext2 filesystem and is not flexible, so by default the \fIzlib\fP compression is +set. The other command sets a property on the file with the given algorithm. +(Note: setting level that way is not yet implemented.) +.SH COMPRESSION LEVELS +.sp +The level support of ZLIB has been added in v4.14, LZO does not support levels +(the kernel implementation provides only one), ZSTD level support has been added +in v5.1. +.sp +There are 9 levels of ZLIB supported (1 to 9), mapping 1:1 from the mount option +to the algorithm defined level. The default is level 3, which provides the +reasonably good compression ratio and is still reasonably fast. The difference +in compression gain of levels 7, 8 and 9 is comparable but the higher levels +take longer. +.sp +The ZSTD support includes levels 1 to 15, a subset of full range of what ZSTD +provides. Levels 1\-3 are real\-time, 4\-8 slower with improved compression and +9\-15 try even harder though the resulting size may not be significantly improved. +.sp +Level 0 always maps to the default. The compression level does not affect +compatibility. +.SH INCOMPRESSIBLE DATA +.sp +Files with already compressed data or with data that won\(aqt compress well with +the CPU and memory constraints of the kernel implementations are using a simple +decision logic. If the first portion of data being compressed is not smaller +than the original, the compression of the file is disabled \-\- unless the +filesystem is mounted with \fIcompress\-force\fP\&. In that case compression will +always be attempted on the file only to be later discarded. This is not optimal +and subject to optimizations and further development. +.sp +If a file is identified as incompressible, a flag is set (\fINOCOMPRESS\fP) and it\(aqs +sticky. On that file compression won\(aqt be performed unless forced. The flag +can be also set by \fBchattr +m\fP (since e2fsprogs 1.46.2) or by properties with +value \fIno\fP or \fInone\fP\&. Empty value will reset it to the default that\(aqs currently +applicable on the mounted filesystem. +.sp +There are two ways to detect incompressible data: +.INDENT 0.0 +.IP \(bu 2 +actual compression attempt \- data are compressed, if the result is not smaller, +it\(aqs discarded, so this depends on the algorithm and level +.IP \(bu 2 +pre\-compression heuristics \- a quick statistical evaluation on the data is +performed and based on the result either compression is performed or skipped, +the NOCOMPRESS bit is not set just by the heuristic, only if the compression +algorithm does not make an improvement +.UNINDENT +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +$ lsattr file +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-m file +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +Using the forcing compression is not recommended, the heuristics are +supposed to decide that and compression algorithms internally detect +incompressible data too. +.SH PRE-COMPRESSION HEURISTICS +.sp +The heuristics aim to do a few quick statistical tests on the compressed data +in order to avoid probably costly compression that would turn out to be +inefficient. Compression algorithms could have internal detection of +incompressible data too but this leads to more overhead as the compression is +done in another thread and has to write the data anyway. The heuristic is +read\-only and can utilize cached memory. +.sp +The tests performed based on the following: data sampling, long repeated +pattern detection, byte frequency, Shannon entropy. +.SH COMPATIBILITY +.sp +Compression is done using the COW mechanism so it\(aqs incompatible with +\fInodatacow\fP\&. Direct IO works on compressed files but will fall back to buffered +writes and leads to recompression. Currently \fInodatasum\fP and compression don\(aqt +work together. +.sp +The compression algorithms have been added over time so the version +compatibility should be also considered, together with other tools that may +access the compressed data like bootloaders. +.SH SYSFS INTERFACE +.sp +Btrfs has a sysfs interface to provide extra knobs. +.sp +The top level path is \fB/sys/fs/btrfs/\fP, and the main directory layout is the following: +.TS +center; +|l|l|l|. +_ +T{ +Relative Path +T} T{ +Description +T} T{ +Version +T} +_ +T{ +features/ +T} T{ +All supported features +T} T{ +3.14+ +T} +_ +T{ +<UUID>/ +T} T{ +Mounted fs UUID +T} T{ +3.14+ +T} +_ +T{ +<UUID>/allocation/ +T} T{ +Space allocation info +T} T{ +3.14+ +T} +_ +T{ +<UUID>/features/ +T} T{ +Features of the filesystem +T} T{ +3.14+ +T} +_ +T{ +<UUID>/devices/<DEVID>/ +T} T{ +Symlink to each block device sysfs +T} T{ +5.6+ +T} +_ +T{ +<UUID>/devinfo/<DEVID>/ +T} T{ +Btrfs specific info for each device +T} T{ +5.6+ +T} +_ +T{ +<UUID>/qgroups/ +T} T{ +Global qgroup info +T} T{ +5.9+ +T} +_ +T{ +<UUID>/qgroups/<LEVEL>_<ID>/ +T} T{ +Info for each qgroup +T} T{ +5.9+ +T} +_ +T{ +<UUID>/discard/ +T} T{ +Discard stats and tunables +T} T{ +6.1+ +T} +_ +.TE +.sp +For \fB/sys/fs/btrfs/features/\fP directory, each file means a supported feature +for the current kernel. +.sp +For \fB/sys/fs/btrfs/<UUID>/features/\fP directory, each file means an enabled +feature for the mounted filesystem. +.sp +The features shares the same name in section +\fI\%FILESYSTEM FEATURES\fP\&. +.sp +Files in \fB/sys/fs/btrfs/<UUID>/\fP directory are: +.INDENT 0.0 +.TP +.B bg_reclaim_threshold +(RW, since: 5.19) +.sp +Used space percentage of total device space to start auto block group claim. +Mostly for zoned devices. +.TP +.B checksum +(RO, since: 5.5) +.sp +The checksum used for the mounted filesystem. +This includes both the checksum type (see section +\fI\%CHECKSUM ALGORITHMS\fP) +and the implemented driver (mostly shows if it\(aqs hardware accelerated). +.TP +.B clone_alignment +(RO, since: 3.16) +.sp +The bytes alignment for \fIclone\fP and \fIdedupe\fP ioctls. +.TP +.B commit_stats +(RW, since: 6.0) +.sp +The performance statistics for btrfs transaction commit. +Mostly for debug purposes. +.sp +Writing into this file will reset the maximum commit duration to +the input value. +.TP +.B exclusive_operation +(RO, since: 5.10) +.sp +Shows the running exclusive operation. +Check section +\fI\%FILESYSTEM EXCLUSIVE OPERATIONS\fP +for details. +.TP +.B generation +(RO, since: 5.11) +.sp +Show the generation of the mounted filesystem. +.TP +.B label +(RW, since: 3.14) +.sp +Show the current label of the mounted filesystem. +.TP +.B metadata_uuid +(RO, since: 5.0) +.sp +Shows the metadata uuid of the mounted filesystem. +Check \fImetadata_uuid\fP feature for more details. +.TP +.B nodesize +(RO, since: 3.14) +.sp +Show the nodesize of the mounted filesystem. +.TP +.B quota_override +(RW, since: 4.13) +.sp +Shows the current quota override status. +0 means no quota override. +1 means quota override, quota can ignore the existing limit settings. +.TP +.B read_policy +(RW, since: 5.11) +.sp +Shows the current balance policy for reads. +Currently only \(dqpid\(dq (balance using pid value) is supported. +.TP +.B sectorsize +(RO, since: 3.14) +.sp +Shows the sectorsize of the mounted filesystem. +.UNINDENT +.sp +Files and directories in \fB/sys/fs/btrfs/<UUID>/allocations\fP directory are: +.INDENT 0.0 +.TP +.B global_rsv_reserved +(RO, since: 3.14) +.sp +The used bytes of the global reservation. +.TP +.B global_rsv_size +(RO, since: 3.14) +.sp +The total size of the global reservation. +.TP +.B \fIdata/\fP, \fImetadata/\fP and \fIsystem/\fP directories +(RO, since: 5.14) +.sp +Space info accounting for the 3 chunk types. +Mostly for debug purposes. +.UNINDENT +.sp +Files in \fB/sys/fs/btrfs/<UUID>/allocations/\fP\fIdata,metadata,system\fP directory are: +.INDENT 0.0 +.TP +.B bg_reclaim_threshold +(RW, since: 5.19) +.sp +Reclaimable space percentage of block group\(aqs size (excluding +permanently unusable space) to reclaim the block group. +Can be used on regular or zoned devices. +.TP +.B chunk_size +(RW, since: 6.0) +.sp +Shows the chunk size. Can be changed for data and metadata. +Cannot be set for zoned devices. +.UNINDENT +.sp +Files in \fB/sys/fs/btrfs/<UUID>/devinfo/<DEVID>\fP directory are: +.INDENT 0.0 +.TP +.B error_stats: +(RO, since: 5.14) +.sp +Shows all the history error numbers of the device. +.TP +.B fsid: +(RO, since: 5.17) +.sp +Shows the fsid which the device belongs to. +It can be different than the \fI<UUID>\fP if it\(aqs a seed device. +.TP +.B in_fs_metadata +(RO, since: 5.6) +.sp +Shows whether we have found the device. +Should always be 1, as if this turns to 0, the \fI<DEVID>\fP directory +would get removed automatically. +.TP +.B missing +(RO, since: 5.6) +.sp +Shows whether the device is missing. +.TP +.B replace_target +(RO, since: 5.6) +.sp +Shows whether the device is the replace target. +If no dev\-replace is running, this value should be 0. +.TP +.B scrub_speed_max +(RW, since: 5.14) +.sp +Shows the scrub speed limit for this device. The unit is Bytes/s. +0 means no limit. +.TP +.B writeable +(RO, since: 5.6) +.sp +Show if the device is writeable. +.UNINDENT +.sp +Files in \fB/sys/fs/btrfs/<UUID>/qgroups/\fP directory are: +.INDENT 0.0 +.TP +.B enabled +(RO, since: 6.1) +.sp +Shows if qgroup is enabled. +Also, if qgroup is disabled, the \fIqgroups\fP directory would +be removed automatically. +.TP +.B inconsistent +(RO, since: 6.1) +.sp +Shows if the qgroup numbers are inconsistent. +If 1, it\(aqs recommended to do a qgroup rescan. +.TP +.B drop_subtree_threshold +(RW, since: 6.1) +.sp +Shows the subtree drop threshold to automatically mark qgroup inconsistent. +.sp +When dropping large subvolumes with qgroup enabled, there would be a huge +load for qgroup accounting. +If we have a subtree whose level is larger than or equal to this value, +we will not trigger qgroup account at all, but mark qgroup inconsistent to +avoid the huge workload. +.sp +Default value is 8, where no subtree drop can trigger qgroup. +.sp +Lower value can reduce qgroup workload, at the cost of extra qgroup rescan +to re\-calculate the numbers. +.UNINDENT +.sp +Files in \fB/sys/fs/btrfs/<UUID>/<LEVEL>_<ID>/\fP directory are: +.INDENT 0.0 +.TP +.B exclusive +(RO, since: 5.9) +.sp +Shows the exclusively owned bytes of the qgroup. +.TP +.B limit_flags +(RO, since: 5.9) +.sp +Shows the numeric value of the limit flags. +If 0, means no limit implied. +.TP +.B max_exclusive +(RO, since: 5.9) +.sp +Shows the limits on exclusively owned bytes. +.TP +.B max_referenced +(RO, since: 5.9) +.sp +Shows the limits on referenced bytes. +.TP +.B referenced +(RO, since: 5.9) +.sp +Shows the referenced bytes of the qgroup. +.TP +.B rsv_data +(RO, since: 5.9) +.sp +Shows the reserved bytes for data. +.TP +.B rsv_meta_pertrans +(RO, since: 5.9) +.sp +Shows the reserved bytes for per transaction metadata. +.TP +.B rsv_meta_prealloc +(RO, since: 5.9) +.sp +Shows the reserved bytes for preallocated metadata. +.UNINDENT +.sp +Files in \fB/sys/fs/btrfs/<UUID>/discard/\fP directory are: +.INDENT 0.0 +.TP +.B discardable_bytes +(RO, since: 6.1) +.sp +Shows amount of bytes that can be discarded in the async discard and +nodiscard mode. +.TP +.B discardable_extents +(RO, since: 6.1) +.sp +Shows number of extents to be discarded in the async discard and +nodiscard mode. +.TP +.B discard_bitmap_bytes +(RO, since: 6.1) +.sp +Shows amount of discarded bytes from data tracked as bitmaps. +.TP +.B discard_extent_bytes +(RO, since: 6.1) +.sp +Shows amount of discarded extents from data tracked as bitmaps. +.TP +.B discard_bytes_saved +(RO, since: 6.1) +.sp +Shows the amount of bytes that were reallocated without being discarded. +.TP +.B kbps_limit +(RW, since: 6.1) +.sp +Tunable limit of kilobytes per second issued as discard IO in the async +discard mode. +.TP +.B iops_limit +(RW, since: 6.1) +.sp +Tunable limit of number of discard IO operations to be issued in the +async discard mode. +.TP +.B max_discard_size +(RW, since: 6.1) +.sp +Tunable limit for size of one IO discard request. +.UNINDENT +.SH FILESYSTEM EXCLUSIVE OPERATIONS +.sp +There are several operations that affect the whole filesystem and cannot be run +in parallel. Attempt to start one while another is running will fail (see +exceptions below). +.sp +Since kernel 5.10 the currently running operation can be obtained from +\fB/sys/fs/UUID/exclusive_operation\fP with following values and operations: +.INDENT 0.0 +.IP \(bu 2 +balance +.IP \(bu 2 +balance paused (since 5.17) +.IP \(bu 2 +device add +.IP \(bu 2 +device delete +.IP \(bu 2 +device replace +.IP \(bu 2 +resize +.IP \(bu 2 +swapfile activate +.IP \(bu 2 +none +.UNINDENT +.sp +Enqueuing is supported for several btrfs subcommands so they can be started +at once and then serialized. +.sp +There\(aqs an exception when a paused balance allows to start a device add +operation as they don\(aqt really collide and this can be used to add more space +for the balance to finish. +.SH FILESYSTEM LIMITS +.INDENT 0.0 +.TP +.B maximum file name length +255 +.sp +This limit is imposed by Linux VFS, the structures of BTRFS could store +larger file names. +.TP +.B maximum symlink target length +depends on the \fInodesize\fP value, for 4KiB it\(aqs 3949 bytes, for larger nodesize +it\(aqs 4095 due to the system limit PATH_MAX +.sp +The symlink target may not be a valid path, i.e. the path name components +can exceed the limits (NAME_MAX), there\(aqs no content validation at \fBsymlink(3)\fP +creation. +.TP +.B maximum number of inodes +2\s-2\u64\d\s0 but depends on the available metadata space as the inodes are created +dynamically +.sp +Each subvolume is an independent namespace of inodes and thus their +numbers, so the limit is per subvolume, not for the whole filesystem. +.TP +.B inode numbers +minimum number: 256 (for subvolumes), regular files and directories: 257, +maximum number: (2\s-2\u64\d\s0 \- 256) +.sp +The inode numbers that can be assigned to user created files are from +the whole 64bit space except first 256 and last 256 in that range that +are reserved for internal b\-tree identifiers. +.TP +.B maximum file length +inherent limit of BTRFS is 2\s-2\u64\d\s0 (16 EiB) but the practical +limit of Linux VFS is 2\s-2\u63\d\s0 (8 EiB) +.TP +.B maximum number of subvolumes +the subvolume ids can go up to 2\s-2\u48\d\s0 but the number of actual subvolumes +depends on the available metadata space +.sp +The space consumed by all subvolume metadata includes bookkeeping of +shared extents can be large (MiB, GiB). The range is not the full 64bit +range because of qgroups that use the upper 16 bits for another +purposes. +.TP +.B maximum number of hardlinks of a file in a directory +65536 when the \fIextref\fP feature is turned on during mkfs (default), roughly +100 otherwise and depends on file name length that fits into one metadata node +.TP +.B minimum filesystem size +the minimal size of each device depends on the \fImixed\-bg\fP feature, without that +(the default) it\(aqs about 109MiB, with mixed\-bg it\(aqs is 16MiB +.UNINDENT +.SH BOOTLOADER SUPPORT +.sp +GRUB2 (\fI\%https://www.gnu.org/software/grub\fP) has the most advanced support of +booting from BTRFS with respect to features. +.sp +U\-Boot (\fI\%https://www.denx.de/wiki/U\-Boot/\fP) has decent support for booting but +not all BTRFS features are implemented, check the documentation. +.sp +In general, the first 1MiB on each device is unused with the exception of +primary superblock that is on the offset 64KiB and spans 4KiB. The rest can be +freely used by bootloaders or for other system information. Note that booting +from a filesystem on \fI\%zoned device\fP is not supported. +.SH FILE ATTRIBUTES +.sp +The btrfs filesystem supports setting file attributes or flags. Note there are +old and new interfaces, with confusing names. The following list should clarify +that: +.INDENT 0.0 +.IP \(bu 2 +\fIattributes\fP: \fBchattr(1)\fP or \fBlsattr(1)\fP utilities (the ioctls are +FS_IOC_GETFLAGS and FS_IOC_SETFLAGS), due to the ioctl names the attributes +are also called flags +.IP \(bu 2 +\fIxflags\fP: to distinguish from the previous, it\(aqs extended flags, with tunable +bits similar to the attributes but extensible and new bits will be added in +the future (the ioctls are FS_IOC_FSGETXATTR and FS_IOC_FSSETXATTR but they +are not related to extended attributes that are also called xattrs), there\(aqs +no standard tool to change the bits, there\(aqs support in \fBxfs_io(8)\fP as +command \fBxfs_io \-c chattr\fP +.UNINDENT +.SS Attributes +.INDENT 0.0 +.TP +.B a +\fIappend only\fP, new writes are always written at the end of the file +.TP +.B A +\fIno atime updates\fP +.TP +.B c +\fIcompress data\fP, all data written after this attribute is set will be compressed. +Please note that compression is also affected by the mount options or the parent +directory attributes. +.sp +When set on a directory, all newly created files will inherit this attribute. +This attribute cannot be set with \(aqm\(aq at the same time. +.TP +.B C +\fIno copy\-on\-write\fP, file data modifications are done in\-place +.sp +When set on a directory, all newly created files will inherit this attribute. +.sp +\fBNOTE:\fP +.INDENT 7.0 +.INDENT 3.5 +Due to implementation limitations, this flag can be set/unset only on +empty files. +.UNINDENT +.UNINDENT +.TP +.B d +\fIno dump\fP, makes sense with 3rd party tools like \fBdump(8)\fP, on BTRFS the +attribute can be set/unset but no other special handling is done +.TP +.B D +\fIsynchronous directory updates\fP, for more details search \fBopen(2)\fP for \fIO_SYNC\fP +and \fIO_DSYNC\fP +.TP +.B i +\fIimmutable\fP, no file data and metadata changes allowed even to the root user as +long as this attribute is set (obviously the exception is unsetting the attribute) +.TP +.B m +\fIno compression\fP, permanently turn off compression on the given file. Any +compression mount options will not affect this file. (\fBchattr\fP support added in +1.46.2) +.sp +When set on a directory, all newly created files will inherit this attribute. +This attribute cannot be set with \fIc\fP at the same time. +.TP +.B S +\fIsynchronous updates\fP, for more details search \fBopen(2)\fP for \fIO_SYNC\fP and +\fIO_DSYNC\fP +.UNINDENT +.sp +No other attributes are supported. For the complete list please refer to the +\fBchattr(1)\fP manual page. +.SS XFLAGS +.sp +There\(aqs an overlap of letters assigned to the bits with the attributes, this list +refers to what \fBxfs_io(8)\fP provides: +.INDENT 0.0 +.TP +.B i +\fIimmutable\fP, same as the attribute +.TP +.B a +\fIappend only\fP, same as the attribute +.TP +.B s +\fIsynchronous updates\fP, same as the attribute \fIS\fP +.TP +.B A +\fIno atime updates\fP, same as the attribute +.TP +.B d +\fIno dump\fP, same as the attribute +.UNINDENT +.SH ZONED MODE +.sp +Since version 5.12 btrfs supports so called \fIzoned mode\fP\&. This is a special +on\-disk format and allocation/write strategy that\(aqs friendly to zoned devices. +In short, a device is partitioned into fixed\-size zones and each zone can be +updated by append\-only manner, or reset. As btrfs has no fixed data structures, +except the super blocks, the zoned mode only requires block placement that +follows the device constraints. You can learn about the whole architecture at +\fI\%https://zonedstorage.io\fP . +.sp +The devices are also called SMR/ZBC/ZNS, in \fIhost\-managed\fP mode. Note that +there are devices that appear as non\-zoned but actually are, this is +\fIdrive\-managed\fP and using zoned mode won\(aqt help. +.sp +The zone size depends on the device, typical sizes are 256MiB or 1GiB. In +general it must be a power of two. Emulated zoned devices like \fInull_blk\fP allow +to set various zone sizes. +.SS Requirements, limitations +.INDENT 0.0 +.IP \(bu 2 +all devices must have the same zone size +.IP \(bu 2 +maximum zone size is 8GiB +.IP \(bu 2 +minimum zone size is 4MiB +.IP \(bu 2 +mixing zoned and non\-zoned devices is possible, the zone writes are emulated, +but this is namely for testing +.IP \(bu 2 +the super block is handled in a special way and is at different locations than on a non\-zoned filesystem: +.INDENT 2.0 +.IP \(bu 2 +primary: 0B (and the next two zones) +.IP \(bu 2 +secondary: 512GiB (and the next two zones) +.IP \(bu 2 +tertiary: 4TiB (4096GiB, and the next two zones) +.UNINDENT +.UNINDENT +.SS Incompatible features +.sp +The main constraint of the zoned devices is lack of in\-place update of the data. +This is inherently incompatible with some features: +.INDENT 0.0 +.IP \(bu 2 +NODATACOW \- overwrite in\-place, cannot create such files +.IP \(bu 2 +fallocate \- preallocating space for in\-place first write +.IP \(bu 2 +mixed\-bg \- unordered writes to data and metadata, fixing that means using +separate data and metadata block groups +.IP \(bu 2 +booting \- the zone at offset 0 contains superblock, resetting the zone would +destroy the bootloader data +.UNINDENT +.sp +Initial support lacks some features but they\(aqre planned: +.INDENT 0.0 +.IP \(bu 2 +only single (data, metadata) and DUP (metadata) profile is supported +.IP \(bu 2 +fstrim \- due to dependency on free space cache v1 +.UNINDENT +.SS Super block +.sp +As said above, super block is handled in a special way. In order to be crash +safe, at least one zone in a known location must contain a valid superblock. +This is implemented as a ring buffer in two consecutive zones, starting from +known offsets 0B, 512GiB and 4TiB. +.sp +The values are different than on non\-zoned devices. Each new super block is +appended to the end of the zone, once it\(aqs filled, the zone is reset and writes +continue to the next one. Looking up the latest super block needs to read +offsets of both zones and determine the last written version. +.sp +The amount of space reserved for super block depends on the zone size. The +secondary and tertiary copies are at distant offsets as the capacity of the +devices is expected to be large, tens of terabytes. Maximum zone size supported +is 8GiB, which would mean that e.g. offset 0\-16GiB would be reserved just for +the super block on a hypothetical device of that zone size. This is wasteful +but required to guarantee crash safety. +.SS Devices +.SS Real hardware +.sp +The WD Ultrastar series 600 advertises HM\-SMR, i.e. the host\-managed zoned +mode. There are two more: DA (device managed, no zoned information exported to +the system), HA (host aware, can be used as regular disk but zoned writes +improve performance). There are not many devices available at the moment, the +information about exact zoned mode is hard to find, check data sheets or +community sources gathering information from real devices. +.sp +Note: zoned mode won\(aqt work with DM\-SMR disks. +.INDENT 0.0 +.IP \(bu 2 +Ultrastar® DC ZN540 NVMe ZNS SSD (\fI\%product +brief\fP) +.UNINDENT +.SS Emulated: null_blk +.sp +The driver \fInull_blk\fP provides memory backed device and is suitable for +testing. There are some quirks setting up the devices. The module must be +loaded with \fInr_devices=0\fP or the numbering of device nodes will be offset. The +\fIconfigfs\fP must be mounted at \fI/sys/kernel/config\fP and the administration of +the null_blk devices is done in \fI/sys/kernel/config/nullb\fP\&. The device nodes +are named like \fB/dev/nullb0\fP and are numbered sequentially. NOTE: the device +name may be different than the named directory in sysfs! +.sp +Setup: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +modprobe\ configfs +modprobe\ null_blk\ nr_devices=0 +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +Create a device \fImydev\fP, assuming no other previously created devices, size is +2048MiB, zone size 256MiB. There are more tunable parameters, this is a minimal +example taking defaults: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +cd\ /sys/kernel/config/nullb/ +mkdir\ mydev +cd\ mydev +echo\ 2048\ >\ size +echo\ 1\ >\ zoned +echo\ 1\ >\ memory_backed +echo\ 256\ >\ zone_size +echo\ 1\ >\ power +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +This will create a device \fB/dev/nullb0\fP and the value of file \fIindex\fP will +match the ending number of the device node. +.sp +Remove the device: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +rmdir\ /sys/kernel/config/nullb/mydev +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +Then continue with \fBmkfs.btrfs /dev/nullb0\fP, the zoned mode is auto\-detected. +.sp +For convenience, there\(aqs a script wrapping the basic null_blk management operations +\fI\%https://github.com/kdave/nullb.git\fP, the above commands become: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +nullb setup +nullb create \-s 2g \-z 256 +mkfs.btrfs /dev/nullb0 +\&... +nullb rm nullb0 +.ft P +.fi +.UNINDENT +.UNINDENT +.SS Emulated: TCMU runner +.sp +TCMU is a framework to emulate SCSI devices in userspace, providing various +backends for the storage, with zoned support as well. A file\-backed zoned +device can provide more options for larger storage and zone size. Please follow +the instructions at \fI\%https://zonedstorage.io/projects/tcmu\-runner/\fP . +.SS Compatibility, incompatibility +.INDENT 0.0 +.IP \(bu 2 +the feature sets an incompat bit and requires new kernel to access the +filesystem (for both read and write) +.IP \(bu 2 +superblock needs to be handled in a special way, there are still 3 copies +but at different offsets (0, 512GiB, 4TiB) and the 2 consecutive zones are a +ring buffer of the superblocks, finding the latest one needs reading it from +the write pointer or do a full scan of the zones +.IP \(bu 2 +mixing zoned and non zoned devices is possible (zones are emulated) but is +recommended only for testing +.IP \(bu 2 +mixing zoned devices with different zone sizes is not possible +.IP \(bu 2 +zone sizes must be power of two, zone sizes of real devices are e.g. 256MiB +or 1GiB, larger size is expected, maximum zone size supported by btrfs is +8GiB +.UNINDENT +.SS Status, stability, reporting bugs +.sp +The zoned mode has been released in 5.12 and there are still some rough edges +and corner cases one can hit during testing. Please report bugs to +\fI\%https://github.com/naota/linux/issues/\fP . +.SS References +.INDENT 0.0 +.IP \(bu 2 +\fI\%https://zonedstorage.io\fP +.INDENT 2.0 +.IP \(bu 2 +\fI\%https://zonedstorage.io/projects/libzbc/\fP \-\- \fIlibzbc\fP is library and set +of tools to directly manipulate devices with ZBC/ZAC support +.IP \(bu 2 +\fI\%https://zonedstorage.io/projects/libzbd/\fP \-\- \fIlibzbd\fP uses the kernel +provided zoned block device interface based on the ioctl() system calls +.UNINDENT +.IP \(bu 2 +\fI\%https://hddscan.com/blog/2020/hdd\-wd\-smr.html\fP \-\- some details about exact device types +.IP \(bu 2 +\fI\%https://lwn.net/Articles/853308/\fP \-\- \fIBtrfs on zoned block devices\fP +.IP \(bu 2 +\fI\%https://www.usenix.org/conference/vault20/presentation/bjorling\fP \-\- Zone +Append: A New Way of Writing to Zoned Storage +.UNINDENT +.SH CONTROL DEVICE +.sp +There\(aqs a character special device \fB/dev/btrfs\-control\fP with major and minor +numbers 10 and 234 (the device can be found under the \fImisc\fP category). +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +$ ls \-l /dev/btrfs\-control +crw\-\-\-\-\-\-\- 1 root root 10, 234 Jan 1 12:00 /dev/btrfs\-control +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +The device accepts some ioctl calls that can perform following actions on the +filesystem module: +.INDENT 0.0 +.IP \(bu 2 +scan devices for btrfs filesystem (i.e. to let multi\-device filesystems mount +automatically) and register them with the kernel module +.IP \(bu 2 +similar to scan, but also wait until the device scanning process is finished +for a given filesystem +.IP \(bu 2 +get the supported features (can be also found under \fB/sys/fs/btrfs/features\fP) +.UNINDENT +.sp +The device is created when btrfs is initialized, either as a module or a +built\-in functionality and makes sense only in connection with that. Running +e.g. mkfs without the module loaded will not register the device and will +probably warn about that. +.sp +In rare cases when the module is loaded but the device is not present (most +likely accidentally deleted), it\(aqs possible to recreate it by +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +# mknod \-\-mode=600 /dev/btrfs\-control c 10 234 +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +or (since 5.11) by a convenience command +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +# btrfs rescue create\-control\-device +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +The control device is not strictly required but the device scanning will not +work and a workaround would need to be used to mount a multi\-device filesystem. +The mount option \fIdevice\fP can trigger the device scanning during mount, see +also \fBbtrfs device scan\fP\&. +.SH FILESYSTEM WITH MULTIPLE PROFILES +.sp +It is possible that a btrfs filesystem contains multiple block group profiles +of the same type. This could happen when a profile conversion using balance +filters is interrupted (see \fI\%btrfs\-balance(8)\fP). Some +\fBbtrfs\fP commands perform +a test to detect this kind of condition and print a warning like this: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +WARNING: Multiple block group profiles detected, see \(aqman btrfs(5)\(aq. +WARNING: Data: single, raid1 +WARNING: Metadata: single, raid1 +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +The corresponding output of \fBbtrfs filesystem df\fP might look like: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +WARNING: Multiple block group profiles detected, see \(aqman btrfs(5)\(aq. +WARNING: Data: single, raid1 +WARNING: Metadata: single, raid1 +Data, RAID1: total=832.00MiB, used=0.00B +Data, single: total=1.63GiB, used=0.00B +System, single: total=4.00MiB, used=16.00KiB +Metadata, single: total=8.00MiB, used=112.00KiB +Metadata, RAID1: total=64.00MiB, used=32.00KiB +GlobalReserve, single: total=16.25MiB, used=0.00B +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +There\(aqs more than one line for type \fIData\fP and \fIMetadata\fP, while the profiles +are \fIsingle\fP and \fIRAID1\fP\&. +.sp +This state of the filesystem OK but most likely needs the user/administrator to +take an action and finish the interrupted tasks. This cannot be easily done +automatically, also the user knows the expected final profiles. +.sp +In the example above, the filesystem started as a single device and \fIsingle\fP +block group profile. Then another device was added, followed by balance with +\fIconvert=raid1\fP but for some reason hasn\(aqt finished. Restarting the balance +with \fIconvert=raid1\fP will continue and end up with filesystem with all block +group profiles \fIRAID1\fP\&. +.sp +\fBNOTE:\fP +.INDENT 0.0 +.INDENT 3.5 +If you\(aqre familiar with balance filters, you can use +\fIconvert=raid1,profiles=single,soft\fP, which will take only the unconverted +\fIsingle\fP profiles and convert them to \fIraid1\fP\&. This may speed up the conversion +as it would not try to rewrite the already convert \fIraid1\fP profiles. +.UNINDENT +.UNINDENT +.sp +Having just one profile is desired as this also clearly defines the profile of +newly allocated block groups, otherwise this depends on internal allocation +policy. When there are multiple profiles present, the order of selection is +RAID56, RAID10, RAID1, RAID0 as long as the device number constraints are +satisfied. +.sp +Commands that print the warning were chosen so they\(aqre brought to user +attention when the filesystem state is being changed in that regard. This is: +\fBdevice add\fP, \fBdevice delete\fP, \fBbalance cancel\fP, \fBbalance pause\fP\&. Commands +that report space usage: \fBfilesystem df\fP, \fBdevice usage\fP\&. The command +\fBfilesystem usage\fP provides a line in the overall summary: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +Multiple profiles: yes (data, metadata) +.ft P +.fi +.UNINDENT +.UNINDENT +.SH SEEDING DEVICE +.sp +The COW mechanism and multiple devices under one hood enable an interesting +concept, called a seeding device: extending a read\-only filesystem on a +device with another device that captures all writes. For example +imagine an immutable golden image of an operating system enhanced with another +device that allows to use the data from the golden image and normal operation. +This idea originated on CD\-ROMs with base OS and allowing to use them for live +systems, but this became obsolete. There are technologies providing similar +functionality, like \fI\%unionmount\fP, +\fI\%overlayfs\fP or +\fI\%qcow2\fP image snapshot. +.sp +The seeding device starts as a normal filesystem, once the contents is ready, +\fBbtrfstune \-S 1\fP is used to flag it as a seeding device. Mounting such device +will not allow any writes, except adding a new device by \fBbtrfs device add\fP\&. +Then the filesystem can be remounted as read\-write. +.sp +Given that the filesystem on the seeding device is always recognized as +read\-only, it can be used to seed multiple filesystems from one device at the +same time. The UUID that is normally attached to a device is automatically +changed to a random UUID on each mount. +.sp +Once the seeding device is mounted, it needs the writable device. After adding +it, unmounting and mounting with \fBumount /path; mount /dev/writable +/path\fP or remounting read\-write with \fBremount \-o remount,rw\fP makes the +filesystem at \fB/path\fP ready for use. +.sp +\fBNOTE:\fP +.INDENT 0.0 +.INDENT 3.5 +There is a known bug with using remount to make the mount writeable: +remount will leave the filesystem in a state where it is unable to +clean deleted snapshots, so it will leak space until it is unmounted +and mounted properly. +.UNINDENT +.UNINDENT +.sp +Furthermore, deleting the seeding device from the filesystem can turn it into +a normal filesystem, provided that the writable device can also contain all the +data from the seeding device. +.sp +The seeding device flag can be cleared again by \fBbtrfstune \-f \-S 0\fP, e.g. +allowing to update with newer data but please note that this will invalidate +all existing filesystems that use this particular seeding device. This works +for some use cases, not for others, and the forcing flag to the command is +mandatory to avoid accidental mistakes. +.sp +Example how to create and use one seeding device: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +# mkfs.btrfs /dev/sda +# mount /dev/sda /mnt/mnt1 +\&... fill mnt1 with data +# umount /mnt/mnt1 + +# btrfstune \-S 1 /dev/sda + +# mount /dev/sda /mnt/mnt1 +# btrfs device add /dev/sdb /mnt/mnt1 +# umount /mnt/mnt1 +# mount /dev/sdb /mnt/mnt1 +\&... /mnt/mnt1 is now writable +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +Now \fB/mnt/mnt1\fP can be used normally. The device \fB/dev/sda\fP can be mounted +again with a another writable device: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +# mount /dev/sda /mnt/mnt2 +# btrfs device add /dev/sdc /mnt/mnt2 +# umount /mnt/mnt2 +# mount /dev/sdc /mnt/mnt2 +\&... /mnt/mnt2 is now writable +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +The writable device (file:\fI/dev/sdb\fP) can be decoupled from the seeding device and +used independently: +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +# btrfs device delete /dev/sda /mnt/mnt1 +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +As the contents originated in the seeding device, it\(aqs possible to turn +\fB/dev/sdb\fP to a seeding device again and repeat the whole process. +.sp +A few things to note: +.INDENT 0.0 +.IP \(bu 2 +it\(aqs recommended to use only single device for the seeding device, it works +for multiple devices but the \fIsingle\fP profile must be used in order to make +the seeding device deletion work +.IP \(bu 2 +block group profiles \fIsingle\fP and \fIdup\fP support the use cases above +.IP \(bu 2 +the label is copied from the seeding device and can be changed by \fBbtrfs filesystem label\fP +.IP \(bu 2 +each new mount of the seeding device gets a new random UUID +.IP \(bu 2 +\fBumount /path; mount /dev/writable /path\fP can be replaced with +\fBmount \-o remount,rw /path\fP +but it won\(aqt reclaim space of deleted subvolumes until the seeding device +is mounted read\-write again before making it seeding again +.UNINDENT +.SS Chained seeding devices +.sp +Though it\(aqs not recommended and is rather an obscure and untested use case, +chaining seeding devices is possible. In the first example, the writable device +\fB/dev/sdb\fP can be turned onto another seeding device again, depending on the +unchanged seeding device \fB/dev/sda\fP\&. Then using \fB/dev/sdb\fP as the primary +seeding device it can be extended with another writable device, say \fB/dev/sdd\fP, +and it continues as before as a simple tree structure on devices. +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +# mkfs.btrfs /dev/sda +# mount /dev/sda /mnt/mnt1 +\&... fill mnt1 with data +# umount /mnt/mnt1 + +# btrfstune \-S 1 /dev/sda + +# mount /dev/sda /mnt/mnt1 +# btrfs device add /dev/sdb /mnt/mnt1 +# mount \-o remount,rw /mnt/mnt1 +\&... /mnt/mnt1 is now writable +# umount /mnt/mnt1 + +# btrfstune \-S 1 /dev/sdb + +# mount /dev/sdb /mnt/mnt1 +# btrfs device add /dev/sdc /mnt +# mount \-o remount,rw /mnt/mnt1 +\&... /mnt/mnt1 is now writable +# umount /mnt/mnt1 +.ft P +.fi +.UNINDENT +.UNINDENT +.sp +As a result we have: +.INDENT 0.0 +.IP \(bu 2 +\fIsda\fP is a single seeding device, with its initial contents +.IP \(bu 2 +\fIsdb\fP is a seeding device but requires \fIsda\fP, the contents are from the time +when \fIsdb\fP is made seeding, i.e. contents of \fIsda\fP with any later changes +.IP \(bu 2 +\fIsdc\fP last writable, can be made a seeding one the same way as was \fIsdb\fP, +preserving its contents and depending on \fIsda\fP and \fIsdb\fP +.UNINDENT +.sp +As long as the seeding devices are unmodified and available, they can be used +to start another branch. +.SH RAID56 STATUS AND RECOMMENDED PRACTICES +.sp +The RAID56 feature provides striping and parity over several devices, same as +the traditional RAID5/6. There are some implementation and design deficiencies +that make it unreliable for some corner cases and the feature \fBshould not be +used in production, only for evaluation or testing\fP\&. The power failure safety +for metadata with RAID56 is not 100%. +.SS Metadata +.sp +Do not use \fIraid5\fP nor \fIraid6\fP for metadata. Use \fIraid1\fP or \fIraid1c3\fP +respectively. +.sp +The substitute profiles provide the same guarantees against loss of 1 or 2 +devices, and in some respect can be an improvement. Recovering from one +missing device will only need to access the remaining 1st or 2nd copy, that in +general may be stored on some other devices due to the way RAID1 works on +btrfs, unlike on a striped profile (similar to \fIraid0\fP) that would need all +devices all the time. +.sp +The space allocation pattern and consumption is different (e.g. on N devices): +for \fIraid5\fP as an example, a 1GiB chunk is reserved on each device, while with +\fIraid1\fP there\(aqs each 1GiB chunk stored on 2 devices. The consumption of each +1GiB of used metadata is then \fIN * 1GiB\fP for vs \fI2 * 1GiB\fP\&. Using \fIraid1\fP +is also more convenient for balancing/converting to other profile due to lower +requirement on the available chunk space. +.SS Missing/incomplete support +.sp +When RAID56 is on the same filesystem with different raid profiles, the space +reporting is inaccurate, e.g. \fBdf\fP, \fBbtrfs filesystem df\fP or +\fBbtrfs filesystem usage\fP\&. When there\(aqs only a one profile per block +group type (e.g. RAID5 for data) the reporting is accurate. +.sp +When scrub is started on a RAID56 filesystem, it\(aqs started on all devices that +degrade the performance. The workaround is to start it on each device +separately. Due to that the device stats may not match the actual state and +some errors might get reported multiple times. +.sp +The \fIwrite hole\fP problem. An unclean shutdown could leave a partially written +stripe in a state where the some stripe ranges and the parity are from the old +writes and some are new. The information which is which is not tracked. Write +journal is not implemented. Alternatively a full read\-modify\-write would make +sure that a full stripe is always written, avoiding the write hole completely, +but performance in that case turned out to be too bad for use. +.sp +The striping happens on all available devices (at the time the chunks were +allocated), so in case a new device is added it may not be utilized +immediately and would require a rebalance. A fixed configured stripe width is +not implemented. +.SH STORAGE MODEL, HARDWARE CONSIDERATIONS +.SS Storage model +.sp +\fIA storage model is a model that captures key physical aspects of data +structure in a data store. A filesystem is the logical structure organizing +data on top of the storage device.\fP +.sp +The filesystem assumes several features or limitations of the storage device +and utilizes them or applies measures to guarantee reliability. BTRFS in +particular is based on a COW (copy on write) mode of writing, i.e. not updating +data in place but rather writing a new copy to a different location and then +atomically switching the pointers. +.sp +In an ideal world, the device does what it promises. The filesystem assumes +that this may not be true so additional mechanisms are applied to either detect +misbehaving hardware or get valid data by other means. The devices may (and do) +apply their own detection and repair mechanisms but we won\(aqt assume any. +.sp +The following assumptions about storage devices are considered (sorted by +importance, numbers are for further reference): +.INDENT 0.0 +.IP 1. 3 +atomicity of reads and writes of blocks/sectors (the smallest unit of data +the device presents to the upper layers) +.IP 2. 3 +there\(aqs a flush command that instructs the device to forcibly order writes +before and after the command; alternatively there\(aqs a barrier command that +facilitates the ordering but may not flush the data +.IP 3. 3 +data sent to write to a given device offset will be written without further +changes to the data and to the offset +.IP 4. 3 +writes can be reordered by the device, unless explicitly serialized by the +flush command +.IP 5. 3 +reads and writes can be freely reordered and interleaved +.UNINDENT +.sp +The consistency model of BTRFS builds on these assumptions. The logical data +updates are grouped, into a generation, written on the device, serialized by +the flush command and then the super block is written ending the generation. +All logical links among metadata comprising a consistent view of the data may +not cross the generation boundary. +.SS When things go wrong +.sp +\fBNo or partial atomicity of block reads/writes (1)\fP +.INDENT 0.0 +.IP \(bu 2 +\fIProblem\fP: a partial block contents is written (\fItorn write\fP), e.g. due to a +power glitch or other electronics failure during the read/write +.IP \(bu 2 +\fIDetection\fP: checksum mismatch on read +.IP \(bu 2 +\fIRepair\fP: use another copy or rebuild from multiple blocks using some encoding +scheme +.UNINDENT +.sp +\fBThe flush command does not flush (2)\fP +.sp +This is perhaps the most serious problem and impossible to mitigate by +filesystem without limitations and design restrictions. What could happen in +the worst case is that writes from one generation bleed to another one, while +still letting the filesystem consider the generations isolated. Crash at any +point would leave data on the device in an inconsistent state without any hint +what exactly got written, what is missing and leading to stale metadata link +information. +.sp +Devices usually honor the flush command, but for performance reasons may do +internal caching, where the flushed data are not yet persistently stored. A +power failure could lead to a similar scenario as above, although it\(aqs less +likely that later writes would be written before the cached ones. This is +beyond what a filesystem can take into account. Devices or controllers are +usually equipped with batteries or capacitors to write the cache contents even +after power is cut. (\fIBattery backed write cache\fP) +.sp +\fBData get silently changed on write (3)\fP +.sp +Such thing should not happen frequently, but still can happen spuriously due +the complex internal workings of devices or physical effects of the storage +media itself. +.INDENT 0.0 +.IP \(bu 2 +\fIProblem\fP: while the data are written atomically, the contents get changed +.IP \(bu 2 +\fIDetection\fP: checksum mismatch on read +.IP \(bu 2 +\fIRepair\fP: use another copy or rebuild from multiple blocks using some +encoding scheme +.UNINDENT +.sp +\fBData get silently written to another offset (3)\fP +.sp +This would be another serious problem as the filesystem has no information +when it happens. For that reason the measures have to be done ahead of time. +This problem is also commonly called \fIghost write\fP\&. +.sp +The metadata blocks have the checksum embedded in the blocks, so a correct +atomic write would not corrupt the checksum. It\(aqs likely that after reading +such block the data inside would not be consistent with the rest. To rule that +out there\(aqs embedded block number in the metadata block. It\(aqs the logical +block number because this is what the logical structure expects and verifies. +.sp +The following is based on information publicly available, user feedback, +community discussions or bug report analyses. It\(aqs not complete and further +research is encouraged when in doubt. +.SS Main memory +.sp +The data structures and raw data blocks are temporarily stored in computer +memory before they get written to the device. It is critical that memory is +reliable because even simple bit flips can have vast consequences and lead to +damaged structures, not only in the filesystem but in the whole operating +system. +.sp +Based on experience in the community, memory bit flips are more common than one +would think. When it happens, it\(aqs reported by the tree\-checker or by a checksum +mismatch after reading blocks. There are some very obvious instances of bit +flips that happen, e.g. in an ordered sequence of keys in metadata blocks. We can +easily infer from the other data what values get damaged and how. However, fixing +that is not straightforward and would require cross\-referencing data from the +entire filesystem to see the scope. +.sp +If available, ECC memory should lower the chances of bit flips, but this +type of memory is not available in all cases. A memory test should be performed +in case there\(aqs a visible bit flip pattern, though this may not detect a faulty +memory module because the actual load of the system could be the factor making +the problems appear. In recent years attacks on how the memory modules operate +have been demonstrated (\fIrowhammer\fP) achieving specific bits to be flipped. +While these were targeted, this shows that a series of reads or writes can +affect unrelated parts of memory. +.sp +Further reading: +.INDENT 0.0 +.IP \(bu 2 +\fI\%https://en.wikipedia.org/wiki/Row_hammer\fP +.UNINDENT +.sp +What to do: +.INDENT 0.0 +.IP \(bu 2 +run \fImemtest\fP, note that sometimes memory errors happen only when the system +is under heavy load that the default memtest cannot trigger +.IP \(bu 2 +memory errors may appear as filesystem going read\-only due to \(dqpre write\(dq +check, that verify meta data before they get written but fail some basic +consistency checks +.UNINDENT +.SS Direct memory access (DMA) +.sp +Another class of errors is related to DMA (direct memory access) performed +by device drivers. While this could be considered a software error, the +data transfers that happen without CPU assistance may accidentally corrupt +other pages. Storage devices utilize DMA for performance reasons, the +filesystem structures and data pages are passed back and forth, making +errors possible in case page life time is not properly tracked. +.sp +There are lots of quirks (device\-specific workarounds) in Linux kernel +drivers (regarding not only DMA) that are added when found. The quirks +may avoid specific errors or disable some features to avoid worse problems. +.sp +What to do: +.INDENT 0.0 +.IP \(bu 2 +use up\-to\-date kernel (recent releases or maintained long term support versions) +.IP \(bu 2 +as this may be caused by faulty drivers, keep the systems up\-to\-date +.UNINDENT +.SS Rotational disks (HDD) +.sp +Rotational HDDs typically fail at the level of individual sectors or small clusters. +Read failures are caught on the levels below the filesystem and are returned to +the user as \fIEIO \- Input/output error\fP\&. Reading the blocks repeatedly may +return the data eventually, but this is better done by specialized tools and +filesystem takes the result of the lower layers. Rewriting the sectors may +trigger internal remapping but this inevitably leads to data loss. +.sp +Disk firmware is technically software but from the filesystem perspective is +part of the hardware. IO requests are processed, and caching or various +other optimizations are performed, which may lead to bugs under high load or +unexpected physical conditions or unsupported use cases. +.sp +Disks are connected by cables with two ends, both of which can cause problems +when not attached properly. Data transfers are protected by checksums and the +lower layers try hard to transfer the data correctly or not at all. The errors +from badly\-connecting cables may manifest as large amount of failed read or +write requests, or as short error bursts depending on physical conditions. +.sp +What to do: +.INDENT 0.0 +.IP \(bu 2 +check \fBsmartctl\fP for potential issues +.UNINDENT +.SS Solid state drives (SSD) +.sp +The mechanism of information storage is different from HDDs and this affects +the failure mode as well. The data are stored in cells grouped in large blocks +with limited number of resets and other write constraints. The firmware tries +to avoid unnecessary resets and performs optimizations to maximize the storage +media lifetime. The known techniques are deduplication (blocks with same +fingerprint/hash are mapped to same physical block), compression or internal +remapping and garbage collection of used memory cells. Due to the additional +processing there are measures to verity the data e.g. by ECC codes. +.sp +The observations of failing SSDs show that the whole electronic fails at once +or affects a lot of data (e.g. stored on one chip). Recovering such data +may need specialized equipment and reading data repeatedly does not help as +it\(aqs possible with HDDs. +.sp +There are several technologies of the memory cells with different +characteristics and price. The lifetime is directly affected by the type and +frequency of data written. Writing \(dqtoo much\(dq distinct data (e.g. encrypted) +may render the internal deduplication ineffective and lead to a lot of rewrites +and increased wear of the memory cells. +.sp +There are several technologies and manufacturers so it\(aqs hard to describe them +but there are some that exhibit similar behaviour: +.INDENT 0.0 +.IP \(bu 2 +expensive SSD will use more durable memory cells and is optimized for +reliability and high load +.IP \(bu 2 +cheap SSD is projected for a lower load (\(dqdesktop user\(dq) and is optimized for +cost, it may employ the optimizations and/or extended error reporting +partially or not at all +.UNINDENT +.sp +It\(aqs not possible to reliably determine the expected lifetime of an SSD due to +lack of information about how it works or due to lack of reliable stats provided +by the device. +.sp +Metadata writes tend to be the biggest component of lifetime writes to a SSD, +so there is some value in reducing them. Depending on the device class (high +end/low end) the features like DUP block group profiles may affect the +reliability in both ways: +.INDENT 0.0 +.IP \(bu 2 +\fIhigh end\fP are typically more reliable and using \fIsingle\fP for data and +metadata could be suitable to reduce device wear +.IP \(bu 2 +\fIlow end\fP could lack ability to identify errors so an additional redundancy +at the filesystem level (checksums, \fIDUP\fP) could help +.UNINDENT +.sp +Only users who consume 50 to 100% of the SSD\(aqs actual lifetime writes need to be +concerned by the write amplification of btrfs DUP metadata. Most users will be +far below 50% of the actual lifetime, or will write the drive to death and +discover how many writes 100% of the actual lifetime was. SSD firmware often +adds its own write multipliers that can be arbitrary and unpredictable and +dependent on application behavior, and these will typically have far greater +effect on SSD lifespan than DUP metadata. It\(aqs more or less impossible to +predict when a SSD will run out of lifetime writes to within a factor of two, so +it\(aqs hard to justify wear reduction as a benefit. +.sp +Further reading: +.INDENT 0.0 +.IP \(bu 2 +\fI\%https://www.snia.org/educational\-library/ssd\-and\-deduplication\-end\-spinning\-disk\-2012\fP +.IP \(bu 2 +\fI\%https://www.snia.org/educational\-library/realities\-solid\-state\-storage\-2013\-2013\fP +.IP \(bu 2 +\fI\%https://www.snia.org/educational\-library/ssd\-performance\-primer\-2013\fP +.IP \(bu 2 +\fI\%https://www.snia.org/educational\-library/how\-controllers\-maximize\-ssd\-life\-2013\fP +.UNINDENT +.sp +What to do: +.INDENT 0.0 +.IP \(bu 2 +run \fBsmartctl\fP or self\-tests to look for potential issues +.IP \(bu 2 +keep the firmware up\-to\-date +.UNINDENT +.SS NVM express, non\-volatile memory (NVMe) +.sp +NVMe is a type of persistent memory usually connected over a system bus (PCIe) +or similar interface and the speeds are an order of magnitude faster than SSD. +It is also a non\-rotating type of storage, and is not typically connected by a +cable. It\(aqs not a SCSI type device either but rather a complete specification +for logical device interface. +.sp +In a way the errors could be compared to a combination of SSD class and regular +memory. Errors may exhibit as random bit flips or IO failures. There are tools +to access the internal log (\fBnvme log\fP and \fBnvme\-cli\fP) for a more detailed +analysis. +.sp +There are separate error detection and correction steps performed e.g. on the +bus level and in most cases never making in to the filesystem level. Once this +happens it could mean there\(aqs some systematic error like overheating or bad +physical connection of the device. You may want to run self\-tests (using +\fBsmartctl\fP). +.INDENT 0.0 +.IP \(bu 2 +\fI\%https://en.wikipedia.org/wiki/NVM_Express\fP +.IP \(bu 2 +\fI\%https://www.smartmontools.org/wiki/NVMe_Support\fP +.UNINDENT +.SS Drive firmware +.sp +Firmware is technically still software but embedded into the hardware. As all +software has bugs, so does firmware. Storage devices can update the firmware +and fix known bugs. In some cases the it\(aqs possible to avoid certain bugs by +quirks (device\-specific workarounds) in Linux kernel. +.sp +A faulty firmware can cause wide range of corruptions from small and localized +to large affecting lots of data. Self\-repair capabilities may not be sufficient. +.sp +What to do: +.INDENT 0.0 +.IP \(bu 2 +check for firmware updates in case there are known problems, note that +updating firmware can be risky on itself +.IP \(bu 2 +use up\-to\-date kernel (recent releases or maintained long term support versions) +.UNINDENT +.SS SD flash cards +.sp +There are a lot of devices with low power consumption and thus using storage +media based on low power consumption too, typically flash memory stored on +a chip enclosed in a detachable card package. An improperly inserted card may be +damaged by electrical spikes when the device is turned on or off. The chips +storing data in turn may be damaged permanently. All types of flash memory +have a limited number of rewrites, so the data are internally translated by FTL +(flash translation layer). This is implemented in firmware (technically a +software) and prone to bugs that manifest as hardware errors. +.sp +Adding redundancy like using DUP profiles for both data and metadata can help +in some cases but a full backup might be the best option once problems appear +and replacing the card could be required as well. +.SS Hardware as the main source of filesystem corruptions +.sp +\fBIf you use unreliable hardware and don\(aqt know about that, don\(aqt blame the +filesystem when it tells you.\fP +.SH SEE ALSO +.sp +\fBacl(5)\fP, +\fI\%btrfs(8)\fP, +\fBchattr(1)\fP, +\fBfstrim(8)\fP, +\fBioctl(2)\fP, +\fI\%mkfs.btrfs(8)\fP, +\fBmount(8)\fP, +\fBswapon(8)\fP +.\" Generated by docutils manpage writer. +. |