diff options
Diffstat (limited to 'USAGE-MKSQUASHFS-4.6')
| -rw-r--r-- | USAGE-MKSQUASHFS-4.6 | 1069 |
1 files changed, 1069 insertions, 0 deletions
diff --git a/USAGE-MKSQUASHFS-4.6 b/USAGE-MKSQUASHFS-4.6 new file mode 100644 index 0000000..37dab44 --- /dev/null +++ b/USAGE-MKSQUASHFS-4.6 @@ -0,0 +1,1069 @@ + MKSQUASHFS - a tool to create Squashfs filesystems + +As Squashfs is a read-only filesystem, the Mksquashfs program must be used to +create populated squashfs filesystems. + +SYNTAX:mksquashfs source1 source2 ... FILESYSTEM [OPTIONS] [-e list of +exclude dirs/files] + +Filesystem compression options: +-b <block_size> set data block to <block_size>. Default 128 Kbytes. + Optionally a suffix of K or M can be given to specify + Kbytes or Mbytes respectively +-comp <comp> select <comp> compression + Compressors available: + gzip (default) + lzo + lz4 + xz + zstd +-noI do not compress inode table +-noId do not compress the uid/gid table (implied by -noI) +-noD do not compress data blocks +-noF do not compress fragment blocks +-noX do not compress extended attributes +-no-compression do not compress any of the data or metadata. This is + equivalent to specifying -noI -noD -noF and -noX + +Filesystem build options: +-tar read uncompressed tar file from standard in (stdin) +-no-strip act like tar, and do not strip leading directories + from source files +-tarstyle alternative name for -no-strip +-cpiostyle act like cpio, and read file pathnames from standard in + (stdin) +-cpiostyle0 like -cpiostyle, but filenames are null terminated. Can + be used with find -print0 action +-reproducible build filesystems that are reproducible (default) +-not-reproducible build filesystems that are not reproducible +-mkfs-time <time> set filesystem creation timestamp to <time>. <time> can + be an unsigned 32-bit int indicating seconds since the + epoch (1970-01-01) or a string value which is passed to + the "date" command to parse. Any string value which the + date command recognises can be used such as "now", + "last week", or "Wed Feb 15 21:02:39 GMT 2023" +-all-time <time> set all file timestamps to <time>. <time> can be an + unsigned 32-bit int indicating seconds since the epoch + (1970-01-01) or a string value which is passed to the + "date" command to parse. Any string value which the date + command recognises can be used such as "now", "last + week", or "Wed Feb 15 21:02:39 GMT 2023" +-root-time <time> set root directory time to <time>. <time> can be an + unsigned 32-bit int indicating seconds since the epoch + (1970-01-01) or a string value which is passed to the + "date" command to parse. Any string value which the date + command recognises can be used such as "now", "last + week", or "Wed Feb 15 21:02:39 GMT 2023" +-root-mode <mode> set root directory permissions to octal <mode> +-root-uid <value> set root directory owner to specified <value>, + <value> can be either an integer uid or user name +-root-gid <value> set root directory group to specified <value>, + <value> can be either an integer gid or group name +-all-root make all files owned by root +-force-uid <value> set all file uids to specified <value>, <value> can be + either an integer uid or user name +-force-gid <value> set all file gids to specified <value>, <value> can be + either an integer gid or group name +-pseudo-override make pseudo file uids and gids override -all-root, + -force-uid and -force-gid options +-no-exports do not make filesystem exportable via NFS (-tar default) +-exports make filesystem exportable via NFS (default) +-no-sparse do not detect sparse files +-no-tailends do not pack tail ends into fragments (default) +-tailends pack tail ends into fragments +-no-fragments do not use fragments +-no-duplicates do not perform duplicate checking +-no-hardlinks do not hardlink files, instead store duplicates +-keep-as-directory if one source directory is specified, create a root + directory containing that directory, rather than the + contents of the directory + +Filesystem filter options: +-p <pseudo-definition> add pseudo file definition. The definition should + be quoted +-pf <pseudo-file> add list of pseudo file definitions from <pseudo-file>, + use - for stdin. Pseudo file definitions should not be + quoted +-sort <sort_file> sort files according to priorities in <sort_file>. One + file or dir with priority per line. Priority -32768 to + 32767, default priority 0 +-ef <exclude_file> list of exclude dirs/files. One per line +-wildcards allow extended shell wildcards (globbing) to be used in + exclude dirs/files +-regex allow POSIX regular expressions to be used in exclude + dirs/files +-max-depth <levels> descend at most <levels> of directories when scanning + filesystem +-one-file-system do not cross filesystem boundaries. If a directory + crosses the boundary, create an empty directory for + each mount point. If a file crosses the boundary + ignore it +-one-file-system-x do not cross filesystem boundaries. Like + -one-file-system option except directories are also + ignored if they cross the boundary + +Filesystem extended attribute (xattrs) options: +-no-xattrs do not store extended attributes +-xattrs store extended attributes (default) +-xattrs-exclude <regex> exclude any xattr names matching <regex>. <regex> is a + POSIX regular expression, e.g. -xattrs-exclude '^user.' + excludes xattrs from the user namespace +-xattrs-include <regex> include any xattr names matching <regex>. <regex> is a + POSIX regular expression, e.g. -xattrs-include '^user.' + includes xattrs from the user namespace +-xattrs-add <name=val> add the xattr <name> with <val> to files. If an + user xattr it will be added to regular files and + directories (see man 7 xattr). Otherwise it will be + added to all files. <val> by default will be treated as + binary (i.e. an uninterpreted byte sequence), but it can + be prefixed with 0s, where it will be treated as base64 + encoded, or prefixed with 0x, where val will be treated + as hexidecimal. Additionally it can be prefixed with + 0t where this encoding is similar to binary encoding, + except backslashes are specially treated, and a + backslash followed by 3 octal digits can be used to + encode any ASCII character, which obviously can be used + to encode control codes. The option can be repeated + multiple times to add multiple xattrs + +Mksquashfs runtime options: +-version print version, licence and copyright message +-exit-on-error treat normally ignored errors as fatal +-quiet no verbose output +-info print files written to filesystem +-no-progress do not display the progress bar +-progress display progress bar when using the -info option +-percentage display a percentage rather than the full progress bar. + Can be used with dialog --gauge etc. +-throttle <percentage> throttle the I/O input rate by the given percentage. + This can be used to reduce the I/O and CPU consumption + of Mksquashfs +-limit <percentage> limit the I/O input rate to the given percentage. + This can be used to reduce the I/O and CPU consumption + of Mksquashfs (alternative to -throttle) +-processors <number> use <number> processors. By default will use number of + processors available +-mem <size> use <size> physical memory for caches. Use K, M or G to + specify Kbytes, Mbytes or Gbytes respectively +-mem-percent <percent> use <percent> physical memory for caches. Default 25% +-mem-default print default memory usage in Mbytes + +Filesystem append options: +-noappend do not append to existing filesystem +-root-becomes <name> when appending source files/directories, make the + original root become a subdirectory in the new root + called <name>, rather than adding the new source items + to the original root +-no-recovery do not generate a recovery file +-recovery-path <name> use <name> as the directory to store the recovery file +-recover <name> recover filesystem data using recovery file <name> + +Filesystem actions options: +-action <action@expr> evaluate <expr> on every file, and execute <action> + if it returns TRUE +-log-action <act@expr> as above, but log expression evaluation results and + actions performed +-true-action <act@expr> as above, but only log expressions which return TRUE +-false-action <act@exp> as above, but only log expressions which return FALSE +-action-file <file> as action, but read actions from <file> +-log-action-file <file> as -log-action, but read actions from <file> +-true-action-file <f> as -true-action, but read actions from <f> +-false-action-file <f> as -false-action, but read actions from <f> + +Tar file only options: +-default-mode <mode> tar files often do not store permissions for + intermediate directories. This option sets the default + directory permissions to octal <mode>, rather than 0755. + This also sets the root inode mode +-default-uid <uid> tar files often do not store uids for intermediate + directories. This option sets the default directory + owner to <uid>, rather than the user running Mksquashfs. + This also sets the root inode uid +-default-gid <gid> tar files often do not store gids for intermediate + directories. This option sets the default directory + group to <gid>, rather than the group of the user + running Mksquashfs. This also sets the root inode gid +-ignore-zeros allow tar files to be concatenated together and fed to + Mksquashfs. Normally a tarfile has two consecutive 512 + byte blocks filled with zeros which means EOF and + Mksquashfs will stop reading after the first tar file on + encountering them. This option makes Mksquashfs ignore + the zero filled blocks + +Expert options (these may make the filesystem unmountable): +-nopad do not pad filesystem to a multiple of 4K +-offset <offset> skip <offset> bytes at the beginning of FILESYSTEM. + Optionally a suffix of K, M or G can be given to specify + Kbytes, Mbytes or Gbytes respectively. + Default 0 bytes +-o <offset> synonym for -offset + +Miscellaneous options: +-fstime <time> alternative name for -mkfs-time +-always-use-fragments alternative name for -tailends +-root-owned alternative name for -all-root +-noInodeCompression alternative name for -noI +-noIdTableCompression alternative name for -noId +-noDataCompression alternative name for -noD +-noFragmentCompression alternative name for -noF +-noXattrCompression alternative name for -noX + +-help output this options text to stdout +-h output this options text to stdout + +-Xhelp print compressor options for selected compressor + +Pseudo file definition format: +"filename d mode uid gid" create a directory +"filename m mode uid gid" modify filename +"filename b mode uid gid major minor" create a block device +"filename c mode uid gid major minor" create a character device +"filename f mode uid gid command" create file from stdout of command +"filename s mode uid gid symlink" create a symbolic link +"filename i mode uid gid [s|f]" create a socket (s) or FIFO (f) +"filename x name=val" create an extended attribute +"filename l linkname" create a hard-link to linkname +"filename L pseudo_filename" same, but link to pseudo file +"filename D time mode uid gid" create a directory with timestamp time +"filename M time mode uid gid" modify a file with timestamp time +"filename B time mode uid gid major minor" + create block device with timestamp time +"filename C time mode uid gid major minor" + create char device with timestamp time +"filename F time mode uid gid command" create file with timestamp time +"filename S time mode uid gid symlink" create symlink with timestamp time +"filename I time mode uid gid [s|f]" create socket/fifo with timestamp time + + +Source1 source2 ... are the source directories/files containing the +files/directories that will form the squashfs filesystem. If a single +directory is specified (i.e. mksquashfs source image.sqfs) the squashfs +filesystem will consist of that directory, with the top-level root +directory containing the contents. + +If multiple source directories or files are specified, Mksquashfs will merge +the specified sources into a single filesystem, with the root directory +containing each of the source files/directories. The name of each directory +entry will be the basename of the source path. If more than one source +entry maps to the same name, the conflicts are named xxx_1, xxx_2, etc. where +xxx is the original name. + +To make this clear, take two example directories. Source directory +"/home/phillip/test" contains "file1", "file2" and "dir1". +Source directory "goodies" contains "goodies1", "goodies2" and "goodies3". + +usage example 1: + +% mksquashfs /home/phillip/test image.sqfs + +This will generate a squashfs filesystem with root entries +"file1", "file2" and "dir1". + +example 2: + +% mksquashfs /home/phillip/test goodies image.sqfs + +This will create a squashfs filesystem with the root containing +entries "test" and "goodies" corresponding to the source +directories "/home/phillip/test" and "goodies". + +example 3: + +% mksquashfs /home/phillip/test goodies test image.sqfs + +This is the same as the previous example, except a third +source directory "test" has been specified. This conflicts +with the first directory named "test" and will be renamed "test_1". + +Multiple sources allow filesystems to be generated without needing to +copy all source files into a common directory. This simplifies creating +filesystems. + +The -keep-as-directory option can be used when only one source directory +is specified, and you wish the root to contain that directory, rather than +the contents of the directory. For example: + +example 4: + +% mksquashfs /home/phillip/test image.sqfs -keep-as-directory + +This is the same as example 1, except for -keep-as-directory. +This will generate a root directory containing directory "test", +rather than the "test" directory contents "file1", "file2" and "dir1". + +If you want the full path to retained (like TAR behaviour), you can specify the +-no-strip option. + +example 5: + +% mksquashfs /home/phillip/test image.sqfs -no-strip + +This will make a filesystem with "home", "home/phillip" and "home/phillip/test" +directories, see section 4 for more details. + +The Dest argument is the destination where the squashfs filesystem will be +written. This can either be a conventional file or a block device. If the file +doesn't exist it will be created, if it does exist and a squashfs +filesystem exists on it, Mksquashfs will append. The -noappend option will +write a new filesystem irrespective of whether an existing filesystem is +present. See section 12 for more details about appending. + +1. CHANGING COMPRESSION ALGORITHM AND COMPRESSION SPECIFIC OPTIONS +------------------------------------------------------------------ + +By default Mksquashfs will compress using the GZIP compression algorithm. This +algorithm offers a good trade-off between compression ratio, and memory and time +taken to decompress. + +Squashfs also supports LZ4, LZO, XZ and ZSTD compression. LZO offers worse +compression ratio than GZIP, but is faster to decompress. XZ offers better +compression ratio than GZIP, but at the expense of greater memory and time +to decompress (and significantly more time to compress). LZ4 is similar +to LZO. ZSTD has been developed by Facebook, and aims to compress well and +be fast to decompress. You should experiment with the compressors to +see which is best for you. + +If you're not building the squashfs-tools and kernel from source, then the tools +and kernel may or may not have been built with support for LZ4, LZO, XZ or ZSTD +compression. The compression algorithms supported by the build of Mksquashfs +can be found by typing mksquashfs -help. + +The full list of compressors available and their compression specific options +are: + +Compressors available and compressor specific options: + gzip (default) + -Xcompression-level <compression-level> + <compression-level> should be 1 .. 9 (default 9) + -Xwindow-size <window-size> + <window-size> should be 8 .. 15 (default 15) + -Xstrategy strategy1,strategy2,...,strategyN + Compress using strategy1,strategy2,...,strategyN in turn + and choose the best compression. + Available strategies: default, filtered, huffman_only, + run_length_encoded and fixed + lzo + -Xalgorithm <algorithm> + Where <algorithm> is one of: + lzo1x_1 + lzo1x_1_11 + lzo1x_1_12 + lzo1x_1_15 + lzo1x_999 (default) + -Xcompression-level <compression-level> + <compression-level> should be 1 .. 9 (default 8) + Only applies to lzo1x_999 algorithm + lz4 + -Xhc + Compress using LZ4 High Compression + xz + -Xbcj filter1,filter2,...,filterN + Compress using filter1,filter2,...,filterN in turn + (in addition to no filter), and choose the best compression. + Available filters: x86, arm, armthumb, powerpc, sparc, ia64 + -Xdict-size <dict-size> + Use <dict-size> as the XZ dictionary size. The dictionary size + can be specified as a percentage of the block size, or as an + absolute value. The dictionary size must be less than or equal + to the block size and 8192 bytes or larger. It must also be + storable in the xz header as either 2^n or as 2^n+2^(n+1). + Example dict-sizes are 75%, 50%, 37.5%, 25%, or 32K, 16K, 8K + etc. + zstd + -Xcompression-level <compression-level> + <compression-level> should be 1 .. 22 (default 15) + +If the compressor offers compression specific options (all the compressors now +have compression specific options except the deprecated lzma1 compressor) +then these options are also displayed (.i.e. in the above XZ is shown with two +compression specific options). The compression specific options are, obviously, +specific to the compressor in question, and the compressor documentation and +web sites should be consulted to understand their behaviour. In general +the Mksquashfs compression defaults for each compressor are optimised to +give the best performance for each compressor, where what constitutes +best depends on the compressor. For GZIP/XZ best means highest compression, +for LZO/LZ4 best means a tradeoff between compression and (de)-compression +overhead (LZO/LZ4 by definition are intended for weaker processors). + + +2. REDUCING CPU AND I/O USAGE +----------------------------- + +By default Mksquashfs will use all the CPUs available to compress and create the +filesystem, and will read from the source files and write to the output +filesystem as fast as possible. This maximises both CPU usage and I/O. + +Sometimes you don't want Mksquashfs to use all CPU and I/O bandwidth. For those +cases Mksquashfs supports two complementary options, -processors and -throttle. + +The -processors option can be used to reduce the number of parallel compression +threads used by Mksquashfs. Reducing this to 1 will create the minimum number +of threads, and this will reduce CPU usage, and that in turn will reduce I/O +(because CPUs are normally the bottleneck). + +The -throttle option reduces the speed Mksquashfs reads from the source files. +The value is a percentage (obviously from 1 - 100), and 50 will reduce the +read rate by half (the read thread will spend half its time idling), and 75 +by three quarters. Reducing the speed of I/O will also reduce the CPU +usage as there is insufficient data rate to use all cores. + +Which option should you use? Both will effectively reduce CPU and I/O in +normal cases where intensive use is being made of both I/O and CPUs. But +in edge cases there can be an imbalance where reducing one has no effect, or +it can't be reduced any further. For instance when there is only 1 or 2 cores +available, setting -processors to the minimum of 1 may still use too much +CPU. Additionally if your input source is slow Mksquashfs may still max it out +with -processors set to the minimum of 1. In this case you can use -throttle +in addition to -processors or on its own. + + +3. CHANGING GLOBAL COMPRESSION DEFAULTS USED IN MKSQUASHFS +---------------------------------------------------------- + +There are a large number of options that can be used to control the +compression in Mksquashfs. By and large the defaults are the most +optimum settings and should rarely need to be changed. + +Note, this does not apply to the block size, increasing the block size from the +default of 128 Kbytes will increase compression (especially for the XZ and ZSTD +compressors) and should increase I/O performance too. However, a block size of +greater than 128 Kbytes may increase latency in certain cases (where the +filesystem contains lots of fragments, and no locality of reference is +observed). For this reason the block size default is configured to the less +optimal 128 Kbytes. Users should experiment with 256 Kbyte sizes or above. + +The -b option allows the block size to be selected, both "K" and "M" postfixes +are supported, this can be either 4K, 8K, 16K, 32K, 64K, 128K, 256K, 512K or +1M bytes. + +The -noI, -noD, -noF and -noX options can be used to force Mksquashfs to not +compress inodes/directories, data, fragments and extended attributes +respectively. Giving all options generates an uncompressed filesystem. + +The -no-fragments option tells Mksquashfs to not generate fragment blocks. A +fragment block consists of multiple small files (all smaller than the block +size) packed and compressed together. This produces much better compression +than storing and compressing these files separately. It also typically +improves I/O as multiple files in the same directory are read at the same time. +You don't want to enable this option unless you fully understand the effects. + +The -tailends option tells Mksquashfs to always generate fragments for files +irrespective of the file length. By default only small files less than the data +block size are packed into fragment blocks. The tail ends of files which do not +fit fully into a block, are NOT by default packed into fragments. This is a +legacy setting when hard disks were mechanical, and had slow seek times. In +general setting this option will gain a little more compression, without +affecting I/O performance. + +The -no-duplicates option tells Mksquashfs to not check the files being added to +the filesystem for duplicates. This can result in quicker filesystem +generation although obviously compression will suffer badly if there is a lot +of duplicates. + + +4. TAR STYLE HANDLING OF SOURCE PATHNAMES IN MKSQUASHFS +------------------------------------------------------- + +Mksquashfs has always stripped the leading directories of any source pathnames +given on the command line. + +For example given the command line + +% mksquashfs dir-a/dir-b/dir-c/file1 dir-A/dir-B/file2 sqfs + +Mksquashfs will strip the leading directories, and put file1 and file2 into +the same root directory. If file1 and file2 are directories it will place the +directories into the same root directory, but, if only one directory is +specified, it will put the contents of that directory into the root directory. +Obviously, for a lot of the time this is what you want. But, if it isn't what +you want it can be quite difficult to get Mksquashfs to do something different. + +A lot of people don't like this, and would prefer Mksquashfs acted like "tar", +which does not strip leading directories. This allows you to create a directory +hierarchy from the pathnames of the supplied files. In the above example, the +tar archive would contain the pathnames "dir-a/dir-b/dir-c/file1" and +"dir-A/dir-B/file2". + +Mksquashfs will act like tar if given the option -no-strip, or -tarstyle. + + +5. CPIO STYLE HANDLING OF SOURCE PATHNAMES IN MKSQUASHFS +-------------------------------------------------------- + +Mksquashfs allows you to pipe the set of files to be added to the filesystem to +standard in (stdin), if the -cpiostyle option is given. + +As with cpio, directories are not recursively scanned and their contents added. +Evey file to be added to the filesystem must be explicitly specified. + +Typically the list of files to be added will be produced via find, or a similar +utility. + +For example + +% find /home/phillip/squashfs-tools | mksquashfs - img.sqfs -cpiostyle + +Will create an image containing everything in squashfs-tools and its +sub-directories. Note, "-" is given as the source pathname in Mksquashfs, and +indicates no commmand line sources. + +The following will add just the files ending in .c, .h and .o. + +% find /home/phillip/squashfs-tools -name "*.[cho]" | mksquashfs - img.sqfs -cpiostyle + + +6. SPECIFYING THE UIDs/GIDs USED IN THE FILESYSTEM +-------------------------------------------------- + +By default files in the generated filesystem inherit the UID and GID ownership +of the original file. However, Mksquashfs provides a number of options which +can be used to override the ownership. + +The -all-root option forces all file uids/gids in the generated Squashfs +filesystem to be root. This allows root owned filesystems to be built without +root access on the host machine. + +The "-force-uid uid" option forces all files in the generated Squashfs +filesystem to be owned by the specified uid. The uid can be specified either by +name (i.e. "root") or by number. + +The "-force-gid gid" option forces all files in the generated Squashfs +filesystem to be group owned by the specified gid. The gid can be specified +either by name (i.e. "root") or by number. + + +7. EXCLUDING FILES FROM THE FILESYSTEM +-------------------------------------- + +The -e and -ef options allow files/directories to be specified which are +excluded from the output filesystem. The -e option takes the exclude +files/directories from the command line, the -ef option takes the +exlude files/directories from the specified exclude file, one file/directory +per line. + +Two styles of exclude file matching are supported: basic exclude matching, and +extended wildcard matching. Basic exclude matching is a legacy feature +retained for backwards compatibility with earlier versions of Mksquashfs. +Extended wildcard matching should be used in preference. + +7.1 BASIC EXCLUDE MATCHING +-------------------------- + +Each exclude file is treated as an exact match of a file/directory in +the source directories. If an exclude file/directory is absolute (i.e. +prefixed with /, ../, or ./) the entry is treated as absolute, however, if an +exclude file/directory is relative, it is treated as being relative to each of +the sources in turn, i.e. + +% mksquashfs /tmp/source1 source2 output_fs -e ex1 /tmp/source1/ex2 out/ex3 + +Will generate exclude files /tmp/source1/ex2, /tmp/source1/ex1, source2/ex1, +/tmp/source1/out/ex3 and source2/out/ex3. + +7.2 EXTENDED EXCLUDE FILE HANDLING +---------------------------------- + +Extended exclude file matching treats each exclude file as a wildcard or +regex expression. To enable wildcard matching specify the -wildcards +option, and to enable regex matching specify the -regex option. In most +cases the -wildcards option should be used rather than -regex because wildcard +matching behaviour is significantly easier to understand! + +In addition to wildcards/regex expressions, exclude files can be "anchored" or +"non-anchored". An anchored exclude is one which matches from the root of the +directory and nowhere else, a non-anchored exclude matches anywhere. For +example given the directory hierarchy "a/b/c/a/b", the anchored exclude +"a/b" will match "a/b" at the root of the directory hierarchy, but +it will not match the "/a/b" sub-directory within directory "c", whereas a +non-anchored exclude would. + +A couple of examples should make this clearer. + +Anchored excludes + + 1. mksquashfs example image.sqsh -wildcards -e 'test/*.gz' + + Exclude all files matching "*.gz" in the top level directory "test". + + 2. mksquashfs example image.sqsh -wildcards -e '*/[Tt]est/example*' + + Exclude all files beginning with "example" inside directories called + "Test" or "test", that occur inside any top level directory. + + Using extended wildcards, negative matching is also possible. + + 3. mksquashfs example image.sqsh -wildcards -e 'test/!(*data*).gz' + + Exclude all files matching "*.gz" in top level directory "test", + except those with "data" in the name. + +Non-anchored excludes + + By default excludes match from the top level directory, but it is + often useful to exclude a file matching anywhere in the source directories. + For this non-anchored excludes can be used, specified by pre-fixing the + exclude with "...". + + Examples: + + 1. mksquashfs example image.sqsh -wildcards -e '... *.gz' + + Exclude files matching "*.gz" anywhere in the source directories. + For example this will match "example.gz", "test/example.gz", and + "test/test/example.gz". + + 2. mksquashfs example image.sqsh -wildcards -e '... [Tt]est/*.gz' + + Exclude files matching "*.gz" inside directories called "Test" or + "test" that occur anywhere in the source directories. + + Again, using extended wildcards, negative matching is also possible. + + 3. mksquashfs example image.sqsh -wildcards -e '... !(*data*).gz' + + Exclude all files matching "*.gz" anywhere in the source directories, + except those with "data" in the name. + + +8. FILTERING AND ADDING EXTENDED ATTRIBUTES (XATTRs) +---------------------------------------------------- + +Mksquashfs has a number of options which allow extended attributes (xattrs) to +be filtered from the source files or added to the created Squashfs filesystem. + +The -no-xattrs option removes any extended attributes which may exist in the +source files, and creates a filesystem without any extended attributes. + +The -xattrs-exclude option specifies a regular expression (regex), which +removes any extended attribute that matches the regular expression from all +files. For example the regex '^user.' will remove all User extended attributes. + +The -xattrs-include option instead specifies a regular expression (regex) +which includes any extended attribute that matches, and removes anything +that does't match. For example the regex '^user.' will only keep User +extended attributes and anything else will be removed. + +Mksquashfs also allows you to add extended attributes to files in the Squashfs +filesystem using the -xattrs-add option. This option takes an xattr name and +value pair separated by the '=' character. + +The extended attribute name can be any valid name and can be in the namespaces +security, system, trusted, or user. User extended attributes are added to files +and directories (see man 7 xattr for explanation), and the others are added to +all files. + +The extended attribute value by default will be treated as binary (i.e. an +uninterpreted byte sequence), but it can be prefixed with 0s, where it will be +treated as base64 encoded, or prefixed with 0x, where it will be treated as +hexidecimal. + +Obviously using base64 or hexidecimal allows values to be used that cannot be +entered on the command line such as non-printable characters etc. But it +renders the string non-human readable. To keep readability and to allow +non-printable characters to be entered, the 0t prefix is supported. This +encoding is similar to binary encoding, except backslashes are specially +treated, and a backslash followed by three octal digits can be used to encode +any ASCII character, which obviously can be used to encode non-printable values. +The following four command lines are equivalent + +mksquashfs dir image.sqfs -xattrs-add "user.comment=hello world" +mksquashfs dir image.sqfs -xattrs-add "user.comment=0saGVsbG8gd29ybGQ=" +mksquashfs dir image.sqfs -xattrs-add "user.comment=0x68656c6c6f20776f726c64" +mksquashfs dir image.sqfs -xattrs-add "user.comment=0thello world" + +Obviously in the above example there are no non-printable characters and so +the 0t prefixed string is identical to the first line. The following three +command lines are identical, but where the space has been replaced by the +non-printable NUL '\0' (null character). + +mksquashfs dir image.sqfs -xattrs-add "user.comment=0thello\000world" +mksquashfs dir image.sqfs -xattrs-add "user.comment=0saGVsbG8Ad29ybGQ=" +mksquashfs dir image.sqfs -xattrs-add "user.comment=0x68656c6c6f00776f726c64" + +9. PSEUDO FILE SUPPORT +---------------------- + +Mksquashfs supports pseudo files, these allow files, directories, character +devices, block devices, fifos, symbolic links, hard links and extended +attributes to be specified and added to the Squashfs filesystem being built, +rather than requiring them to be present in the source files. This, for +example, allows device nodes to be added to the filesystem without requiring +root access. + +Pseudo files also support "dynamic pseudo files" and a modify operation. +Dynamic pseudo files allow files to be dynamically created when Mksquashfs +is run, their contents being the result of running a command or piece of +shell script. The modify operation allows the mode/uid/gid of an existing +file in the source filesystem to be modified. + +Two Mksquashfs options are supported, -p allows one pseudo file to be specified +on the command line, and -pf allows a pseudo file to be specified containing a +list of pseduo definitions, one per line. + +9.1 CREATING A DYNAMIC FILE +--------------------------- + +Pseudo definition + +Filename f mode uid gid command + +mode is the octal mode specifier, similar to that expected by chmod. + +uid and gid can be either specified as a decimal number, or by name. + +command can be an executable or a piece of shell script, and it is executed +by running "/bin/sh -c command". The stdout becomes the contents of +"Filename". + +Examples: + +Running a basic command +----------------------- + +/somedir/dmesg f 444 root root dmesg + +creates a file "/somedir/dmesg" containing the output from dmesg. + +Executing shell script +---------------------- + +RELEASE f 444 root root \ + if [ ! -e /tmp/ver ]; then \ + echo 0 > /tmp/ver; \ + fi; \ + ver=`cat /tmp/ver`; \ + ver=$((ver +1)); \ + echo $ver > /tmp/ver; \ + echo -n `cat /tmp/release`; \ + echo "-dev #"$ver `date` "Build host" `hostname` + +Creates a file RELEASE containing the release name, date, build host, and +an incrementing version number. The incrementing version is a side-effect +of executing the shell script, and ensures every time Sqfstar is run a +new version number is used without requiring any other shell scripting. + +The above example also shows that commands can be split across multiple lines +using "\". Obviously as the script will be presented to the shell as a single +line, a semicolon is need to separate individual shell commands within the +shell script. + +Reading from a device (or fifo/named socket) +-------------------------------------------- + +input f 444 root root dd if=/dev/sda1 bs=1024 count=10 + +Copies 10K from the device /dev/sda1 into the file input. Ordinarily Sqfstar +given a device, fifo, or named socket will place that special file within the +Squashfs filesystem, the above allows input from these special files to be +captured and placed in the Squashfs filesystem. + +9.2 CREATING A BLOCK OR CHARACTER DEVICE +---------------------------------------- + +Pseudo definition + +Filename type mode uid gid major minor + +Where type is either + b - for block devices, and + c - for character devices + +mode is the octal mode specifier, similar to that expected by chmod. + +uid and gid can be either specified as a decimal number, or by name. + +For example: + +/dev/chr_dev c 666 root root 100 1 +/dev/blk_dev b 666 0 0 200 200 + +creates a character device "/dev/chr_dev" with major:minor 100:1 and +a block device "/dev/blk_dev" with major:minor 200:200, both with root +uid/gid and a mode of rw-rw-rw. + +9.3 CREATING A DIRECTORY +------------------------- + +Pseudo definition + +Filename d mode uid gid + +mode is the octal mode specifier, similar to that expected by chmod. + +uid and gid can be either specified as a decimal number, or by name. + +For example: + +/pseudo_dir d 666 root root + +creates a directory "/pseudo_dir" with root uid/gid and mode of rw-rw-rw. + +9.4 CREATING A SYMBOLIC LINK +----------------------------- + +Pseudo definition + +Filename s mode uid gid symlink + +uid and gid can be either specified as a decimal number, or by name. + +Note mode is ignored, as symlinks always have "rwxrwxrwx" permissions. + +For example: + +symlink s 0 root root example + +8reates a symlink "symlink" to file "example" with root uid/gid. + +9.5 CREATING HARD LINKS (FILE REFERENCES) +----------------------------------------- + +The "f" Pseudo definition allows a regular file to be created from the output of +a command (or shell). Often this is used to reference a file outside the source +directories by executing "cat", e.g. + +README f 0555 0 0 cat /home/phillip/latest-version/README + +Because this is a quite frequent use of the definition, an alternative faster +"File reference" or Hard Link Pseudo definition exists: + +README l /home/phillip/latest-version/README + +Will create a reference to "/home/phillip/latest-version/README", +and obviously the timestamp/mode and owership will be used. + +The definition also be used to create additional references to +files within the source directories. For instance if "phillip/latest/README" +was a file being added to the filesystem, then + +README l phillip/latest/README + +Will create a Hard Link (and increment the nlink count on the inode). + +In both cases, the path to the file being referenced is the system +filesystem path, and can be absolute (prefixed with /), or relative +to the current working directory. + +There is an additional 'L' Pseudo definition, which closes a loophole in +the above 'l' definition. The 'l' Pseudo definition cannot create references +or Hard Links to files created by Pseudo definitions, because by +definition they do not exist in the system filesystem. + +with 'L' the referenced file is expected to be a Pseudo file, and in this case +the path is taken to be from the root of the Squashfs filesystem being created, +e.g. + +char-dev c 0555 0 0 1 2 + +link L char-dev + +Will create a Hard Link named "link" to the character device called "char-dev" +created by the previous Pseudo definition. + +9.6 CREATING SOCKETS/FIFOS +--------------------------- + +Pseudo definition + +filename i mode uid gid [s|f] + +To create a Unix domain socket, 's' should be used, i.e. + +filename i 0777 root root s + +and to create a FIFO, 'f' should be used, i.e. + +filename i 0777 root root f + +9.7 ADDING EXTENDED ATTRIBUTES TO FILES +--------------------------------------- + +Pseudo definition + +filename x name=val + +Will add the extended attribute <name> to <filename> with <val> contents. See +Section 7 for a description of the <val> formats supported. + +9.8 MODIFYING ATTRIBUTES OF AN EXISTING FILE +-------------------------------------------- + +Pseudo definition + +Filename m mode uid gid + +mode is the octal mode specifier, similar to that expected by chmod. + +uid and gid can be either specified as a decimal number, or by name. + +For example: + +dmesg m 666 root root + +Changes the attributes of the file "dmesg" in the filesystem to have +root uid/gid and a mode of rw-rw-rw, overriding the attributes obtained +from the source filesystem. + +10. EXTENDED PSEUDO FILE DEFINITIONS WITH TIMESTAMPS +---------------------------------------------------- + +The Pseudo file definitions described above do not allow the timestamp +of the created file to be specified, and the files will be timestamped +with the current time. + +Extended versions of the Pseudo file definitions are supported which +take a <time> timestamp. These are distinquished from the previous +definitions by using an upper-case type character. For example the "D" +definition is identical to the "d" definition, but it takes a <time> +timestamp. + +The list of extended definitions are: + + filename F time mode uid gid command + filename D time mode uid gid + filename B time mode uid gid major minor + filename C time mode uid gid major minor + filename S time mode uid gid symlink + filename I time mode uid gid [s|f] + filename M time mode uid gid + +<time> can be either an unsigned decimal integer (which represents the +seconds since the epoch of 1970-01-01 00:00 UTC), or a "date string" +which is parsed and converted into an integer since the epoch, by calling +the "date" command. + +Because most date strings have spaces, they will need to be quoted, and if +entered on the command line, these quotes will need to be protected from the +shell by backslashes, i.e. + +% mksquashfs dir image.sqsh -p "file D \"1 jan 1980\" 0777 phillip phillip" + +Obviously anything "date" accepts as a valid string can be used, such as +"yesterday", "last week" etc. + + +11. APPENDING TO SQUASHFS FILESYSTEMS +------------------------------------- + +Running Mksquashfs with the destination directory containing an existing +filesystem will add the source items to the existing filesystem. By default, +the source items are added to the existing root directory. + +To make this clear... An existing filesystem "image" contains root entries +"old1", and "old2". Source directory "/home/phillip/test" contains "file1", +"file2" and "dir1". + +example 1: + +% mksquashfs /home/phillip/test image + +Will create a new "image" with root entries "old1", "old2", "file1", "file2" and +"dir1" + +example 2: + +% mksquashfs /home/phillip/test image -keep-as-directory + +Will create a new "image" with root entries "old1", "old2", and "test". +As shown in the previous section, for single source directories +'-keep-as-directory' adds the source directory rather than the +contents of the directory. + +example 3: + +% mksquashfs /home/phillip/test image -keep-as-directory -root-becomes +original-root + +Will create a new "image" with root entries "original-root", and "test". The +'-root-becomes' option specifies that the original root becomes a subdirectory +in the new root, with the specified name. + +The append option with file duplicate detection, means squashfs can be +used as a simple versioning archiving filesystem. A squashfs filesystem can +be created with for example the linux-2.4.19 source. Appending the linux-2.4.20 +source will create a filesystem with the two source trees, but only the +changed files will take extra room, the unchanged files will be detected as +duplicates. + +12. APPENDING RECOVERY FILE FEATURE +----------------------------------- + +Recovery files are created when appending to existing Squashfs +filesystems. This allows the original filesystem to be recovered +if Mksquashfs aborts unexpectedly (i.e. power failure). + +The recovery files are called squashfs_recovery_xxx_yyy, where +"xxx" is the name of the filesystem being appended to, and "yyy" is a +number to guarantee filename uniqueness (the PID of the parent Mksquashfs +process). + +Normally if Mksquashfs exits correctly the recovery file is deleted to +avoid cluttering the filesystem. If Mksquashfs aborts, the "-recover" +option can be used to recover the filesystem, giving the previously +created recovery file as a parameter, i.e. + +mksquashfs dummy image.sqsh -recover squashfs_recovery_image.sqsh_1234 + +The writing of the recovery file can be disabled by specifying the +"-no-recovery" option. + + +13. MKSQUASHFS ACTIONS INTRODUCTION +----------------------------------- + +The Mksquashfs Actions code allows an "action" to be executed on a file if one +or more "tests" succeed. If you're familiar with the "find" command, then an +action is similar to "-print", and a test is similar to say "-name" or "-type". + +To illustrate this it is useful to give two concrete examples. + +example 1: the fragment action + +% mksquashfs /home/phillip/github github.sqsh -action "fragment(cfiles) @ name(*.[ch])" -action "fragment(ofiles) @ name(*.o)" + +This example defines two "fragment actions" which control the packing of files +within fragments. Specifically, it creates a specialised fragment called +"cfiles" which packs files matching the wildcard name "*.[ch]". + +It also creates another specialised fragment called "ofiles" which packs files +matching the wilcard name "*.o". + +Producing specialised fragments which only pack files which match a range of +tests, can produce better compression and/or I/O performance as it can optimise +similarity or access patterns. But it can also produce worse compression, and +so you should always test the effect. + +Additionally, you should be able to see that an action definition is split into +an action function before the "@", and one or more test functions after the @. +Quoting is necessary here to protect it from interpretation by the shell. Also +the spacing before and after the "@" isn't necessary and is used here for added +readability. + +example 2: the uncompressed action + +% mksquashfs /home/phillip backup.sqsh -action "uncompressed @ ( name(*.jpg) || name(*.mpg) ) || ( name(*.img) && filesize(+1G) )" + +This is a more complex example. It tells Mksquashfs to not try and compress any +file which matches the wildcard names "*.jpg" and "*.mpg". But it also tells +Mksquashfs not to try and compress files which match the wildcard name "*.img" +and are also 1 Gigabyte in size or larger. + +This example introduces the fact that tests can be combined using the logical +operators && (and), || (or) and ! (not), and can be bracketed. + +Please see the ACTIONS-README file for syntax and extra information. + + +14. MISCELLANEOUS OPTIONS +------------------------ + +The -info option displays the files/directories as they are compressed and +added to the filesystem. The original uncompressed size of each file +is printed, along with DUPLICATE if the file is a duplicate of a +file in the filesystem. + +The -nopad option informs Mksquashfs to not pad the filesystem to a 4K multiple. +This is performed by default to enable the output filesystem file to be mounted +by loopback, which requires files to be a 4K multiple. If the filesystem is +being written to a block device, or is to be stored in a bootimage, the extra +pad bytes are not needed. |