diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/lzip.1 | 87 | ||||
| -rw-r--r-- | doc/lzip.info | 572 | ||||
| -rw-r--r-- | doc/lzip.texinfo | 598 |
3 files changed, 1257 insertions, 0 deletions
diff --git a/doc/lzip.1 b/doc/lzip.1 new file mode 100644 index 0000000..a1a7591 --- /dev/null +++ b/doc/lzip.1 @@ -0,0 +1,87 @@ +.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.36. +.TH LZIP "1" "April 2009" "Lzip 1.6-pre1" "User Commands" +.SH NAME +Lzip \- manual page for Lzip 1.6-pre1 +.SH SYNOPSIS +.B lzip +[\fIoptions\fR] [\fIfiles\fR] +.SH DESCRIPTION +Lzip \- A data compressor based on the LZMA algorithm. +.SH OPTIONS +.TP +\fB\-h\fR, \fB\-\-help\fR +display this help and exit +.TP +\fB\-V\fR, \fB\-\-version\fR +output version information and exit +.TP +\fB\-b\fR, \fB\-\-member\-size=\fR<n> +set member size limit in bytes +.TP +\fB\-c\fR, \fB\-\-stdout\fR +send output to standard output +.TP +\fB\-d\fR, \fB\-\-decompress\fR +decompress +.TP +\fB\-f\fR, \fB\-\-force\fR +overwrite existing output files +.TP +\fB\-k\fR, \fB\-\-keep\fR +keep (don't delete) input files +.TP +\fB\-m\fR, \fB\-\-match\-length=\fR<n> +set match length limit in bytes [80] +.TP +\fB\-o\fR, \fB\-\-output=\fR<file> +if reading stdin, place the output into <file> +.TP +\fB\-q\fR, \fB\-\-quiet\fR +suppress all messages +.TP +\fB\-s\fR, \fB\-\-dictionary\-size=\fR<n> +set dictionary size limit in bytes [8MiB] +.TP +\fB\-S\fR, \fB\-\-volume\-size=\fR<n> +set volume size limit in bytes +.TP +\fB\-t\fR, \fB\-\-test\fR +test compressed file integrity +.TP +\fB\-v\fR, \fB\-\-verbose\fR +be verbose (a 2nd \fB\-v\fR gives more) +.TP +\fB\-1\fR .. \fB\-9\fR +set compression level [default 6] +.TP +\fB\-\-fast\fR +alias for \fB\-1\fR +.TP +\fB\-\-best\fR +alias for \fB\-9\fR +.PP +If no file names are given, lzip compresses or decompresses +from standard input to standard output. +Numbers may be followed by a multiplier: k = kB = 10^3 = 1000, +Ki = KiB = 2^10 = 1024, M = 10^6, Mi = 2^20, G = 10^9, Gi = 2^30, etc... +.SH "REPORTING BUGS" +Report bugs to lzip\-bug@nongnu.org +Lzip home page: http://www.nongnu.org/lzip/lzip.html +.SH COPYRIGHT +Copyright \(co 2009 Antonio Diaz Diaz. +License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> +.br +This is free software: you are free to change and redistribute it. +There is NO WARRANTY, to the extent permitted by law. +.SH "SEE ALSO" +The full documentation for +.B Lzip +is maintained as a Texinfo manual. If the +.B info +and +.B Lzip +programs are properly installed at your site, the command +.IP +.B info Lzip +.PP +should give you access to the complete manual. diff --git a/doc/lzip.info b/doc/lzip.info new file mode 100644 index 0000000..d40f84f --- /dev/null +++ b/doc/lzip.info @@ -0,0 +1,572 @@ +This is lzip.info, produced by makeinfo version 4.13 from lzip.texinfo. + +INFO-DIR-SECTION Data Compression +START-INFO-DIR-ENTRY +* Lzip: (lzip). Data compressor based on the LZMA algorithm +END-INFO-DIR-ENTRY + + +File: lzip.info, Node: Top, Next: Introduction, Up: (dir) + +Lzip +**** + +This manual is for Lzip (version 1.6-pre1, 27 April 2009). + +* Menu: + +* Introduction:: Purpose and features of lzip +* Algorithm:: How lzip compresses the data +* Invoking Lzip:: Command line interface +* File Format:: Detailed format of the compressed file +* Examples:: A small tutorial with examples +* Lzdiff:: Comparing compressed files +* Lzgrep:: Searching inside compressed files +* Lziprecover:: Recovering data from damaged compressed files +* Problems:: Reporting bugs +* Concept Index:: Index of concepts + + + Copyright (C) 2008, 2009 Antonio Diaz Diaz. + + This manual is free documentation: you have unlimited permission to +copy, distribute and modify it. + + +File: lzip.info, Node: Introduction, Next: Algorithm, Prev: Top, Up: Top + +1 Introduction +************** + +Lzip is a lossless data compressor based on the LZMA algorithm, with +very safe integrity checking and a user interface similar to the one of +gzip or bzip2. Lzip decompresses almost as fast as gzip and compresses +better than bzip2, which makes it well suited for software distribution +and data archiving. + + Lzip replaces every file given in the command line with a compressed +version of itself, with the name "original_name.lz". Each compressed +file has the same modification date, permissions, and, when possible, +ownership as the corresponding original, so that these properties can be +correctly restored at decompression time. Lzip is able to read from some +types of non regular files if the `--stdout' option is specified. + + If no file names are specified, lzip compresses (or decompresses) +from standard input to standard output. In this case, lzip will decline +to write compressed output to a terminal, as this would be entirely +incomprehensible and therefore pointless. + + Lzip will correctly decompress a file which is the concatenation of +two or more compressed files. The result is the concatenation of the +corresponding uncompressed files. Integrity testing of concatenated +compressed files is also supported. + + Lzip can produce multimember files and safely recover, with +lziprecover, the undamaged members in case of file damage. Lzip can +also split the compressed output in volumes of a given size, even when +reading from standard input. This allows the direct creation of +multivolume compressed tar archives. + + The amount of memory required for compression is about 2 times the +dictionary size limit plus 8 times the dictionary size really used. For +decompression is a little more than the dictionary size really used. +Lzip will automatically use the smallest possible dictionary size for +each member without exceeding the given limit. It is important to +appreciate that the decompression memory requirement is affected at +compression time by the choice of dictionary size limit. + + When decompressing, lzip attempts to guess the name for the +decompressed file from that of the compressed file as follows: + +filename.lz becomes filename +filename.tlz becomes filename.tar +anyothername becomes anyothername.out + + As a self-check for your protection, lzip stores in the member +trailer the 32-bit CRC of the original data and the size of the +original data, to make sure that the decompressed version of the data +is identical to the original. This guards against corruption of the +compressed data, and against undetected bugs in lzip (hopefully very +unlikely). The chances of data corruption going undetected are +microscopic, less than one chance in 4000 million for each member +processed. Be aware, though, that the check occurs upon decompression, +so it can only tell you that something is wrong. It can't help you +recover the original uncompressed data. + + Return values: 0 for a normal exit, 1 for environmental problems +(file not found, invalid flags, I/O errors, etc), 2 to indicate a +corrupt or invalid input file, 3 for an internal consistency error (eg, +bug) which caused lzip to panic. + + +File: lzip.info, Node: Algorithm, Next: Invoking Lzip, Prev: Introduction, Up: Top + +2 Algorithm +*********** + +Lzip implements a simplified version of the LZMA (Lempel-Ziv-Markov +chain-Algorithm) algorithm. The original LZMA algorithm was designed by +Igor Pavlov. + + The high compression of LZMA comes from combining two basic, +well-proven compression ideas: sliding dictionaries (LZ77/78) and +markov models (the thing used by every compression algorithm that uses +a range encoder or similar order-0 entropy coder as its last stage) +with segregation of contexts according to what the bits are used for. + + Lzip is a two stage compressor. The first stage is a Lempel-Ziv +coder, which reduces redundancy by translating chunks of data to their +corresponding distance-length pairs. The second stage is a range encoder +that uses a different probability model for each type of data; +distances, lengths, literal bytes, etc. + + The match finder, part of the LZ coder, is the most important piece +of the LZMA algorithm, as it is in many Lempel-Ziv based algorithms. +Most of lzip's execution time is spent in the match finder, and it has +the greatest influence on the compression ratio. + + Here is how it works, step by step: + + 1) The member header is written to the output stream. + + 2) The first byte is coded literally, because there are no previous +bytes to which the match finder can refer to. + + 3) The main encoder advances to the next byte in the input data and +calls the match finder. + + 4) The match finder fills an array with the minimum distances before +the current byte where a match of a given length can be found. + + 5) Go back to step 3 until a sequence (formed of pairs, repeated +distances and literal bytes) of minimum price has been formed. Where the +price represents the number of output bits produced. + + 6) The range encoder encodes the sequence produced by the main +encoder and sends the produced bytes to the output stream. + + 7) Go back to step 3 until the input data is finished or until the +member or volume size limits are reached. + + 8) The range encoder is flushed. + + 9) The member trailer is written to the output stream. + + 10) If there are more data to compress, go back to step 1. + + +File: lzip.info, Node: Invoking Lzip, Next: File Format, Prev: Algorithm, Up: Top + +3 Invoking Lzip +*************** + +The format for running lzip is: + + lzip [OPTIONS] [FILES] + + Lzip supports the following options: + +`--help' +`-h' + Print an informative help message describing the options and exit. + +`--version' +`-V' + Print the version number of lzip on the standard output and exit. + +`--member-size=SIZE' +`-b SIZE' + Produce a multimember file and set the member size limit to SIZE + bytes. Minimum member size limit is 100kB. Small member size may + degrade compression ratio, so use it only when needed. The default + is to produce single member files. + +`--stdout' +`-c' + Compress or decompress to standard output. Needed when reading + from a named pipe (fifo) or from a device. Use it to recover as + much of the uncompressed data as possible when decompressing a + corrupt file. + +`--decompress' +`-d' + Decompress. + +`--force' +`-f' + Force overwrite of output file. + +`--keep' +`-k' + Keep (don't delete) input files during compression or + decompression. + +`--match-length=LENGTH' +`-m LENGTH' + Set the match length limit in bytes. Valid values range from 5 to + 273. Larger values usually give better compression ratios but + longer compression times. + +`--output=FILE' +`-o FILE' + When reading from standard input and `--stdout' has not been + specified, use `FILE' as the virtual name of the uncompressed + file. This produces a file named `FILE' when decompressing, a file + named `FILE.lz' when compressing, and several files named + `FILE00001.lz', `FILE00002.lz', etc, when compressing and + splitting the output in volumes. + +`--quiet' +`-q' + Quiet operation. Suppress all messages. + +`--dictionary-size=SIZE' +`-s SIZE' + Set the dictionary size limit in bytes. Valid values range from + 4KiB to 512MiB. Lzip will use the smallest possible dictionary + size for each member without exceeding this limit. Note that + dictionary sizes are quantized. If the specified size does not + match one of the valid sizes, it will be rounded upwards. + +`--volume-size=SIZE' +`-S SIZE' + Split the compressed output into several volume files with names + `original_name00001.lz', `original_name00002.lz', etc, and set the + volume size limit to SIZE bytes. Each volume is a complete, maybe + multimember, lzip file. Minimum volume size limit is 100kB. Small + volume size may degrade compression ratio, so use it only when + needed. + +`--test' +`-t' + Check integrity of the specified file(s), but don't decompress + them. This really performs a trial decompression and throws away + the result. Use `-tvv' or `-tvvv' to see information about the + file. + +`--verbose' +`-v' + Verbose mode. Show the compression ratio for each file processed. + Further -v's increase the verbosity level. + +`-1 .. -9' + Set the compression parameters (dictionary size and match length + limit) as shown in the table below. Note that `-9' can be much + slower than `-1'. These options have no effect when decompressing. + + Level Dictionary size Match length limit + -1 4MiB 10 bytes + -2 4MiB 12 bytes + -3 4MiB 17 bytes + -4 4MiB 26 bytes + -5 4MiB 44 bytes + -6 8MiB 80 bytes + -7 16MiB 108 bytes + -8 16MiB 163 bytes + -9 32MiB 273 bytes + +`--fast' +`--best' + Aliases for GNU gzip compatibility. + + + + Numbers given as arguments to options may be followed by a multiplier +and an optional `B' for "byte". + + Table of SI and binary prefixes (unit multipliers): + +Prefix Value | Prefix Value +k kilobyte (10^3 = 1000) | Ki kibibyte (2^10 = 1024) +M megabyte (10^6) | Mi mebibyte (2^20) +G gigabyte (10^9) | Gi gibibyte (2^30) +T terabyte (10^12) | Ti tebibyte (2^40) +P petabyte (10^15) | Pi pebibyte (2^50) +E exabyte (10^18) | Ei exbibyte (2^60) +Z zettabyte (10^21) | Zi zebibyte (2^70) +Y yottabyte (10^24) | Yi yobibyte (2^80) + + +File: lzip.info, Node: File Format, Next: Examples, Prev: Invoking Lzip, Up: Top + +4 File Format +************* + +In the diagram below, a box like this: ++---+ +| | <-- the vertical bars might be missing ++---+ + + represents one byte; a box like this: ++==============+ +| | ++==============+ + + represents a variable number of bytes. + + + A lzip file consists of a series of "members" (compressed data sets). +The members simply appear one after another in the file, with no +additional information before, between, or after them. + + Each member has the following structure: ++--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| ID string | VN | DS | Lzma stream | CRC32 | Data size | Member size | ++--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + + All multibyte values are stored in little endian order. + +`ID string' + A four byte string, identifying the member type, with the value + "LZIP". + +`VN (version number, 1 byte)' + Just in case something needs to be modified in the future. Valid + values are 0 and 1. Version 0 files have only one member and lack + `Member size'. + +`DS (coded dictionary size, 1 byte)' + Bits 4-0 contain the base 2 logarithm of the base dictionary size. + Bits 7-5 contain the number of "wedges" to substract from the base + dictionary size to obtain the dictionary size. The size of a wedge + is (base dictionary size / 16). + Valid values for dictionary size range from 4KiB to 512MiB. + +`Lzma stream' + The lzma stream, finished by an end of stream marker. Uses default + values for encoder properties. + +`CRC32 (4 bytes)' + CRC of the uncompressed original data. + +`Data size (8 bytes)' + Size of the uncompressed original data. + +`Member size (8 bytes)' + Total size of the member, including header and trailer. This + facilitates safe recovery of undamaged members from multimember + files. + + + +File: lzip.info, Node: Examples, Next: Lzdiff, Prev: File Format, Up: Top + +5 A small tutorial with examples +******************************** + +WARNING! If your data is important, give the `--keep' option to lzip +and do not remove the original file until you verify the compressed +file with a command like `lzip -cd file.lz | cmp file -'. + + +Example 1: Replace a regular file with its compressed version file.lz +and show the compression ratio. + + lzip -v file + + +Example 2: Like example 1 but the created file.lz is multimember with a +member size of 1MiB. + + lzip -b 1MiB file + + +Example 3: Compress a whole floppy in /dev/fd0 and send the output to +file.lz. + + lzip -c /dev/fd0 > file.lz + + +Example 4: Create a multivolume compressed tar archive with a volume +size of 1440KiB. + + tar -c some_directory | lzip -S 1440KiB -o volume_name + + +Example 5: Extract a multivolume compressed tar archive. + + lzip -cd volume_name*.lz | tar -xf - + + +Example 6: Create a multivolume compressed backup of a big database file +with a volume size of 650MB, where each volume is a multimember file +with a member size of 32MiB. + + lzip -b 32MiB -S 650MB big_database + + +Example 7: Recover the first volume of those created in example 6 from +two copies, `big_database1_00001.lz' and `big_database2_00001.lz', with +member 00007 damaged in the first copy and member 00018 damaged in the +second copy. (Indented lines are lzip error messages). + + lziprecover big_database1_00001.lz + lziprecover big_database2_00001.lz + lzip -t rec*big_database1_00001.lz + rec00007big_database1_00001.lz: crc mismatch + lzip -t rec*big_database2_00001.lz + rec00018big_database1_00001.lz: crc mismatch + cp rec00007big_database2_00001.lz rec00007big_database1_00001.lz + cat rec*big_database1_00001.lz > big_database3_00001.lz + + +File: lzip.info, Node: Lzdiff, Next: Lzgrep, Prev: Examples, Up: Top + +6 Lzdiff +******** + +Lzdiff is a wrapper script around the diff and cmp commands that allows +transparent comparison of any combination of compressed and +non-compressed files. If any given file is compressed, its uncompressed +content is used. The supported compressors are gzip, bzip2 and lzip. + + The format for running lzdiff is: + + lzdiff [OPTIONS] [DIFF_OPTIONS] FILE1 [FILE2] + +Compares FILE1 to FILE2. If FILE2 is omitted, compares FILE1 to the +uncompressed contents of FILE1.[gz|bz2|lz] (depending on the default +compressor selected). DIFF_OPTIONS are passed directly to diff or cmp. +The exit status from diff or cmp is preserved. + + Lzdiff supports the following options: + +`--help' +`-h' + Print an informative help message describing the options and exit. + +`--version' +`-V' + Print the version number of lzdiff on the standard output and exit. + +`--gzip' + Use gzip as default decompressor. + +`--bzip2' + Use bzip2 as default decompressor. + +`--lzip' + Use lzip as default decompressor (default). + +`--diff' + Use diff to compare files (default). + +`--cmp' + Use cmp to compare files. + + + Lzdiff has the limitation that messages from the diff or cmp programs +refer to temporary filenames instead of those specified. + + +File: lzip.info, Node: Lzgrep, Next: Lziprecover, Prev: Lzdiff, Up: Top + +7 Lzgrep +******** + +Lzgrep is a wrapper script around the grep command that allows +transparent search on any combination of compressed and non-compressed +files. If any given file is compressed, its uncompressed content is +used. If a given file does not exist, lzgrep tries the compressed file +name corresponding to the default compressor selected. The supported +compressors are gzip, bzip2 and lzip. + + The format for running lzgrep is: + + lzgrep [OPTIONS] [GREP_OPTIONS] PATTERN [FILES] + +GREP_OPTIONS are passed directly to grep. The exit status from grep is +preserved. + + Lzgrep supports the following options: + +`--help' +`-h' + Print an informative help message describing the options and exit. + +`--version' +`-V' + Print the version number of lzgrep on the standard output and exit. + +`--gzip' + Use gzip as default decompressor. + +`--bzip2' + Use bzip2 as default decompressor. + +`--lzip' + Use lzip as default decompressor (default). + + + +File: lzip.info, Node: Lziprecover, Next: Problems, Prev: Lzgrep, Up: Top + +8 Lziprecover +************* + +Lziprecover is a program that searches for members in .lz files, and +writes each member in its own .lz file. You can then use `lzip -t' to +test the integrity of the resulting files, and decompress those which +are undamaged. + + Lziprecover takes a single argument, the name of the damaged file, +and writes a number of files `rec00001file.lz', `rec00002file.lz', etc, +containing the extracted members. The output filenames are designed so +that the use of wildcards in subsequent processing, for example, +`lzip -dc rec*file.lz > recovered_data', processes the files in the +correct order. + + +File: lzip.info, Node: Problems, Next: Concept Index, Prev: Lziprecover, Up: Top + +9 Reporting Bugs +**************** + +There are probably bugs in lzip. There are certainly errors and +omissions in this manual. If you report them, they will get fixed. If +you don't, no one will ever know about them and they will remain unfixed +for all eternity, if not longer. + + If you find a bug in lzip, please send electronic mail to +<lzip-bug@nongnu.org>. Include the version number, which you can find +by running `lzip --version'. + + +File: lzip.info, Node: Concept Index, Prev: Problems, Up: Top + +Concept Index +************* + + +* Menu: + +* algorithm: Algorithm. (line 6) +* bugs: Problems. (line 6) +* examples: Examples. (line 6) +* file format: File Format. (line 6) +* getting help: Problems. (line 6) +* introduction: Introduction. (line 6) +* invoking: Invoking Lzip. (line 6) +* lzdiff: Lzdiff. (line 6) +* lzgrep: Lzgrep. (line 6) +* lziprecover: Lziprecover. (line 6) +* options: Invoking Lzip. (line 6) +* usage: Invoking Lzip. (line 6) +* version: Invoking Lzip. (line 6) + + + +Tag Table: +Node: Top224 +Node: Introduction967 +Node: Algorithm4208 +Node: Invoking Lzip6434 +Node: File Format10781 +Node: Examples12735 +Node: Lzdiff14568 +Node: Lzgrep15887 +Node: Lziprecover16922 +Node: Problems17619 +Node: Concept Index18144 + +End Tag Table diff --git a/doc/lzip.texinfo b/doc/lzip.texinfo new file mode 100644 index 0000000..f29b29e --- /dev/null +++ b/doc/lzip.texinfo @@ -0,0 +1,598 @@ +\input texinfo @c -*-texinfo-*- +@c %**start of header +@setfilename lzip.info +@settitle Lzip +@finalout +@c %**end of header + +@set UPDATED 27 April 2009 +@set VERSION 1.6-pre1 + +@dircategory Data Compression +@direntry +* Lzip: (lzip). Data compressor based on the LZMA algorithm +@end direntry + + +@titlepage +@title Lzip +@subtitle A data compressor based on the LZMA algorithm +@subtitle for Lzip version @value{VERSION}, @value{UPDATED} +@author by Antonio Diaz Diaz + +@page +@vskip 0pt plus 1filll +@end titlepage + +@contents + +@node Top +@top + +This manual is for Lzip (version @value{VERSION}, @value{UPDATED}). + +@menu +* Introduction:: Purpose and features of lzip +* Algorithm:: How lzip compresses the data +* Invoking Lzip:: Command line interface +* File Format:: Detailed format of the compressed file +* Examples:: A small tutorial with examples +* Lzdiff:: Comparing compressed files +* Lzgrep:: Searching inside compressed files +* Lziprecover:: Recovering data from damaged compressed files +* Problems:: Reporting bugs +* Concept Index:: Index of concepts +@end menu + +@sp 1 +Copyright @copyright{} 2008, 2009 Antonio Diaz Diaz. + +This manual is free documentation: you have unlimited permission +to copy, distribute and modify it. + + +@node Introduction +@chapter Introduction +@cindex introduction + +Lzip is a lossless data compressor based on the LZMA algorithm, with +very safe integrity checking and a user interface similar to the one of +gzip or bzip2. Lzip decompresses almost as fast as gzip and compresses +better than bzip2, which makes it well suited for software distribution +and data archiving. + +Lzip replaces every file given in the command line with a compressed +version of itself, with the name "original_name.lz". Each compressed +file has the same modification date, permissions, and, when possible, +ownership as the corresponding original, so that these properties can be +correctly restored at decompression time. Lzip is able to read from some +types of non regular files if the @samp{--stdout} option is specified. + +If no file names are specified, lzip compresses (or decompresses) from +standard input to standard output. In this case, lzip will decline to +write compressed output to a terminal, as this would be entirely +incomprehensible and therefore pointless. + +Lzip will correctly decompress a file which is the concatenation of two +or more compressed files. The result is the concatenation of the +corresponding uncompressed files. Integrity testing of concatenated +compressed files is also supported. + +Lzip can produce multimember files and safely recover, with lziprecover, +the undamaged members in case of file damage. Lzip can also split the +compressed output in volumes of a given size, even when reading from +standard input. This allows the direct creation of multivolume +compressed tar archives. + +The amount of memory required for compression is about 2 times the +dictionary size limit plus 8 times the dictionary size really used. For +decompression is a little more than the dictionary size really used. +Lzip will automatically use the smallest possible dictionary size for +each member without exceeding the given limit. It is important to +appreciate that the decompression memory requirement is affected at +compression time by the choice of dictionary size limit. + +When decompressing, lzip attempts to guess the name for the decompressed +file from that of the compressed file as follows: + +@multitable {anyothername} {becomes} {anyothername.out} +@item filename.lz @tab becomes @tab filename +@item filename.tlz @tab becomes @tab filename.tar +@item anyothername @tab becomes @tab anyothername.out +@end multitable + +As a self-check for your protection, lzip stores in the member trailer +the 32-bit CRC of the original data and the size of the original data, +to make sure that the decompressed version of the data is identical to +the original. This guards against corruption of the compressed data, and +against undetected bugs in lzip (hopefully very unlikely). The chances +of data corruption going undetected are microscopic, less than one +chance in 4000 million for each member processed. Be aware, though, that +the check occurs upon decompression, so it can only tell you that +something is wrong. It can't help you recover the original uncompressed +data. + +Return values: 0 for a normal exit, 1 for environmental problems (file +not found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or +invalid input file, 3 for an internal consistency error (eg, bug) which +caused lzip to panic. + + +@node Algorithm +@chapter Algorithm +@cindex algorithm + +Lzip implements a simplified version of the LZMA (Lempel-Ziv-Markov +chain-Algorithm) algorithm. The original LZMA algorithm was designed by +Igor Pavlov. + +The high compression of LZMA comes from combining two basic, well-proven +compression ideas: sliding dictionaries (LZ77/78) and markov models (the +thing used by every compression algorithm that uses a range encoder or +similar order-0 entropy coder as its last stage) with segregation of +contexts according to what the bits are used for. + +Lzip is a two stage compressor. The first stage is a Lempel-Ziv coder, +which reduces redundancy by translating chunks of data to their +corresponding distance-length pairs. The second stage is a range encoder +that uses a different probability model for each type of data; +distances, lengths, literal bytes, etc. + +The match finder, part of the LZ coder, is the most important piece of +the LZMA algorithm, as it is in many Lempel-Ziv based algorithms. Most +of lzip's execution time is spent in the match finder, and it has the +greatest influence on the compression ratio. + +Here is how it works, step by step: + +1) The member header is written to the output stream. + +2) The first byte is coded literally, because there are no previous +bytes to which the match finder can refer to. + +3) The main encoder advances to the next byte in the input data and +calls the match finder. + +4) The match finder fills an array with the minimum distances before the +current byte where a match of a given length can be found. + +5) Go back to step 3 until a sequence (formed of pairs, repeated +distances and literal bytes) of minimum price has been formed. Where the +price represents the number of output bits produced. + +6) The range encoder encodes the sequence produced by the main encoder +and sends the produced bytes to the output stream. + +7) Go back to step 3 until the input data is finished or until the +member or volume size limits are reached. + +8) The range encoder is flushed. + +9) The member trailer is written to the output stream. + +10) If there are more data to compress, go back to step 1. + + +@node Invoking Lzip +@chapter Invoking Lzip +@cindex invoking +@cindex options +@cindex usage +@cindex version + +The format for running lzip is: + +@example +lzip [@var{options}] [@var{files}] +@end example + +Lzip supports the following options: + +@table @samp +@item --help +@itemx -h +Print an informative help message describing the options and exit. + +@item --version +@itemx -V +Print the version number of lzip on the standard output and exit. + +@item --member-size=@var{size} +@itemx -b @var{size} +Produce a multimember file and set the member size limit to @var{size} +bytes. Minimum member size limit is 100kB. Small member size may degrade +compression ratio, so use it only when needed. The default is to produce +single member files. + +@item --stdout +@itemx -c +Compress or decompress to standard output. Needed when reading from a +named pipe (fifo) or from a device. Use it to recover as much of the +uncompressed data as possible when decompressing a corrupt file. + +@item --decompress +@itemx -d +Decompress. + +@item --force +@itemx -f +Force overwrite of output file. + +@item --keep +@itemx -k +Keep (don't delete) input files during compression or decompression. + +@item --match-length=@var{length} +@itemx -m @var{length} +Set the match length limit in bytes. Valid values range from 5 to 273. +Larger values usually give better compression ratios but longer +compression times. + +@item --output=@var{file} +@itemx -o @var{file} +When reading from standard input and @samp{--stdout} has not been +specified, use @samp{@var{file}} as the virtual name of the uncompressed +file. This produces a file named @samp{@var{file}} when decompressing, a +file named @samp{@var{file}.lz} when compressing, and several files +named @samp{@var{file}00001.lz}, @samp{@var{file}00002.lz}, etc, when +compressing and splitting the output in volumes. + +@item --quiet +@itemx -q +Quiet operation. Suppress all messages. + +@item --dictionary-size=@var{size} +@itemx -s @var{size} +Set the dictionary size limit in bytes. Valid values range from 4KiB to +512MiB. Lzip will use the smallest possible dictionary size for each +member without exceeding this limit. Note that dictionary sizes are +quantized. If the specified size does not match one of the valid sizes, +it will be rounded upwards. + +@item --volume-size=@var{size} +@itemx -S @var{size} +Split the compressed output into several volume files with names +@samp{original_name00001.lz}, @samp{original_name00002.lz}, etc, and set +the volume size limit to @var{size} bytes. Each volume is a complete, +maybe multimember, lzip file. Minimum volume size limit is 100kB. Small +volume size may degrade compression ratio, so use it only when needed. + +@item --test +@itemx -t +Check integrity of the specified file(s), but don't decompress them. +This really performs a trial decompression and throws away the result. +Use @samp{-tvv} or @samp{-tvvv} to see information about the file. + +@item --verbose +@itemx -v +Verbose mode. Show the compression ratio for each file processed. +Further -v's increase the verbosity level. + +@item -1 .. -9 +Set the compression parameters (dictionary size and match length limit) +as shown in the table below. Note that @samp{-9} can be much slower than +@samp{-1}. These options have no effect when decompressing. + +@multitable {Level} {Dictionary size} {Match length limit} +@item Level @tab Dictionary size @tab Match length limit +@item -1 @tab 4MiB @tab 10 bytes +@item -2 @tab 4MiB @tab 12 bytes +@item -3 @tab 4MiB @tab 17 bytes +@item -4 @tab 4MiB @tab 26 bytes +@item -5 @tab 4MiB @tab 44 bytes +@item -6 @tab 8MiB @tab 80 bytes +@item -7 @tab 16MiB @tab 108 bytes +@item -8 @tab 16MiB @tab 163 bytes +@item -9 @tab 32MiB @tab 273 bytes +@end multitable + +@item --fast +@itemx --best +Aliases for GNU gzip compatibility. + +@end table + +@sp 1 +Numbers given as arguments to options may be followed by a multiplier +and an optional @samp{B} for "byte". + +Table of SI and binary prefixes (unit multipliers): + +@multitable {Prefix} {kilobyte (10^3 = 1000)} {|} {Prefix} {kibibyte (2^10 = 1024)} +@item Prefix @tab Value @tab | @tab Prefix @tab Value +@item k @tab kilobyte (10^3 = 1000) @tab | @tab Ki @tab kibibyte (2^10 = 1024) +@item M @tab megabyte (10^6) @tab | @tab Mi @tab mebibyte (2^20) +@item G @tab gigabyte (10^9) @tab | @tab Gi @tab gibibyte (2^30) +@item T @tab terabyte (10^12) @tab | @tab Ti @tab tebibyte (2^40) +@item P @tab petabyte (10^15) @tab | @tab Pi @tab pebibyte (2^50) +@item E @tab exabyte (10^18) @tab | @tab Ei @tab exbibyte (2^60) +@item Z @tab zettabyte (10^21) @tab | @tab Zi @tab zebibyte (2^70) +@item Y @tab yottabyte (10^24) @tab | @tab Yi @tab yobibyte (2^80) +@end multitable + + +@node File Format +@chapter File Format +@cindex file format + +In the diagram below, a box like this: +@verbatim ++---+ +| | <-- the vertical bars might be missing ++---+ +@end verbatim + +represents one byte; a box like this: +@verbatim ++==============+ +| | ++==============+ +@end verbatim + +represents a variable number of bytes. + +@sp 1 +A lzip file consists of a series of "members" (compressed data sets). +The members simply appear one after another in the file, with no +additional information before, between, or after them. + +Each member has the following structure: +@verbatim ++--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| ID string | VN | DS | Lzma stream | CRC32 | Data size | Member size | ++--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +@end verbatim + +All multibyte values are stored in little endian order. + +@table @samp +@item ID string +A four byte string, identifying the member type, with the value "LZIP". + +@item VN (version number, 1 byte) +Just in case something needs to be modified in the future. Valid values +are 0 and 1. Version 0 files have only one member and lack @samp{Member +size}. + +@item DS (coded dictionary size, 1 byte) +Bits 4-0 contain the base 2 logarithm of the base dictionary size.@* +Bits 7-5 contain the number of "wedges" to substract from the base +dictionary size to obtain the dictionary size. The size of a wedge is +(base dictionary size / 16).@* +Valid values for dictionary size range from 4KiB to 512MiB. + +@item Lzma stream +The lzma stream, finished by an end of stream marker. Uses default values +for encoder properties. + +@item CRC32 (4 bytes) +CRC of the uncompressed original data. + +@item Data size (8 bytes) +Size of the uncompressed original data. + +@item Member size (8 bytes) +Total size of the member, including header and trailer. This facilitates +safe recovery of undamaged members from multimember files. + +@end table + + +@node Examples +@chapter A small tutorial with examples +@cindex examples + +WARNING! If your data is important, give the @samp{--keep} option to +lzip and do not remove the original file until you verify the compressed +file with a command like @samp{lzip -cd file.lz | cmp file -}. + +@sp 1 +@noindent +Example 1: Replace a regular file with its compressed version file.lz +and show the compression ratio. + +@example +lzip -v file +@end example + +@sp 1 +@noindent +Example 2: Like example 1 but the created file.lz is multimember with a +member size of 1MiB. + +@example +lzip -b 1MiB file +@end example + +@sp 1 +@noindent +Example 3: Compress a whole floppy in /dev/fd0 and send the output to +file.lz. + +@example +lzip -c /dev/fd0 > file.lz +@end example + +@sp 1 +@noindent +Example 4: Create a multivolume compressed tar archive with a volume +size of 1440KiB. + +@example +tar -c some_directory | lzip -S 1440KiB -o volume_name +@end example + +@sp 1 +@noindent +Example 5: Extract a multivolume compressed tar archive. + +@example +lzip -cd volume_name*.lz | tar -xf - +@end example + +@sp 1 +@noindent +Example 6: Create a multivolume compressed backup of a big database file +with a volume size of 650MB, where each volume is a multimember file +with a member size of 32MiB. + +@example +lzip -b 32MiB -S 650MB big_database +@end example + +@sp 1 +@noindent +Example 7: Recover the first volume of those created in example 6 from +two copies, @samp{big_database1_00001.lz} and +@samp{big_database2_00001.lz}, with member 00007 damaged in the first +copy and member 00018 damaged in the second copy. (Indented lines are +lzip error messages). + +@example +lziprecover big_database1_00001.lz +lziprecover big_database2_00001.lz +lzip -t rec*big_database1_00001.lz + rec00007big_database1_00001.lz: crc mismatch +lzip -t rec*big_database2_00001.lz + rec00018big_database1_00001.lz: crc mismatch +cp rec00007big_database2_00001.lz rec00007big_database1_00001.lz +cat rec*big_database1_00001.lz > big_database3_00001.lz +@end example + + +@node Lzdiff +@chapter Lzdiff +@cindex lzdiff + +Lzdiff is a wrapper script around the diff and cmp commands that allows +transparent comparison of any combination of compressed and +non-compressed files. If any given file is compressed, its uncompressed +content is used. The supported compressors are gzip, bzip2 and lzip. + +The format for running lzdiff is: + +@example +lzdiff [@var{options}] [@var{diff_options}] @var{file1} [@var{file2}] +@end example + +@noindent +Compares @var{file1} to @var{file2}. If @var{file2} is omitted, compares +@var{file1} to the uncompressed contents of @var{file1}.[gz|bz2|lz] +(depending on the default compressor selected). @var{diff_options} are +passed directly to diff or cmp. The exit status from diff or cmp is +preserved. + +Lzdiff supports the following options: + +@table @samp +@item --help +@itemx -h +Print an informative help message describing the options and exit. + +@item --version +@itemx -V +Print the version number of lzdiff on the standard output and exit. + +@item --gzip +Use gzip as default decompressor. + +@item --bzip2 +Use bzip2 as default decompressor. + +@item --lzip +Use lzip as default decompressor (default). + +@item --diff +Use diff to compare files (default). + +@item --cmp +Use cmp to compare files. + +@end table + +Lzdiff has the limitation that messages from the diff or cmp programs +refer to temporary filenames instead of those specified. + + +@node Lzgrep +@chapter Lzgrep +@cindex lzgrep + +Lzgrep is a wrapper script around the grep command that allows +transparent search on any combination of compressed and non-compressed +files. If any given file is compressed, its uncompressed content is +used. If a given file does not exist, lzgrep tries the compressed file +name corresponding to the default compressor selected. The supported +compressors are gzip, bzip2 and lzip. + +The format for running lzgrep is: + +@example +lzgrep [@var{options}] [@var{grep_options}] @var{pattern} [@var{files}] +@end example + +@noindent +@var{grep_options} are passed directly to grep. The exit status from +grep is preserved. + +Lzgrep supports the following options: + +@table @samp +@item --help +@itemx -h +Print an informative help message describing the options and exit. + +@item --version +@itemx -V +Print the version number of lzgrep on the standard output and exit. + +@item --gzip +Use gzip as default decompressor. + +@item --bzip2 +Use bzip2 as default decompressor. + +@item --lzip +Use lzip as default decompressor (default). + +@end table + + +@node Lziprecover +@chapter Lziprecover +@cindex lziprecover + +Lziprecover is a program that searches for members in .lz files, and +writes each member in its own .lz file. You can then use +@w{@samp{lzip -t}} to test the integrity of the resulting files, and +decompress those which are undamaged. + +Lziprecover takes a single argument, the name of the damaged file, and +writes a number of files @samp{rec00001file.lz}, @samp{rec00002file.lz}, +etc, containing the extracted members. The output filenames are designed +so that the use of wildcards in subsequent processing, for example, +@w{@samp{lzip -dc rec*file.lz > recovered_data}}, processes the files in +the correct order. + + +@node Problems +@chapter Reporting Bugs +@cindex bugs +@cindex getting help + +There are probably bugs in lzip. There are certainly errors and +omissions in this manual. If you report them, they will get fixed. If +you don't, no one will ever know about them and they will remain unfixed +for all eternity, if not longer. + +If you find a bug in lzip, please send electronic mail to +@email{lzip-bug@@nongnu.org}. Include the version number, which you can +find by running @w{@samp{lzip --version}}. + + +@node Concept Index +@unnumbered Concept Index + +@printindex cp + +@bye |