From 083c902a1bf6cd876dd4f683b7936a8246ec0cd9 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Sat, 7 Nov 2015 08:49:43 +0100 Subject: Merging upstream version 1.11. Signed-off-by: Daniel Baumann --- doc/lzip.1 | 12 +- doc/lzip.info | 325 +++++++++++++++++++++++++++++++++++++++--------------- doc/lzip.texinfo | 313 +++++++++++++++++++++++++++++++++++++++------------- doc/lziprecover.1 | 27 +++-- 4 files changed, 499 insertions(+), 178 deletions(-) (limited to 'doc') diff --git a/doc/lzip.1 b/doc/lzip.1 index 3254f33..670d15e 100644 --- a/doc/lzip.1 +++ b/doc/lzip.1 @@ -1,12 +1,12 @@ .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.37.1. -.TH LZIP "1" "April 2010" "Lzip 1.10" "User Commands" +.TH LZIP "1" "September 2010" "Lzip 1.11" "User Commands" .SH NAME -Lzip \- data compressor based on the LZMA algorithm +Lzip \- reduces the size of files .SH SYNOPSIS .B lzip [\fIoptions\fR] [\fIfiles\fR] .SH DESCRIPTION -Lzip \- A data compressor based on the LZMA algorithm. +Lzip \- Data compressor based on the LZMA algorithm. .SH OPTIONS .TP \fB\-h\fR, \fB\-\-help\fR @@ -31,7 +31,7 @@ overwrite existing output files keep (don't delete) input files .TP \fB\-m\fR, \fB\-\-match\-length=\fR -set match length limit in bytes [80] +set match length limit in bytes [36] .TP \fB\-o\fR, \fB\-\-output=\fR if reading stdin, place the output into @@ -51,11 +51,11 @@ test compressed file integrity \fB\-v\fR, \fB\-\-verbose\fR be verbose (a 2nd \fB\-v\fR gives more) .TP -\fB\-1\fR .. \fB\-9\fR +\fB\-0\fR .. \fB\-9\fR set compression level [default 6] .TP \fB\-\-fast\fR -alias for \fB\-1\fR +alias for \fB\-0\fR .TP \fB\-\-best\fR alias for \fB\-9\fR diff --git a/doc/lzip.info b/doc/lzip.info index 7914b91..00cf933 100644 --- a/doc/lzip.info +++ b/doc/lzip.info @@ -11,18 +11,19 @@ File: lzip.info, Node: Top, Next: Introduction, Up: (dir) Lzip Manual *********** -This manual is for Lzip (version 1.10, 5 April 2010). +This manual is for Lzip (version 1.11, 16 September 2010). * Menu: -* Introduction:: Purpose and features of lzip -* Algorithm:: How lzip compresses the data -* Invoking Lzip:: Command line interface -* File Format:: Detailed format of the compressed file -* Examples:: A small tutorial with examples -* Lziprecover:: Recovering data from damaged compressed files -* Problems:: Reporting bugs -* Concept Index:: Index of concepts +* Introduction:: Purpose and features of lzip +* Algorithm:: How lzip compresses the data +* Invoking Lzip:: Command line interface +* File Format:: Detailed format of the compressed file +* Examples:: A small tutorial with examples +* Lziprecover:: Recovering data from damaged compressed files +* Invoking Lziprecover:: Command line interface +* Problems:: Reporting bugs +* Concept Index:: Index of concepts Copyright (C) 2008, 2009, 2010 Antonio Diaz Diaz. @@ -68,11 +69,14 @@ multivolume compressed tar archives. The amount of memory required for compression is about 5 MiB plus 1 or 2 times the dictionary size limit (1 if input file size is less than dictionary size limit, else 2) plus 8 times the dictionary size really -used. For decompression it is a little more than the dictionary size -really used. Lzip will automatically use the smallest possible -dictionary size without exceeding the given limit. It is important to -appreciate that the decompression memory requirement is affected at -compression time by the choice of dictionary size limit. +used. The option `-0' is special and only requires about 1.5 MiB at +most. The amount of memory required for decompression is a little more +than the dictionary size really used. + + Lzip will automatically use the smallest possible dictionary size +without exceeding the given limit. Keep in mind that the decompression +memory requirement is affected at compression time by the choice of +dictionary size limit. When decompressing, lzip attempts to guess the name for the decompressed file from that of the compressed file as follows: @@ -104,14 +108,12 @@ File: lzip.info, Node: Algorithm, Next: Invoking Lzip, Prev: Introduction, U *********** Lzip implements a simplified version of the LZMA (Lempel-Ziv-Markov -chain-Algorithm) algorithm. The original LZMA algorithm was designed by -Igor Pavlov. - - The high compression of LZMA comes from combining two basic, -well-proven compression ideas: sliding dictionaries (LZ77/78) and -markov models (the thing used by every compression algorithm that uses -a range encoder or similar order-0 entropy coder as its last stage) -with segregation of contexts according to what the bits are used for. +chain-Algorithm) algorithm. The high compression of LZMA comes from +combining two basic, well-proven compression ideas: sliding dictionaries +(LZ77/78) and markov models (the thing used by every compression +algorithm that uses a range encoder or similar order-0 entropy coder as +its last stage) with segregation of contexts according to what the bits +are used for. Lzip is a two stage compressor. The first stage is a Lempel-Ziv coder, which reduces redundancy by translating chunks of data to their @@ -153,6 +155,13 @@ member or volume size limits are reached. 10) If there are more data to compress, go back to step 1. + +The ideas embodied in lzip are due to (at least) the following people: +Abraham Lempel and Jacob Ziv (for the LZ algorithm), Andrey Markov (for +the definition of Markov chains), G.N.N. Martin (for the definition of +range encoding), Igor Pavlov (for putting all the above together in +LZMA), and Julian Seward (for bzip2's CLI and the idea of unzcrash). +  File: lzip.info, Node: Invoking Lzip, Next: File Format, Prev: Algorithm, Up: Top @@ -178,7 +187,7 @@ The format for running lzip is: Produce a multimember file and set the member size limit to SIZE bytes. Minimum member size limit is 100kB. Small member size may degrade compression ratio, so use it only when needed. The default - is to produce single member files. + is to produce single-member files. `--stdout' `-c' @@ -202,9 +211,10 @@ The format for running lzip is: `--match-length=LENGTH' `-m LENGTH' - Set the match length limit in bytes. Valid values range from 5 to - 273. Larger values usually give better compression ratios but - longer compression times. + Set the match length limit in bytes. After a match this long is + found, the search is finished. Valid values range from 5 to 273. + Larger values usually give better compression ratios but longer + compression times. `--output=FILE' `-o FILE' @@ -227,6 +237,11 @@ The format for running lzip is: dictionary sizes are quantized. If the specified size does not match one of the valid sizes, it will be rounded upwards. + For maximum compression you should use a dictionary size limit as + large as possible, but keep in mind that the decompression memory + requirement is affected at compression time by the choice of + dictionary size limit. + `--volume-size=SIZE' `-S SIZE' Split the compressed output into several volume files with names @@ -240,28 +255,35 @@ The format for running lzip is: `-t' Check integrity of the specified file(s), but don't decompress them. This really performs a trial decompression and throws away - the result. Use `-tvv' or `-tvvv' to see information about the - file. + the result. Use it together with `-v' to see information about + the file. `--verbose' `-v' Verbose mode. Show the compression ratio for each file processed. Further -v's increase the verbosity level. -`-1 .. -9' +`-0 .. -9' Set the compression parameters (dictionary size and match length limit) as shown in the table below. Note that `-9' can be much - slower than `-1'. These options have no effect when decompressing. + slower than `-0'. These options have no effect when decompressing. + + The bidimensional parameter space of LZMA can't be mapped to a + linear scale optimal for all files. If your files are large, very + repetitive, etc, you may need to use the `--match-length' and + `--dictionary-size' options directly to achieve optimal + performance. Level Dictionary size Match length limit - -1 1 MiB 10 bytes - -2 1.5 MiB 12 bytes - -3 2 MiB 17 bytes - -4 3 MiB 26 bytes - -5 4 MiB 44 bytes - -6 8 MiB 80 bytes - -7 16 MiB 108 bytes - -8 24 MiB 163 bytes + -0 64 KiB 16 bytes + -1 1 MiB 5 bytes + -2 1.5 MiB 6 bytes + -3 2 MiB 8 bytes + -4 3 MiB 12 bytes + -5 4 MiB 20 bytes + -6 8 MiB 36 bytes + -7 16 MiB 68 bytes + -8 24 MiB 132 bytes -9 32 MiB 273 bytes `--fast' @@ -316,7 +338,7 @@ additional information before, between, or after them. All multibyte values are stored in little endian order. `ID string' - A four byte string, identifying the member type, with the value + A four byte string, identifying the lzip format, with the value "LZIP". `VN (version number, 1 byte)' @@ -353,9 +375,12 @@ File: lzip.info, Node: Examples, Next: Lziprecover, Prev: File Format, Up: T 5 A small tutorial with examples ******************************** -WARNING! If your data is important, give the `--keep' option to lzip -and do not remove the original file until you verify the compressed -file with a command like `lzip -cd file.lz | cmp file -'. +WARNING! Even if lzip is bug-free, other causes may result in a corrupt +compressed file (bugs in the system libraries, memory errors, etc). +Therefore, if the data you are going to compress is important give the +`--keep' option to lzip and do not remove the original file until you +verify the compressed file with a command like +`lzip -cd file.lz | cmp file -'. Example 1: Replace a regular file with its compressed version file.lz @@ -365,77 +390,198 @@ and show the compression ratio. Example 2: Like example 1 but the created file.lz is multimember with a -member size of 1MiB. +member size of 1MiB. The compression ratio is not shown. lzip -b 1MiB file -Example 3: Compress a whole floppy in /dev/fd0 and send the output to +Example 3: Restore a regular file from its compressed version file.lz. +If the operation is successful, file.lz is removed. + + lzip -d file.lz + + +Example 4: Verify the integrity of the compressed file file.lz and show +status. + + lzip -tv file.lz + + +Example 5: Compress a whole floppy in /dev/fd0 and send the output to file.lz. lzip -c /dev/fd0 > file.lz -Example 4: Create a multivolume compressed tar archive with a volume +Example 6: Decompress file.lz partially until 10KiB of decompressed data +are produced. + + lzip -cd file.lz | dd bs=1024 count=10 + + +Example 7: Create a multivolume compressed tar archive with a volume size of 1440KiB. tar -c some_directory | lzip -S 1440KiB -o volume_name -Example 5: Extract a multivolume compressed tar archive. +Example 8: Extract a multivolume compressed tar archive. lzip -cd volume_name*.lz | tar -xf - -Example 6: Create a multivolume compressed backup of a big database file +Example 9: Create a multivolume compressed backup of a big database file with a volume size of 650MB, where each volume is a multimember file with a member size of 32MiB. - lzip -b 32MiB -S 650MB big_database - - -Example 7: Recover the first volume of those created in example 6 from -two copies, `big_database1_00001.lz' and `big_database2_00001.lz', with -member 00007 damaged in the first copy and member 00018 damaged in the -second copy. (Indented lines are lzip error messages). - - lziprecover big_database1_00001.lz - lziprecover big_database2_00001.lz - lzip -t rec*big_database1_00001.lz - rec00007big_database1_00001.lz: crc mismatch - lzip -t rec*big_database2_00001.lz - rec00018big_database1_00001.lz: crc mismatch - cp rec00007big_database2_00001.lz rec00007big_database1_00001.lz - cat rec*big_database1_00001.lz > big_database3_00001.lz + lzip -b 32MiB -S 650MB big_db + + +Example 10: Recover a compressed backup from two copies on CD-ROM (see +the GNU ddrescue manual for details about ddrescue) + + ddrescue -b2048 /dev/cdrom cdimage1 logfile1 + mount -t iso9660 -o loop,ro cdimage1 /mnt/cdimage + cp /mnt/cdimage/backup.tar.lz rescued1.tar.lz + umount /mnt/cdimage + (insert second copy in the CD drive) + ddrescue -b2048 /dev/cdrom cdimage2 logfile2 + mount -t iso9660 -o loop,ro cdimage2 /mnt/cdimage + cp /mnt/cdimage/backup.tar.lz rescued2.tar.lz + umount /mnt/cdimage + lziprecover -m -o rescued.tar.lz rescued1.tar.lz rescued2.tar.lz + + +Example 11: Recover the first volume of those created in example 9 from +two copies, `big_db1_00001.lz' and `big_db2_00001.lz', with member +00007 damaged in the first copy, member 00018 damaged in the second +copy, and member 00012 damaged in both copies. (Indented lines are +abridged error messages from lzip/lziprecover). Two correct copies are +produced and compared. + + lziprecover -s big_db1_00001.lz + lziprecover -s big_db2_00001.lz + lzip -t rec*big_db1_00001.lz + rec00007big_db1_00001.lz: crc mismatch + rec00012big_db1_00001.lz: crc mismatch + lzip -t rec*big_db2_00001.lz + rec00012big_db2_00001.lz: crc mismatch + rec00018big_db2_00001.lz: crc mismatch + lziprecover -m rec00012big_db1_00001.lz rec00012big_db2_00001.lz + Input files merged successfully + cp rec00007big_db2_00001.lz rec00007big_db1_00001.lz + cp rec00012big_db1_00001_fixed.lz rec00012big_db1_00001.lz + cp rec00012big_db1_00001_fixed.lz rec00012big_db2_00001.lz + cp rec00018big_db1_00001.lz rec00018big_db2_00001.lz + cat rec*big_db1_00001.lz > big_db3_00001.lz + cat rec*big_db2_00001.lz > big_db4_00001.lz + zcmp big_db3_00001.lz big_db4_00001.lz  -File: lzip.info, Node: Lziprecover, Next: Problems, Prev: Examples, Up: Top +File: lzip.info, Node: Lziprecover, Next: Invoking Lziprecover, Prev: Examples, Up: Top 6 Lziprecover ************* -Lziprecover is a program that searches for members in .lz files, and -writes each member in its own .lz file. You can then use `lzip -t' to -test the integrity of the resulting files, and decompress those which -are undamaged. +Lziprecover is a data recovery tool for lzip compressed files able to +repair slightly damaged files, recover badly damaged files from two or +more copies, and extract undamaged members from multi-member files. + + Lziprecover takes as arguments the names of the damaged files and +writes zero or more recovered files depending on the operation selected +and whether the recovery succeeded or not. The damaged files themselves +are never modified. - Data from damaged members can be partially recovered writing it to -stdout as shown in the following example (the resulting file may contain -garbage data at the end): + If the files are too damaged for lziprecover to repair them, data +from damaged members can be partially recovered writing it to stdout as +shown in the following example (the resulting file may contain garbage +data at the end): lzip -cd rec00001file.lz > rec00001file - Lziprecover takes a single argument, the name of the damaged file, -and writes a number of files `rec00001file.lz', `rec00002file.lz', etc, -containing the extracted members. The output filenames are designed so -that the use of wildcards in subsequent processing, for example, -`lzip -dc rec*file.lz > recovered_data', processes the files in the -correct order. + If the cause of file corruption is damaged media, the combination GNU +ddrescue + lziprecover is the best option for recovering data from +multiple damaged copies. *Note ddrescue-example::, for an example. + + +File: lzip.info, Node: Invoking Lziprecover, Next: Problems, Prev: Lziprecover, Up: Top + +7 Invoking Lziprecover +********************** + +The format for running lziprecover is: + + lziprecover [OPTIONS] [FILES] + + Lziprecover supports the following options: + +`--help' +`-h' + Print an informative help message describing the options and exit. + +`--version' +`-V' + Print the version number of lziprecover on the standard output and + exit. + +`--force' +`-f' + Force overwrite of output file. + +`--merge' +`-m' + Try to produce a correct file merging the good parts of two or more + damaged copies. The copies must be single-member files. The merge + will fail if the copies have too many damaged areas or if the same + byte is damaged in all copies. If successful, a repaired copy is + written to the file `FILE_fixed.lz'. + + To give you an idea of its possibilities, when merging two copies + each of them with one damaged area affecting 1 percent of the + copy, the probability of obtaining a correct file is about 98 + percent. With three such copies the probability rises to 99.97 + percent. For large files with small errors, the probability + approaches 100 percent even with only two copies. + +`--output=FILE' +`-o FILE' + Place the output into `FILE' instead of into `FILE_fixed.lz'. + + If splitting, the names of the files produced are in the form + `rec00001FILE', etc. + +`--quiet' +`-q' + Quiet operation. Suppress all messages. + +`--repair' +`-R' + Try to repair a small error, affecting only one byte, in a + single-member FILE. If successful, a repaired copy is written to + the file `FILE_fixed.lz'. `FILE' is not modified at all. + +`--split' +`-s' + Search for members in `FILE' and write each member in its own + `.lz' file. You can then use `lzip -t' to test the integrity of + the resulting files, decompress those which are undamaged, and try + to repair or partially decompress those which are damaged. + + The names of the files produced are in the form `rec00001FILE.lz', + `rec00002FILE.lz', etc, and are designed so that the use of + wildcards in subsequent processing, for example, + `lzip -cd rec*FILE.lz > recovered_data', processes the files in + the correct order. + +`--verbose' +`-v' + Verbose mode. Further -v's increase the verbosity level. +  -File: lzip.info, Node: Problems, Next: Concept Index, Prev: Lziprecover, Up: Top +File: lzip.info, Node: Problems, Next: Concept Index, Prev: Invoking Lziprecover, Up: Top -7 Reporting Bugs +8 Reporting Bugs **************** There are probably bugs in lzip. There are certainly errors and @@ -462,7 +608,8 @@ Concept Index * file format: File Format. (line 6) * getting help: Problems. (line 6) * introduction: Introduction. (line 6) -* invoking: Invoking Lzip. (line 6) +* invoking lzip: Invoking Lzip. (line 6) +* invoking lziprecover: Invoking Lziprecover. (line 6) * lziprecover: Lziprecover. (line 6) * options: Invoking Lzip. (line 6) * usage: Invoking Lzip. (line 6) @@ -472,13 +619,15 @@ Concept Index  Tag Table: Node: Top224 -Node: Introduction897 -Node: Algorithm4207 -Node: Invoking Lzip6433 -Node: File Format10780 -Node: Examples12734 -Node: Lziprecover14572 -Node: Problems15487 -Node: Concept Index16012 +Node: Introduction1029 +Node: Algorithm4421 +Node: Invoking Lzip6939 +Node: File Format11911 +Node: Examples13865 +Ref: ddrescue-example15619 +Node: Lziprecover17412 +Node: Invoking Lziprecover18465 +Node: Problems20826 +Node: Concept Index21360  End Tag Table diff --git a/doc/lzip.texinfo b/doc/lzip.texinfo index 9cacd16..5c62d2f 100644 --- a/doc/lzip.texinfo +++ b/doc/lzip.texinfo @@ -5,8 +5,8 @@ @finalout @c %**end of header -@set UPDATED 5 April 2010 -@set VERSION 1.10 +@set UPDATED 16 September 2010 +@set VERSION 1.11 @dircategory Data Compression @direntry @@ -16,7 +16,7 @@ @titlepage @title Lzip -@subtitle A data compressor based on the LZMA algorithm +@subtitle Data compressor based on the LZMA algorithm @subtitle for Lzip version @value{VERSION}, @value{UPDATED} @author by Antonio Diaz Diaz @@ -24,7 +24,9 @@ @vskip 0pt plus 1filll @end titlepage +@ifnothtml @contents +@end ifnothtml @node Top @top @@ -32,14 +34,15 @@ This manual is for Lzip (version @value{VERSION}, @value{UPDATED}). @menu -* Introduction:: Purpose and features of lzip -* Algorithm:: How lzip compresses the data -* Invoking Lzip:: Command line interface -* File Format:: Detailed format of the compressed file -* Examples:: A small tutorial with examples -* Lziprecover:: Recovering data from damaged compressed files -* Problems:: Reporting bugs -* Concept Index:: Index of concepts +* Introduction:: Purpose and features of lzip +* Algorithm:: How lzip compresses the data +* Invoking Lzip:: Command line interface +* File Format:: Detailed format of the compressed file +* Examples:: A small tutorial with examples +* Lziprecover:: Recovering data from damaged compressed files +* Invoking Lziprecover:: Command line interface +* Problems:: Reporting bugs +* Concept Index:: Index of concepts @end menu @sp 1 @@ -85,11 +88,14 @@ compressed tar archives. The amount of memory required for compression is about 5 MiB plus 1 or 2 times the dictionary size limit (1 if input file size is less than dictionary size limit, else 2) plus 8 times the dictionary size really -used. For decompression it is a little more than the dictionary size -really used. Lzip will automatically use the smallest possible -dictionary size without exceeding the given limit. It is important to -appreciate that the decompression memory requirement is affected at -compression time by the choice of dictionary size limit. +used. The option @samp{-0} is special and only requires about 1.5 MiB at +most. The amount of memory required for decompression is a little more +than the dictionary size really used. + +Lzip will automatically use the smallest possible dictionary size +without exceeding the given limit. Keep in mind that the decompression +memory requirement is affected at compression time by the choice of +dictionary size limit. When decompressing, lzip attempts to guess the name for the decompressed file from that of the compressed file as follows: @@ -122,14 +128,12 @@ caused lzip to panic. @cindex algorithm Lzip implements a simplified version of the LZMA (Lempel-Ziv-Markov -chain-Algorithm) algorithm. The original LZMA algorithm was designed by -Igor Pavlov. - -The high compression of LZMA comes from combining two basic, well-proven -compression ideas: sliding dictionaries (LZ77/78) and markov models (the -thing used by every compression algorithm that uses a range encoder or -similar order-0 entropy coder as its last stage) with segregation of -contexts according to what the bits are used for. +chain-Algorithm) algorithm. The high compression of LZMA comes from +combining two basic, well-proven compression ideas: sliding dictionaries +(LZ77/78) and markov models (the thing used by every compression +algorithm that uses a range encoder or similar order-0 entropy coder as +its last stage) with segregation of contexts according to what the bits +are used for. Lzip is a two stage compressor. The first stage is a Lempel-Ziv coder, which reduces redundancy by translating chunks of data to their @@ -171,10 +175,18 @@ member or volume size limits are reached. 10) If there are more data to compress, go back to step 1. +@sp 1 +@noindent +The ideas embodied in lzip are due to (at least) the following people: +Abraham Lempel and Jacob Ziv (for the LZ algorithm), Andrey Markov (for +the definition of Markov chains), G.N.N. Martin (for the definition of +range encoding), Igor Pavlov (for putting all the above together in +LZMA), and Julian Seward (for bzip2's CLI and the idea of unzcrash). + @node Invoking Lzip @chapter Invoking Lzip -@cindex invoking +@cindex invoking lzip @cindex options @cindex usage @cindex version @@ -201,7 +213,7 @@ Print the version number of lzip on the standard output and exit. Produce a multimember file and set the member size limit to @var{size} bytes. Minimum member size limit is 100kB. Small member size may degrade compression ratio, so use it only when needed. The default is to produce -single member files. +single-member files. @item --stdout @itemx -c @@ -223,9 +235,9 @@ Keep (don't delete) input files during compression or decompression. @item --match-length=@var{length} @itemx -m @var{length} -Set the match length limit in bytes. Valid values range from 5 to 273. -Larger values usually give better compression ratios but longer -compression times. +Set the match length limit in bytes. After a match this long is found, +the search is finished. Valid values range from 5 to 273. Larger values +usually give better compression ratios but longer compression times. @item --output=@var{file} @itemx -o @var{file} @@ -248,6 +260,10 @@ member without exceeding this limit. Note that dictionary sizes are quantized. If the specified size does not match one of the valid sizes, it will be rounded upwards. +For maximum compression you should use a dictionary size limit as large +as possible, but keep in mind that the decompression memory requirement +is affected at compression time by the choice of dictionary size limit. + @item --volume-size=@var{size} @itemx -S @var{size} Split the compressed output into several volume files with names @@ -260,28 +276,35 @@ volume size may degrade compression ratio, so use it only when needed. @itemx -t Check integrity of the specified file(s), but don't decompress them. This really performs a trial decompression and throws away the result. -Use @samp{-tvv} or @samp{-tvvv} to see information about the file. +Use it together with @samp{-v} to see information about the file. @item --verbose @itemx -v Verbose mode. Show the compression ratio for each file processed. Further -v's increase the verbosity level. -@item -1 .. -9 +@item -0 .. -9 Set the compression parameters (dictionary size and match length limit) as shown in the table below. Note that @samp{-9} can be much slower than -@samp{-1}. These options have no effect when decompressing. +@samp{-0}. These options have no effect when decompressing. + +The bidimensional parameter space of LZMA can't be mapped to a linear +scale optimal for all files. If your files are large, very repetitive, +etc, you may need to use the @samp{--match-length} and +@samp{--dictionary-size} options directly to achieve optimal +performance. @multitable {Level} {Dictionary size} {Match length limit} @item Level @tab Dictionary size @tab Match length limit -@item -1 @tab 1 MiB @tab 10 bytes -@item -2 @tab 1.5 MiB @tab 12 bytes -@item -3 @tab 2 MiB @tab 17 bytes -@item -4 @tab 3 MiB @tab 26 bytes -@item -5 @tab 4 MiB @tab 44 bytes -@item -6 @tab 8 MiB @tab 80 bytes -@item -7 @tab 16 MiB @tab 108 bytes -@item -8 @tab 24 MiB @tab 163 bytes +@item -0 @tab 64 KiB @tab 16 bytes +@item -1 @tab 1 MiB @tab 5 bytes +@item -2 @tab 1.5 MiB @tab 6 bytes +@item -3 @tab 2 MiB @tab 8 bytes +@item -4 @tab 3 MiB @tab 12 bytes +@item -5 @tab 4 MiB @tab 20 bytes +@item -6 @tab 8 MiB @tab 36 bytes +@item -7 @tab 16 MiB @tab 68 bytes +@item -8 @tab 24 MiB @tab 132 bytes @item -9 @tab 32 MiB @tab 273 bytes @end multitable @@ -346,7 +369,7 @@ All multibyte values are stored in little endian order. @table @samp @item ID string -A four byte string, identifying the member type, with the value "LZIP". +A four byte string, identifying the lzip format, with the value "LZIP". @item VN (version number, 1 byte) Just in case something needs to be modified in the future. Valid values @@ -381,9 +404,12 @@ safe recovery of undamaged members from multimember files. @chapter A small tutorial with examples @cindex examples -WARNING! If your data is important, give the @samp{--keep} option to -lzip and do not remove the original file until you verify the compressed -file with a command like @samp{lzip -cd file.lz | cmp file -}. +WARNING! Even if lzip is bug-free, other causes may result in a corrupt +compressed file (bugs in the system libraries, memory errors, etc). +Therefore, if the data you are going to compress is important give the +@samp{--keep} option to lzip and do not remove the original file until +you verify the compressed file with a command like @w{@samp{lzip -cd +file.lz | cmp file -}}. @sp 1 @noindent @@ -397,7 +423,7 @@ lzip -v file @sp 1 @noindent Example 2: Like example 1 but the created file.lz is multimember with a -member size of 1MiB. +member size of 1MiB. The compression ratio is not shown. @example lzip -b 1MiB file @@ -405,7 +431,25 @@ lzip -b 1MiB file @sp 1 @noindent -Example 3: Compress a whole floppy in /dev/fd0 and send the output to +Example 3: Restore a regular file from its compressed version file.lz. +If the operation is successful, file.lz is removed. + +@example +lzip -d file.lz +@end example + +@sp 1 +@noindent +Example 4: Verify the integrity of the compressed file file.lz and show +status. + +@example +lzip -tv file.lz +@end example + +@sp 1 +@noindent +Example 5: Compress a whole floppy in /dev/fd0 and send the output to file.lz. @example @@ -414,7 +458,16 @@ lzip -c /dev/fd0 > file.lz @sp 1 @noindent -Example 4: Create a multivolume compressed tar archive with a volume +Example 6: Decompress file.lz partially until 10KiB of decompressed data +are produced. + +@example +lzip -cd file.lz | dd bs=1024 count=10 +@end example + +@sp 1 +@noindent +Example 7: Create a multivolume compressed tar archive with a volume size of 1440KiB. @example @@ -423,7 +476,7 @@ tar -c some_directory | lzip -S 1440KiB -o volume_name @sp 1 @noindent -Example 5: Extract a multivolume compressed tar archive. +Example 8: Extract a multivolume compressed tar archive. @example lzip -cd volume_name*.lz | tar -xf - @@ -431,31 +484,60 @@ lzip -cd volume_name*.lz | tar -xf - @sp 1 @noindent -Example 6: Create a multivolume compressed backup of a big database file +Example 9: Create a multivolume compressed backup of a big database file with a volume size of 650MB, where each volume is a multimember file with a member size of 32MiB. @example -lzip -b 32MiB -S 650MB big_database +lzip -b 32MiB -S 650MB big_db @end example @sp 1 +@anchor{ddrescue-example} @noindent -Example 7: Recover the first volume of those created in example 6 from -two copies, @samp{big_database1_00001.lz} and -@samp{big_database2_00001.lz}, with member 00007 damaged in the first -copy and member 00018 damaged in the second copy. (Indented lines are -lzip error messages). +Example 10: Recover a compressed backup from two copies on CD-ROM (see +the GNU ddrescue manual for details about ddrescue) @example -lziprecover big_database1_00001.lz -lziprecover big_database2_00001.lz -lzip -t rec*big_database1_00001.lz - rec00007big_database1_00001.lz: crc mismatch -lzip -t rec*big_database2_00001.lz - rec00018big_database1_00001.lz: crc mismatch -cp rec00007big_database2_00001.lz rec00007big_database1_00001.lz -cat rec*big_database1_00001.lz > big_database3_00001.lz +ddrescue -b2048 /dev/cdrom cdimage1 logfile1 +mount -t iso9660 -o loop,ro cdimage1 /mnt/cdimage +cp /mnt/cdimage/backup.tar.lz rescued1.tar.lz +umount /mnt/cdimage + (insert second copy in the CD drive) +ddrescue -b2048 /dev/cdrom cdimage2 logfile2 +mount -t iso9660 -o loop,ro cdimage2 /mnt/cdimage +cp /mnt/cdimage/backup.tar.lz rescued2.tar.lz +umount /mnt/cdimage +lziprecover -m -o rescued.tar.lz rescued1.tar.lz rescued2.tar.lz +@end example + +@sp 1 +@noindent +Example 11: Recover the first volume of those created in example 9 from +two copies, @samp{big_db1_00001.lz} and @samp{big_db2_00001.lz}, with +member 00007 damaged in the first copy, member 00018 damaged in the +second copy, and member 00012 damaged in both copies. (Indented lines +are abridged error messages from lzip/lziprecover). Two correct copies +are produced and compared. + +@example +lziprecover -s big_db1_00001.lz +lziprecover -s big_db2_00001.lz +lzip -t rec*big_db1_00001.lz + rec00007big_db1_00001.lz: crc mismatch + rec00012big_db1_00001.lz: crc mismatch +lzip -t rec*big_db2_00001.lz + rec00012big_db2_00001.lz: crc mismatch + rec00018big_db2_00001.lz: crc mismatch +lziprecover -m rec00012big_db1_00001.lz rec00012big_db2_00001.lz + Input files merged successfully +cp rec00007big_db2_00001.lz rec00007big_db1_00001.lz +cp rec00012big_db1_00001_fixed.lz rec00012big_db1_00001.lz +cp rec00012big_db1_00001_fixed.lz rec00012big_db2_00001.lz +cp rec00018big_db1_00001.lz rec00018big_db2_00001.lz +cat rec*big_db1_00001.lz > big_db3_00001.lz +cat rec*big_db2_00001.lz > big_db4_00001.lz +zcmp big_db3_00001.lz big_db4_00001.lz @end example @@ -463,25 +545,104 @@ cat rec*big_database1_00001.lz > big_database3_00001.lz @chapter Lziprecover @cindex lziprecover -Lziprecover is a program that searches for members in .lz files, and -writes each member in its own .lz file. You can then use -@w{@samp{lzip -t}} to test the integrity of the resulting files, and -decompress those which are undamaged. +Lziprecover is a data recovery tool for lzip compressed files able to +repair slightly damaged files, recover badly damaged files from two or +more copies, and extract undamaged members from multi-member files. -Data from damaged members can be partially recovered writing it to -stdout as shown in the following example (the resulting file may contain -garbage data at the end): +Lziprecover takes as arguments the names of the damaged files and writes +zero or more recovered files depending on the operation selected and +whether the recovery succeeded or not. The damaged files themselves are +never modified. + +If the files are too damaged for lziprecover to repair them, data from +damaged members can be partially recovered writing it to stdout as shown +in the following example (the resulting file may contain garbage data at +the end): @example lzip -cd rec00001file.lz > rec00001file @end example -Lziprecover takes a single argument, the name of the damaged file, and -writes a number of files @samp{rec00001file.lz}, @samp{rec00002file.lz}, -etc, containing the extracted members. The output filenames are designed -so that the use of wildcards in subsequent processing, for example, -@w{@samp{lzip -dc rec*file.lz > recovered_data}}, processes the files in -the correct order. +If the cause of file corruption is damaged media, the combination GNU +ddrescue + lziprecover is the best option for recovering data from +multiple damaged copies. @xref{ddrescue-example}, for an example. + +@node Invoking Lziprecover +@chapter Invoking Lziprecover +@cindex invoking lziprecover + +The format for running lziprecover is: + +@example +lziprecover [@var{options}] [@var{files}] +@end example + +Lziprecover supports the following options: + +@table @samp +@item --help +@itemx -h +Print an informative help message describing the options and exit. + +@item --version +@itemx -V +Print the version number of lziprecover on the standard output and exit. + +@item --force +@itemx -f +Force overwrite of output file. + +@item --merge +@itemx -m +Try to produce a correct file merging the good parts of two or more +damaged copies. The copies must be single-member files. The merge will +fail if the copies have too many damaged areas or if the same byte is +damaged in all copies. If successful, a repaired copy is written to the +file @samp{@var{file}_fixed.lz}. + +To give you an idea of its possibilities, when merging two copies each +of them with one damaged area affecting 1 percent of the copy, the +probability of obtaining a correct file is about 98 percent. With three +such copies the probability rises to 99.97 percent. For large files with +small errors, the probability approaches 100 percent even with only two +copies. + +@item --output=@var{file} +@itemx -o @var{file} +Place the output into @samp{@var{file}} instead of into +@samp{@var{file}_fixed.lz}. + +If splitting, the names of the files produced are in the form +@samp{rec00001@var{file}}, etc. + +@item --quiet +@itemx -q +Quiet operation. Suppress all messages. + +@item --repair +@itemx -R +Try to repair a small error, affecting only one byte, in a single-member +@var{file}. If successful, a repaired copy is written to the file +@samp{@var{file}_fixed.lz}. @samp{@var{file}} is not modified at all. + +@item --split +@itemx -s +Search for members in @samp{@var{file}} and write each member in its own +@samp{.lz} file. You can then use @samp{lzip -t} to test the integrity +of the resulting files, decompress those which are undamaged, and try to +repair or partially decompress those which are damaged. + +The names of the files produced are in the form +@samp{rec00001@var{file}.lz}, @samp{rec00002@var{file}.lz}, etc, and are +designed so that the use of wildcards in subsequent processing, for +example, @w{@samp{lzip -cd rec*@var{file}.lz > recovered_data}}, +processes the files in the correct order. + +@item --verbose +@itemx -v +Verbose mode. Further -v's increase the verbosity level. + +@end table @node Problems diff --git a/doc/lziprecover.1 b/doc/lziprecover.1 index 0a60e62..95ddb29 100644 --- a/doc/lziprecover.1 +++ b/doc/lziprecover.1 @@ -1,16 +1,12 @@ .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.37.1. -.TH LZIPRECOVER "1" "April 2010" "Lziprecover 1.10" "User Commands" +.TH LZIPRECOVER "1" "September 2010" "Lziprecover 1.11" "User Commands" .SH NAME -Lziprecover \- recover undamaged members from lzip files +Lziprecover \- recovers data from damaged lzip files .SH SYNOPSIS .B lziprecover -[\fIoptions\fR] \fIfile\fR +[\fIoptions\fR] [\fIfiles\fR] .SH DESCRIPTION -Lziprecover \- Member recoverer program for lzip compressed files. -.PP -Searches for members in .lz files, and writes each member in its own .lz -file. You can then use `lzip \fB\-t\fR' to test the integrity of the resulting -files, and decompress those which are undamaged. +Lziprecover \- Data recovery tool for lzip compressed files. .SH OPTIONS .TP \fB\-h\fR, \fB\-\-help\fR @@ -19,9 +15,24 @@ display this help and exit \fB\-V\fR, \fB\-\-version\fR output version information and exit .TP +\fB\-f\fR, \fB\-\-force\fR +overwrite existing output files +.TP +\fB\-m\fR, \fB\-\-merge\fR +correct errors in file using several copies +.TP +\fB\-o\fR, \fB\-\-output=\fR +place the output into +.TP \fB\-q\fR, \fB\-\-quiet\fR suppress all messages .TP +\fB\-R\fR, \fB\-\-repair\fR +try to repair a small error in file +.TP +\fB\-s\fR, \fB\-\-split\fR +split a multimember file in single\-member files +.TP \fB\-v\fR, \fB\-\-verbose\fR be verbose (a 2nd \fB\-v\fR gives more) .SH "REPORTING BUGS" -- cgit v1.2.3