diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/lziprecover.1 | 9 | ||||
| -rw-r--r-- | doc/lziprecover.info | 123 | ||||
| -rw-r--r-- | doc/lziprecover.texi | 84 |
3 files changed, 137 insertions, 79 deletions
diff --git a/doc/lziprecover.1 b/doc/lziprecover.1 index ffd4a6d..7eefb7a 100644 --- a/doc/lziprecover.1 +++ b/doc/lziprecover.1 @@ -1,5 +1,5 @@ .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.46.1. -.TH LZIPRECOVER "1" "August 2014" "lziprecover 1.16" "User Commands" +.TH LZIPRECOVER "1" "October 2014" "lziprecover 1.17-pre1" "User Commands" .SH NAME lziprecover \- recovers data from damaged lzip files .SH SYNOPSIS @@ -7,6 +7,13 @@ lziprecover \- recovers data from damaged lzip files [\fI\,options\/\fR] [\fI\,files\/\fR] .SH DESCRIPTION Lziprecover \- Data recovery tool and decompressor for the lzip format. +Lziprecover can repair perfectly most files with small errors (up to one +single\-byte error per member), without the need of any extra redundance +at all. Losing an entire archive just because of a corrupt byte near the +beginning is a thing of the past. +Lziprecover can also produce a correct file by merging the good parts of +two or more damaged copies, extract data from damaged files, decompress +files and test integrity of files. .SH OPTIONS .TP \fB\-h\fR, \fB\-\-help\fR diff --git a/doc/lziprecover.info b/doc/lziprecover.info index b97224a..6c636e8 100644 --- a/doc/lziprecover.info +++ b/doc/lziprecover.info @@ -12,7 +12,7 @@ File: lziprecover.info, Node: Top, Next: Introduction, Up: (dir) Lziprecover Manual ****************** -This manual is for Lziprecover (version 1.16, 29 August 2014). +This manual is for Lziprecover (version 1.17-pre1, 16 October 2014). * Menu: @@ -20,6 +20,7 @@ This manual is for Lziprecover (version 1.16, 29 August 2014). * Invoking lziprecover:: Command line interface * Repairing files:: Fixing bit-flip and similar errors * Merging files:: Fixing several damaged copies +* File names:: Names of the files produced by lziprecover * File format:: Detailed format of the compressed file * Examples:: A small tutorial with examples * Unzcrash:: Testing the robustness of decompressors @@ -40,11 +41,13 @@ File: lziprecover.info, Node: Introduction, Next: Invoking lziprecover, Prev: Lziprecover is a data recovery tool and decompressor for files in the lzip compressed data format (.lz), able to repair slightly damaged -files, recover badly damaged files from two or more copies, extract data -from damaged files, decompress files and test integrity of files. +files, produce a correct file by merging the good parts of two or more +damaged copies, extract data from damaged files, decompress files and +test integrity of files. - The lzip file format is designed for long-term data archiving, taking -into account both data integrity and decoder availability: + The lzip file format is designed for data sharing and long-term +archiving, taking into account both data integrity and decoder +availability: * The lzip format provides very safe integrity checking and some data recovery means. The lziprecover program can repair bit-flip errors @@ -59,8 +62,8 @@ into account both data integrity and decoder availability: archaeologist to extract the data from a lzip file long after quantum computers eventually render LZMA obsolete. - * Additionally lzip is copylefted, which guarantees that it will - remain free forever. + * Additionally the lzip reference implementation is copylefted, which + guarantees that it will remain free forever. A nice feature of the lzip format is that a corrupt byte is easier to repair the nearer it is from the beginning of the file. Therefore, with @@ -168,11 +171,12 @@ The format for running lziprecover is: '-m' '--merge' - Try to produce a correct file merging the good parts of two or more - damaged copies. If successful, a repaired copy is written to the - file 'FILE_fixed.lz'. The exit status is 0 if a correct file could - be produced, 2 otherwise. See the chapter 'Merging files' (*note - Merging files::) for a complete description of the merge mode. + Try to produce a correct file by merging the good parts of two or + more damaged copies. If successful, a repaired copy is written to + the file 'FILE_fixed.lz'. The exit status is 0 if a correct file + could be produced, 2 otherwise. See the chapter 'Merging files' + (*note Merging files::) for a complete description of the merge + mode. '-o FILE' '--output=FILE' @@ -203,11 +207,11 @@ The format for running lziprecover is: undamaged, and try to repair or partially decompress those which are damaged. - The names of the files produced are in the form 'rec01FILE.lz', - 'rec02FILE.lz', etc, and are designed so that the use of wildcards - in subsequent processing, for example, - 'lziprecover -cd rec*FILE.lz > recovered_data', processes the - files in the correct order. The number of digits used in the names + The names of the files produced are in the form 'rec01FILE', + 'rec02FILE', etc, and are designed so that the use of wildcards in + subsequent processing, for example, + 'lziprecover -cd rec*FILE > recovered_data', processes the files + in the correct order. The number of digits used in the names varies depending on the number of members in 'FILE'. '-t' @@ -253,17 +257,26 @@ File: lziprecover.info, Node: Repairing files, Next: Merging files, Prev: Inv 3 Repairing files ***************** -Lziprecover is usually able to repair files with small errors (up to one -byte error per member). The error may be located anywhere in the file -except in the header (first 6 bytes of each member) or in the 'Member -size' field of the trailer (last 8 bytes of each member). This makes -lzip files resistant to bit-flip, one of the most common forms of data -corruption. +Lziprecover can repair perfectly most files with small errors (up to one +single-byte error per member), without the need of any extra redundance +at all. If the reparation is successful, the repaired file will be +identical bit for bit to the original. + + The error may be located anywhere in the file except in the header +(first 6 bytes of each member) or in the 'Member size' field of the +trailer (last 8 bytes of each member). This makes lzip files resistant +to bit-flip, one of the most common forms of data corruption. Bit-flip happens when one bit in the file is changed from 0 to 1 or vice versa. It may be caused by bad RAM or even by natural radiation. I have seen a case of bit-flip in a file stored on an USB flash drive. + One byte may seem small, but most file corruptions not produced by +I/O errors just affect one byte, or even one bit, of the file. Also, +unlike magnetic media, where errors usually affect a whole sector, +solid-state storage devices tend to produce single-byte errors, making +of lzip the perfect format for data stored on such devices. + Repairing a file can take some time. Small files or files with the error located near the beginning can be repaired in a few seconds. But repairing a large file compressed with a large dictionary size and with @@ -274,14 +287,14 @@ cause much more loss of data than errors located near the end. So lziprecover repairs more efficiently the worst errors. -File: lziprecover.info, Node: Merging files, Next: File format, Prev: Repairing files, Up: Top +File: lziprecover.info, Node: Merging files, Next: File names, Prev: Repairing files, Up: Top 4 Merging files *************** If you have several copies of a file but all of them are too damaged to repair them (*note Repairing files::), lziprecover can try to produce a -correct file merging the good parts of the damaged copies. +correct file by merging the good parts of the damaged copies. The merge may succeed even if some copies of the file have all the headers and trailers damaged, as long as there is at least one copy of @@ -293,14 +306,14 @@ is damaged in all copies), or are adjacent and the boundary can't be determined, or if the copies have too many damaged areas. All the copies must have the same size. If some of them have been -truncated and are therefore smaller than they should, you can extend -them to the correct size with the following command before merging them -with the other copies: +truncated and are therefore smaller than they should, they can be +extended to the correct size with the following command before merging +them with the other copies: ddrescue --extend-outfile=<correct_size> small_file.lz extended_file.lz If some of the copies have got garbage data at the end and are -therefore larger than they should, you can reduce their sizes to the +therefore larger than they should, their sizes can be reduced to the correct value with the following command before merging them with the other copies: @@ -311,12 +324,24 @@ each of them with one damaged area affecting 1 percent of the copy, the probability of obtaining a correct file is about 98 percent. With three such copies the probability rises to 99.97 percent. For large files (a few MB) with small errors (one sector damaged per copy), the probability -approaches 100 percent even with only two copies. +approaches 100 percent even with only two copies. (Supposing that the +errors are randomly located inside each copy). + + +File: lziprecover.info, Node: File names, Next: File format, Prev: Merging files, Up: Top + +5 Names of the files produced by lziprecover +******************************************** + +The name of the fixed file produced by '--merge' and '--repair' is made +by appending the string '_fixed.lz' to the original file name. If the +original file name ends with one of the extensions '.tar.lz', '.lz' or +'.tlz', the string '_fixed' is inserted before the extension. -File: lziprecover.info, Node: File format, Next: Examples, Prev: Merging files, Up: Top +File: lziprecover.info, Node: File format, Next: Examples, Prev: File names, Up: Top -5 File format +6 File format ************* Perfection is reached, not when there is no longer anything to add, but @@ -389,7 +414,7 @@ additional information before, between, or after them. File: lziprecover.info, Node: Examples, Next: Unzcrash, Prev: File format, Up: Top -6 A small tutorial with examples +7 A small tutorial with examples ******************************** Example 1: Restore a regular file from its compressed version @@ -460,7 +485,7 @@ correct file produced is saved in 'big_db_00001.lz'. File: lziprecover.info, Node: Unzcrash, Next: Problems, Prev: Examples, Up: Top -7 Testing the robustness of decompressors +8 Testing the robustness of decompressors ***************************************** The lziprecover package also includes unzcrash, a program written to @@ -476,9 +501,9 @@ memory accesses. If it does, please, report it as a bug. Unzcrash really executes as a subprocess the shell command specified in the first non-option argument, and then writes the file specified in the second non-option argument to the standard input of the subprocess, -modifying the corresponding byte each time. Therefore you can use -unzcrash to test any decompressor (not only lzip), or even other decoder -programs with a suitable command line syntax. +modifying the corresponding byte each time. Therefore unzcrash can be +used to test any decompressor (not only lzip), or even other decoder +programs having a suitable command line syntax. The format for running unzcrash is: @@ -537,7 +562,7 @@ caused unzcrash to panic. File: lziprecover.info, Node: Problems, Next: Concept index, Prev: Unzcrash, Up: Top -8 Reporting bugs +9 Reporting bugs **************** There are probably bugs in lziprecover. There are certainly errors and @@ -561,6 +586,7 @@ Concept index * bugs: Problems. (line 6) * examples: Examples. (line 6) * file format: File format. (line 6) +* file names: File names. (line 6) * getting help: Problems. (line 6) * introduction: Introduction. (line 6) * invoking: Invoking lziprecover. (line 6) @@ -572,16 +598,17 @@ Concept index Tag Table: Node: Top231 -Node: Introduction1077 -Node: Invoking lziprecover4105 -Node: Repairing files9543 -Node: Merging files10733 -Node: File format12504 -Node: Examples15014 -Ref: ddrescue-example16215 -Node: Unzcrash17324 -Node: Problems19876 -Node: Concept index20426 +Node: Introduction1153 +Node: Invoking lziprecover4249 +Node: Repairing files9686 +Node: Merging files11371 +Node: File names13212 +Node: File format13676 +Node: Examples16183 +Ref: ddrescue-example17384 +Node: Unzcrash18493 +Node: Problems21047 +Node: Concept index21597 End Tag Table diff --git a/doc/lziprecover.texi b/doc/lziprecover.texi index 80d6eb4..08d4312 100644 --- a/doc/lziprecover.texi +++ b/doc/lziprecover.texi @@ -6,8 +6,8 @@ @finalout @c %**end of header -@set UPDATED 29 August 2014 -@set VERSION 1.16 +@set UPDATED 16 October 2014 +@set VERSION 1.17-pre1 @dircategory Data Compression @direntry @@ -39,6 +39,7 @@ This manual is for Lziprecover (version @value{VERSION}, @value{UPDATED}). * Invoking lziprecover:: Command line interface * Repairing files:: Fixing bit-flip and similar errors * Merging files:: Fixing several damaged copies +* File names:: Names of the files produced by lziprecover * File format:: Detailed format of the compressed file * Examples:: A small tutorial with examples * Unzcrash:: Testing the robustness of decompressors @@ -59,11 +60,13 @@ to copy, distribute and modify it. Lziprecover is a data recovery tool and decompressor for files in the lzip compressed data format (.lz), able to repair slightly damaged -files, recover badly damaged files from two or more copies, extract data -from damaged files, decompress files and test integrity of files. +files, produce a correct file by merging the good parts of two or more +damaged copies, extract data from damaged files, decompress files and +test integrity of files. -The lzip file format is designed for long-term data archiving, taking -into account both data integrity and decoder availability: +The lzip file format is designed for data sharing and long-term +archiving, taking into account both data integrity and decoder +availability: @itemize @bullet @item @@ -82,8 +85,8 @@ data from a lzip file long after quantum computers eventually render LZMA obsolete. @item -Additionally lzip is copylefted, which guarantees that it will remain -free forever. +Additionally the lzip reference implementation is copylefted, which +guarantees that it will remain free forever. @end itemize A nice feature of the lzip format is that a corrupt byte is easier to @@ -196,7 +199,7 @@ information about the members in the file. @item -m @itemx --merge -Try to produce a correct file merging the good parts of two or more +Try to produce a correct file by merging the good parts of two or more damaged copies. If successful, a repaired copy is written to the file @samp{@var{file}_fixed.lz}. The exit status is 0 if a correct file could be produced, 2 otherwise. See the chapter @samp{Merging files} @@ -231,12 +234,12 @@ Search for members in @samp{@var{file}} and write each member in its own integrity of the resulting files, decompress those which are undamaged, and try to repair or partially decompress those which are damaged. -The names of the files produced are in the form -@samp{rec01@var{file}.lz}, @samp{rec02@var{file}.lz}, etc, and are -designed so that the use of wildcards in subsequent processing, for -example, @w{@samp{lziprecover -cd rec*@var{file}.lz > recovered_data}}, -processes the files in the correct order. The number of digits used in -the names varies depending on the number of members in @samp{@var{file}}. +The names of the files produced are in the form @samp{rec01@var{file}}, +@samp{rec02@var{file}}, etc, and are designed so that the use of +wildcards in subsequent processing, for example, @w{@samp{lziprecover +-cd rec*@var{file} > recovered_data}}, processes the files in the +correct order. The number of digits used in the names varies depending +on the number of members in @samp{@var{file}}. @item -t @itemx --test @@ -282,17 +285,26 @@ caused lziprecover to panic. @chapter Repairing files @cindex repairing files -Lziprecover is usually able to repair files with small errors (up to one -byte error per member). The error may be located anywhere in the file -except in the header (first 6 bytes of each member) or in the -@samp{Member size} field of the trailer (last 8 bytes of each member). -This makes lzip files resistant to bit-flip, one of the most common -forms of data corruption. +Lziprecover can repair perfectly most files with small errors (up to one +single-byte error per member), without the need of any extra redundance +at all. If the reparation is successful, the repaired file will be +identical bit for bit to the original. + +The error may be located anywhere in the file except in the header +(first 6 bytes of each member) or in the @samp{Member size} field of the +trailer (last 8 bytes of each member). This makes lzip files resistant +to bit-flip, one of the most common forms of data corruption. Bit-flip happens when one bit in the file is changed from 0 to 1 or vice versa. It may be caused by bad RAM or even by natural radiation. I have seen a case of bit-flip in a file stored on an USB flash drive. +One byte may seem small, but most file corruptions not produced by I/O +errors just affect one byte, or even one bit, of the file. Also, unlike +magnetic media, where errors usually affect a whole sector, solid-state +storage devices tend to produce single-byte errors, making of lzip the +perfect format for data stored on such devices. + Repairing a file can take some time. Small files or files with the error located near the beginning can be repaired in a few seconds. But repairing a large file compressed with a large dictionary size and with @@ -309,7 +321,7 @@ repairs more efficiently the worst errors. If you have several copies of a file but all of them are too damaged to repair them (@pxref{Repairing files}), lziprecover can try to produce a -correct file merging the good parts of the damaged copies. +correct file by merging the good parts of the damaged copies. The merge may succeed even if some copies of the file have all the headers and trailers damaged, as long as there is at least one copy of @@ -321,16 +333,16 @@ damaged in all copies), or are adjacent and the boundary can't be determined, or if the copies have too many damaged areas. All the copies must have the same size. If some of them have been -truncated and are therefore smaller than they should, you can extend -them to the correct size with the following command before merging them -with the other copies: +truncated and are therefore smaller than they should, they can be +extended to the correct size with the following command before merging +them with the other copies: @example ddrescue --extend-outfile=<correct_size> small_file.lz extended_file.lz @end example If some of the copies have got garbage data at the end and are therefore -larger than they should, you can reduce their sizes to the correct value +larger than they should, their sizes can be reduced to the correct value with the following command before merging them with the other copies: @example @@ -342,7 +354,19 @@ of them with one damaged area affecting 1 percent of the copy, the probability of obtaining a correct file is about 98 percent. With three such copies the probability rises to 99.97 percent. For large files (a few MB) with small errors (one sector damaged per copy), the probability -approaches 100 percent even with only two copies. +approaches 100 percent even with only two copies. (Supposing that the +errors are randomly located inside each copy). + + +@node File names +@chapter Names of the files produced by lziprecover +@cindex file names + +The name of the fixed file produced by @samp{--merge} and +@samp{--repair} is made by appending the string @samp{_fixed.lz} to the +original file name. If the original file name ends with one of the +extensions @samp{.tar.lz}, @samp{.lz} or @samp{.tlz}, the string +@samp{_fixed} is inserted before the extension. @node File format @@ -541,9 +565,9 @@ accesses. If it does, please, report it as a bug. Unzcrash really executes as a subprocess the shell command specified in the first non-option argument, and then writes the file specified in the second non-option argument to the standard input of the subprocess, -modifying the corresponding byte each time. Therefore you can use -unzcrash to test any decompressor (not only lzip), or even other decoder -programs with a suitable command line syntax. +modifying the corresponding byte each time. Therefore unzcrash can be +used to test any decompressor (not only lzip), or even other decoder +programs having a suitable command line syntax. The format for running unzcrash is: |