Merging upstream version 1.18~pre2.

Signed-off-by: Daniel Baumann <mail@daniel-baumann.ch>

3 files changed, 179 insertions, 56 deletions

diff --git a/doc/lziprecover.1 b/doc/lziprecover.1
index 99b61dd..87c0598 100644
--- a/doc/lziprecover.1
+++ b/doc/lziprecover.1
@@ -1,5 +1,5 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.46.1.
-.TH LZIPRECOVER "1" "June 2015" "lziprecover 1.18-pre1" "User Commands"
+.TH LZIPRECOVER "1" "September 2015" "lziprecover 1.18-pre2" "User Commands"
 .SH NAME
 lziprecover \- recovers data from damaged lzip files
 .SH SYNOPSIS
@@ -23,6 +23,9 @@ display this help and exit
 \fB\-V\fR, \fB\-\-version\fR
 output version information and exit
 .TP
+\fB\-a\fR, \fB\-\-trailing\-error\fR
+exit with error status if trailing data
+.TP
 \fB\-c\fR, \fB\-\-stdout\fR
 send decompressed output to standard output
 .TP
diff --git a/doc/lziprecover.info b/doc/lziprecover.info
index 654e60c..8d7bc66 100644
--- a/doc/lziprecover.info
+++ b/doc/lziprecover.info
@@ -12,7 +12,7 @@ File: lziprecover.info,  Node: Top,  Next: Introduction,  Up: (dir)
 Lziprecover Manual
 ******************
 
-This manual is for Lziprecover (version 1.18-pre1, 30 June 2015).
+This manual is for Lziprecover (version 1.18-pre2, 16 September 2015).
 
 * Menu:
 
@@ -23,6 +23,7 @@ This manual is for Lziprecover (version 1.18-pre1, 30 June 2015).
 * Merging files::          Fixing several damaged copies
 * File names::             Names of the files produced by lziprecover
 * File format::            Detailed format of the compressed file
+* Trailing data::          Extra data appended to the file
 * Examples::               A small tutorial with examples
 * Unzcrash::               Testing the robustness of decompressors
 * Problems::               Reporting bugs
@@ -54,7 +55,7 @@ availability:
      recovery means. The lziprecover program can repair bit-flip errors
      (one of the most common forms of data corruption) in lzip files,
      and provides data recovery capabilities, including error-checked
-     merging of damaged copies of a file.
+     merging of damaged copies of a file. *Note Data safety::.
 
    * The lzip format is as simple as possible (but not simpler). The
      lzip manual provides the code of a simple decompressor along with
@@ -125,6 +126,13 @@ The format for running lziprecover is:
      Print the version number of lziprecover on the standard output and
      exit.
 
+'-a'
+'--trailing-error'
+     Exit with error status 2 if any remaining input is detected after
+     decompressing the last member. Such remaining input is usually
+     trailing garbage that can be safely ignored. *Note
+     concat-example::.
+
 '-c'
 '--stdout'
      Decompress to standard output. Needed when reading from a named
@@ -133,7 +141,9 @@ The format for running lziprecover is:
 
 '-d'
 '--decompress'
-     Decompress.
+     Decompress the specified file(s). If a file fails to decompress,
+     lziprecover exits immediately without decompressing the rest of the
+     files.
 
 '-D RANGE'
 '--range-decompress=RANGE'
@@ -220,7 +230,8 @@ The format for running lziprecover is:
      Check integrity of the specified file(s), but don't decompress
      them.  This really performs a trial decompression and throws away
      the result.  Use it together with '-v' to see information about
-     the file.
+     the file(s). If a file fails the test, lziprecover continues
+     checking the rest of the files.
 
 '-v'
 '--verbose'
@@ -228,7 +239,7 @@ The format for running lziprecover is:
      When decompressing or testing, further -v's (up to 4) increase the
      verbosity level, showing status, compression ratio, dictionary
      size, trailer contents (CRC, data size, member size), and up to 6
-     bytes of trailing garbage (if any).
+     bytes of trailing data (if any).
 
 
    Numbers given as arguments to options may be followed by a multiplier
@@ -387,7 +398,7 @@ original file name ends with one of the extensions '.tar.lz', '.lz' or
 '.tlz', the string '_fixed' is inserted before the extension.
 
 
-File: lziprecover.info,  Node: File format,  Next: Examples,  Prev: File names,  Up: Top
+File: lziprecover.info,  Node: File format,  Next: Trailing data,  Prev: File names,  Up: Top
 
 7 File format
 *************
@@ -421,7 +432,7 @@ additional information before, between, or after them.
 
    All multibyte values are stored in little endian order.
 
-'ID string'
+'ID string (the "magic" bytes)'
      A four byte string, identifying the lzip format, with the value
      "LZIP" (0x4C, 0x5A, 0x49, 0x50).
 
@@ -458,9 +469,42 @@ additional information before, between, or after them.
 
 
 
-File: lziprecover.info,  Node: Examples,  Next: Unzcrash,  Prev: File format,  Up: Top
+File: lziprecover.info,  Node: Trailing data,  Next: Examples,  Prev: File format,  Up: Top
+
+8 Extra data appended to the file
+*********************************
+
+Sometimes extra data is found appended to a lzip file after the last
+member. Such trailing data may be:
+
+   * Padding added to make the file size a multiple of some block size,
+     for example when writing to a tape.
+
+   * Garbage added by some not totally successful copy operation.
+
+   * Useful data added by the user; a cryptographically secure hash, a
+     description of file contents, etc.
+
+   * Malicious data added to the file in order to make its total size
+     and hash value (for a chosen hash) coincide with those of another
+     file.
+
+   * In very rare cases, trailing data could be the corrupt header of
+     another member. In multi-member or concatenated files the
+     probability of corruption happening in the magic bytes is 5 times
+     smaller than the probability of getting a false positive caused by
+     the corruption of the integrity information itself. Therefore it
+     can be considered to be below the noise level.
+
+   Trailing data can be safely ignored in most cases. In some cases,
+like user-added data, it is expected to be ignored. In those cases
+where a file containing trailing data must be rejected, the option
+'--trailing-error' can be used. *Note --trailing-error::.
+
+
+File: lziprecover.info,  Node: Examples,  Next: Unzcrash,  Prev: Trailing data,  Up: Top
 
-8 A small tutorial with examples
+9 A small tutorial with examples
 ********************************
 
 Example 1: Restore a regular file from its compressed version
@@ -475,29 +519,38 @@ show status.
      lziprecover -tv file.lz
 
 
-Example 3: Decompress 'file.lz' partially until 10 KiB of decompressed
+Example 3: The right way of concatenating compressed files.  *Note
+Trailing data::.
+
+     Don't do this
+       cat file1.lz file2.lz file3.lz | lziprecover -d
+     Do this instead
+       lziprecover -cd file1.lz file2.lz file3.lz
+
+
+Example 4: Decompress 'file.lz' partially until 10 KiB of decompressed
 data are produced.
 
      lziprecover -D 0,10KiB file.lz
 
 
-Example 4: Decompress 'file.lz' partially from decompressed byte 10000
+Example 5: Decompress 'file.lz' partially from decompressed byte 10000
 to decompressed byte 15000 (5000 bytes are produced).
 
      lziprecover -D 10000-15000 file.lz
 
 
-Example 5: Repair small errors in the file 'file.lz'. (Indented lines
+Example 6: Repair small errors in the file 'file.lz'. (Indented lines
 are abridged diagnostic messages from lziprecover).
 
      lziprecover -v -R file.lz
        Copy of input file repaired successfully.
      lziprecover -tv file_fixed.lz
-       ok
+       file_fixed.lz: ok
      mv file_fixed.lz file.lz
 
 
-Example 6: Split the multi-member file 'file.lz' and write each member
+Example 7: Split the multi-member file 'file.lz' and write each member
 in its own 'recXXXfile.lz' file. Then use 'lziprecover -t' to test the
 integrity of the resulting files.
 
@@ -505,26 +558,26 @@ integrity of the resulting files.
      lziprecover -tv rec*file.lz
 
 
-Example 7: Recover a compressed backup from two copies on CD-ROM with
-error-checked merging of copies (*Note GNU ddrescue manual:
+Example 8: Recover a compressed backup from two copies on CD-ROM with
+error-checked merging of copies.  (*Note GNU ddrescue manual:
 (ddrescue)Top, for details about ddrescue).
 
-     ddrescue -d -r1 -b2048 /dev/cdrom cdimage1 logfile1
+     ddrescue -d -r1 -b2048 /dev/cdrom cdimage1 mapfile1
      mount -t iso9660 -o loop,ro cdimage1 /mnt/cdimage
      cp /mnt/cdimage/backup.tar.lz rescued1.tar.lz
      umount /mnt/cdimage
        (insert second copy in the CD drive)
-     ddrescue -d -r1 -b2048 /dev/cdrom cdimage2 logfile2
+     ddrescue -d -r1 -b2048 /dev/cdrom cdimage2 mapfile2
      mount -t iso9660 -o loop,ro cdimage2 /mnt/cdimage
      cp /mnt/cdimage/backup.tar.lz rescued2.tar.lz
      umount /mnt/cdimage
      lziprecover -m -v -o backup.tar.lz rescued1.tar.lz rescued2.tar.lz
        Input files merged successfully.
      lziprecover -tv backup.tar.lz
-       ok
+       backup.tar.lz: ok
 
 
-Example 8: Recover the first volume of those created with the command
+Example 9: Recover the first volume of those created with the command
 'lzip -b 32MiB -S 650MB big_db' from two copies, 'big_db1_00001.lz' and
 'big_db2_00001.lz', with member 07 damaged in the first copy, member 18
 damaged in the second copy, and member 12 damaged in both copies. The
@@ -533,13 +586,13 @@ correct file produced is saved in 'big_db_00001.lz'.
      lziprecover -m -v -o big_db_00001.lz big_db1_00001.lz big_db2_00001.lz
        Input files merged successfully.
      lziprecover -tv big_db_00001.lz
-       ok
+       big_db_00001.lz: ok
 
 
 File: lziprecover.info,  Node: Unzcrash,  Next: Problems,  Prev: Examples,  Up: Top
 
-9 Testing the robustness of decompressors
-*****************************************
+10 Testing the robustness of decompressors
+******************************************
 
 The lziprecover package also includes unzcrash, a program written to
 test robustness to decompression of corrupted data, inspired by
@@ -615,7 +668,7 @@ caused unzcrash to panic.
 
 File: lziprecover.info,  Node: Problems,  Next: Concept index,  Prev: Unzcrash,  Up: Top
 
-10 Reporting bugs
+11 Reporting bugs
 *****************
 
 There are probably bugs in lziprecover. There are certainly errors and
@@ -646,24 +699,28 @@ Concept index
 * invoking:                              Invoking lziprecover.  (line 6)
 * merging files:                         Merging files.         (line 6)
 * repairing files:                       Repairing files.       (line 6)
+* trailing data:                         Trailing data.         (line 6)
 * unzcrash:                              Unzcrash.              (line 6)
 
 
 
 Tag Table:
 Node: Top231
-Node: Introduction1214
-Node: Invoking lziprecover4310
-Node: Data safety9743
-Node: Repairing files11667
-Node: Merging files13569
-Node: File names15410
-Node: File format15874
-Node: Examples18278
-Ref: ddrescue-example19524
-Node: Unzcrash20780
-Node: Problems23334
-Node: Concept index23886
+Node: Introduction1278
+Node: Invoking lziprecover4395
+Ref: --trailing-error4860
+Node: Data safety10294
+Node: Repairing files12218
+Node: Merging files14120
+Node: File names15961
+Node: File format16425
+Node: Trailing data18854
+Node: Examples20230
+Ref: concat-example20661
+Ref: ddrescue-example21725
+Node: Unzcrash23015
+Node: Problems25571
+Node: Concept index26123
 
 End Tag Table
 
diff --git a/doc/lziprecover.texi b/doc/lziprecover.texi
index 29045e7..e29a59f 100644
--- a/doc/lziprecover.texi
+++ b/doc/lziprecover.texi
@@ -6,8 +6,8 @@
 @finalout
 @c %**end of header
 
-@set UPDATED 30 June 2015
-@set VERSION 1.18-pre1
+@set UPDATED 16 September 2015
+@set VERSION 1.18-pre2
 
 @dircategory Data Compression
 @direntry
@@ -42,6 +42,7 @@ This manual is for Lziprecover (version @value{VERSION}, @value{UPDATED}).
 * Merging files::          Fixing several damaged copies
 * File names::             Names of the files produced by lziprecover
 * File format::            Detailed format of the compressed file
+* Trailing data::          Extra data appended to the file
 * Examples::               A small tutorial with examples
 * Unzcrash::               Testing the robustness of decompressors
 * Problems::               Reporting bugs
@@ -75,7 +76,7 @@ The lzip format provides very safe integrity checking and some data
 recovery means. The lziprecover program can repair bit-flip errors (one
 of the most common forms of data corruption) in lzip files, and provides
 data recovery capabilities, including error-checked merging of damaged
-copies of a file.
+copies of a file. @xref{Data safety}.
 
 @item
 The lzip format is as simple as possible (but not simpler). The lzip
@@ -152,6 +153,13 @@ Print an informative help message describing the options and exit.
 @itemx --version
 Print the version number of lziprecover on the standard output and exit.
 
+@anchor{--trailing-error}
+@item -a
+@itemx --trailing-error
+Exit with error status 2 if any remaining input is detected after
+decompressing the last member. Such remaining input is usually trailing
+garbage that can be safely ignored. @xref{concat-example}.
+
 @item -c
 @itemx --stdout
 Decompress to standard output. Needed when reading from a named pipe
@@ -160,7 +168,9 @@ data as possible when decompressing a corrupt file.
 
 @item -d
 @itemx --decompress
-Decompress.
+Decompress the specified file(s). If a file fails to decompress,
+lziprecover exits immediately without decompressing the rest of the
+files.
 
 @item -D @var{range}
 @itemx --range-decompress=@var{range}
@@ -246,7 +256,9 @@ on the number of members in @samp{@var{file}}.
 @itemx --test
 Check integrity of the specified file(s), but don't decompress them.
 This really performs a trial decompression and throws away the result.
-Use it together with @samp{-v} to see information about the file.
+Use it together with @samp{-v} to see information about the file(s). If
+a file fails the test, lziprecover continues checking the rest of the
+files.
 
 @item -v
 @itemx --verbose
@@ -254,7 +266,7 @@ Verbose mode.@*
 When decompressing or testing, further -v's (up to 4) increase the
 verbosity level, showing status, compression ratio, dictionary size,
 trailer contents (CRC, data size, member size), and up to 6 bytes of
-trailing garbage (if any).
+trailing data (if any).
 
 @end table
 
@@ -456,7 +468,7 @@ Each member has the following structure:
 All multibyte values are stored in little endian order.
 
 @table @samp
-@item ID string
+@item ID string (the "magic" bytes)
 A four byte string, identifying the lzip format, with the value "LZIP"
 (0x4C, 0x5A, 0x49, 0x50).
 
@@ -499,6 +511,44 @@ facilitates safe recovery of undamaged members from multi-member files.
 @end table
 
 
+@node Trailing data
+@chapter Extra data appended to the file
+@cindex trailing data
+
+Sometimes extra data is found appended to a lzip file after the last
+member. Such trailing data may be:
+
+@itemize @bullet
+@item
+Padding added to make the file size a multiple of some block size, for
+example when writing to a tape.
+
+@item
+Garbage added by some not totally successful copy operation.
+
+@item
+Useful data added by the user; a cryptographically secure hash, a
+description of file contents, etc.
+
+@item
+Malicious data added to the file in order to make its total size and
+hash value (for a chosen hash) coincide with those of another file.
+
+@item
+In very rare cases, trailing data could be the corrupt header of another
+member. In multi-member or concatenated files the probability of
+corruption happening in the magic bytes is 5 times smaller than the
+probability of getting a false positive caused by the corruption of the
+integrity information itself. Therefore it can be considered to be below
+the noise level.
+@end itemize
+
+Trailing data can be safely ignored in most cases. In some cases, like
+user-added data, it is expected to be ignored. In those cases where a
+file containing trailing data must be rejected, the option
+@samp{--trailing-error} can be used. @xref{--trailing-error}.
+
+
 @node Examples
 @chapter A small tutorial with examples
 @cindex examples
@@ -521,8 +571,21 @@ lziprecover -tv file.lz
 @end example
 
 @sp 1
+@anchor{concat-example}
+@noindent
+Example 3: The right way of concatenating compressed files.
+@xref{Trailing data}.
+
+@example
+Don't do this
+  cat file1.lz file2.lz file3.lz | lziprecover -d
+Do this instead
+  lziprecover -cd file1.lz file2.lz file3.lz
+@end example
+
+@sp 1
 @noindent
-Example 3: Decompress @samp{file.lz} partially until 10 KiB of
+Example 4: Decompress @samp{file.lz} partially until 10 KiB of
 decompressed data are produced.
 
 @example
@@ -531,7 +594,7 @@ lziprecover -D 0,10KiB file.lz
 
 @sp 1
 @noindent
-Example 4: Decompress @samp{file.lz} partially from decompressed byte
+Example 5: Decompress @samp{file.lz} partially from decompressed byte
 10000 to decompressed byte 15000 (5000 bytes are produced).
 
 @example
@@ -540,20 +603,20 @@ lziprecover -D 10000-15000 file.lz
 
 @sp 1
 @noindent
-Example 5: Repair small errors in the file @samp{file.lz}. (Indented
+Example 6: Repair small errors in the file @samp{file.lz}. (Indented
 lines are abridged diagnostic messages from lziprecover).
 
 @example
 lziprecover -v -R file.lz
   Copy of input file repaired successfully.
 lziprecover -tv file_fixed.lz
-  ok
+  file_fixed.lz: ok
 mv file_fixed.lz file.lz
 @end example
 
 @sp 1
 @noindent
-Example 6: Split the multi-member file @samp{file.lz} and write each
+Example 7: Split the multi-member file @samp{file.lz} and write each
 member in its own @samp{recXXXfile.lz} file. Then use
 @w{@samp{lziprecover -t}} to test the integrity of the resulting files.
 
@@ -565,8 +628,8 @@ lziprecover -tv rec*file.lz
 @sp 1
 @anchor{ddrescue-example}
 @noindent
-Example 7: Recover a compressed backup from two copies on CD-ROM with
-error-checked merging of copies
+Example 8: Recover a compressed backup from two copies on CD-ROM with
+error-checked merging of copies.
 @ifnothtml
 (@xref{Top,GNU ddrescue manual,,ddrescue},
 @end ifnothtml
@@ -577,24 +640,24 @@ error-checked merging of copies
 for details about ddrescue).
 
 @example
-ddrescue -d -r1 -b2048 /dev/cdrom cdimage1 logfile1
+ddrescue -d -r1 -b2048 /dev/cdrom cdimage1 mapfile1
 mount -t iso9660 -o loop,ro cdimage1 /mnt/cdimage
 cp /mnt/cdimage/backup.tar.lz rescued1.tar.lz
 umount /mnt/cdimage
   (insert second copy in the CD drive)
-ddrescue -d -r1 -b2048 /dev/cdrom cdimage2 logfile2
+ddrescue -d -r1 -b2048 /dev/cdrom cdimage2 mapfile2
 mount -t iso9660 -o loop,ro cdimage2 /mnt/cdimage
 cp /mnt/cdimage/backup.tar.lz rescued2.tar.lz
 umount /mnt/cdimage
 lziprecover -m -v -o backup.tar.lz rescued1.tar.lz rescued2.tar.lz
   Input files merged successfully.
 lziprecover -tv backup.tar.lz
-  ok
+  backup.tar.lz: ok
 @end example
 
 @sp 1
 @noindent
-Example 8: Recover the first volume of those created with the command
+Example 9: Recover the first volume of those created with the command
 @w{@samp{lzip -b 32MiB -S 650MB big_db}} from two copies,
 @samp{big_db1_00001.lz} and @samp{big_db2_00001.lz}, with member 07
 damaged in the first copy, member 18 damaged in the second copy, and
@@ -605,7 +668,7 @@ member 12 damaged in both copies. The correct file produced is saved in
 lziprecover -m -v -o big_db_00001.lz big_db1_00001.lz big_db2_00001.lz
   Input files merged successfully.
 lziprecover -tv big_db_00001.lz
-  ok
+  big_db_00001.lz: ok
 @end example