Adding upstream version 0.21.upstream/0.21

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

3 files changed, 212 insertions, 73 deletions

diff --git a/doc/tarlz.1 b/doc/tarlz.1
index e2ed3de..4df1fa1 100644
--- a/doc/tarlz.1
+++ b/doc/tarlz.1
@@ -1,5 +1,5 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.47.16.
-.TH TARLZ "1" "January 2021" "tarlz 0.19" "User Commands"
+.TH TARLZ "1" "June 2021" "tarlz 0.21" "User Commands"
 .SH NAME
 tarlz \- creates tar archives with multimember lzip compression
 .SH SYNOPSIS
@@ -72,6 +72,9 @@ use <date> as mtime for files added to archive
 \fB\-n\fR, \fB\-\-threads=\fR<n>
 set number of (de)compression threads [2]
 .TP
+\fB\-o\fR, \fB\-\-output=\fR<file>
+compress to <file>
+.TP
 \fB\-p\fR, \fB\-\-preserve\-permissions\fR
 don't subtract the umask on extraction
 .TP
@@ -90,6 +93,9 @@ verbosely list files processed
 \fB\-x\fR, \fB\-\-extract\fR
 extract files/directories from an archive
 .TP
+\fB\-z\fR, \fB\-\-compress\fR
+compress existing POSIX tar archives
+.TP
 \fB\-0\fR .. \fB\-9\fR
 set compression level [default 6]
 .TP
@@ -131,10 +137,13 @@ number of 1 MiB output packets buffered [64]
 .TP
 \fB\-\-check\-lib\fR
 compare version of lzlib.h with liblz.{a,so}
+.TP
+\fB\-\-warn\-newer\fR
+warn if any file is newer than the archive
 .PP
 Exit status: 0 for a normal exit, 1 for environmental problems (file not
 found, files differ, invalid flags, I/O errors, etc), 2 to indicate a
-corrupt or invalid input file, 3 for an internal consistency error (eg, bug)
+corrupt or invalid input file, 3 for an internal consistency error (e.g. bug)
 which caused tarlz to panic.
 .SH "REPORTING BUGS"
 Report bugs to lzip\-bug@nongnu.org
diff --git a/doc/tarlz.info b/doc/tarlz.info
index d287697..c797f0b 100644
--- a/doc/tarlz.info
+++ b/doc/tarlz.info
@@ -11,7 +11,7 @@ File: tarlz.info,  Node: Top,  Next: Introduction,  Up: (dir)
 Tarlz Manual
 ************
 
-This manual is for Tarlz (version 0.19, 8 January 2021).
+This manual is for Tarlz (version 0.21, 14 June 2021).
 
 * Menu:
 
@@ -102,7 +102,7 @@ The format for running tarlz is:
      tarlz [OPTIONS] [FILES]
 
 All operations except '--concatenate' operate on whole trees if any FILE is
-a directory.
+a directory. Tarlz overwrites output files without warning.
 
    On archive creation or appending tarlz archives the files specified, but
 removes from member names any leading and trailing slashes and any file name
@@ -119,7 +119,7 @@ member names in the archive or given in the command line, so that
 
    If several compression levels or '--*solid' options are given, the last
 setting is used. For example '-9 --solid --uncompressed -1' is equivalent
-to '-1 --solid'
+to '-1 --solid'.
 
    tarlz supports the following options: *Note Argument syntax:
 (arg_parser)Argument syntax.
@@ -134,16 +134,17 @@ to '-1 --solid'
 
 '-A'
 '--concatenate'
-     Append one or more archives to the end of an archive. All the archives
-     involved must be regular (seekable) files, and must be either all
-     compressed or all uncompressed. Compressed and uncompressed archives
-     can't be mixed. Compressed archives must be multimember lzip files
-     with the two end-of-file blocks plus any zero padding contained in the
-     last lzip member of each archive. The intermediate end-of-file blocks
-     are removed as each new archive is concatenated. If the archive is
-     uncompressed, tarlz parses and skips tar headers until it finds the
-     end-of-file blocks. Exit with status 0 without modifying the archive
-     if no FILES have been specified.
+     Append one or more archives to the end of an archive. If no archive is
+     specified with the option '-f', the input archives are concatenated to
+     standard output. All the archives involved must be regular (seekable)
+     files, and must be either all compressed or all uncompressed.
+     Compressed and uncompressed archives can't be mixed. Compressed
+     archives must be multimember lzip files with the two end-of-file
+     blocks plus any zero padding contained in the last lzip member of each
+     archive. The intermediate end-of-file blocks are removed as each new
+     archive is concatenated. If the archive is uncompressed, tarlz parses
+     and skips tar headers until it finds the end-of-file blocks. Exit with
+     status 0 without modifying the archive if no FILES have been specified.
 
 '-B BYTES'
 '--data-size=BYTES'
@@ -249,6 +250,12 @@ to '-1 --solid'
      and during decompression to the number of lzip members in the tar.lz
      archive, which you can find by running 'lzip -lv archive.tar.lz'.
 
+'-o FILE'
+'--output=FILE'
+     Write the compressed output to FILE. '-o -' writes the compressed
+     output to standard output. Currently '--output' only works with
+     '--compress'.
+
 '-p'
 '--preserve-permissions'
      On extraction, set file permissions as they appear in the archive.
@@ -295,11 +302,35 @@ to '-1 --solid'
      example, extracting a link over a directory will usually fail.
      (Principle of least surprise).
 
+'-z'
+'--compress'
+     Compress existing POSIX tar archives aligning the lzip members to the
+     tar members with choice of granularity (--bsolid by default, --dsolid
+     works like --asolid). The input archives are kept unchanged. Existing
+     compressed archives are not overwritten. A hyphen '-' used as the name
+     of an input archive reads from standard input and writes to standard
+     output (unless the option '--output' is used). Tarlz can be used as
+     compressor for GNU tar using a command like
+     'tar -c -Hustar foo | tarlz -z -o foo.tar.lz'. Note that tarlz only
+     works reliably on archives without global headers, or with global
+     headers whose content can be ignored.
+
+     The compression is reversible, including any garbage present after the
+     EOF blocks. Tarlz stops parsing after the first EOF block is found,
+     and then compresses the rest of the archive. Unless solid compression
+     is requested, the EOF blocks are compressed in a lzip member separated
+     from the preceding members and from any non-zero garbage following the
+     EOF blocks. '--compress' implies plzip argument style, not tar style.
+     Each input archive is compressed to a file with the extension '.lz'
+     added unless the option '--output' is used. When '--output' is used,
+     only one input archive can be specified. '-f' can't be used with
+     '--compress'.
+
 '-0 .. -9'
-     Set the compression level for '--create' and '--append'. The default
-     compression level is '-6'. Like lzip, tarlz also minimizes the
-     dictionary size of the lzip members it creates, reducing the amount of
-     memory required for decompression.
+     Set the compression level for '--create', '--append', and
+     '--compress'. The default compression level is '-6'. Like lzip, tarlz
+     also minimizes the dictionary size of the lzip members it creates,
+     reducing the amount of memory required for decompression.
 
      Level   Dictionary size   Match length limit
      -0      64 KiB            16 bytes
@@ -408,11 +439,20 @@ to '-1 --solid'
      the version of lzlib being used and the value of 'LZ_API_VERSION' (if
      defined). *Note Library version: (lzlib)Library version.
 
+'--warn-newer'
+     During archive creation, warn if any file being archived has a
+     modification time newer than the archive creation time. This option
+     may slow archive creation somewhat because it makes an extra call to
+     'stat' after archiving each file, but it guarantees that file contents
+     were not modified during the creation of the archive. Note that the
+     file must be at least one second newer than the archive for it to be
+     detected as newer.
+
 
    Exit status: 0 for a normal exit, 1 for environmental problems (file not
 found, files differ, invalid flags, I/O errors, etc), 2 to indicate a
-corrupt or invalid input file, 3 for an internal consistency error (eg, bug)
-which caused tarlz to panic.
+corrupt or invalid input file, 3 for an internal consistency error (e.g.
+bug) which caused tarlz to panic.
 
 
 File: tarlz.info,  Node: Portable character set,  Next: File format,  Prev: Invoking tarlz,  Up: Top
@@ -430,7 +470,9 @@ The set of characters from which portable file names are constructed.
 characters, respectively.
 
    File names are identifiers. Therefore, archiving works better when file
-names use only the portable character set without spaces added.
+names use only the portable character set without spaces added. Unicode is
+for human consumption. It should be avoided in computing environments,
+specially in file names. *Note why not Unicode: (moe)why not Unicode.
 
 
 File: tarlz.info,  Node: File format,  Next: Amendments to pax format,  Prev: Portable character set,  Up: Top
@@ -718,8 +760,9 @@ integrity checking of lzip may not be able to detect the corruption before
 the metadata has been used, for example, to create a new file in the wrong
 place.
 
-   Because of the above, tarlz protects the extended records with a CRC in a
-way compatible with standard tar tools. *Note key_crc32::.
+   Because of the above, tarlz protects the extended records with a Cyclic
+Redundancy Check (CRC) in a way compatible with standard tar tools. *Note
+key_crc32::.
 
 
 5.2 Remove flawed backward compatibility
@@ -740,7 +783,9 @@ to extract a file with a truncated file name.
 zeroed except size, chksum, typeflag, magic and version. This prevents old
 tar programs from extracting the extended records as a file in the wrong
 place. Tarlz also sets to zero those fields of the ustar header overridden
-by extended records.
+by extended records. Finally, tarlz skips members without name when decoding
+except when listing. This is needed to detect certain format violations
+during parallel extraction.
 
    If an extended header is required for any reason (for example a file size
 larger than 8 GiB or a link name longer than 100 bytes), tarlz moves the
@@ -860,11 +905,20 @@ File: tarlz.info,  Node: Multi-threaded decoding,  Next: Minimum archive sizes,
 7 Limitations of parallel tar decoding
 **************************************
 
-Safely decoding an arbitrary tar archive in parallel is impossible. For
-example, if a tar archive containing another tar archive is decoded starting
-from some position other than the beginning, there is no way to know if the
-first header found there belongs to the outer tar archive or to the inner
-tar archive. Tar is a format inherently serial; it was designed for tapes.
+Safely decoding an arbitrary tar archive in parallel is only possible if one
+decodes the headers sequentially first. For example, if a tar archive
+containing another tar archive is decoded starting from some position other
+than the beginning, there is no way to know if the first header found there
+belongs to the outer tar archive or to the inner tar archive. Tar is a
+format inherently serial; it was designed for tapes.
+
+   The pax format is even more serial than the ustar format. Two headers
+need to be decoded sequentially for each file. The extended header may even
+need parsing to reveal something as basic as file size. If a thread decodes
+the ustar header skipping the preceding extended header, it may extract a
+file of incorrect size at the wrong place. Moreover, a pax archive with
+global headers can't be decoded in parallel because each thread can't know
+about the global headers decoded by other threads.
 
    In the case of compressed tar archives, the start of each compressed
 block determines one point through which the tar archive can be decoded in
@@ -1026,6 +1080,14 @@ Example 8: Copy the contents of directory 'sourcedir' to the directory
 
      tarlz -C sourcedir -c . | tarlz -C destdir -x
 
+
+Example 9: Compress the existing POSIX archive 'archive.tar' and write the
+output to 'archive.tar.lz'. Compress each member individually for maximum
+availability. (If one member in the compressed archive gets damaged, the
+other members can still be extracted).
+
+     tarlz -z --no-solid archive.tar
+
 
 File: tarlz.info,  Node: Problems,  Next: Concept index,  Prev: Examples,  Up: Top
 
@@ -1069,22 +1131,22 @@ Concept index
 
 Tag Table:
 Node: Top223
-Node: Introduction1214
-Node: Invoking tarlz4022
-Ref: --data-size6233
-Ref: --bsolid14593
-Node: Portable character set18852
-Node: File format19495
-Ref: key_crc3224420
-Node: Amendments to pax format30021
-Ref: crc3230685
-Ref: flawed-compat31970
-Node: Program design34615
-Node: Multi-threaded decoding38540
-Node: Minimum archive sizes42482
-Node: Examples44620
-Node: Problems46335
-Node: Concept index46863
+Node: Introduction1212
+Node: Invoking tarlz4020
+Ref: --data-size6389
+Ref: --bsolid16341
+Node: Portable character set21079
+Node: File format21874
+Ref: key_crc3226799
+Node: Amendments to pax format32400
+Ref: crc3233064
+Ref: flawed-compat34375
+Node: Program design37176
+Node: Multi-threaded decoding41101
+Node: Minimum archive sizes45590
+Node: Examples47728
+Node: Problems49744
+Node: Concept index50272
 
 End Tag Table
 
diff --git a/doc/tarlz.texi b/doc/tarlz.texi
index c6e7e89..f451e5a 100644
--- a/doc/tarlz.texi
+++ b/doc/tarlz.texi
@@ -6,8 +6,8 @@
 @finalout
 @c %**end of header
 
-@set UPDATED 8 January 2021
-@set VERSION 0.19
+@set UPDATED 14 June 2021
+@set VERSION 0.21
 
 @dircategory Data Compression
 @direntry
@@ -138,7 +138,7 @@ tarlz [@var{options}] [@var{files}]
 
 @noindent
 All operations except @samp{--concatenate} operate on whole trees if any
-@var{file} is a directory.
+@var{file} is a directory. Tarlz overwrites output files without warning.
 
 On archive creation or appending tarlz archives the files specified, but
 removes from member names any leading and trailing slashes and any file name
@@ -155,7 +155,7 @@ member names in the archive or given in the command line, so that
 
 If several compression levels or @samp{--*solid} options are given, the last
 setting is used. For example @w{@samp{-9 --solid --uncompressed -1}} is
-equivalent to @samp{-1 --solid}
+equivalent to @w{@samp{-1 --solid}}.
 
 tarlz supports the following
 @uref{http://www.nongnu.org/arg-parser/manual/arg_parser_manual.html#Argument-syntax,,options}:
@@ -174,15 +174,17 @@ This version number should be included in all bug reports.
 
 @item -A
 @itemx --concatenate
-Append one or more archives to the end of an archive. All the archives
-involved must be regular (seekable) files, and must be either all compressed
-or all uncompressed. Compressed and uncompressed archives can't be mixed.
-Compressed archives must be multimember lzip files with the two end-of-file
-blocks plus any zero padding contained in the last lzip member of each
-archive. The intermediate end-of-file blocks are removed as each new archive
-is concatenated. If the archive is uncompressed, tarlz parses and skips tar
-headers until it finds the end-of-file blocks. Exit with status 0 without
-modifying the archive if no @var{files} have been specified.
+Append one or more archives to the end of an archive. If no archive is
+specified with the option @samp{-f}, the input archives are concatenated to
+standard output. All the archives involved must be regular (seekable) files,
+and must be either all compressed or all uncompressed. Compressed and
+uncompressed archives can't be mixed. Compressed archives must be
+multimember lzip files with the two end-of-file blocks plus any zero padding
+contained in the last lzip member of each archive. The intermediate
+end-of-file blocks are removed as each new archive is concatenated. If the
+archive is uncompressed, tarlz parses and skips tar headers until it finds
+the end-of-file blocks. Exit with status 0 without modifying the archive if
+no @var{files} have been specified.
 
 @anchor{--data-size}
 @item -B @var{bytes}
@@ -285,6 +287,12 @@ Note that the number of usable threads is limited during compression to
 and during decompression to the number of lzip members in the tar.lz
 archive, which you can find by running @w{@samp{lzip -lv archive.tar.lz}}.
 
+@item -o @var{file}
+@itemx --output=@var{file}
+Write the compressed output to @var{file}. @w{@samp{-o -}} writes the
+compressed output to standard output. Currently @samp{--output} only works
+with @samp{--compress}.
+
 @item -p
 @itemx --preserve-permissions
 On extraction, set file permissions as they appear in the archive. This is
@@ -331,11 +339,34 @@ special effort to extract a file over an incompatible type of file. For
 example, extracting a link over a directory will usually fail. (Principle of
 least surprise).
 
+@item -z
+@itemx --compress
+Compress existing POSIX tar archives aligning the lzip members to the tar
+members with choice of granularity (---bsolid by default, ---dsolid works
+like ---asolid). The input archives are kept unchanged. Existing compressed
+archives are not overwritten. A hyphen @samp{-} used as the name of an input
+archive reads from standard input and writes to standard output (unless the
+option @samp{--output} is used). Tarlz can be used as compressor for GNU tar
+using a command like @w{@samp{tar -c -Hustar foo | tarlz -z -o foo.tar.lz}}.
+Note that tarlz only works reliably on archives without global headers, or
+with global headers whose content can be ignored.
+
+The compression is reversible, including any garbage present after the EOF
+blocks. Tarlz stops parsing after the first EOF block is found, and then
+compresses the rest of the archive. Unless solid compression is requested,
+the EOF blocks are compressed in a lzip member separated from the preceding
+members and from any non-zero garbage following the EOF blocks.
+@samp{--compress} implies plzip argument style, not tar style. Each input
+archive is compressed to a file with the extension @samp{.lz} added unless
+the option @samp{--output} is used. When @samp{--output} is used, only one
+input archive can be specified. @samp{-f} can't be used with
+@samp{--compress}.
+
 @item -0 .. -9
-Set the compression level for @samp{--create} and @samp{--append}. The
-default compression level is @samp{-6}. Like lzip, tarlz also minimizes the
-dictionary size of the lzip members it creates, reducing the amount of
-memory required for decompression.
+Set the compression level for @samp{--create}, @samp{--append}, and
+@samp{--compress}. The default compression level is @samp{-6}. Like lzip,
+tarlz also minimizes the dictionary size of the lzip members it creates,
+reducing the amount of memory required for decompression.
 
 @multitable {Level} {Dictionary size} {Match length limit}
 @item Level @tab Dictionary size @tab Match length limit
@@ -446,6 +477,14 @@ the value of @samp{LZ_API_VERSION} (if defined).
 @xref{Library version,,,lzlib}.
 @end ifnothtml
 
+@item --warn-newer
+During archive creation, warn if any file being archived has a modification
+time newer than the archive creation time. This option may slow archive
+creation somewhat because it makes an extra call to @samp{stat} after
+archiving each file, but it guarantees that file contents were not modified
+during the creation of the archive. Note that the file must be at least one
+second newer than the archive for it to be detected as newer.
+
 @ignore
 @item --permissive
 Allow some violations of the archive format, like consecutive extended
@@ -457,7 +496,7 @@ keyword appearing in the same block of extended records.
 
 Exit status: 0 for a normal exit, 1 for environmental problems (file not
 found, files differ, invalid flags, I/O errors, etc), 2 to indicate a
-corrupt or invalid input file, 3 for an internal consistency error (eg, bug)
+corrupt or invalid input file, 3 for an internal consistency error (e.g. bug)
 which caused tarlz to panic.
 
 
@@ -477,7 +516,13 @@ The last three characters are the period, underscore, and hyphen-minus
 characters, respectively.
 
 File names are identifiers. Therefore, archiving works better when file
-names use only the portable character set without spaces added.
+names use only the portable character set without spaces added. Unicode is
+for human consumption. It should be
+@uref{http://www.gnu.org/software/moe/manual/moe_manual.html#why-not-Unicode,,avoided}
+in computing environments, specially in file names.
+@ifnothtml
+@xref{why not Unicode,,,moe}.
+@end ifnothtml
 
 
 @node File format
@@ -796,8 +841,9 @@ integrity checking of lzip may not be able to detect the corruption before
 the metadata has been used, for example, to create a new file in the wrong
 place.
 
-Because of the above, tarlz protects the extended records with a CRC in a
-way compatible with standard tar tools. @xref{key_crc32}.
+Because of the above, tarlz protects the extended records with a Cyclic
+Redundancy Check (CRC) in a way compatible with standard tar tools.
+@xref{key_crc32}.
 
 @sp 1
 @anchor{flawed-compat}
@@ -818,7 +864,9 @@ To avoid this problem, tarlz writes extended headers with all fields zeroed
 except size, chksum, typeflag, magic and version. This prevents old tar
 programs from extracting the extended records as a file in the wrong place.
 Tarlz also sets to zero those fields of the ustar header overridden by
-extended records.
+extended records. Finally, tarlz skips members without name when decoding
+except when listing. This is needed to detect certain format violations
+during parallel extraction.
 
 If an extended header is required for any reason (for example a file size
 larger than @w{8 GiB} or a link name longer than 100 bytes), tarlz moves the
@@ -940,11 +988,20 @@ error be avoided.
 @chapter Limitations of parallel tar decoding
 @cindex parallel tar decoding
 
-Safely decoding an arbitrary tar archive in parallel is impossible. For
-example, if a tar archive containing another tar archive is decoded starting
-from some position other than the beginning, there is no way to know if the
-first header found there belongs to the outer tar archive or to the inner
-tar archive. Tar is a format inherently serial; it was designed for tapes.
+Safely decoding an arbitrary tar archive in parallel is only possible if one
+decodes the headers sequentially first. For example, if a tar archive
+containing another tar archive is decoded starting from some position other
+than the beginning, there is no way to know if the first header found there
+belongs to the outer tar archive or to the inner tar archive. Tar is a
+format inherently serial; it was designed for tapes.
+
+The pax format is even more serial than the ustar format. Two headers need
+to be decoded sequentially for each file. The extended header may even need
+parsing to reveal something as basic as file size. If a thread decodes the
+ustar header skipping the preceding extended header, it may extract a file
+of incorrect size at the wrong place. Moreover, a pax archive with global
+headers can't be decoded in parallel because each thread can't know about
+the global headers decoded by other threads.
 
 In the case of compressed tar archives, the start of each compressed block
 determines one point through which the tar archive can be decoded in
@@ -1131,6 +1188,17 @@ directory @samp{destdir}.
 tarlz -C sourcedir -c . | tarlz -C destdir -x
 @end example
 
+@sp 1
+@noindent
+Example 9: Compress the existing POSIX archive @samp{archive.tar} and write
+the output to @samp{archive.tar.lz}. Compress each member individually for
+maximum availability. (If one member in the compressed archive gets damaged,
+the other members can still be extracted).
+
+@example
+tarlz -z --no-solid archive.tar
+@end example
+
 
 @node Problems
 @chapter Reporting bugs