Merging upstream version 1.2~pre1.

Signed-off-by: Daniel Baumann <mail@daniel-baumann.ch>

1 files changed, 415 insertions, 0 deletions

diff --git a/doc/plzip.texi b/doc/plzip.texi
new file mode 100644
index 0000000..413a9e3
--- /dev/null
+++ b/doc/plzip.texi
@@ -0,0 +1,415 @@
+\input texinfo @c -*-texinfo-*-
+@c %**start of header
+@setfilename plzip.info
+@documentencoding ISO-8859-15
+@settitle Plzip Manual
+@finalout
+@c %**end of header
+
+@set UPDATED 20 January 2014
+@set VERSION 1.2-pre1
+
+@dircategory Data Compression
+@direntry
+* Plzip: (plzip).               Parallel compressor compatible with lzip
+@end direntry
+
+
+@ifnothtml
+@titlepage
+@title Plzip
+@subtitle Parallel compressor compatible with lzip
+@subtitle for Plzip version @value{VERSION}, @value{UPDATED}
+@author by Antonio Diaz Diaz
+
+@page
+@vskip 0pt plus 1filll
+@end titlepage
+
+@contents
+@end ifnothtml
+
+@node Top
+@top
+
+This manual is for Plzip (version @value{VERSION}, @value{UPDATED}).
+
+@menu
+* Introduction::           Purpose and features of plzip
+* Program design::         Internal structure of plzip
+* Invoking plzip::         Command line interface
+* File format::            Detailed format of the compressed file
+* Problems::               Reporting bugs
+* Concept index::          Index of concepts
+@end menu
+
+@sp 1
+Copyright @copyright{} 2009, 2010, 2011, 2012, 2013, 2014
+Antonio Diaz Diaz.
+
+This manual is free documentation: you have unlimited permission
+to copy, distribute and modify it.
+
+
+@node Introduction
+@chapter Introduction
+@cindex introduction
+
+Plzip is a massively parallel (multi-threaded), lossless data compressor
+based on the lzlib compression library, with a user interface similar to
+the one of lzip, bzip2 or gzip.
+
+Plzip can compress/decompress large files on multiprocessor machines
+much faster than lzip, at the cost of a slightly reduced compression
+ratio. Note that the number of usable threads is limited by file size,
+so on files larger than a few GB plzip can use hundreds of processors,
+but on files of only a few MB plzip is no faster than lzip.
+
+Plzip uses the lzip file format; the files produced by plzip are fully
+compatible with lzip-1.4 or newer, and can be rescued with lziprecover.
+
+The lzip file format is designed for long-term data archiving and
+provides very safe integrity checking. The member trailer stores the
+32-bit CRC of the original data, the size of the original data and the
+size of the member. These values, together with the value remaining in
+the range decoder and the end-of-stream marker, provide a 4 factor
+integrity checking which guarantees that the decompressed version of the
+data is identical to the original. This guards against corruption of the
+compressed data, and against undetected bugs in plzip (hopefully very
+unlikely). The chances of data corruption going undetected are
+microscopic. Be aware, though, that the check occurs upon decompression,
+so it can only tell you that something is wrong. It can't help you
+recover the original uncompressed data.
+
+If you ever need to recover data from a damaged lzip file, try the
+lziprecover program. Lziprecover makes lzip files resistant to bit-flip
+(one of the most common forms of data corruption), and provides data
+recovery capabilities, including error-checked merging of damaged copies
+of a file.
+
+Plzip uses the same well-defined exit status values used by lzip and
+bzip2, which makes it safer than compressors returning ambiguous warning
+values (like gzip) when it is used as a back end for tar or zutils.
+
+When compressing, plzip replaces every file given in the command line
+with a compressed version of itself, with the name "original_name.lz".
+When decompressing, plzip attempts to guess the name for the decompressed
+file from that of the compressed file as follows:
+
+@multitable {anyothername} {becomes} {anyothername.out}
+@item filename.lz  @tab becomes @tab filename
+@item filename.tlz @tab becomes @tab filename.tar
+@item anyothername @tab becomes @tab anyothername.out
+@end multitable
+
+(De)compressing a file is much like copying or moving it; therefore plzip
+preserves the access and modification dates, permissions, and, when
+possible, ownership of the file just as "cp -p" does. (If the user ID or
+the group ID can't be duplicated, the file permission bits S_ISUID and
+S_ISGID are cleared).
+
+Plzip is able to read from some types of non regular files if the
+@samp{--stdout} option is specified.
+
+If no file names are specified, plzip compresses (or decompresses) from
+standard input to standard output. In this case, plzip will decline to
+write compressed output to a terminal, as this would be entirely
+incomprehensible and therefore pointless.
+
+Plzip will correctly decompress a file which is the concatenation of two
+or more compressed files. The result is the concatenation of the
+corresponding uncompressed files. Integrity testing of concatenated
+compressed files is also supported.
+
+WARNING! Even if plzip is bug-free, other causes may result in a corrupt
+compressed file (bugs in the system libraries, memory errors, etc).
+Therefore, if the data you are going to compress is important, give the
+@samp{--keep} option to plzip and do not remove the original file until
+you verify the compressed file with a command like
+@w{@samp{plzip -cd file.lz | cmp file -}}.
+
+
+@node Program design
+@chapter Program design
+@cindex program design
+
+For each input file, a splitter thread and several worker threads are
+created, acting the main thread as muxer (multiplexer) thread. A "packet
+courier" takes care of data transfers among threads and limits the
+maximum number of data blocks (packets) being processed simultaneously.
+
+The splitter reads data blocks from the input file, and distributes them
+to the workers. The workers (de)compress the blocks received from the
+splitter. The muxer collects processed packets from the workers, and
+writes them to the output file.
+
+When decompressing from a regular file, the splitter is removed and the
+workers read directly from the input file. If the output file is also a
+regular file, the muxer is also removed, and the workers write directly
+to the output file. With these optimizations, decompression speed of
+large files with many members is only limited by the number of
+processors available and by I/O speed.
+
+
+@node Invoking plzip
+@chapter Invoking plzip
+@cindex invoking
+@cindex options
+@cindex usage
+@cindex version
+
+The format for running plzip is:
+
+@example
+plzip [@var{options}] [@var{files}]
+@end example
+
+Plzip supports the following options:
+
+@table @samp
+@item -h
+@itemx --help
+Print an informative help message describing the options and exit.
+
+@item -V
+@itemx --version
+Print the version number of plzip on the standard output and exit.
+
+@item -B @var{bytes}
+@itemx --data-size=@var{bytes}
+@anchor{--data-size}
+Set the input data block size in bytes. The input file will be divided
+in chunks of this size before compression is performed. Valid values
+range from 8 KiB to 1 GiB. Default value is two times the dictionary
+size. Plzip will reduce the dictionary size if it is larger than the
+chosen data size.
+
+@item -c
+@itemx --stdout
+Compress or decompress to standard output. Needed when reading from a
+named pipe (fifo) or from a device.
+
+@item -d
+@itemx --decompress
+Decompress.
+
+@item -f
+@itemx --force
+Force overwrite of output files.
+
+@item -F
+@itemx --recompress
+Force recompression of files whose name already has the @samp{.lz} or
+@samp{.tlz} suffix.
+
+@item -k
+@itemx --keep
+Keep (don't delete) input files during compression or decompression.
+
+@item -m @var{bytes}
+@itemx --match-length=@var{bytes}
+Set the match length limit in bytes. After a match this long is found,
+the search is finished. Valid values range from 5 to 273. Larger values
+usually give better compression ratios but longer compression times.
+
+@item -n @var{n}
+@itemx --threads=@var{n}
+Set the number of worker threads. Valid values range from 1 to "as many
+as your system can support". If this option is not used, plzip tries to
+detect the number of processors in the system and use it as default
+value. @w{@samp{plzip --help}} shows the system's default value.
+
+Note that the number of usable threads is limited to @w{ceil( file_size
+/ data_size )} during compression (@pxref{--data-size}), and to the
+number of members in the input during decompression.
+
+@item -o @var{file}
+@itemx --output=@var{file}
+When reading from standard input and @samp{--stdout} has not been
+specified, use @samp{@var{file}} as the virtual name of the uncompressed
+file. This produces a file named @samp{@var{file}} when decompressing,
+and a file named @samp{@var{file}.lz} when compressing.
+
+@item -q
+@itemx --quiet
+Quiet operation. Suppress all messages.
+
+@item -s @var{bytes}
+@itemx --dictionary-size=@var{bytes}
+Set the dictionary size limit in bytes. Valid values range from 4 KiB to
+512 MiB. Plzip will use the smallest possible dictionary size for each
+member without exceeding this limit. Note that dictionary sizes are
+quantized. If the specified size does not match one of the valid sizes,
+it will be rounded upwards by adding up to (@var{bytes} / 16) to it.
+
+For maximum compression you should use a dictionary size limit as large
+as possible, but keep in mind that the decompression memory requirement
+is affected at compression time by the choice of dictionary size limit.
+
+@item -t
+@itemx --test
+Check integrity of the specified file(s), but don't decompress them.
+This really performs a trial decompression and throws away the result.
+Use it together with @samp{-v} to see information about the file.
+
+@item -v
+@itemx --verbose
+Verbose mode.@*
+When compressing, show the compression ratio for each file processed. A
+second @samp{-v} shows the progress of compression.@*
+When decompressing or testing, further -v's (up to 4) increase the
+verbosity level, showing status, compression ratio, decompressed size,
+and compressed size.
+
+@item -1 .. -9
+Set the compression parameters (dictionary size and match length limit)
+as shown in the table below. Note that @samp{-9} can be much slower than
+@samp{-1}. These options have no effect when decompressing.
+
+The bidimensional parameter space of LZMA can't be mapped to a linear
+scale optimal for all files. If your files are large, very repetitive,
+etc, you may need to use the @samp{--match-length} and
+@samp{--dictionary-size} options directly to achieve optimal
+performance. For example, @samp{-9m64} usually compresses executables
+more (and faster) than @samp{-9}.
+
+@multitable {Level} {Dictionary size} {Match length limit}
+@item Level @tab Dictionary size @tab Match length limit
+@item -1 @tab  1 MiB @tab   5 bytes
+@item -2 @tab  1.5 MiB @tab   6 bytes
+@item -3 @tab  2 MiB @tab   8 bytes
+@item -4 @tab  3 MiB @tab  12 bytes
+@item -5 @tab  4 MiB @tab  20 bytes
+@item -6 @tab  8 MiB @tab  36 bytes
+@item -7 @tab 16 MiB @tab  68 bytes
+@item -8 @tab 24 MiB @tab 132 bytes
+@item -9 @tab 32 MiB @tab 273 bytes
+@end multitable
+
+@item --fast
+@itemx --best
+Aliases for GNU gzip compatibility.
+
+@end table
+
+Numbers given as arguments to options may be followed by a multiplier
+and an optional @samp{B} for "byte".
+
+Table of SI and binary prefixes (unit multipliers):
+
+@multitable {Prefix} {kilobyte  (10^3 = 1000)} {|} {Prefix} {kibibyte (2^10 = 1024)}
+@item Prefix @tab Value               @tab | @tab Prefix @tab Value
+@item k @tab kilobyte  (10^3 = 1000)  @tab | @tab Ki @tab kibibyte (2^10 = 1024)
+@item M @tab megabyte  (10^6)         @tab | @tab Mi @tab mebibyte (2^20)
+@item G @tab gigabyte  (10^9)         @tab | @tab Gi @tab gibibyte (2^30)
+@item T @tab terabyte  (10^12)        @tab | @tab Ti @tab tebibyte (2^40)
+@item P @tab petabyte  (10^15)        @tab | @tab Pi @tab pebibyte (2^50)
+@item E @tab exabyte   (10^18)        @tab | @tab Ei @tab exbibyte (2^60)
+@item Z @tab zettabyte (10^21)        @tab | @tab Zi @tab zebibyte (2^70)
+@item Y @tab yottabyte (10^24)        @tab | @tab Yi @tab yobibyte (2^80)
+@end multitable
+
+@sp 1
+Exit status: 0 for a normal exit, 1 for environmental problems (file not
+found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or
+invalid input file, 3 for an internal consistency error (eg, bug) which
+caused plzip to panic.
+
+
+@node File format
+@chapter File format
+@cindex file format
+
+Perfection is reached, not when there is no longer anything to add, but
+when there is no longer anything to take away.@*
+--- Antoine de Saint-Exupery
+
+@sp 1
+In the diagram below, a box like this:
+@verbatim
++---+
+|   | <-- the vertical bars might be missing
++---+
+@end verbatim
+
+represents one byte; a box like this:
+@verbatim
++==============+
+|              |
++==============+
+@end verbatim
+
+represents a variable number of bytes.
+
+@sp 1
+A lzip file consists of a series of "members" (compressed data sets).
+The members simply appear one after another in the file, with no
+additional information before, between, or after them.
+
+Each member has the following structure:
+@verbatim
++--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+| ID string | VN | DS | Lzma stream | CRC32 |   Data size   |  Member size  |
++--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+@end verbatim
+
+All multibyte values are stored in little endian order.
+
+@table @samp
+@item ID string
+A four byte string, identifying the lzip format, with the value "LZIP"
+(0x4C, 0x5A, 0x49, 0x50).
+
+@item VN (version number, 1 byte)
+Just in case something needs to be modified in the future. 1 for now.
+
+@item DS (coded dictionary size, 1 byte)
+Lzip divides the distance between any two powers of 2 into 8 equally
+spaced intervals, named "wedges". The dictionary size is calculated by
+taking a power of 2 (the base size) and substracting from it a number of
+wedges between 0 and 7. The size of a wedge is (base_size / 16).@*
+Bits 4-0 contain the base 2 logarithm of the base size (12 to 29).@*
+Bits 7-5 contain the number of wedges (0 to 7) to substract from the
+base size to obtain the dictionary size.@*
+Example: 0xD3 = 2^19 - 6 * 2^15 = 512 KiB - 6 * 32 KiB = 320 KiB@*
+Valid values for dictionary size range from 4 KiB to 512 MiB.
+
+@item Lzma stream
+The lzma stream, finished by an end of stream marker. Uses default values
+for encoder properties. See the lzip manual for a full description.
+
+@item CRC32 (4 bytes)
+CRC of the uncompressed original data.
+
+@item Data size (8 bytes)
+Size of the uncompressed original data.
+
+@item Member size (8 bytes)
+Total size of the member, including header and trailer. This field acts
+as a distributed index, allows the verification of stream integrity, and
+facilitates safe recovery of undamaged members from multi-member files.
+
+@end table
+
+
+@node Problems
+@chapter Reporting bugs
+@cindex bugs
+@cindex getting help
+
+There are probably bugs in plzip. There are certainly errors and
+omissions in this manual. If you report them, they will get fixed. If
+you don't, no one will ever know about them and they will remain unfixed
+for all eternity, if not longer.
+
+If you find a bug in plzip, please send electronic mail to
+@email{lzip-bug@@nongnu.org}. Include the version number, which you can
+find by running @w{@samp{plzip --version}}.
+
+
+@node Concept index
+@unnumbered Concept index
+
+@printindex cp
+
+@bye