summaryrefslogtreecommitdiffstats
path: root/doc
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2022-02-21 16:16:13 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2022-02-21 16:16:27 +0000
commite19c8398c1946bd4616c3fa1c095149462346d15 (patch)
tree85f55afe5b89928e59ea76bdff18478bd1cc6b82 /doc
parentReleasing debian version 1.12-3. (diff)
downloadlzlib-e19c8398c1946bd4616c3fa1c095149462346d15.tar.xz
lzlib-e19c8398c1946bd4616c3fa1c095149462346d15.zip
Merging upstream version 1.13.
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'doc')
-rw-r--r--doc/lzlib.info243
-rw-r--r--doc/lzlib.texi244
-rw-r--r--doc/minilzip.135
3 files changed, 270 insertions, 252 deletions
diff --git a/doc/lzlib.info b/doc/lzlib.info
index bef1859..d81bc88 100644
--- a/doc/lzlib.info
+++ b/doc/lzlib.info
@@ -1,6 +1,6 @@
This is lzlib.info, produced by makeinfo version 4.13+ from lzlib.texi.
-INFO-DIR-SECTION Data Compression
+INFO-DIR-SECTION Compression
START-INFO-DIR-ENTRY
* Lzlib: (lzlib). Compression library for the lzip format
END-INFO-DIR-ENTRY
@@ -11,7 +11,7 @@ File: lzlib.info, Node: Top, Next: Introduction, Up: (dir)
Lzlib Manual
************
-This manual is for Lzlib (version 1.12, 2 January 2021).
+This manual is for Lzlib (version 1.13, 23 January 2022).
* Menu:
@@ -30,7 +30,7 @@ This manual is for Lzlib (version 1.12, 2 January 2021).
* Concept index:: Index of concepts
- Copyright (C) 2009-2021 Antonio Diaz Diaz.
+ Copyright (C) 2009-2022 Antonio Diaz Diaz.
This manual is free documentation: you have unlimited permission to copy,
distribute, and modify it.
@@ -73,8 +73,12 @@ byte near the beginning is a thing of the past.
The functions and variables forming the interface of the compression
library are declared in the file 'lzlib.h'. Usage examples of the library
-are given in the files 'bbexample.c', 'ffexample.c', and 'main.c' from the
-source distribution.
+are given in the files 'bbexample.c', 'ffexample.c', and 'minilzip.c' from
+the source distribution.
+
+ All the library functions are thread safe. The library does not install
+any signal handler. The decoder checks the consistency of the compressed
+data, so the library should never crash even in case of corrupted input.
Compression/decompression is done by repeatedly calling a couple of
read/write functions until all the data have been processed by the library.
@@ -102,20 +106,16 @@ concatenated compressed data streams is also supported.
automatically creating multimember output. The members so created are large,
about 2 PiB each.
- All the library functions are thread safe. The library does not install
-any signal handler. The decoder checks the consistency of the compressed
-data, so the library should never crash even in case of corrupted input.
-
In spite of its name (Lempel-Ziv-Markov chain-Algorithm), LZMA is not a
concrete algorithm; it is more like "any algorithm using the LZMA coding
scheme". For example, the option '-0' of lzip uses the scheme in almost the
simplest way possible; issuing the longest match it can find, or a literal
byte if it can't find a match. Inversely, a much more elaborated way of
-finding coding sequences of minimum size than the one currently used by
-lzip could be developed, and the resulting sequence could also be coded
-using the LZMA coding scheme.
+finding coding sequences of minimum size than the one currently used by lzip
+could be developed, and the resulting sequence could also be coded using the
+LZMA coding scheme.
- Lzlib currently implements two variants of the LZMA algorithm; fast
+ Lzlib currently implements two variants of the LZMA algorithm: fast
(used by option '-0' of minilzip) and normal (used by all other compression
levels).
@@ -145,7 +145,8 @@ One goal of lzlib is to keep perfect backward compatibility with older
versions of itself down to 1.0. Any application working with an older lzlib
should work with a newer lzlib. Installing a newer lzlib should not break
anything. This chapter describes the constants and functions that the
-application can use to discover the version of the library being used.
+application can use to discover the version of the library being used. All
+of them are declared in 'lzlib.h'.
-- Constant: LZ_API_VERSION
This constant is defined in 'lzlib.h' and works as a version test
@@ -325,13 +326,13 @@ except 'LZ_compress_open' whose return value must be verified by calling
'LZ_compress_sync_flush'. Then call 'LZ_compress_read' until it
returns 0.
- This function writes a LZMA marker '3' ("Sync Flush" marker) to the
- compressed output. Note that the sync flush marker is not allowed in
- lzip files; it is a device for interactive communication between
- applications using lzlib, but is useless and wasteful in a file, and
- is excluded from the media type 'application/lzip'. The LZMA marker
- '2' ("End Of Stream" marker) is the only marker allowed in lzip files.
- *Note Data format::.
+ This function writes at least one LZMA marker '3' ("Sync Flush" marker)
+ to the compressed output. Note that the sync flush marker is not
+ allowed in lzip files; it is a device for interactive communication
+ between applications using lzlib, but is useless and wasteful in a
+ file, and is excluded from the media type 'application/lzip'. The LZMA
+ marker '2' ("End Of Stream" marker) is the only marker allowed in lzip
+ files. *Note Data format::.
Repeated use of 'LZ_compress_sync_flush' may degrade compression
ratio, so use it only when needed. If the interval between calls to
@@ -347,34 +348,30 @@ except 'LZ_compress_open' whose return value must be verified by calling
-- Function: int LZ_compress_read ( struct LZ_Encoder * const ENCODER,
uint8_t * const BUFFER, const int SIZE )
- The function 'LZ_compress_read' reads up to SIZE bytes from the stream
- pointed to by ENCODER, storing the results in BUFFER. If
- LZ_API_VERSION >= 1012, BUFFER may be a null pointer, in which case
- the bytes read are discarded.
-
- The return value is the number of bytes actually read. This might be
- less than SIZE; for example, if there aren't that many bytes left in
- the stream or if more bytes have to be yet written with the function
+ Reads up to SIZE bytes from the stream pointed to by ENCODER, storing
+ the results in BUFFER. If LZ_API_VERSION >= 1012, BUFFER may be a null
+ pointer, in which case the bytes read are discarded.
+
+ Returns the number of bytes actually read. This might be less than
+ SIZE; for example, if there aren't that many bytes left in the stream
+ or if more bytes have to be yet written with the function
'LZ_compress_write'. Note that reading less than SIZE bytes is not an
error.
-- Function: int LZ_compress_write ( struct LZ_Encoder * const ENCODER,
uint8_t * const BUFFER, const int SIZE )
- The function 'LZ_compress_write' writes up to SIZE bytes from BUFFER
- to the stream pointed to by ENCODER.
-
- The return value is the number of bytes actually written. This might be
+ Writes up to SIZE bytes from BUFFER to the stream pointed to by
+ ENCODER. Returns the number of bytes actually written. This might be
less than SIZE. Note that writing less than SIZE bytes is not an error.
-- Function: int LZ_compress_write_size ( struct LZ_Encoder * const
ENCODER )
- The function 'LZ_compress_write_size' returns the maximum number of
- bytes that can be immediately written through 'LZ_compress_write'. For
- efficiency reasons, once the input buffer is full and
- 'LZ_compress_write_size' returns 0, almost all the buffer must be
- compressed before a size greater than 0 is returned again. (This is
- done to minimize the amount of data that must be copied to the
- beginning of the buffer before new data can be accepted).
+ Returns the maximum number of bytes that can be immediately written
+ through 'LZ_compress_write'. For efficiency reasons, once the input
+ buffer is full and 'LZ_compress_write_size' returns 0, almost all the
+ buffer must be compressed before a size greater than 0 is returned
+ again. (This is done to minimize the amount of data that must be
+ copied to the beginning of the buffer before new data can be accepted).
It is guaranteed that an immediate call to 'LZ_compress_write' will
accept a SIZE up to the returned number of bytes.
@@ -472,14 +469,13 @@ except 'LZ_decompress_open' whose return value must be verified by calling
-- Function: int LZ_decompress_read ( struct LZ_Decoder * const DECODER,
uint8_t * const BUFFER, const int SIZE )
- The function 'LZ_decompress_read' reads up to SIZE bytes from the
- stream pointed to by DECODER, storing the results in BUFFER. If
- LZ_API_VERSION >= 1012, BUFFER may be a null pointer, in which case
- the bytes read are discarded.
-
- The return value is the number of bytes actually read. This might be
- less than SIZE; for example, if there aren't that many bytes left in
- the stream or if more bytes have to be yet written with the function
+ Reads up to SIZE bytes from the stream pointed to by DECODER, storing
+ the results in BUFFER. If LZ_API_VERSION >= 1012, BUFFER may be a null
+ pointer, in which case the bytes read are discarded.
+
+ Returns the number of bytes actually read. This might be less than
+ SIZE; for example, if there aren't that many bytes left in the stream
+ or if more bytes have to be yet written with the function
'LZ_decompress_write'. Note that reading less than SIZE bytes is not
an error.
@@ -499,18 +495,16 @@ except 'LZ_decompress_open' whose return value must be verified by calling
-- Function: int LZ_decompress_write ( struct LZ_Decoder * const DECODER,
uint8_t * const BUFFER, const int SIZE )
- The function 'LZ_decompress_write' writes up to SIZE bytes from BUFFER
- to the stream pointed to by DECODER.
-
- The return value is the number of bytes actually written. This might be
+ Writes up to SIZE bytes from BUFFER to the stream pointed to by
+ DECODER. Returns the number of bytes actually written. This might be
less than SIZE. Note that writing less than SIZE bytes is not an error.
-- Function: int LZ_decompress_write_size ( struct LZ_Decoder * const
DECODER )
- The function 'LZ_decompress_write_size' returns the maximum number of
- bytes that can be immediately written through 'LZ_decompress_write'.
- This number varies smoothly; each compressed byte consumed may be
- overwritten immediately, increasing by 1 the value returned.
+ Returns the maximum number of bytes that can be immediately written
+ through 'LZ_decompress_write'. This number varies smoothly; each
+ compressed byte consumed may be overwritten immediately, increasing by
+ 1 the value returned.
It is guaranteed that an immediate call to 'LZ_decompress_write' will
accept a SIZE up to the returned number of bytes.
@@ -530,24 +524,24 @@ except 'LZ_decompress_open' whose return value must be verified by calling
-- Function: int LZ_decompress_member_finished ( struct LZ_Decoder * const
DECODER )
Returns 1 if the previous call to 'LZ_decompress_read' finished reading
- the current member, indicating that final values for member are
+ the current member, indicating that final values for the member are
available through 'LZ_decompress_data_crc',
'LZ_decompress_data_position', and 'LZ_decompress_member_position'.
Otherwise it returns 0.
-- Function: int LZ_decompress_member_version ( struct LZ_Decoder * const
DECODER )
- Returns the version of current member from member header.
+ Returns the version of the current member, read from the member header.
-- Function: int LZ_decompress_dictionary_size ( struct LZ_Decoder * const
DECODER )
- Returns the dictionary size of the current member, read from the member
- header.
+ Returns the dictionary size of the current member, read from the
+ member header.
-- Function: unsigned LZ_decompress_data_crc ( struct LZ_Decoder * const
DECODER )
Returns the 32 bit Cyclic Redundancy Check of the data decompressed
- from the current member. The returned value is valid only when
+ from the current member. The value returned is valid only when
'LZ_decompress_member_finished' returns 1.
-- Function: unsigned long long LZ_decompress_data_position ( struct
@@ -650,13 +644,14 @@ compatible with lzip 1.4 or newer.
Lzip is a lossless data compressor with a user interface similar to the
one of gzip or bzip2. Lzip uses a simplified form of the 'Lempel-Ziv-Markov
-chain-Algorithm' (LZMA) stream format, chosen to maximize safety and
-interoperability. Lzip can compress about as fast as gzip (lzip -0) or
-compress most files more than bzip2 (lzip -9). Decompression speed is
-intermediate between gzip and bzip2. Lzip is better than gzip and bzip2
-from a data recovery perspective. Lzip has been designed, written, and
-tested with great care to replace gzip and bzip2 as the standard
-general-purpose compressed format for unix-like systems.
+chain-Algorithm' (LZMA) stream format and provides a 3 factor integrity
+checking to maximize interoperability and optimize safety. Lzip can compress
+about as fast as gzip (lzip -0) or compress most files more than bzip2
+(lzip -9). Decompression speed is intermediate between gzip and bzip2. Lzip
+is better than gzip and bzip2 from a data recovery perspective. Lzip has
+been designed, written, and tested with great care to replace gzip and
+bzip2 as the standard general-purpose compressed format for unix-like
+systems.
The format for running minilzip is:
@@ -705,10 +700,13 @@ once, the first time it appears in the command line.
'-d'
'--decompress'
- Decompress the files specified. If a file does not exist or can't be
- opened, minilzip continues decompressing the rest of the files. If a
- file fails to decompress, or is a terminal, minilzip exits immediately
- without decompressing the rest of the files.
+ Decompress the files specified. If a file does not exist, can't be
+ opened, or the destination file already exists and '--force' has not
+ been specified, minilzip continues decompressing the rest of the files
+ and exits with error status 1. If a file fails to decompress, or is a
+ terminal, minilzip exits immediately with error status 2 without
+ decompressing the rest of the files. A terminal is considered an
+ uncompressed file, and therefore invalid.
'-f'
'--force'
@@ -831,12 +829,14 @@ once, the first time it appears in the command line.
'--check-lib'
Compare the version of lzlib used to compile minilzip with the version
- actually being used and exit. Report any differences found. Exit with
- error status 1 if differences are found. A mismatch may indicate that
- lzlib is not correctly installed or that a different version of lzlib
- has been installed after compiling the shared version of minilzip.
- 'minilzip -v --check-lib' shows the version of lzlib being used and
- the value of 'LZ_API_VERSION' (if defined). *Note Library version::.
+ actually being used at run time and exit. Report any differences
+ found. Exit with error status 1 if differences are found. A mismatch
+ may indicate that lzlib is not correctly installed or that a different
+ version of lzlib has been installed after compiling the shared version
+ of minilzip. Exit with error status 2 if LZ_API_VERSION and
+ LZ_version_string don't match. 'minilzip -v --check-lib' shows the
+ version of lzlib being used and the value of LZ_API_VERSION (if
+ defined). *Note Library version::.
Numbers given as arguments to options may be followed by a multiplier
@@ -857,7 +857,7 @@ Y yottabyte (10^24) | Yi yobibyte (2^80)
Exit status: 0 for a normal exit, 1 for environmental problems (file not
found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or invalid
-input file, 3 for an internal consistency error (eg, bug) which caused
+input file, 3 for an internal consistency error (e.g., bug) which caused
minilzip to panic.

@@ -886,9 +886,11 @@ when there is no longer anything to take away.
represents a variable number of bytes.
- A lzip data stream consists of a series of "members" (compressed data
+ Lzip data consist of a series of independent "members" (compressed data
sets). The members simply appear one after another in the data stream, with
-no additional information before, between, or after them.
+no additional information before, between, or after them. Each member can
+encode in compressed form up to 16 EiB - 1 byte of uncompressed data. The
+size of a multimember data stream is unlimited.
Each member has the following structure:
@@ -916,7 +918,7 @@ no additional information before, between, or after them.
Valid values for dictionary size range from 4 KiB to 512 MiB.
'LZMA stream'
- The LZMA stream, finished by an end of stream marker. Uses default
+ The LZMA stream, finished by an "End Of Stream" marker. Uses default
values for encoder properties. *Note Stream format: (lzip)Stream
format, for a complete description.
Lzip only uses the LZMA marker '2' ("End Of Stream" marker). Lzlib
@@ -924,16 +926,17 @@ no additional information before, between, or after them.
sync_flush::.
'CRC32 (4 bytes)'
- Cyclic Redundancy Check (CRC) of the uncompressed original data.
+ Cyclic Redundancy Check (CRC) of the original uncompressed data.
'Data size (8 bytes)'
- Size of the uncompressed original data.
+ Size of the original uncompressed data.
'Member size (8 bytes)'
Total size of the member, including header and trailer. This field acts
as a distributed index, allows the verification of stream integrity,
- and facilitates safe recovery of undamaged members from multimember
- files.
+ and facilitates the safe recovery of undamaged members from
+ multimember files. Member size should be limited to 2 PiB to prevent
+ the data size field from overflowing.

@@ -967,10 +970,10 @@ File: lzlib.info, Node: Buffer compression, Next: Buffer decompression, Up: E
Buffer-to-buffer single-member compression (MEMBER_SIZE > total output).
-/* Compresses 'insize' bytes from 'inbuf' to 'outbuf'.
- Returns the size of the compressed data in '*outlenp'.
- In case of error, or if 'outsize' is too small, returns false and does
- not modify '*outlenp'.
+/* Compress 'insize' bytes from 'inbuf' to 'outbuf'.
+ Return the size of the compressed data in '*outlenp'.
+ In case of error, or if 'outsize' is too small, return false and do not
+ modify '*outlenp'.
*/
bool bbcompress( const uint8_t * const inbuf, const int insize,
const int dictionary_size, const int match_len_limit,
@@ -1011,10 +1014,10 @@ File: lzlib.info, Node: Buffer decompression, Next: File compression, Prev: B
Buffer-to-buffer decompression.
-/* Decompresses 'insize' bytes from 'inbuf' to 'outbuf'.
- Returns the size of the decompressed data in '*outlenp'.
- In case of error, or if 'outsize' is too small, returns false and does
- not modify '*outlenp'.
+/* Decompress 'insize' bytes from 'inbuf' to 'outbuf'.
+ Return the size of the decompressed data in '*outlenp'.
+ In case of error, or if 'outsize' is too small, return false and do not
+ modify '*outlenp'.
*/
bool bbdecompress( const uint8_t * const inbuf, const int insize,
uint8_t * const outbuf, const int outsize,
@@ -1159,9 +1162,9 @@ int ffmmcompress( FILE * const infile, FILE * const outfile )
Example 2: Multimember compression (user-restarted members). (Call
LZ_compress_open with MEMBER_SIZE > largest member).
-/* Compresses 'infile' to 'outfile' as a multimember stream with one member
+/* Compress 'infile' to 'outfile' as a multimember stream with one member
for each line of text terminated by a newline character or by EOF.
- Returns 0 if success, 1 if error.
+ Return 0 if success, 1 if error.
*/
int fflfcompress( struct LZ_Encoder * const encoder,
FILE * const infile, FILE * const outfile )
@@ -1205,7 +1208,7 @@ File: lzlib.info, Node: Skipping data errors, Prev: File compression mm, Up:
11.6 Skipping data errors
=========================
-/* Decompresses 'infile' to 'outfile' with automatic resynchronization to
+/* Decompress 'infile' to 'outfile' with automatic resynchronization to
next member in case of data error, including the automatic removal of
leading garbage.
*/
@@ -1253,7 +1256,7 @@ eternity, if not longer.
If you find a bug in lzlib, please send electronic mail to
<lzip-bug@nongnu.org>. Include the version number, which you can find by
-running 'minilzip --version' or in 'LZ_version_string' from 'lzlib.h'.
+running 'minilzip --version' and 'minilzip -v --check-lib'.

File: lzlib.info, Node: Concept index, Prev: Problems, Up: Top
@@ -1288,29 +1291,29 @@ Concept index

Tag Table:
-Node: Top220
-Node: Introduction1342
+Node: Top215
+Node: Introduction1338
Node: Library version6413
-Node: Buffering8918
-Node: Parameter limits10143
-Node: Compression functions11097
-Ref: member_size12907
-Ref: sync_flush14673
-Node: Decompression functions19493
-Node: Error codes27187
-Node: Error messages29478
-Node: Invoking minilzip30057
-Node: Data format39651
-Ref: coded-dict-size40957
-Node: Examples42267
-Node: Buffer compression43228
-Node: Buffer decompression44754
-Node: File compression46174
-Node: File decompression47157
-Node: File compression mm48161
-Node: Skipping data errors51193
-Node: Problems52505
-Node: Concept index53077
+Node: Buffering8957
+Node: Parameter limits10182
+Node: Compression functions11136
+Ref: member_size12946
+Ref: sync_flush14712
+Node: Decompression functions19400
+Node: Error codes26968
+Node: Error messages29259
+Node: Invoking minilzip29838
+Node: Data format39786
+Ref: coded-dict-size41232
+Node: Examples42641
+Node: Buffer compression43602
+Node: Buffer decompression45122
+Node: File compression46536
+Node: File decompression47519
+Node: File compression mm48523
+Node: Skipping data errors51552
+Node: Problems52862
+Node: Concept index53423

End Tag Table
diff --git a/doc/lzlib.texi b/doc/lzlib.texi
index 644a3d7..3caf9dd 100644
--- a/doc/lzlib.texi
+++ b/doc/lzlib.texi
@@ -6,10 +6,10 @@
@finalout
@c %**end of header
-@set UPDATED 2 January 2021
-@set VERSION 1.12
+@set UPDATED 23 January 2022
+@set VERSION 1.13
-@dircategory Data Compression
+@dircategory Compression
@direntry
* Lzlib: (lzlib). Compression library for the lzip format
@end direntry
@@ -52,7 +52,7 @@ This manual is for Lzlib (version @value{VERSION}, @value{UPDATED}).
@end menu
@sp 1
-Copyright @copyright{} 2009-2021 Antonio Diaz Diaz.
+Copyright @copyright{} 2009-2022 Antonio Diaz Diaz.
This manual is free documentation: you have unlimited permission to copy,
distribute, and modify it.
@@ -77,9 +77,9 @@ taking into account both data integrity and decoder availability:
The lzip format provides very safe integrity checking and some data
recovery means. The program
@uref{http://www.nongnu.org/lzip/manual/lziprecover_manual.html#Data-safety,,lziprecover}
-can repair bit flip errors (one of the most common forms of data
-corruption) in lzip files, and provides data recovery capabilities,
-including error-checked merging of damaged copies of a file.
+can repair bit flip errors (one of the most common forms of data corruption)
+in lzip files, and provides data recovery capabilities, including
+error-checked merging of damaged copies of a file.
@ifnothtml
@xref{Data safety,,,lziprecover}.
@end ifnothtml
@@ -89,8 +89,8 @@ The lzip format is as simple as possible (but not simpler). The lzip
manual provides the source code of a simple decompressor along with a
detailed explanation of how it works, so that with the only help of the
lzip manual it would be possible for a digital archaeologist to extract
-the data from a lzip file long after quantum computers eventually render
-LZMA obsolete.
+the data from a lzip file long after quantum computers eventually
+render LZMA obsolete.
@item
Additionally the lzip reference implementation is copylefted, which
@@ -104,8 +104,12 @@ the beginning is a thing of the past.
The functions and variables forming the interface of the compression library
are declared in the file @samp{lzlib.h}. Usage examples of the library are
-given in the files @samp{bbexample.c}, @samp{ffexample.c}, and @samp{main.c}
-from the source distribution.
+given in the files @samp{bbexample.c}, @samp{ffexample.c}, and
+@samp{minilzip.c} from the source distribution.
+
+All the library functions are thread safe. The library does not install any
+signal handler. The decoder checks the consistency of the compressed data,
+so the library should never crash even in case of corrupted input.
Compression/decompression is done by repeatedly calling a couple of
read/write functions until all the data have been processed by the library.
@@ -134,22 +138,17 @@ Lzlib is able to compress and decompress streams of unlimited size by
automatically creating multimember output. The members so created are large,
about @w{2 PiB} each.
-All the library functions are thread safe. The library does not install
-any signal handler. The decoder checks the consistency of the compressed
-data, so the library should never crash even in case of corrupted input.
-
In spite of its name (Lempel-Ziv-Markov chain-Algorithm), LZMA is not a
concrete algorithm; it is more like "any algorithm using the LZMA coding
-scheme". For example, the option @samp{-0} of lzip uses the scheme in almost
-the simplest way possible; issuing the longest match it can find, or a
-literal byte if it can't find a match. Inversely, a much more elaborated way
-of finding coding sequences of minimum size than the one currently used by
-lzip could be developed, and the resulting sequence could also be coded
-using the LZMA coding scheme.
+scheme". For example, the option @samp{-0} of lzip uses the scheme in almost the
+simplest way possible; issuing the longest match it can find, or a literal
+byte if it can't find a match. Inversely, a much more elaborated way of
+finding coding sequences of minimum size than the one currently used by lzip
+could be developed, and the resulting sequence could also be coded using the
+LZMA coding scheme.
-Lzlib currently implements two variants of the LZMA algorithm; fast (used by
-option @samp{-0} of minilzip) and normal (used by all other compression
-levels).
+Lzlib currently implements two variants of the LZMA algorithm: fast (used by
+option @samp{-0} of minilzip) and normal (used by all other compression levels).
The high compression of LZMA comes from combining two basic, well-proven
compression ideas: sliding dictionaries (LZ77/78) and markov models (the
@@ -176,7 +175,8 @@ One goal of lzlib is to keep perfect backward compatibility with older
versions of itself down to 1.0. Any application working with an older lzlib
should work with a newer lzlib. Installing a newer lzlib should not break
anything. This chapter describes the constants and functions that the
-application can use to discover the version of the library being used.
+application can use to discover the version of the library being used. All
+of them are declared in @samp{lzlib.h}.
@defvr Constant LZ_API_VERSION
This constant is defined in @samp{lzlib.h} and works as a version test
@@ -372,12 +372,13 @@ already written with the function @samp{LZ_compress_write}. First call
@samp{LZ_compress_sync_flush}. Then call @samp{LZ_compress_read} until it
returns 0.
-This function writes a LZMA marker @samp{3} ("Sync Flush" marker) to the
-compressed output. Note that the sync flush marker is not allowed in lzip
-files; it is a device for interactive communication between applications
-using lzlib, but is useless and wasteful in a file, and is excluded from the
-media type @samp{application/lzip}. The LZMA marker @samp{2} ("End Of
-Stream" marker) is the only marker allowed in lzip files. @xref{Data format}.
+This function writes at least one LZMA marker @samp{3} ("Sync Flush" marker)
+to the compressed output. Note that the sync flush marker is not allowed in
+lzip files; it is a device for interactive communication between
+applications using lzlib, but is useless and wasteful in a file, and is
+excluded from the media type @samp{application/lzip}. The LZMA marker
+@samp{2} ("End Of Stream" marker) is the only marker allowed in lzip files.
+@xref{Data format}.
Repeated use of @samp{LZ_compress_sync_flush} may degrade compression
ratio, so use it only when needed. If the interval between calls to
@@ -394,36 +395,33 @@ are more bytes available than those needed to complete @var{member_size},
@deftypefun int LZ_compress_read ( struct LZ_Encoder * const @var{encoder}, uint8_t * const @var{buffer}, const int @var{size} )
-The function @samp{LZ_compress_read} reads up to @var{size} bytes from the
-stream pointed to by @var{encoder}, storing the results in @var{buffer}.
-If @w{LZ_API_VERSION >= 1012}, @var{buffer} may be a null pointer, in which
-case the bytes read are discarded.
-
-The return value is the number of bytes actually read. This might be less
-than @var{size}; for example, if there aren't that many bytes left in the
-stream or if more bytes have to be yet written with the function
+Reads up to @var{size} bytes from the stream pointed to by @var{encoder},
+storing the results in @var{buffer}. If @w{LZ_API_VERSION >= 1012},
+@var{buffer} may be a null pointer, in which case the bytes read are
+discarded.
+
+Returns the number of bytes actually read. This might be less than
+@var{size}; for example, if there aren't that many bytes left in the stream
+or if more bytes have to be yet written with the function
@samp{LZ_compress_write}. Note that reading less than @var{size} bytes is
not an error.
@end deftypefun
@deftypefun int LZ_compress_write ( struct LZ_Encoder * const @var{encoder}, uint8_t * const @var{buffer}, const int @var{size} )
-The function @samp{LZ_compress_write} writes up to @var{size} bytes from
-@var{buffer} to the stream pointed to by @var{encoder}.
-
-The return value is the number of bytes actually written. This might be
-less than @var{size}. Note that writing less than @var{size} bytes is
-not an error.
+Writes up to @var{size} bytes from @var{buffer} to the stream pointed to by
+@var{encoder}. Returns the number of bytes actually written. This might be
+less than @var{size}. Note that writing less than @var{size} bytes is not an
+error.
@end deftypefun
@deftypefun int LZ_compress_write_size ( struct LZ_Encoder * const @var{encoder} )
-The function @samp{LZ_compress_write_size} returns the maximum number of
-bytes that can be immediately written through @samp{LZ_compress_write}.
-For efficiency reasons, once the input buffer is full and
-@samp{LZ_compress_write_size} returns 0, almost all the buffer must be
-compressed before a size greater than 0 is returned again. (This is done to
-minimize the amount of data that must be copied to the beginning of the
+Returns the maximum number of bytes that can be immediately written through
+@samp{LZ_compress_write}. For efficiency reasons, once the input buffer is
+full and @samp{LZ_compress_write_size} returns 0, almost all the buffer must
+be compressed before a size greater than 0 is returned again. (This is done
+to minimize the amount of data that must be copied to the beginning of the
buffer before new data can be accepted).
It is guaranteed that an immediate call to @samp{LZ_compress_write} will
@@ -478,10 +476,10 @@ perhaps not yet read.
@chapter Decompression functions
@cindex decompression functions
-These are the functions used to decompress data. In case of error, all
-of them return -1 or 0, for signed and unsigned return values
-respectively, except @samp{LZ_decompress_open} whose return value must
-be verified by calling @samp{LZ_decompress_errno} before using it.
+These are the functions used to decompress data. In case of error, all of
+them return -1 or 0, for signed and unsigned return values respectively,
+except @samp{LZ_decompress_open} whose return value must be verified by
+calling @samp{LZ_decompress_errno} before using it.
@deftypefun {struct LZ_Decoder *} LZ_decompress_open ( void )
@@ -539,14 +537,14 @@ function does nothing.
@deftypefun int LZ_decompress_read ( struct LZ_Decoder * const @var{decoder}, uint8_t * const @var{buffer}, const int @var{size} )
-The function @samp{LZ_decompress_read} reads up to @var{size} bytes from the
-stream pointed to by @var{decoder}, storing the results in @var{buffer}.
-If @w{LZ_API_VERSION >= 1012}, @var{buffer} may be a null pointer, in which
-case the bytes read are discarded.
-
-The return value is the number of bytes actually read. This might be less
-than @var{size}; for example, if there aren't that many bytes left in the
-stream or if more bytes have to be yet written with the function
+Reads up to @var{size} bytes from the stream pointed to by @var{decoder},
+storing the results in @var{buffer}. If @w{LZ_API_VERSION >= 1012},
+@var{buffer} may be a null pointer, in which case the bytes read are
+discarded.
+
+Returns the number of bytes actually read. This might be less than
+@var{size}; for example, if there aren't that many bytes left in the stream
+or if more bytes have to be yet written with the function
@samp{LZ_decompress_write}. Note that reading less than @var{size} bytes is
not an error.
@@ -571,20 +569,18 @@ recover as much data as possible from each damaged member.
@deftypefun int LZ_decompress_write ( struct LZ_Decoder * const @var{decoder}, uint8_t * const @var{buffer}, const int @var{size} )
-The function @samp{LZ_decompress_write} writes up to @var{size} bytes from
-@var{buffer} to the stream pointed to by @var{decoder}.
-
-The return value is the number of bytes actually written. This might be
-less than @var{size}. Note that writing less than @var{size} bytes is
-not an error.
+Writes up to @var{size} bytes from @var{buffer} to the stream pointed to by
+@var{decoder}. Returns the number of bytes actually written. This might be
+less than @var{size}. Note that writing less than @var{size} bytes is not an
+error.
@end deftypefun
@deftypefun int LZ_decompress_write_size ( struct LZ_Decoder * const @var{decoder} )
-The function @samp{LZ_decompress_write_size} returns the maximum number of
-bytes that can be immediately written through @samp{LZ_decompress_write}.
-This number varies smoothly; each compressed byte consumed may be
-overwritten immediately, increasing by 1 the value returned.
+Returns the maximum number of bytes that can be immediately written through
+@samp{LZ_decompress_write}. This number varies smoothly; each compressed
+byte consumed may be overwritten immediately, increasing by 1 the value
+returned.
It is guaranteed that an immediate call to @samp{LZ_decompress_write} will
accept a @var{size} up to the returned number of bytes.
@@ -607,26 +603,25 @@ does not imply @samp{LZ_decompress_member_finished}.
@deftypefun int LZ_decompress_member_finished ( struct LZ_Decoder * const @var{decoder} )
Returns 1 if the previous call to @samp{LZ_decompress_read} finished reading
-the current member, indicating that final values for member are available
+the current member, indicating that final values for the member are available
through @samp{LZ_decompress_data_crc}, @samp{LZ_decompress_data_position},
and @samp{LZ_decompress_member_position}. Otherwise it returns 0.
@end deftypefun
@deftypefun int LZ_decompress_member_version ( struct LZ_Decoder * const @var{decoder} )
-Returns the version of current member from member header.
+Returns the version of the current member, read from the member header.
@end deftypefun
@deftypefun int LZ_decompress_dictionary_size ( struct LZ_Decoder * const @var{decoder} )
-Returns the dictionary size of the current member, read from the member
-header.
+Returns the dictionary size of the current member, read from the member header.
@end deftypefun
@deftypefun {unsigned} LZ_decompress_data_crc ( struct LZ_Decoder * const @var{decoder} )
Returns the 32 bit Cyclic Redundancy Check of the data decompressed from
-the current member. The returned value is valid only when
+the current member. The value returned is valid only when
@samp{LZ_decompress_member_finished} returns 1.
@end deftypefun
@@ -672,8 +667,7 @@ examine @samp{LZ_(de)compress_errno}.
The error codes are defined in the header file @samp{lzlib.h}.
@deftypevr Constant {enum LZ_Errno} LZ_ok
-The value of this constant is 0 and is used to indicate that there is no
-error.
+The value of this constant is 0 and is used to indicate that there is no error.
@end deftypevr
@deftypevr Constant {enum LZ_Errno} LZ_bad_argument
@@ -737,16 +731,17 @@ The value of @var{lz_errno} normally comes from a call to
Minilzip is a test program for the compression library lzlib, fully
compatible with lzip 1.4 or newer.
-@uref{http://www.nongnu.org/lzip/lzip.html,,Lzip} is a lossless data
-compressor with a user interface similar to the one of gzip or bzip2. Lzip
-uses a simplified form of the 'Lempel-Ziv-Markov chain-Algorithm' (LZMA)
-stream format, chosen to maximize safety and interoperability. Lzip can
-compress about as fast as gzip @w{(lzip -0)} or compress most files more
-than bzip2 @w{(lzip -9)}. Decompression speed is intermediate between gzip
-and bzip2. Lzip is better than gzip and bzip2 from a data recovery
-perspective. Lzip has been designed, written, and tested with great care to
-replace gzip and bzip2 as the standard general-purpose compressed format for
-unix-like systems.
+@uref{http://www.nongnu.org/lzip/lzip.html,,Lzip}
+is a lossless data compressor with a user interface similar to the one
+of gzip or bzip2. Lzip uses a simplified form of the 'Lempel-Ziv-Markov
+chain-Algorithm' (LZMA) stream format and provides a 3 factor integrity
+checking to maximize interoperability and optimize safety. Lzip can compress
+about as fast as gzip @w{(lzip -0)} or compress most files more than bzip2
+@w{(lzip -9)}. Decompression speed is intermediate between gzip and bzip2.
+Lzip is better than gzip and bzip2 from a data recovery perspective. Lzip
+has been designed, written, and tested with great care to replace gzip and
+bzip2 as the standard general-purpose compressed format for unix-like
+systems.
@noindent
The format for running minilzip is:
@@ -803,10 +798,12 @@ and @samp{-S}. @samp{-c} has no effect when testing or listing.
@item -d
@itemx --decompress
-Decompress the files specified. If a file does not exist or can't be
-opened, minilzip continues decompressing the rest of the files. If a file
-fails to decompress, or is a terminal, minilzip exits immediately without
-decompressing the rest of the files.
+Decompress the files specified. If a file does not exist, can't be opened,
+or the destination file already exists and @samp{--force} has not been
+specified, minilzip continues decompressing the rest of the files and exits with
+error status 1. If a file fails to decompress, or is a terminal, minilzip exits
+immediately with error status 2 without decompressing the rest of the files.
+A terminal is considered an uncompressed file, and therefore invalid.
@item -f
@itemx --force
@@ -932,12 +929,13 @@ header" error and the cause is not indeed a corrupt header.
@item --check-lib
Compare the @uref{#Library-version,,version of lzlib} used to compile
-minilzip with the version actually being used and exit. Report any
-differences found. Exit with error status 1 if differences are found. A
+minilzip with the version actually being used at run time and exit. Report
+any differences found. Exit with error status 1 if differences are found. A
mismatch may indicate that lzlib is not correctly installed or that a
different version of lzlib has been installed after compiling the shared
-version of minilzip. @w{@samp{minilzip -v --check-lib}} shows the version of
-lzlib being used and the value of @samp{LZ_API_VERSION} (if defined).
+version of minilzip. Exit with error status 2 if LZ_API_VERSION and
+LZ_version_string don't match. @w{@samp{minilzip -v --check-lib}} shows the
+version of lzlib being used and the value of LZ_API_VERSION (if defined).
@ifnothtml
@xref{Library version}.
@end ifnothtml
@@ -963,9 +961,9 @@ Table of SI and binary prefixes (unit multipliers):
@sp 1
Exit status: 0 for a normal exit, 1 for environmental problems (file not
-found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or
-invalid input file, 3 for an internal consistency error (eg, bug) which
-caused minilzip to panic.
+found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or invalid
+input file, 3 for an internal consistency error (e.g., bug) which caused
+minilzip to panic.
@node Data format
@@ -996,9 +994,11 @@ represents one byte; a box like this:
represents a variable number of bytes.
@sp 1
-A lzip data stream consists of a series of "members" (compressed data sets).
-The members simply appear one after another in the data stream, with no
-additional information before, between, or after them.
+Lzip data consist of a series of independent "members" (compressed data
+sets). The members simply appear one after another in the data stream, with
+no additional information before, between, or after them. Each member can
+encode in compressed form up to @w{16 EiB - 1 byte} of uncompressed data.
+The size of a multimember data stream is unlimited.
Each member has the following structure:
@@ -1029,7 +1029,7 @@ Example: 0xD3 = 2^19 - 6 * 2^15 = 512 KiB - 6 * 32 KiB = 320 KiB@*
Valid values for dictionary size range from 4 KiB to 512 MiB.
@item LZMA stream
-The LZMA stream, finished by an end of stream marker. Uses default values
+The LZMA stream, finished by an "End Of Stream" marker. Uses default values
for encoder properties.
@ifnothtml
@xref{Stream format,,,lzip},
@@ -1043,15 +1043,17 @@ Lzip only uses the LZMA marker @samp{2} ("End Of Stream" marker). Lzlib
also uses the LZMA marker @samp{3} ("Sync Flush" marker). @xref{sync_flush}.
@item CRC32 (4 bytes)
-Cyclic Redundancy Check (CRC) of the uncompressed original data.
+Cyclic Redundancy Check (CRC) of the original uncompressed data.
@item Data size (8 bytes)
-Size of the uncompressed original data.
+Size of the original uncompressed data.
@item Member size (8 bytes)
Total size of the member, including header and trailer. This field acts
as a distributed index, allows the verification of stream integrity, and
-facilitates safe recovery of undamaged members from multimember files.
+facilitates the safe recovery of undamaged members from multimember files.
+Member size should be limited to @w{2 PiB} to prevent the data size field
+from overflowing.
@end table
@@ -1086,10 +1088,10 @@ Buffer-to-buffer single-member compression
@w{(@var{member_size} > total output)}.
@verbatim
-/* Compresses 'insize' bytes from 'inbuf' to 'outbuf'.
- Returns the size of the compressed data in '*outlenp'.
- In case of error, or if 'outsize' is too small, returns false and does
- not modify '*outlenp'.
+/* Compress 'insize' bytes from 'inbuf' to 'outbuf'.
+ Return the size of the compressed data in '*outlenp'.
+ In case of error, or if 'outsize' is too small, return false and do not
+ modify '*outlenp'.
*/
bool bbcompress( const uint8_t * const inbuf, const int insize,
const int dictionary_size, const int match_len_limit,
@@ -1131,10 +1133,10 @@ bool bbcompress( const uint8_t * const inbuf, const int insize,
Buffer-to-buffer decompression.
@verbatim
-/* Decompresses 'insize' bytes from 'inbuf' to 'outbuf'.
- Returns the size of the decompressed data in '*outlenp'.
- In case of error, or if 'outsize' is too small, returns false and does
- not modify '*outlenp'.
+/* Decompress 'insize' bytes from 'inbuf' to 'outbuf'.
+ Return the size of the decompressed data in '*outlenp'.
+ In case of error, or if 'outsize' is too small, return false and do not
+ modify '*outlenp'.
*/
bool bbdecompress( const uint8_t * const inbuf, const int insize,
uint8_t * const outbuf, const int outsize,
@@ -1285,9 +1287,9 @@ Example 2: Multimember compression (user-restarted members).
(Call LZ_compress_open with @var{member_size} > largest member).
@verbatim
-/* Compresses 'infile' to 'outfile' as a multimember stream with one member
+/* Compress 'infile' to 'outfile' as a multimember stream with one member
for each line of text terminated by a newline character or by EOF.
- Returns 0 if success, 1 if error.
+ Return 0 if success, 1 if error.
*/
int fflfcompress( struct LZ_Encoder * const encoder,
FILE * const infile, FILE * const outfile )
@@ -1332,7 +1334,7 @@ int fflfcompress( struct LZ_Encoder * const encoder,
@cindex skipping data errors
@verbatim
-/* Decompresses 'infile' to 'outfile' with automatic resynchronization to
+/* Decompress 'infile' to 'outfile' with automatic resynchronization to
next member in case of data error, including the automatic removal of
leading garbage.
*/
@@ -1381,8 +1383,8 @@ for all eternity, if not longer.
If you find a bug in lzlib, please send electronic mail to
@email{lzip-bug@@nongnu.org}. Include the version number, which you can
-find by running @w{@samp{minilzip --version}} or in
-@samp{LZ_version_string} from @samp{lzlib.h}.
+find by running @w{@samp{minilzip --version}} and
+@w{@samp{minilzip -v --check-lib}}.
@node Concept index
diff --git a/doc/minilzip.1 b/doc/minilzip.1
index 13a2d6d..0c4c06d 100644
--- a/doc/minilzip.1
+++ b/doc/minilzip.1
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.16.
-.TH MINILZIP "1" "January 2021" "minilzip 1.12" "User Commands"
+.TH MINILZIP "1" "January 2022" "minilzip 1.13" "User Commands"
.SH NAME
minilzip \- reduces the size of files
.SH SYNOPSIS
@@ -11,13 +11,14 @@ compatible with lzip 1.4 or newer.
.PP
Lzip is a lossless data compressor with a user interface similar to the one
of gzip or bzip2. Lzip uses a simplified form of the 'Lempel\-Ziv\-Markov
-chain\-Algorithm' (LZMA) stream format, chosen to maximize safety and
-interoperability. Lzip can compress about as fast as gzip (lzip \fB\-0\fR) or
-compress most files more than bzip2 (lzip \fB\-9\fR). Decompression speed is
-intermediate between gzip and bzip2. Lzip is better than gzip and bzip2 from
-a data recovery perspective. Lzip has been designed, written, and tested
-with great care to replace gzip and bzip2 as the standard general\-purpose
-compressed format for unix\-like systems.
+chain\-Algorithm' (LZMA) stream format and provides a 3 factor integrity
+checking to maximize interoperability and optimize safety. Lzip can compress
+about as fast as gzip (lzip \fB\-0\fR) or compress most files more than bzip2
+(lzip \fB\-9\fR). Decompression speed is intermediate between gzip and bzip2.
+Lzip is better than gzip and bzip2 from a data recovery perspective. Lzip
+has been designed, written, and tested with great care to replace gzip and
+bzip2 as the standard general\-purpose compressed format for unix\-like
+systems.
.SH OPTIONS
.TP
\fB\-h\fR, \fB\-\-help\fR
@@ -100,7 +101,7 @@ To extract all the files from archive 'foo.tar.lz', use the commands
.PP
Exit status: 0 for a normal exit, 1 for environmental problems (file
not found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or
-invalid input file, 3 for an internal consistency error (eg, bug) which
+invalid input file, 3 for an internal consistency error (e.g., bug) which
caused minilzip to panic.
.PP
The ideas embodied in lzlib are due to (at least) the following people:
@@ -113,9 +114,21 @@ Report bugs to lzip\-bug@nongnu.org
.br
Lzlib home page: http://www.nongnu.org/lzip/lzlib.html
.SH COPYRIGHT
-Copyright \(co 2021 Antonio Diaz Diaz.
-Using lzlib 1.12
+Copyright \(co 2022 Antonio Diaz Diaz.
+Using lzlib 1.13
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>
.br
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
+.SH "SEE ALSO"
+The full documentation for
+.B minilzip
+is maintained as a Texinfo manual. If the
+.B info
+and
+.B minilzip
+programs are properly installed at your site, the command
+.IP
+.B info lzlib
+.PP
+should give you access to the complete manual.