summaryrefslogtreecommitdiffstats
path: root/doc/clzip.info
diff options
context:
space:
mode:
Diffstat (limited to 'doc/clzip.info')
-rw-r--r--doc/clzip.info443
1 files changed, 443 insertions, 0 deletions
diff --git a/doc/clzip.info b/doc/clzip.info
new file mode 100644
index 0000000..d4a51b3
--- /dev/null
+++ b/doc/clzip.info
@@ -0,0 +1,443 @@
+This is clzip.info, produced by makeinfo version 4.13 from
+clzip.texinfo.
+
+INFO-DIR-SECTION Data Compression
+START-INFO-DIR-ENTRY
+* Clzip: (clzip). Data compressor based on the LZMA algorithm
+END-INFO-DIR-ENTRY
+
+
+File: clzip.info, Node: Top, Next: Introduction, Up: (dir)
+
+Clzip Manual
+************
+
+This manual is for Clzip (version 1.0-rc2, 21 February 2010).
+
+* Menu:
+
+* Introduction:: Purpose and features of clzip
+* Algorithm:: How clzip compresses the data
+* Invoking Clzip:: Command line interface
+* File Format:: Detailed format of the compressed file
+* Examples:: A small tutorial with examples
+* Problems:: Reporting bugs
+* Concept Index:: Index of concepts
+
+
+ Copyright (C) 2010 Antonio Diaz Diaz.
+
+ This manual is free documentation: you have unlimited permission to
+copy, distribute and modify it.
+
+
+File: clzip.info, Node: Introduction, Next: Algorithm, Prev: Top, Up: Top
+
+1 Introduction
+**************
+
+Clzip is a lossless data compressor based on the LZMA algorithm, with
+very safe integrity checking and a user interface similar to the one of
+gzip or bzip2. Clzip decompresses almost as fast as gzip and compresses
+better than bzip2, which makes it well suited for software distribution
+and data archiving.
+
+ Clzip replaces every file given in the command line with a compressed
+version of itself, with the name "original_name.lz". Each compressed
+file has the same modification date, permissions, and, when possible,
+ownership as the corresponding original, so that these properties can be
+correctly restored at decompression time. Clzip is able to read from
+some types of non regular files if the `--stdout' option is specified.
+
+ If no file names are specified, clzip compresses (or decompresses)
+from standard input to standard output. In this case, clzip will
+decline to write compressed output to a terminal, as this would be
+entirely incomprehensible and therefore pointless.
+
+ Clzip will correctly decompress a file which is the concatenation of
+two or more compressed files. The result is the concatenation of the
+corresponding uncompressed files. Integrity testing of concatenated
+compressed files is also supported.
+
+ Clzip can produce multimember files and safely recover, with
+lziprecover, the undamaged members in case of file damage. Clzip can
+also split the compressed output in volumes of a given size, even when
+reading from standard input. This allows the direct creation of
+multivolume compressed tar archives.
+
+ The amount of memory required for compression is about 5 MiB plus 1
+or 2 times the dictionary size limit (1 if input file size is less than
+dictionary size limit, else 2) plus 8 times the dictionary size really
+used. For decompression is a little more than the dictionary size really
+used. Clzip will automatically use the smallest possible dictionary size
+without exceeding the given limit. It is important to appreciate that
+the decompression memory requirement is affected at compression time by
+the choice of dictionary size limit.
+
+ When decompressing, clzip attempts to guess the name for the
+decompressed file from that of the compressed file as follows:
+
+filename.lz becomes filename
+filename.tlz becomes filename.tar
+anyothername becomes anyothername.out
+
+ As a self-check for your protection, clzip stores in the member
+trailer the 32-bit CRC of the original data and the size of the
+original data, to make sure that the decompressed version of the data
+is identical to the original. This guards against corruption of the
+compressed data, and against undetected bugs in clzip (hopefully very
+unlikely). The chances of data corruption going undetected are
+microscopic, less than one chance in 4000 million for each member
+processed. Be aware, though, that the check occurs upon decompression,
+so it can only tell you that something is wrong. It can't help you
+recover the original uncompressed data.
+
+ Return values: 0 for a normal exit, 1 for environmental problems
+(file not found, invalid flags, I/O errors, etc), 2 to indicate a
+corrupt or invalid input file, 3 for an internal consistency error (eg,
+bug) which caused clzip to panic.
+
+
+File: clzip.info, Node: Algorithm, Next: Invoking Clzip, Prev: Introduction, Up: Top
+
+2 Algorithm
+***********
+
+Clzip implements a simplified version of the LZMA (Lempel-Ziv-Markov
+chain-Algorithm) algorithm. The original LZMA algorithm was designed by
+Igor Pavlov.
+
+ The high compression of LZMA comes from combining two basic,
+well-proven compression ideas: sliding dictionaries (LZ77/78) and
+markov models (the thing used by every compression algorithm that uses
+a range encoder or similar order-0 entropy coder as its last stage)
+with segregation of contexts according to what the bits are used for.
+
+ Clzip is a two stage compressor. The first stage is a Lempel-Ziv
+coder, which reduces redundancy by translating chunks of data to their
+corresponding distance-length pairs. The second stage is a range encoder
+that uses a different probability model for each type of data;
+distances, lengths, literal bytes, etc.
+
+ The match finder, part of the LZ coder, is the most important piece
+of the LZMA algorithm, as it is in many Lempel-Ziv based algorithms.
+Most of clzip's execution time is spent in the match finder, and it has
+the greatest influence on the compression ratio.
+
+ Here is how it works, step by step:
+
+ 1) The member header is written to the output stream.
+
+ 2) The first byte is coded literally, because there are no previous
+bytes to which the match finder can refer to.
+
+ 3) The main encoder advances to the next byte in the input data and
+calls the match finder.
+
+ 4) The match finder fills an array with the minimum distances before
+the current byte where a match of a given length can be found.
+
+ 5) Go back to step 3 until a sequence (formed of pairs, repeated
+distances and literal bytes) of minimum price has been formed. Where the
+price represents the number of output bits produced.
+
+ 6) The range encoder encodes the sequence produced by the main
+encoder and sends the produced bytes to the output stream.
+
+ 7) Go back to step 3 until the input data is finished or until the
+member or volume size limits are reached.
+
+ 8) The range encoder is flushed.
+
+ 9) The member trailer is written to the output stream.
+
+ 10) If there are more data to compress, go back to step 1.
+
+
+File: clzip.info, Node: Invoking Clzip, Next: File Format, Prev: Algorithm, Up: Top
+
+3 Invoking Clzip
+****************
+
+The format for running clzip is:
+
+ clzip [OPTIONS] [FILES]
+
+ Clzip supports the following options:
+
+`--help'
+`-h'
+ Print an informative help message describing the options and exit.
+
+`--version'
+`-V'
+ Print the version number of clzip on the standard output and exit.
+
+`--member-size=SIZE'
+`-b SIZE'
+ Produce a multimember file and set the member size limit to SIZE
+ bytes. Minimum member size limit is 100kB. Small member size may
+ degrade compression ratio, so use it only when needed. The default
+ is to produce single member files.
+
+`--stdout'
+`-c'
+ Compress or decompress to standard output. Needed when reading
+ from a named pipe (fifo) or from a device. Use it to recover as
+ much of the uncompressed data as possible when decompressing a
+ corrupt file.
+
+`--decompress'
+`-d'
+ Decompress.
+
+`--force'
+`-f'
+ Force overwrite of output file.
+
+`--keep'
+`-k'
+ Keep (don't delete) input files during compression or
+ decompression.
+
+`--match-length=LENGTH'
+`-m LENGTH'
+ Set the match length limit in bytes. Valid values range from 5 to
+ 273. Larger values usually give better compression ratios but
+ longer compression times.
+
+`--output=FILE'
+`-o FILE'
+ When reading from standard input and `--stdout' has not been
+ specified, use `FILE' as the virtual name of the uncompressed
+ file. This produces a file named `FILE' when decompressing, a file
+ named `FILE.lz' when compressing, and several files named
+ `FILE00001.lz', `FILE00002.lz', etc, when compressing and
+ splitting the output in volumes.
+
+`--quiet'
+`-q'
+ Quiet operation. Suppress all messages.
+
+`--dictionary-size=SIZE'
+`-s SIZE'
+ Set the dictionary size limit in bytes. Valid values range from
+ 4KiB to 512MiB. Clzip will use the smallest possible dictionary
+ size for each member without exceeding this limit. Note that
+ dictionary sizes are quantized. If the specified size does not
+ match one of the valid sizes, it will be rounded upwards.
+
+`--volume-size=SIZE'
+`-S SIZE'
+ Split the compressed output into several volume files with names
+ `original_name00001.lz', `original_name00002.lz', etc, and set the
+ volume size limit to SIZE bytes. Each volume is a complete, maybe
+ multimember, lzip file. Minimum volume size limit is 100kB. Small
+ volume size may degrade compression ratio, so use it only when
+ needed.
+
+`--test'
+`-t'
+ Check integrity of the specified file(s), but don't decompress
+ them. This really performs a trial decompression and throws away
+ the result. Use `-tvv' or `-tvvv' to see information about the
+ file.
+
+`--verbose'
+`-v'
+ Verbose mode. Show the compression ratio for each file processed.
+ Further -v's increase the verbosity level.
+
+`-1 .. -9'
+ Set the compression parameters (dictionary size and match length
+ limit) as shown in the table below. Note that `-9' can be much
+ slower than `-1'. These options have no effect when decompressing.
+
+ Level Dictionary size Match length limit
+ -1 1 MiB 10 bytes
+ -2 1.5 MiB 12 bytes
+ -3 2 MiB 17 bytes
+ -4 3 MiB 26 bytes
+ -5 4 MiB 44 bytes
+ -6 8 MiB 80 bytes
+ -7 16 MiB 108 bytes
+ -8 24 MiB 163 bytes
+ -9 32 MiB 273 bytes
+
+`--fast'
+`--best'
+ Aliases for GNU gzip compatibility.
+
+
+
+ Numbers given as arguments to options may be followed by a multiplier
+and an optional `B' for "byte".
+
+ Table of SI and binary prefixes (unit multipliers):
+
+Prefix Value | Prefix Value
+k kilobyte (10^3 = 1000) | Ki kibibyte (2^10 = 1024)
+M megabyte (10^6) | Mi mebibyte (2^20)
+G gigabyte (10^9) | Gi gibibyte (2^30)
+T terabyte (10^12) | Ti tebibyte (2^40)
+P petabyte (10^15) | Pi pebibyte (2^50)
+E exabyte (10^18) | Ei exbibyte (2^60)
+Z zettabyte (10^21) | Zi zebibyte (2^70)
+Y yottabyte (10^24) | Yi yobibyte (2^80)
+
+
+File: clzip.info, Node: File Format, Next: Examples, Prev: Invoking Clzip, Up: Top
+
+4 File Format
+*************
+
+In the diagram below, a box like this:
++---+
+| | <-- the vertical bars might be missing
++---+
+
+ represents one byte; a box like this:
++==============+
+| |
++==============+
+
+ represents a variable number of bytes.
+
+
+ A lzip file consists of a series of "members" (compressed data sets).
+The members simply appear one after another in the file, with no
+additional information before, between, or after them.
+
+ Each member has the following structure:
++--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+| ID string | VN | DS | Lzma stream | CRC32 | Data size | Member size |
++--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+ All multibyte values are stored in little endian order.
+
+`ID string'
+ A four byte string, identifying the member type, with the value
+ "LZIP".
+
+`VN (version number, 1 byte)'
+ Just in case something needs to be modified in the future. Valid
+ values are 0 and 1. Version 0 files have only one member and lack
+ `Member size'.
+
+`DS (coded dictionary size, 1 byte)'
+ Bits 4-0 contain the base 2 logarithm of the base dictionary size.
+ Bits 7-5 contain the number of "wedges" to substract from the base
+ dictionary size to obtain the dictionary size. The size of a wedge
+ is (base dictionary size / 16).
+ Valid values for dictionary size range from 4KiB to 512MiB.
+
+`Lzma stream'
+ The lzma stream, finished by an end of stream marker. Uses default
+ values for encoder properties.
+
+`CRC32 (4 bytes)'
+ CRC of the uncompressed original data.
+
+`Data size (8 bytes)'
+ Size of the uncompressed original data.
+
+`Member size (8 bytes)'
+ Total size of the member, including header and trailer. This
+ facilitates safe recovery of undamaged members from multimember
+ files.
+
+
+
+File: clzip.info, Node: Examples, Next: Problems, Prev: File Format, Up: Top
+
+5 A small tutorial with examples
+********************************
+
+WARNING! If your data is important, give the `--keep' option to clzip
+and do not remove the original file until you verify the compressed
+file with a command like `clzip -cd file.lz | cmp file -'.
+
+
+Example 1: Replace a regular file with its compressed version file.lz
+and show the compression ratio.
+
+ clzip -v file
+
+
+Example 2: Like example 1 but the created file.lz is multimember with a
+member size of 1MiB.
+
+ clzip -b 1MiB file
+
+
+Example 3: Compress a whole floppy in /dev/fd0 and send the output to
+file.lz.
+
+ clzip -c /dev/fd0 > file.lz
+
+
+Example 4: Create a multivolume compressed tar archive with a volume
+size of 1440KiB.
+
+ tar -c some_directory | clzip -S 1440KiB -o volume_name
+
+
+Example 5: Extract a multivolume compressed tar archive.
+
+ clzip -cd volume_name*.lz | tar -xf -
+
+
+Example 6: Create a multivolume compressed backup of a big database file
+with a volume size of 650MB, where each volume is a multimember file
+with a member size of 32MiB.
+
+ clzip -b 32MiB -S 650MB big_database
+
+
+File: clzip.info, Node: Problems, Next: Concept Index, Prev: Examples, Up: Top
+
+6 Reporting Bugs
+****************
+
+There are probably bugs in clzip. There are certainly errors and
+omissions in this manual. If you report them, they will get fixed. If
+you don't, no one will ever know about them and they will remain unfixed
+for all eternity, if not longer.
+
+ If you find a bug in clzip, please send electronic mail to
+<lzip-bug@nongnu.org>. Include the version number, which you can find
+by running `clzip --version'.
+
+
+File: clzip.info, Node: Concept Index, Prev: Problems, Up: Top
+
+Concept Index
+*************
+
+
+* Menu:
+
+* algorithm: Algorithm. (line 6)
+* bugs: Problems. (line 6)
+* examples: Examples. (line 6)
+* file format: File Format. (line 6)
+* getting help: Problems. (line 6)
+* introduction: Introduction. (line 6)
+* invoking: Invoking Clzip. (line 6)
+* options: Invoking Clzip. (line 6)
+* usage: Invoking Clzip. (line 6)
+* version: Invoking Clzip. (line 6)
+
+
+
+Tag Table:
+Node: Top226
+Node: Introduction838
+Node: Algorithm4160
+Node: Invoking Clzip6391
+Node: File Format10747
+Node: Examples12703
+Node: Problems13880
+Node: Concept Index14406
+
+End Tag Table
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-21 14:09:41 +0000