From f86afc6aa8317a9ed84be9f8c719dcc64cbd22d1 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Fri, 6 Nov 2015 12:24:27 +0100 Subject: Adding upstream version 1.0~rc2. Signed-off-by: Daniel Baumann --- doc/clzip.info | 443 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 443 insertions(+) create mode 100644 doc/clzip.info (limited to 'doc/clzip.info') diff --git a/doc/clzip.info b/doc/clzip.info new file mode 100644 index 0000000..d4a51b3 --- /dev/null +++ b/doc/clzip.info @@ -0,0 +1,443 @@ +This is clzip.info, produced by makeinfo version 4.13 from +clzip.texinfo. + +INFO-DIR-SECTION Data Compression +START-INFO-DIR-ENTRY +* Clzip: (clzip). Data compressor based on the LZMA algorithm +END-INFO-DIR-ENTRY + + +File: clzip.info, Node: Top, Next: Introduction, Up: (dir) + +Clzip Manual +************ + +This manual is for Clzip (version 1.0-rc2, 21 February 2010). + +* Menu: + +* Introduction:: Purpose and features of clzip +* Algorithm:: How clzip compresses the data +* Invoking Clzip:: Command line interface +* File Format:: Detailed format of the compressed file +* Examples:: A small tutorial with examples +* Problems:: Reporting bugs +* Concept Index:: Index of concepts + + + Copyright (C) 2010 Antonio Diaz Diaz. + + This manual is free documentation: you have unlimited permission to +copy, distribute and modify it. + + +File: clzip.info, Node: Introduction, Next: Algorithm, Prev: Top, Up: Top + +1 Introduction +************** + +Clzip is a lossless data compressor based on the LZMA algorithm, with +very safe integrity checking and a user interface similar to the one of +gzip or bzip2. Clzip decompresses almost as fast as gzip and compresses +better than bzip2, which makes it well suited for software distribution +and data archiving. + + Clzip replaces every file given in the command line with a compressed +version of itself, with the name "original_name.lz". Each compressed +file has the same modification date, permissions, and, when possible, +ownership as the corresponding original, so that these properties can be +correctly restored at decompression time. Clzip is able to read from +some types of non regular files if the `--stdout' option is specified. + + If no file names are specified, clzip compresses (or decompresses) +from standard input to standard output. In this case, clzip will +decline to write compressed output to a terminal, as this would be +entirely incomprehensible and therefore pointless. + + Clzip will correctly decompress a file which is the concatenation of +two or more compressed files. The result is the concatenation of the +corresponding uncompressed files. Integrity testing of concatenated +compressed files is also supported. + + Clzip can produce multimember files and safely recover, with +lziprecover, the undamaged members in case of file damage. Clzip can +also split the compressed output in volumes of a given size, even when +reading from standard input. This allows the direct creation of +multivolume compressed tar archives. + + The amount of memory required for compression is about 5 MiB plus 1 +or 2 times the dictionary size limit (1 if input file size is less than +dictionary size limit, else 2) plus 8 times the dictionary size really +used. For decompression is a little more than the dictionary size really +used. Clzip will automatically use the smallest possible dictionary size +without exceeding the given limit. It is important to appreciate that +the decompression memory requirement is affected at compression time by +the choice of dictionary size limit. + + When decompressing, clzip attempts to guess the name for the +decompressed file from that of the compressed file as follows: + +filename.lz becomes filename +filename.tlz becomes filename.tar +anyothername becomes anyothername.out + + As a self-check for your protection, clzip stores in the member +trailer the 32-bit CRC of the original data and the size of the +original data, to make sure that the decompressed version of the data +is identical to the original. This guards against corruption of the +compressed data, and against undetected bugs in clzip (hopefully very +unlikely). The chances of data corruption going undetected are +microscopic, less than one chance in 4000 million for each member +processed. Be aware, though, that the check occurs upon decompression, +so it can only tell you that something is wrong. It can't help you +recover the original uncompressed data. + + Return values: 0 for a normal exit, 1 for environmental problems +(file not found, invalid flags, I/O errors, etc), 2 to indicate a +corrupt or invalid input file, 3 for an internal consistency error (eg, +bug) which caused clzip to panic. + + +File: clzip.info, Node: Algorithm, Next: Invoking Clzip, Prev: Introduction, Up: Top + +2 Algorithm +*********** + +Clzip implements a simplified version of the LZMA (Lempel-Ziv-Markov +chain-Algorithm) algorithm. The original LZMA algorithm was designed by +Igor Pavlov. + + The high compression of LZMA comes from combining two basic, +well-proven compression ideas: sliding dictionaries (LZ77/78) and +markov models (the thing used by every compression algorithm that uses +a range encoder or similar order-0 entropy coder as its last stage) +with segregation of contexts according to what the bits are used for. + + Clzip is a two stage compressor. The first stage is a Lempel-Ziv +coder, which reduces redundancy by translating chunks of data to their +corresponding distance-length pairs. The second stage is a range encoder +that uses a different probability model for each type of data; +distances, lengths, literal bytes, etc. + + The match finder, part of the LZ coder, is the most important piece +of the LZMA algorithm, as it is in many Lempel-Ziv based algorithms. +Most of clzip's execution time is spent in the match finder, and it has +the greatest influence on the compression ratio. + + Here is how it works, step by step: + + 1) The member header is written to the output stream. + + 2) The first byte is coded literally, because there are no previous +bytes to which the match finder can refer to. + + 3) The main encoder advances to the next byte in the input data and +calls the match finder. + + 4) The match finder fills an array with the minimum distances before +the current byte where a match of a given length can be found. + + 5) Go back to step 3 until a sequence (formed of pairs, repeated +distances and literal bytes) of minimum price has been formed. Where the +price represents the number of output bits produced. + + 6) The range encoder encodes the sequence produced by the main +encoder and sends the produced bytes to the output stream. + + 7) Go back to step 3 until the input data is finished or until the +member or volume size limits are reached. + + 8) The range encoder is flushed. + + 9) The member trailer is written to the output stream. + + 10) If there are more data to compress, go back to step 1. + + +File: clzip.info, Node: Invoking Clzip, Next: File Format, Prev: Algorithm, Up: Top + +3 Invoking Clzip +**************** + +The format for running clzip is: + + clzip [OPTIONS] [FILES] + + Clzip supports the following options: + +`--help' +`-h' + Print an informative help message describing the options and exit. + +`--version' +`-V' + Print the version number of clzip on the standard output and exit. + +`--member-size=SIZE' +`-b SIZE' + Produce a multimember file and set the member size limit to SIZE + bytes. Minimum member size limit is 100kB. Small member size may + degrade compression ratio, so use it only when needed. The default + is to produce single member files. + +`--stdout' +`-c' + Compress or decompress to standard output. Needed when reading + from a named pipe (fifo) or from a device. Use it to recover as + much of the uncompressed data as possible when decompressing a + corrupt file. + +`--decompress' +`-d' + Decompress. + +`--force' +`-f' + Force overwrite of output file. + +`--keep' +`-k' + Keep (don't delete) input files during compression or + decompression. + +`--match-length=LENGTH' +`-m LENGTH' + Set the match length limit in bytes. Valid values range from 5 to + 273. Larger values usually give better compression ratios but + longer compression times. + +`--output=FILE' +`-o FILE' + When reading from standard input and `--stdout' has not been + specified, use `FILE' as the virtual name of the uncompressed + file. This produces a file named `FILE' when decompressing, a file + named `FILE.lz' when compressing, and several files named + `FILE00001.lz', `FILE00002.lz', etc, when compressing and + splitting the output in volumes. + +`--quiet' +`-q' + Quiet operation. Suppress all messages. + +`--dictionary-size=SIZE' +`-s SIZE' + Set the dictionary size limit in bytes. Valid values range from + 4KiB to 512MiB. Clzip will use the smallest possible dictionary + size for each member without exceeding this limit. Note that + dictionary sizes are quantized. If the specified size does not + match one of the valid sizes, it will be rounded upwards. + +`--volume-size=SIZE' +`-S SIZE' + Split the compressed output into several volume files with names + `original_name00001.lz', `original_name00002.lz', etc, and set the + volume size limit to SIZE bytes. Each volume is a complete, maybe + multimember, lzip file. Minimum volume size limit is 100kB. Small + volume size may degrade compression ratio, so use it only when + needed. + +`--test' +`-t' + Check integrity of the specified file(s), but don't decompress + them. This really performs a trial decompression and throws away + the result. Use `-tvv' or `-tvvv' to see information about the + file. + +`--verbose' +`-v' + Verbose mode. Show the compression ratio for each file processed. + Further -v's increase the verbosity level. + +`-1 .. -9' + Set the compression parameters (dictionary size and match length + limit) as shown in the table below. Note that `-9' can be much + slower than `-1'. These options have no effect when decompressing. + + Level Dictionary size Match length limit + -1 1 MiB 10 bytes + -2 1.5 MiB 12 bytes + -3 2 MiB 17 bytes + -4 3 MiB 26 bytes + -5 4 MiB 44 bytes + -6 8 MiB 80 bytes + -7 16 MiB 108 bytes + -8 24 MiB 163 bytes + -9 32 MiB 273 bytes + +`--fast' +`--best' + Aliases for GNU gzip compatibility. + + + + Numbers given as arguments to options may be followed by a multiplier +and an optional `B' for "byte". + + Table of SI and binary prefixes (unit multipliers): + +Prefix Value | Prefix Value +k kilobyte (10^3 = 1000) | Ki kibibyte (2^10 = 1024) +M megabyte (10^6) | Mi mebibyte (2^20) +G gigabyte (10^9) | Gi gibibyte (2^30) +T terabyte (10^12) | Ti tebibyte (2^40) +P petabyte (10^15) | Pi pebibyte (2^50) +E exabyte (10^18) | Ei exbibyte (2^60) +Z zettabyte (10^21) | Zi zebibyte (2^70) +Y yottabyte (10^24) | Yi yobibyte (2^80) + + +File: clzip.info, Node: File Format, Next: Examples, Prev: Invoking Clzip, Up: Top + +4 File Format +************* + +In the diagram below, a box like this: ++---+ +| | <-- the vertical bars might be missing ++---+ + + represents one byte; a box like this: ++==============+ +| | ++==============+ + + represents a variable number of bytes. + + + A lzip file consists of a series of "members" (compressed data sets). +The members simply appear one after another in the file, with no +additional information before, between, or after them. + + Each member has the following structure: ++--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| ID string | VN | DS | Lzma stream | CRC32 | Data size | Member size | ++--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + + All multibyte values are stored in little endian order. + +`ID string' + A four byte string, identifying the member type, with the value + "LZIP". + +`VN (version number, 1 byte)' + Just in case something needs to be modified in the future. Valid + values are 0 and 1. Version 0 files have only one member and lack + `Member size'. + +`DS (coded dictionary size, 1 byte)' + Bits 4-0 contain the base 2 logarithm of the base dictionary size. + Bits 7-5 contain the number of "wedges" to substract from the base + dictionary size to obtain the dictionary size. The size of a wedge + is (base dictionary size / 16). + Valid values for dictionary size range from 4KiB to 512MiB. + +`Lzma stream' + The lzma stream, finished by an end of stream marker. Uses default + values for encoder properties. + +`CRC32 (4 bytes)' + CRC of the uncompressed original data. + +`Data size (8 bytes)' + Size of the uncompressed original data. + +`Member size (8 bytes)' + Total size of the member, including header and trailer. This + facilitates safe recovery of undamaged members from multimember + files. + + + +File: clzip.info, Node: Examples, Next: Problems, Prev: File Format, Up: Top + +5 A small tutorial with examples +******************************** + +WARNING! If your data is important, give the `--keep' option to clzip +and do not remove the original file until you verify the compressed +file with a command like `clzip -cd file.lz | cmp file -'. + + +Example 1: Replace a regular file with its compressed version file.lz +and show the compression ratio. + + clzip -v file + + +Example 2: Like example 1 but the created file.lz is multimember with a +member size of 1MiB. + + clzip -b 1MiB file + + +Example 3: Compress a whole floppy in /dev/fd0 and send the output to +file.lz. + + clzip -c /dev/fd0 > file.lz + + +Example 4: Create a multivolume compressed tar archive with a volume +size of 1440KiB. + + tar -c some_directory | clzip -S 1440KiB -o volume_name + + +Example 5: Extract a multivolume compressed tar archive. + + clzip -cd volume_name*.lz | tar -xf - + + +Example 6: Create a multivolume compressed backup of a big database file +with a volume size of 650MB, where each volume is a multimember file +with a member size of 32MiB. + + clzip -b 32MiB -S 650MB big_database + + +File: clzip.info, Node: Problems, Next: Concept Index, Prev: Examples, Up: Top + +6 Reporting Bugs +**************** + +There are probably bugs in clzip. There are certainly errors and +omissions in this manual. If you report them, they will get fixed. If +you don't, no one will ever know about them and they will remain unfixed +for all eternity, if not longer. + + If you find a bug in clzip, please send electronic mail to +. Include the version number, which you can find +by running `clzip --version'. + + +File: clzip.info, Node: Concept Index, Prev: Problems, Up: Top + +Concept Index +************* + +[index] +* Menu: + +* algorithm: Algorithm. (line 6) +* bugs: Problems. (line 6) +* examples: Examples. (line 6) +* file format: File Format. (line 6) +* getting help: Problems. (line 6) +* introduction: Introduction. (line 6) +* invoking: Invoking Clzip. (line 6) +* options: Invoking Clzip. (line 6) +* usage: Invoking Clzip. (line 6) +* version: Invoking Clzip. (line 6) + + + +Tag Table: +Node: Top226 +Node: Introduction838 +Node: Algorithm4160 +Node: Invoking Clzip6391 +Node: File Format10747 +Node: Examples12703 +Node: Problems13880 +Node: Concept Index14406 + +End Tag Table -- cgit v1.2.3