From b9c3804bd95fa4adf48dd177743ccedf6f35b321 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Sun, 26 Aug 2018 09:57:14 +0200 Subject: Adding upstream version 0.3. Signed-off-by: Daniel Baumann --- AUTHORS | 1 + COPYING | 338 +++++++++++++++++++ ChangeLog | 25 ++ INSTALL | 67 ++++ Makefile.in | 131 ++++++++ NEWS | 11 + README | 47 +++ carg_parser.c | 285 ++++++++++++++++ carg_parser.h | 93 ++++++ configure | 194 +++++++++++ decompress_lunzip.c | 100 ++++++ doc/xlunzip.1 | 90 ++++++ in_place.c | 220 +++++++++++++ linux_lunzip.h | 18 ++ linux_lzip.h | 53 +++ linux_mm.h | 36 +++ lzip.c | 881 ++++++++++++++++++++++++++++++++++++++++++++++++++ lzip.h | 173 ++++++++++ main.c | 720 +++++++++++++++++++++++++++++++++++++++++ testsuite/check.sh | 289 +++++++++++++++++ testsuite/test.txt | 676 ++++++++++++++++++++++++++++++++++++++ testsuite/test.txt.lz | Bin 0 -> 7376 bytes testsuite/zero.lz | Bin 0 -> 36 bytes 23 files changed, 4448 insertions(+) create mode 100644 AUTHORS create mode 100644 COPYING create mode 100644 ChangeLog create mode 100644 INSTALL create mode 100644 Makefile.in create mode 100644 NEWS create mode 100644 README create mode 100644 carg_parser.c create mode 100644 carg_parser.h create mode 100755 configure create mode 100644 decompress_lunzip.c create mode 100644 doc/xlunzip.1 create mode 100644 in_place.c create mode 100644 linux_lunzip.h create mode 100644 linux_lzip.h create mode 100644 linux_mm.h create mode 100644 lzip.c create mode 100644 lzip.h create mode 100644 main.c create mode 100755 testsuite/check.sh create mode 100644 testsuite/test.txt create mode 100644 testsuite/test.txt.lz create mode 100644 testsuite/zero.lz diff --git a/AUTHORS b/AUTHORS new file mode 100644 index 0000000..8a0bf38 --- /dev/null +++ b/AUTHORS @@ -0,0 +1 @@ +Xlunzip was written by Antonio Diaz Diaz. diff --git a/COPYING b/COPYING new file mode 100644 index 0000000..4ad17ae --- /dev/null +++ b/COPYING @@ -0,0 +1,338 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Lesser General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. diff --git a/ChangeLog b/ChangeLog new file mode 100644 index 0000000..2988e55 --- /dev/null +++ b/ChangeLog @@ -0,0 +1,25 @@ +2018-07-10 Antonio Diaz Diaz + + * Version 0.3 released. + * lzip.c: Use a precalculated CRC table as lzlib does. + +2018-07-09 Antonio Diaz Diaz + + * Version 0.2 released. + * lzip_decompress module updated to version 4.14.40-2. + * Improved corrupt header detection to HD=3. + * --in-place now works with both --decompress and --test. + * main.c: Show final diagnostic when testing multiple files. + +2018-06-30 Antonio Diaz Diaz + + * Version 0.1 released. + * Initial release. + * Tests the code shipped in linux patches before june 2018. + + +Copyright (C) 2016-2018 Antonio Diaz Diaz. + +This file is a collection of facts, and thus it is not copyrightable, +but just in case, you have unlimited permission to copy, distribute and +modify it. diff --git a/INSTALL b/INSTALL new file mode 100644 index 0000000..dbf34cb --- /dev/null +++ b/INSTALL @@ -0,0 +1,67 @@ +Requirements +------------ +You will need a C compiler. +I use gcc 5.3.0 and 4.1.2, but the code should compile with any +standards compliant compiler. +Gcc is available at http://gcc.gnu.org. + + +Procedure +--------- +1. Unpack the archive if you have not done so already: + + tar -xf xlunzip[version].tar.lz +or + lzip -cd xlunzip[version].tar.lz | tar -xf - + +This creates the directory ./xlunzip[version] containing the source from +the main archive. + +2. Change to xlunzip directory and run configure. + (Try 'configure --help' for usage instructions). + + cd xlunzip[version] + ./configure + +3. Run make. + + make + +4. Optionally, type 'make check' to run the tests that come with xlunzip. + +5. Type 'make install' to install the program and any data files and + documentation. + + Or type 'make install-compress', which additionally compresses the + man page after installation. (Installing compressed docs may become + the default in the future). + + You can install only the program or the man page by typing 'make + install-bin' or 'make install-man' respectively. + + Instead of 'make install', you can type 'make install-as-lzip' to + install the program and any data files and documentation, and link + the program to the name 'lzip'. + + +Another way +----------- +You can also compile xlunzip into a separate directory. +To do this, you must use a version of 'make' that supports the 'VPATH' +variable, such as GNU 'make'. 'cd' to the directory where you want the +object files and executables to go and run the 'configure' script. +'configure' automatically checks for the source code in '.', in '..' and +in the directory that 'configure' is in. + +'configure' recognizes the option '--srcdir=DIR' to control where to +look for the sources. Usually 'configure' can determine that directory +automatically. + +After running 'configure', you can run 'make' and 'make install' as +explained above. + + +Copyright (C) 2016-2018 Antonio Diaz Diaz. + +This file is free documentation: you have unlimited permission to copy, +distribute and modify it. diff --git a/Makefile.in b/Makefile.in new file mode 100644 index 0000000..7db7662 --- /dev/null +++ b/Makefile.in @@ -0,0 +1,131 @@ + +DISTNAME = $(pkgname)-$(pkgversion) +INSTALL = install +INSTALL_PROGRAM = $(INSTALL) -m 755 +INSTALL_DATA = $(INSTALL) -m 644 +INSTALL_DIR = $(INSTALL) -d -m 755 +SHELL = /bin/sh +CAN_RUN_INSTALLINFO = $(SHELL) -c "install-info --version" > /dev/null 2>&1 + +objs = carg_parser.o decompress_lunzip.o in_place.o lzip.o main.o + + +.PHONY : all install install-bin install-info install-man \ + install-strip install-compress install-strip-compress \ + install-bin-strip install-info-compress install-man-compress \ + install-as-lzip \ + uninstall uninstall-bin uninstall-info uninstall-man \ + doc info man check dist clean distclean + +all : $(progname) + +$(progname) : $(objs) + $(CC) $(LDFLAGS) $(CFLAGS) -o $@ $(objs) + +main.o : main.c + $(CC) $(CPPFLAGS) $(CFLAGS) -DPROGVERSION=\"$(pkgversion)\" -c -o $@ $< + +%.o : %.c + $(CC) $(CPPFLAGS) $(CFLAGS) -c -o $@ $< + +$(objs) : Makefile +carg_parser.o : carg_parser.h +decompress_lunzip.o : linux_lzip.h linux_lunzip.h linux_mm.h lzip.c +in_place.o : linux_lunzip.h lzip.h +lzip.o : linux_lzip.h linux_mm.h +main.o : carg_parser.h linux_lzip.h linux_lunzip.h lzip.h + + +doc : man + +info : $(VPATH)/doc/$(pkgname).info + +$(VPATH)/doc/$(pkgname).info : $(VPATH)/doc/$(pkgname).texi + cd $(VPATH)/doc && makeinfo $(pkgname).texi + +man : $(VPATH)/doc/$(progname).1 + +$(VPATH)/doc/$(progname).1 : $(progname) + help2man -n 'test tool for the lunzip linux module' -o $@ --no-info ./$(progname) + +Makefile : $(VPATH)/configure $(VPATH)/Makefile.in + ./config.status + +check : all + @$(VPATH)/testsuite/check.sh $(VPATH)/testsuite $(pkgversion) + +install : install-bin install-man +install-strip : install-bin-strip install-man +install-compress : install-bin install-man-compress +install-strip-compress : install-bin-strip install-man-compress + +install-bin : all + if [ ! -d "$(DESTDIR)$(bindir)" ] ; then $(INSTALL_DIR) "$(DESTDIR)$(bindir)" ; fi + $(INSTALL_PROGRAM) ./$(progname) "$(DESTDIR)$(bindir)/$(progname)" + +install-bin-strip : all + $(MAKE) INSTALL_PROGRAM='$(INSTALL_PROGRAM) -s' install-bin + +install-info : + if [ ! -d "$(DESTDIR)$(infodir)" ] ; then $(INSTALL_DIR) "$(DESTDIR)$(infodir)" ; fi + -rm -f "$(DESTDIR)$(infodir)/$(pkgname).info"* + $(INSTALL_DATA) $(VPATH)/doc/$(pkgname).info "$(DESTDIR)$(infodir)/$(pkgname).info" + -if $(CAN_RUN_INSTALLINFO) ; then \ + install-info --info-dir="$(DESTDIR)$(infodir)" "$(DESTDIR)$(infodir)/$(pkgname).info" ; \ + fi + +install-info-compress : install-info + lzip -v -9 "$(DESTDIR)$(infodir)/$(pkgname).info" + +install-man : + if [ ! -d "$(DESTDIR)$(mandir)/man1" ] ; then $(INSTALL_DIR) "$(DESTDIR)$(mandir)/man1" ; fi + -rm -f "$(DESTDIR)$(mandir)/man1/$(progname).1"* + $(INSTALL_DATA) $(VPATH)/doc/$(progname).1 "$(DESTDIR)$(mandir)/man1/$(progname).1" + +install-man-compress : install-man + lzip -v -9 "$(DESTDIR)$(mandir)/man1/$(progname).1" + +install-as-lzip : install + -rm -f "$(DESTDIR)$(bindir)/lzip" + cd "$(DESTDIR)$(bindir)" && ln -s $(progname) lzip + +uninstall : uninstall-man uninstall-bin + +uninstall-bin : + -rm -f "$(DESTDIR)$(bindir)/$(progname)" + +uninstall-info : + -if $(CAN_RUN_INSTALLINFO) ; then \ + install-info --info-dir="$(DESTDIR)$(infodir)" --remove "$(DESTDIR)$(infodir)/$(pkgname).info" ; \ + fi + -rm -f "$(DESTDIR)$(infodir)/$(pkgname).info"* + +uninstall-man : + -rm -f "$(DESTDIR)$(mandir)/man1/$(progname).1"* + +dist : doc + ln -sf $(VPATH) $(DISTNAME) + tar -Hustar --owner=root --group=root -cvf $(DISTNAME).tar \ + $(DISTNAME)/AUTHORS \ + $(DISTNAME)/COPYING \ + $(DISTNAME)/ChangeLog \ + $(DISTNAME)/INSTALL \ + $(DISTNAME)/Makefile.in \ + $(DISTNAME)/NEWS \ + $(DISTNAME)/README \ + $(DISTNAME)/configure \ + $(DISTNAME)/doc/$(progname).1 \ + $(DISTNAME)/*.h \ + $(DISTNAME)/*.c \ + $(DISTNAME)/testsuite/check.sh \ + $(DISTNAME)/testsuite/test.txt \ + $(DISTNAME)/testsuite/test.txt.lz \ + $(DISTNAME)/testsuite/zero.lz + rm -f $(DISTNAME) + lzip -v -9 $(DISTNAME).tar + +clean : + -rm -f $(progname) $(objs) + +distclean : clean + -rm -f Makefile config.status *.tar *.tar.lz diff --git a/NEWS b/NEWS new file mode 100644 index 0000000..47d2d69 --- /dev/null +++ b/NEWS @@ -0,0 +1,11 @@ +Changes in version 0.3: + +lzip_decompress module updated to version 4.14.40-3. + +Use a precalculated CRC table as lzlib does. + +Improved corrupt header detection to HD=3. + +'--in-place' now works with both '--decompress' and '--test'. + +Show final diagnostic when testing multiple files. diff --git a/README b/README new file mode 100644 index 0000000..1b26b05 --- /dev/null +++ b/README @@ -0,0 +1,47 @@ +Description + +Xlunzip is a test tool for the lzip decompression code of my lzip patch +for linux. Xlunzip is similar to lunzip, but it uses the lzip_decompress +linux module as a backend. Xlunzip tests the module for stream, +buffer-to-buffer and mixed decompression modes, including in-place +decompression (using the same buffer for input and output). You can use +xlunzip to verify that the module produces correct results when +decompressing single member files, multimember files, or the +concatenation of two or more compressed files. Xlunzip can be used with +unzcrash to test the robustness of the module to the decompression of +corrupted data. + +Note that the in-place decompression of concatenated files can't be +guaranteed to work because an arbitrarily low compression ratio of the +last part of the data can be achieved by appending enough empty +compressed members to a file. + +The xlunzip tarball contains a copy of the lzip_decompress module and +can be compiled and tested without downloading or applying the patch to +the kernel. + +My lzip patch for linux can be found at +http://download.savannah.gnu.org/releases/lzip/kernel/ + +Lzip related components in the kernel +===================================== + +The lzip_decompress module in lib/lzip.c provides a versatile lzip +decompression function able to do buffer to buffer decompression or +stream decompression with fill and flush callback functions. The usage +of the function is documented in include/linux/lzip.h. + +For decompressing the kernel image, initramfs, and initrd, there is a +wrapper function in lib/decompress_lunzip.c providing the same common +interface as the other decompress_*.c files, which is defined in +include/linux/decompress/generic.h. + + +Copyright (C) 2016-2018 Antonio Diaz Diaz. + +This file is free documentation: you have unlimited permission to copy, +distribute and modify it. + +The file Makefile.in is a data file used by configure to produce the +Makefile. It has the same copyright owner and permissions that configure +itself. diff --git a/carg_parser.c b/carg_parser.c new file mode 100644 index 0000000..10ad4dc --- /dev/null +++ b/carg_parser.c @@ -0,0 +1,285 @@ +/* Arg_parser - POSIX/GNU command line argument parser. (C version) + Copyright (C) 2006-2018 Antonio Diaz Diaz. + + This library is free software. Redistribution and use in source and + binary forms, with or without modification, are permitted provided + that the following conditions are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + This library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. +*/ + +#include +#include + +#include "carg_parser.h" + + +/* assure at least a minimum size for buffer 'buf' */ +static void * ap_resize_buffer( void * buf, const int min_size ) + { + if( buf ) buf = realloc( buf, min_size ); + else buf = malloc( min_size ); + return buf; + } + + +static char push_back_record( struct Arg_parser * const ap, + const int code, const char * const argument ) + { + const int len = strlen( argument ); + struct ap_Record * p; + void * tmp = ap_resize_buffer( ap->data, + ( ap->data_size + 1 ) * sizeof (struct ap_Record) ); + if( !tmp ) return 0; + ap->data = (struct ap_Record *)tmp; + p = &(ap->data[ap->data_size]); + p->code = code; + p->argument = 0; + tmp = ap_resize_buffer( p->argument, len + 1 ); + if( !tmp ) return 0; + p->argument = (char *)tmp; + strncpy( p->argument, argument, len + 1 ); + ++ap->data_size; + return 1; + } + + +static char add_error( struct Arg_parser * const ap, const char * const msg ) + { + const int len = strlen( msg ); + void * tmp = ap_resize_buffer( ap->error, ap->error_size + len + 1 ); + if( !tmp ) return 0; + ap->error = (char *)tmp; + strncpy( ap->error + ap->error_size, msg, len + 1 ); + ap->error_size += len; + return 1; + } + + +static void free_data( struct Arg_parser * const ap ) + { + int i; + for( i = 0; i < ap->data_size; ++i ) free( ap->data[i].argument ); + if( ap->data ) { free( ap->data ); ap->data = 0; } + ap->data_size = 0; + } + + +static char parse_long_option( struct Arg_parser * const ap, + const char * const opt, const char * const arg, + const struct ap_Option options[], + int * const argindp ) + { + unsigned len; + int index = -1, i; + char exact = 0, ambig = 0; + + for( len = 0; opt[len+2] && opt[len+2] != '='; ++len ) ; + + /* Test all long options for either exact match or abbreviated matches. */ + for( i = 0; options[i].code != 0; ++i ) + if( options[i].name && strncmp( options[i].name, &opt[2], len ) == 0 ) + { + if( strlen( options[i].name ) == len ) /* Exact match found */ + { index = i; exact = 1; break; } + else if( index < 0 ) index = i; /* First nonexact match found */ + else if( options[index].code != options[i].code || + options[index].has_arg != options[i].has_arg ) + ambig = 1; /* Second or later nonexact match found */ + } + + if( ambig && !exact ) + { + add_error( ap, "option '" ); add_error( ap, opt ); + add_error( ap, "' is ambiguous" ); + return 1; + } + + if( index < 0 ) /* nothing found */ + { + add_error( ap, "unrecognized option '" ); add_error( ap, opt ); + add_error( ap, "'" ); + return 1; + } + + ++*argindp; + + if( opt[len+2] ) /* '--=' syntax */ + { + if( options[index].has_arg == ap_no ) + { + add_error( ap, "option '--" ); add_error( ap, options[index].name ); + add_error( ap, "' doesn't allow an argument" ); + return 1; + } + if( options[index].has_arg == ap_yes && !opt[len+3] ) + { + add_error( ap, "option '--" ); add_error( ap, options[index].name ); + add_error( ap, "' requires an argument" ); + return 1; + } + return push_back_record( ap, options[index].code, &opt[len+3] ); + } + + if( options[index].has_arg == ap_yes ) + { + if( !arg || !arg[0] ) + { + add_error( ap, "option '--" ); add_error( ap, options[index].name ); + add_error( ap, "' requires an argument" ); + return 1; + } + ++*argindp; + return push_back_record( ap, options[index].code, arg ); + } + + return push_back_record( ap, options[index].code, "" ); + } + + +static char parse_short_option( struct Arg_parser * const ap, + const char * const opt, const char * const arg, + const struct ap_Option options[], + int * const argindp ) + { + int cind = 1; /* character index in opt */ + + while( cind > 0 ) + { + int index = -1, i; + const unsigned char code = opt[cind]; + char code_str[2]; + code_str[0] = code; code_str[1] = 0; + + if( code != 0 ) + for( i = 0; options[i].code; ++i ) + if( code == options[i].code ) + { index = i; break; } + + if( index < 0 ) + { + add_error( ap, "invalid option -- '" ); add_error( ap, code_str ); + add_error( ap, "'" ); + return 1; + } + + if( opt[++cind] == 0 ) { ++*argindp; cind = 0; } /* opt finished */ + + if( options[index].has_arg != ap_no && cind > 0 && opt[cind] ) + { + if( !push_back_record( ap, code, &opt[cind] ) ) return 0; + ++*argindp; cind = 0; + } + else if( options[index].has_arg == ap_yes ) + { + if( !arg || !arg[0] ) + { + add_error( ap, "option requires an argument -- '" ); + add_error( ap, code_str ); add_error( ap, "'" ); + return 1; + } + ++*argindp; cind = 0; + if( !push_back_record( ap, code, arg ) ) return 0; + } + else if( !push_back_record( ap, code, "" ) ) return 0; + } + return 1; + } + + +char ap_init( struct Arg_parser * const ap, + const int argc, const char * const argv[], + const struct ap_Option options[], const char in_order ) + { + const char ** non_options = 0; /* skipped non-options */ + int non_options_size = 0; /* number of skipped non-options */ + int argind = 1; /* index in argv */ + int i; + + ap->data = 0; + ap->error = 0; + ap->data_size = 0; + ap->error_size = 0; + if( argc < 2 || !argv || !options ) return 1; + + while( argind < argc ) + { + const unsigned char ch1 = argv[argind][0]; + const unsigned char ch2 = ch1 ? argv[argind][1] : 0; + + if( ch1 == '-' && ch2 ) /* we found an option */ + { + const char * const opt = argv[argind]; + const char * const arg = ( argind + 1 < argc ) ? argv[argind+1] : 0; + if( ch2 == '-' ) + { + if( !argv[argind][2] ) { ++argind; break; } /* we found "--" */ + else if( !parse_long_option( ap, opt, arg, options, &argind ) ) return 0; + } + else if( !parse_short_option( ap, opt, arg, options, &argind ) ) return 0; + if( ap->error ) break; + } + else + { + if( in_order ) + { if( !push_back_record( ap, 0, argv[argind++] ) ) return 0; } + else + { + void * tmp = ap_resize_buffer( non_options, + ( non_options_size + 1 ) * sizeof *non_options ); + if( !tmp ) return 0; + non_options = (const char **)tmp; + non_options[non_options_size++] = argv[argind++]; + } + } + } + if( ap->error ) free_data( ap ); + else + { + for( i = 0; i < non_options_size; ++i ) + if( !push_back_record( ap, 0, non_options[i] ) ) return 0; + while( argind < argc ) + if( !push_back_record( ap, 0, argv[argind++] ) ) return 0; + } + if( non_options ) free( non_options ); + return 1; + } + + +void ap_free( struct Arg_parser * const ap ) + { + free_data( ap ); + if( ap->error ) { free( ap->error ); ap->error = 0; } + ap->error_size = 0; + } + + +const char * ap_error( const struct Arg_parser * const ap ) + { return ap->error; } + + +int ap_arguments( const struct Arg_parser * const ap ) + { return ap->data_size; } + + +int ap_code( const struct Arg_parser * const ap, const int i ) + { + if( i >= 0 && i < ap_arguments( ap ) ) return ap->data[i].code; + else return 0; + } + + +const char * ap_argument( const struct Arg_parser * const ap, const int i ) + { + if( i >= 0 && i < ap_arguments( ap ) ) return ap->data[i].argument; + else return ""; + } diff --git a/carg_parser.h b/carg_parser.h new file mode 100644 index 0000000..e1c70dd --- /dev/null +++ b/carg_parser.h @@ -0,0 +1,93 @@ +/* Arg_parser - POSIX/GNU command line argument parser. (C version) + Copyright (C) 2006-2018 Antonio Diaz Diaz. + + This library is free software. Redistribution and use in source and + binary forms, with or without modification, are permitted provided + that the following conditions are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + This library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. +*/ + +/* Arg_parser reads the arguments in 'argv' and creates a number of + option codes, option arguments and non-option arguments. + + In case of error, 'ap_error' returns a non-null pointer to an error + message. + + 'options' is an array of 'struct ap_Option' terminated by an element + containing a code which is zero. A null name means a short-only + option. A code value outside the unsigned char range means a + long-only option. + + Arg_parser normally makes it appear as if all the option arguments + were specified before all the non-option arguments for the purposes + of parsing, even if the user of your program intermixed option and + non-option arguments. If you want the arguments in the exact order + the user typed them, call 'ap_init' with 'in_order' = true. + + The argument '--' terminates all options; any following arguments are + treated as non-option arguments, even if they begin with a hyphen. + + The syntax for optional option arguments is '-' + (without whitespace), or '--='. +*/ + +#ifdef __cplusplus +extern "C" { +#endif + +enum ap_Has_arg { ap_no, ap_yes, ap_maybe }; + +struct ap_Option + { + int code; /* Short option letter or code ( code != 0 ) */ + const char * name; /* Long option name (maybe null) */ + enum ap_Has_arg has_arg; + }; + + +struct ap_Record + { + int code; + char * argument; + }; + + +struct Arg_parser + { + struct ap_Record * data; + char * error; + int data_size; + int error_size; + }; + + +char ap_init( struct Arg_parser * const ap, + const int argc, const char * const argv[], + const struct ap_Option options[], const char in_order ); + +void ap_free( struct Arg_parser * const ap ); + +const char * ap_error( const struct Arg_parser * const ap ); + + /* The number of arguments parsed (may be different from argc) */ +int ap_arguments( const struct Arg_parser * const ap ); + + /* If ap_code( i ) is 0, ap_argument( i ) is a non-option. + Else ap_argument( i ) is the option's argument (or empty). */ +int ap_code( const struct Arg_parser * const ap, const int i ); + +const char * ap_argument( const struct Arg_parser * const ap, const int i ); + +#ifdef __cplusplus +} +#endif diff --git a/configure b/configure new file mode 100755 index 0000000..10a327b --- /dev/null +++ b/configure @@ -0,0 +1,194 @@ +#! /bin/sh +# configure script for Xlunzip - Test tool for the lunzip linux module +# Copyright (C) 2016-2018 Antonio Diaz Diaz. +# +# This configure script is free software: you have unlimited permission +# to copy, distribute and modify it. + +pkgname=xlunzip +pkgversion=0.3 +progname=xlunzip +srctrigger=doc/${progname}.1 + +# clear some things potentially inherited from environment. +LC_ALL=C +export LC_ALL +srcdir= +prefix=/usr/local +exec_prefix='$(prefix)' +bindir='$(exec_prefix)/bin' +datarootdir='$(prefix)/share' +infodir='$(datarootdir)/info' +mandir='$(datarootdir)/man' +CC=gcc +CPPFLAGS= +CFLAGS='-Wall -W -O2' +LDFLAGS= + +# checking whether we are using GNU C. +/bin/sh -c "${CC} --version" > /dev/null 2>&1 || + { + CC=cc + CFLAGS=-O2 + } + +# Loop over all args +args= +no_create= +while [ $# != 0 ] ; do + + # Get the first arg, and shuffle + option=$1 ; arg2=no + shift + + # Add the argument quoted to args + args="${args} \"${option}\"" + + # Split out the argument for options that take them + case ${option} in + *=*) optarg=`echo ${option} | sed -e 's,^[^=]*=,,;s,/$,,'` ;; + esac + + # Process the options + case ${option} in + --help | -h) + echo "Usage: $0 [OPTION]... [VAR=VALUE]..." + echo + echo "To assign makefile variables (e.g., CC, CFLAGS...), specify them as" + echo "arguments to configure in the form VAR=VALUE." + echo + echo "Options and variables: [defaults in brackets]" + echo " -h, --help display this help and exit" + echo " -V, --version output version information and exit" + echo " --srcdir=DIR find the sources in DIR [. or ..]" + echo " --prefix=DIR install into DIR [${prefix}]" + echo " --exec-prefix=DIR base directory for arch-dependent files [${exec_prefix}]" + echo " --bindir=DIR user executables directory [${bindir}]" + echo " --datarootdir=DIR base directory for doc and data [${datarootdir}]" + echo " --infodir=DIR info files directory [${infodir}]" + echo " --mandir=DIR man pages directory [${mandir}]" + echo " CC=COMPILER C compiler to use [${CC}]" + echo " CPPFLAGS=OPTIONS command line options for the preprocessor [${CPPFLAGS}]" + echo " CFLAGS=OPTIONS command line options for the C compiler [${CFLAGS}]" + echo " LDFLAGS=OPTIONS command line options for the linker [${LDFLAGS}]" + echo + exit 0 ;; + --version | -V) + echo "Configure script for ${pkgname} version ${pkgversion}" + exit 0 ;; + --srcdir) srcdir=$1 ; arg2=yes ;; + --prefix) prefix=$1 ; arg2=yes ;; + --exec-prefix) exec_prefix=$1 ; arg2=yes ;; + --bindir) bindir=$1 ; arg2=yes ;; + --datarootdir) datarootdir=$1 ; arg2=yes ;; + --infodir) infodir=$1 ; arg2=yes ;; + --mandir) mandir=$1 ; arg2=yes ;; + + --srcdir=*) srcdir=${optarg} ;; + --prefix=*) prefix=${optarg} ;; + --exec-prefix=*) exec_prefix=${optarg} ;; + --bindir=*) bindir=${optarg} ;; + --datarootdir=*) datarootdir=${optarg} ;; + --infodir=*) infodir=${optarg} ;; + --mandir=*) mandir=${optarg} ;; + --no-create) no_create=yes ;; + + CC=*) CC=${optarg} ;; + CPPFLAGS=*) CPPFLAGS=${optarg} ;; + CFLAGS=*) CFLAGS=${optarg} ;; + LDFLAGS=*) LDFLAGS=${optarg} ;; + + --*) + echo "configure: WARNING: unrecognized option: '${option}'" 1>&2 ;; + *=* | *-*-*) ;; + *) + echo "configure: unrecognized option: '${option}'" 1>&2 + echo "Try 'configure --help' for more information." 1>&2 + exit 1 ;; + esac + + # Check if the option took a separate argument + if [ "${arg2}" = yes ] ; then + if [ $# != 0 ] ; then args="${args} \"$1\"" ; shift + else echo "configure: Missing argument to '${option}'" 1>&2 + exit 1 + fi + fi +done + +# Find the source files, if location was not specified. +srcdirtext= +if [ -z "${srcdir}" ] ; then + srcdirtext="or . or .." ; srcdir=. + if [ ! -r "${srcdir}/${srctrigger}" ] ; then srcdir=.. ; fi + if [ ! -r "${srcdir}/${srctrigger}" ] ; then + ## the sed command below emulates the dirname command + srcdir=`echo $0 | sed -e 's,[^/]*$,,;s,/$,,;s,^$,.,'` + fi +fi + +if [ ! -r "${srcdir}/${srctrigger}" ] ; then + echo "configure: Can't find sources in ${srcdir} ${srcdirtext}" 1>&2 + echo "configure: (At least ${srctrigger} is missing)." 1>&2 + exit 1 +fi + +# Set srcdir to . if that's what it is. +if [ "`pwd`" = "`cd "${srcdir}" ; pwd`" ] ; then srcdir=. ; fi + +echo +if [ -z "${no_create}" ] ; then + echo "creating config.status" + rm -f config.status + cat > config.status << EOF +#! /bin/sh +# This file was generated automatically by configure. Don't edit. +# Run this file to recreate the current configuration. +# +# This script is free software: you have unlimited permission +# to copy, distribute and modify it. + +exec /bin/sh $0 ${args} --no-create +EOF + chmod +x config.status +fi + +echo "creating Makefile" +echo "VPATH = ${srcdir}" +echo "prefix = ${prefix}" +echo "exec_prefix = ${exec_prefix}" +echo "bindir = ${bindir}" +echo "datarootdir = ${datarootdir}" +echo "infodir = ${infodir}" +echo "mandir = ${mandir}" +echo "CC = ${CC}" +echo "CPPFLAGS = ${CPPFLAGS}" +echo "CFLAGS = ${CFLAGS}" +echo "LDFLAGS = ${LDFLAGS}" +rm -f Makefile +cat > Makefile << EOF +# Makefile for Xlunzip - Test tool for the lunzip linux module +# Copyright (C) 2016-2018 Antonio Diaz Diaz. +# This file was generated automatically by configure. Don't edit. +# +# This Makefile is free software: you have unlimited permission +# to copy, distribute and modify it. + +pkgname = ${pkgname} +pkgversion = ${pkgversion} +progname = ${progname} +VPATH = ${srcdir} +prefix = ${prefix} +exec_prefix = ${exec_prefix} +bindir = ${bindir} +datarootdir = ${datarootdir} +infodir = ${infodir} +mandir = ${mandir} +CC = ${CC} +CPPFLAGS = ${CPPFLAGS} +CFLAGS = ${CFLAGS} +LDFLAGS = ${LDFLAGS} +EOF +cat "${srcdir}/Makefile.in" >> Makefile + +echo "OK. Now you can run make." diff --git a/decompress_lunzip.c b/decompress_lunzip.c new file mode 100644 index 0000000..106c95b --- /dev/null +++ b/decompress_lunzip.c @@ -0,0 +1,100 @@ +/* + * Wrapper for decompressing LZIP-compressed kernel, initramfs, and initrd + * + * Copyright (C) 2016-2018 Antonio Diaz Diaz. + * + * Licensed under GPLv2 or later, see file LICENSE in this source tree. + */ + +#ifdef STATIC +#define PREBOOT +#include "lzip.c" +#else +#include "linux_lzip.h" +#include "linux_lunzip.h" +#include "linux_mm.h" +#endif + +STATIC int INIT __lunzip(unsigned char *inbuf, long in_len, + long (*fill)(void*, unsigned long), + long (*flush)(void*, unsigned long), + unsigned char *outbuf, long out_size, + long *in_posp, long *out_posp, + void (*error)(char *x)) +{ + const int retval = lzip_decompress(inbuf, in_len, fill, flush, + outbuf, out_size, in_posp, out_posp); + switch (retval) { + case 0: break; + case LZIP_OOM_INBUF: + error("Out of memory while allocating input buffer."); + break; + case LZIP_HEADER1_EOF: + error("File ends unexpectedly at member header."); + break; + case LZIP_HEADER2_EOF: + error("Truncated header in multimember file."); + break; + case LZIP_BAD_MAGIC1: + error("Bad magic number (file not in lzip format)."); + break; + case LZIP_BAD_MAGIC2: + error("Corrupt header in multimember file."); + break; + case LZIP_BAD_VERSION: + error("Version of lzip member format not supported."); + break; + case LZIP_BAD_DICT_SIZE: + error("Invalid dictionary size in member header."); + break; + case LZIP_OOM_OUTBUF: + error("Out of memory while allocating output buffer."); + break; + case LZIP_WRITE_ERROR: + error("Write error."); + break; + case LZIP_BAD_DATA: + error("LZIP-compressed data is corrupt."); + break; + case LZIP_DATA_EOF: + error("LZIP-compressed data ends unexpectedly."); + break; + case LZIP_BAD_CRC: + error("CRC mismatch in LZIP-compressed data."); + break; + default: + error("Bug in the LZIP decompressor."); + } + return retval; +} + +#ifndef PREBOOT +/* decompress_fn (see linux/decompress/generic.h) should have an out_size + * argument to prevent overrunning outbuf in case of corruption of the + * compressed data. + */ +STATIC int INIT lunzip(unsigned char *inbuf, long in_len, + long (*fill)(void*, unsigned long), + long (*flush)(void*, unsigned long), + unsigned char *outbuf, + long *in_posp, + void (*error)(char *x)) +{ + return __lunzip(inbuf, in_len, fill, flush, outbuf, LONG_MAX, + in_posp, 0, error); +} +#else +STATIC int INIT __decompress(unsigned char *inbuf, long in_len, + long (*fill)(void*, unsigned long), + long (*flush)(void*, unsigned long), + unsigned char *outbuf, long out_size, + long *in_posp, + void (*error)(char *x)) +{ +/* Some archs pass out_size = 0 (to mean unlimited size), which is unsafe + * in case of corruption of the compressed data. + */ + return __lunzip(inbuf, in_len - 4, fill, flush, outbuf, + out_size ? out_size : LONG_MAX, in_posp, 0, error); +} +#endif diff --git a/doc/xlunzip.1 b/doc/xlunzip.1 new file mode 100644 index 0000000..905b59a --- /dev/null +++ b/doc/xlunzip.1 @@ -0,0 +1,90 @@ +.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.46.1. +.TH XLUNZIP "1" "July 2018" "xlunzip 0.3" "User Commands" +.SH NAME +xlunzip \- test tool for the lunzip linux module +.SH SYNOPSIS +.B xlunzip +[\fI\,options\/\fR] [\fI\,files\/\fR] +.SH DESCRIPTION +Xlunzip is a test tool for the lzip decompression code of my lzip patch +for linux. Xlunzip is similar to lunzip, but it uses the lzip_decompress +linux module as a backend. Xlunzip tests the module for stream, +buffer\-to\-buffer and mixed decompression modes, including in\-place +decompression (using the same buffer for input and output). You can use +xlunzip to verify that the module produces correct results when +decompressing single member files, multimember files, or the +concatenation of two or more compressed files. Xlunzip can be used with +unzcrash to test the robustness of the module to the decompression of +corrupted data. +.PP +Note that the in\-place decompression of concatenated files can't be +guaranteed to work because an arbitrarily low compression ratio of the +last part of the data can be achieved by appending enough empty +compressed members to a file. +.SH OPTIONS +.TP +\fB\-h\fR, \fB\-\-help\fR +display this help and exit +.TP +\fB\-V\fR, \fB\-\-version\fR +output version information and exit +.TP +\fB\-c\fR, \fB\-\-stdout\fR +write to standard output, keep input files +.TP +\fB\-d\fR, \fB\-\-decompress\fR +decompress (this is the default) +.TP +\fB\-f\fR, \fB\-\-force\fR +overwrite existing output files +.TP +\fB\-I\fR, \fB\-\-in\-place\fR +decompress or test using only one buffer +.TP +\fB\-k\fR, \fB\-\-keep\fR +keep (don't delete) input files +.TP +\fB\-o\fR, \fB\-\-output=\fR +if reading standard input, write to +.TP +\fB\-q\fR, \fB\-\-quiet\fR +suppress all messages +.TP +\fB\-t\fR, \fB\-\-test\fR +test compressed file integrity +.TP +\fB\-v\fR, \fB\-\-verbose\fR +be verbose (a 2nd \fB\-v\fR gives more) +.SS "These options are ignored when --in-place is in effect:" +.TP +\fB\-\-insize[=\fR] +pre\-allocate and fill inbuf [default 16 KiB] +.TP +\fB\-\-outsize[=\fR] +pre\-allocate outbuf [default 512 MiB] +.TP +\fB\-\-nofill\fR +do not pass a fill function; requires \fB\-\-insize\fR +.TP +\fB\-\-noflush\fR +do not pass a flush function; requires \fB\-\-outsize\fR +.PP +If no file names are given, or if a file is '\-', xlunzip decompresses +from standard input to standard output. +Numbers may be followed by a multiplier: k = kB = 10^3 = 1000, +Ki = KiB = 2^10 = 1024, M = 10^6, Mi = 2^20, G = 10^9, Gi = 2^30, etc... +.PP +Exit status: 0 for a normal exit, 1 for environmental problems (file +not found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or +invalid input file, 3 for an internal consistency error (eg, bug) which +caused xlunzip to panic. +.SH "REPORTING BUGS" +Report bugs to lzip\-bug@nongnu.org +.br +Xlunzip home page: http://www.nongnu.org/lzip/xlunzip.html +.SH COPYRIGHT +Copyright \(co 2018 Antonio Diaz Diaz. +License GPLv2+: GNU GPL version 2 or later +.br +This is free software: you are free to change and redistribute it. +There is NO WARRANTY, to the extent permitted by law. diff --git a/in_place.c b/in_place.c new file mode 100644 index 0000000..65a10dc --- /dev/null +++ b/in_place.c @@ -0,0 +1,220 @@ +/* Xlunzip - Test tool for the lunzip linux module + Copyright (C) 2016-2018 Antonio Diaz Diaz. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +*/ + +#define _FILE_OFFSET_BITS 64 + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "linux_lunzip.h" +#include "lzip.h" + + +/* Returns the number of bytes really read. + If (returned value < size) and (errno == 0), means EOF was reached. +*/ +long readblock( const int fd, uint8_t * const buf, const long size ) + { + long sz = 0; + errno = 0; + while( sz < size ) + { + const int n = read( fd, buf + sz, min( 1L << 20, size - sz ) ); + if( n > 0 ) sz += n; + else if( n == 0 ) break; /* EOF */ + else if( errno != EINTR ) break; + errno = 0; + } + return sz; + } + + +/* Returns the address of a malloc'd buffer containing the file data and + the buffer and file sizes in '*buffer_sizep' and '*file_sizep'. + In case of error, returns 0 and does not modify '*size'. +*/ +uint8_t * read_file( const int infd, long * const buffer_sizep, + long * const file_sizep, struct Pretty_print * const pp ) + { + long buffer_size = 1 << 20; + uint8_t * buffer = (uint8_t *)malloc( buffer_size ); + if( !buffer ) + { show_file_error( pp->name, "Not enough memory.", 0 ); return 0; } + + long file_size = readblock( infd, buffer, buffer_size ); + while( file_size >= buffer_size && !errno ) + { + if( buffer_size >= LONG_MAX ) + { show_file_error( pp->name, "File is too large.", 0 ); free( buffer ); + return 0; } + buffer_size = ( buffer_size <= LONG_MAX / 2 ) ? 2 * buffer_size : LONG_MAX; + uint8_t * const tmp = (uint8_t *)realloc( buffer, buffer_size ); + if( !tmp ) + { show_file_error( pp->name, "Not enough memory.", 0 ); free( buffer ); + return 0; } + buffer = tmp; + file_size += readblock( infd, buffer + file_size, buffer_size - file_size ); + } + if( errno ) + { show_file_error( pp->name, "Error reading file", errno ); free( buffer ); + return 0; } + close( infd ); + *buffer_sizep = buffer_size; + *file_sizep = file_size; + return buffer; + } + + +struct File_sizes + { + long long csize; + long long dsize; + long trailing; + }; + +const char * set_file_sizes( struct File_sizes * const file_sizes, + const uint8_t * const buffer, const long file_size ) + { + if( file_size < min_member_size ) return "Input file is too short."; + const Lzip_header * header = (Lzip_header *)buffer; + if( !Lh_verify_magic( *header ) ) + return "Bad magic number (file not in lzip format)."; + if( !Lh_verify_version( *header ) ) + return "Version of lzip member format not supported."; + + file_sizes->csize = file_sizes->dsize = file_sizes->trailing = 0; + long long pos = file_size; /* always points to a header or to EOF */ + while( pos >= min_member_size ) + { + const Lzip_trailer * const trailer = + (Lzip_trailer *)( buffer + pos - Lt_size ); + const long long member_size = Lt_get_member_size( *trailer ); + if( member_size < min_member_size || member_size > pos ) + { + if( file_sizes->csize == 0 ) { --pos; continue; } /* maybe trailing data */ + return "Member size in trailer is corrupt."; + } + header = (Lzip_header *)( buffer + pos - member_size ); + if( !Lh_verify_magic( *header ) || !Lh_verify_version( *header ) ) + { + if( file_sizes->csize == 0 ) { --pos; continue; } /* maybe trailing data */ + return "Bad member header inside file."; + } + if( file_sizes->csize == 0 && file_size - pos > 0 ) + { + file_sizes->trailing = file_size - pos; + header = (Lzip_header *)( buffer + pos ); + if( file_size - pos > Lh_size && + Lh_verify_magic( *header ) && Lh_verify_version( *header ) ) + return "Last member in input file is truncated or corrupt."; + } + pos -= member_size; + file_sizes->csize += member_size; + file_sizes->dsize += Lt_get_data_size( *trailer ); + } + if( pos != 0 || file_sizes->csize == 0 ) return "Can't get file sizes."; + if( file_sizes->csize + file_sizes->trailing != file_size ) + return "Error getting file sizes."; + return 0; + } + + +const char * global_name; +static void error(char *x) { show_file_error( global_name, x, 0 ); } + + +/* + * Load the compressed file at the end of the buffer used to hold the + * decompressed data. Verify that the in-place decompression does not + * overwrite the compressed data. + * + * |----- compressed data ------| + * V V + * |---------------|-------------------|--------| + * ^ ^ + * |------- decompressed data ---------| + */ + +int decompress_in_place( const int infd, struct Pretty_print * const pp, + const bool testing ) + { + long buffer_size = 0, file_size = 0; + uint8_t * buffer = read_file( infd, &buffer_size, &file_size, pp ); + if( !buffer ) return 1; + struct File_sizes file_sizes; + const char * emsg = set_file_sizes( &file_sizes, buffer, file_size ); + if( emsg ) { show_file_error( pp->name, emsg, 0 ); return 2; } + + const long long csize = file_sizes.csize; + const long long dsize = file_sizes.dsize; + /* const long trailing = file_sizes.trailing; */ + if( csize <= 0 || csize > LONG_MAX ) + { show_file_error( pp->name, "File is larger than LONG_MAX.", 0 ); + return 2; } + if( dsize < 0 || dsize > LONG_MAX ) + { show_file_error( pp->name, "Data is larger than LONG_MAX.", 0 ); + return 2; } + /* ( (csize-36+63) >> 6 ) + 36 never failed with single member */ + const long rextra = ( csize >> 5 ) + 72; + if( buffer_size < dsize + rextra ) /* avoid realloc if big enough */ + { + buffer_size = dsize + rextra; + buffer = (uint8_t *)realloc( buffer, buffer_size ); + if( !buffer ) + { show_file_error( pp->name, "Not enough memory.", 0 ); return 1; } + } + else buffer_size = max( dsize + rextra, csize ); + const long cbegin = buffer_size - csize; + if( cbegin > 0 ) memmove( buffer + cbegin, buffer, csize ); + + long in_pos, out_pos; + int retval; + global_name = pp->name; + retval = convert_retval( __lunzip( buffer + cbegin, csize, 0, 0, buffer, + buffer_size, &in_pos, &out_pos, error ) ); + if( retval == 0 && !testing ) + { + const long len = flush( buffer, out_pos ); + if( len < out_pos ) + { show_file_error( pp->name, "Write error", errno ); return 1; } + } + free( buffer ); + if( retval ) return retval; + if( verbosity >= 1 ) Pp_show_msg( pp, 0 ); + if( verbosity >= 2 ) + { + if( out_pos <= 0 || in_pos <= 0 ) + fputs( "no data compressed. ", stderr ); + else + fprintf( stderr, "%6.3f:1, %5.2f%% ratio, %5.2f%% saved. ", + (double)out_pos / in_pos, + ( 100.0 * in_pos ) / out_pos, + 100.0 - ( ( 100.0 * in_pos ) / out_pos ) ); + if( verbosity >= 3 ) + fprintf( stderr, "decompressed %9lu, compressed %8lu. ", + out_pos, in_pos ); + } + if( verbosity >= 1 ) + fputs( testing ? "ok\n" : "done\n", stderr ); + return 0; + } diff --git a/linux_lunzip.h b/linux_lunzip.h new file mode 100644 index 0000000..8aafbad --- /dev/null +++ b/linux_lunzip.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef LINUX_DECOMPRESS_LUNZIP_H +#define LINUX_DECOMPRESS_LUNZIP_H + +int lunzip(unsigned char *inbuf, long in_len, + long (*fill)(void*, unsigned long), + long (*flush)(void*, unsigned long), + unsigned char *outbuf, + long *in_posp, + void (*error)(char *x)); + +int __lunzip(unsigned char *inbuf, long in_len, + long (*fill)(void*, unsigned long), + long (*flush)(void*, unsigned long), + unsigned char *outbuf, long out_size, + long *in_posp, long *out_posp, + void (*error)(char *x)); +#endif diff --git a/linux_lzip.h b/linux_lzip.h new file mode 100644 index 0000000..be5ad50 --- /dev/null +++ b/linux_lzip.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __LZIP_H__ +#define __LZIP_H__ +/* + * LZIP decompressor + * + * Copyright (C) 2016-2018 Antonio Diaz Diaz. + */ + +/* Return values (< 0 = Error) */ +enum { + LZIP_OOM_INBUF = -1, + LZIP_HEADER1_EOF = -2, + LZIP_HEADER2_EOF = -3, + LZIP_BAD_MAGIC1 = -4, + LZIP_BAD_MAGIC2 = -5, + LZIP_BAD_VERSION = -6, + LZIP_BAD_DICT_SIZE = -7, + LZIP_OOM_OUTBUF = -8, + LZIP_WRITE_ERROR = -9, + LZIP_BAD_DATA = -10, + LZIP_DATA_EOF = -11, + LZIP_BAD_CRC = -12 +}; + +int lzip_decompress(unsigned char *inbuf, long in_len, + long (*fill)(void*, unsigned long), + long (*flush)(void*, unsigned long), + unsigned char *outbuf, long out_size, + long *in_posp, long *out_posp); + +/* inbuf - input buffer. If null or in_len <= 0, fill must be non-null + * in_len - len of pre-read data in inbuf if inbuf is non-null + * fill - if non-null, function to fill inbuf when empty + * flush - if non-null, function to write out outbuf when full + * outbuf - output buffer. If null or out_size <= 0, flush must be non-null + * out_size - size of outbuf if outbuf is non-null + * in_posp - if non-null, the number of bytes consumed will be returned here + * out_posp - if non-null, the number of bytes produced will be returned here + * + * fill will be called (repeatedly) to read data. in_len bytes will be read + * per call (or 16384 bytes per call if inbuf is null or in_len <= 0). + * + * If flush is null, outbuf must be large enough to buffer all the expected + * output. Else the flush function will be called to flush the output buffer + * at the appropriate time (stream dependent). + * If out_size > 0 but is not large enough to buffer all the expected output, + * it must be at least as large as the dictionary size of the data. + * + * inbuf and outbuf may overlap (in-place decompression). + */ + +#endif diff --git a/linux_mm.h b/linux_mm.h new file mode 100644 index 0000000..d3b3954 --- /dev/null +++ b/linux_mm.h @@ -0,0 +1,36 @@ +#ifndef DECOMPR_MM_H +#define DECOMPR_MM_H + +#define large_malloc(a) malloc(a) +#define large_free(a) free(a) + +#define INIT +#define STATIC + +#define _FILE_OFFSET_BITS 64 + +#include +#include +#include +#include +#include + +/* + * min()/max() macros that also do strict type-checking.. See the + * "unnecessary" pointer comparison. + */ +#define min(x, y) ({ \ + __typeof__(x) _min1 = (x); \ + __typeof__(y) _min2 = (y); \ + (void) (&_min1 == &_min2); \ + _min1 < _min2 ? _min1 : _min2; }) + +#define max(x, y) ({ \ + __typeof__(x) _max1 = (x); \ + __typeof__(y) _max2 = (y); \ + (void) (&_max1 == &_max2); \ + _max1 > _max2 ? _max1 : _max2; }) + +#define min_t(type, x, y) min((type) x, (type) y) + +#endif diff --git a/lzip.c b/lzip.c new file mode 100644 index 0000000..7c96975 --- /dev/null +++ b/lzip.c @@ -0,0 +1,881 @@ +/* + * LZIP decompressor + * + * Copyright (C) 2016-2018 Antonio Diaz Diaz. + * + * Licensed under GPLv2 or later, see file LICENSE in this source tree. + */ + +#include "linux_lzip.h" +#include "linux_mm.h" + +/* + * STATIC_RW_DATA is used in the pre-boot environment on some architectures. + * See for details. + */ +#ifndef STATIC_RW_DATA +#define STATIC_RW_DATA static +#endif + +typedef int State; + +enum { states = 12 }; + +static inline bool St_is_char(const State st) { return st < 7; } + +static inline State St_set_char(const State st) +{ + STATIC_RW_DATA const State next[states] = { 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 4, 5 }; + return next[st]; +} + +static inline State St_set_match(const State st) +{ + return ((st < 7) ? 7 : 10); +} + +static inline State St_set_rep(const State st) +{ + return ((st < 7) ? 8 : 11); +} + +static inline State St_set_short_rep(const State st) +{ + return ((st < 7) ? 9 : 11); +} + + +enum { + min_dictionary_bits = 12, + min_dictionary_size = 1 << min_dictionary_bits, + max_dictionary_bits = 29, + max_dictionary_size = 1 << max_dictionary_bits, + literal_context_bits = 3, + pos_state_bits = 2, + pos_states = 1 << pos_state_bits, + pos_state_mask = pos_states - 1, + + len_states = 4, + dis_slot_bits = 6, + start_dis_model = 4, + end_dis_model = 14, + modeled_distances = 1 << (end_dis_model / 2), /* 128 */ + dis_align_bits = 4, + dis_align_size = 1 << dis_align_bits, + + len_low_bits = 3, + len_mid_bits = 3, + len_high_bits = 8, + len_low_symbols = 1 << len_low_bits, + len_mid_symbols = 1 << len_mid_bits, + len_high_symbols = 1 << len_high_bits, + max_len_symbols = len_low_symbols + len_mid_symbols + len_high_symbols, + + min_match_len = 2, /* must be 2 */ + max_match_len = min_match_len + max_len_symbols - 1, /* 273 */ + min_match_len_limit = 5 +}; + +static inline int get_len_state(const int len) +{ + return min(len - min_match_len, len_states - 1); +} + +static inline int get_lit_state(const uint8_t prev_byte) +{ + return (prev_byte >> (8 - literal_context_bits)); +} + + +enum { bit_model_move_bits = 5, + bit_model_total_bits = 11, + bit_model_total = 1 << bit_model_total_bits +}; + +typedef int Bit_model; + +static inline void Bm_init(Bit_model * const probability) +{ + *probability = bit_model_total / 2; +} + +static inline void Bm_array_init(Bit_model bm[], const int size) +{ + int i; + + for (i = 0; i < size; ++i) + Bm_init(&bm[i]); +} + +struct Len_model { + Bit_model choice1; + Bit_model choice2; + Bit_model bm_low[pos_states][len_low_symbols]; + Bit_model bm_mid[pos_states][len_mid_symbols]; + Bit_model bm_high[len_high_symbols]; +}; + +static inline void Lm_init(struct Len_model * const lm) +{ + Bm_init(&lm->choice1); + Bm_init(&lm->choice2); + Bm_array_init(lm->bm_low[0], pos_states * len_low_symbols); + Bm_array_init(lm->bm_mid[0], pos_states * len_mid_symbols); + Bm_array_init(lm->bm_high, len_high_symbols); +} + + +/* Table of CRCs of all 8-bit messages. */ +STATIC_RW_DATA const uint32_t crc32[256] = + { + 0x00000000, 0x77073096, 0xEE0E612C, 0x990951BA, 0x076DC419, 0x706AF48F, + 0xE963A535, 0x9E6495A3, 0x0EDB8832, 0x79DCB8A4, 0xE0D5E91E, 0x97D2D988, + 0x09B64C2B, 0x7EB17CBD, 0xE7B82D07, 0x90BF1D91, 0x1DB71064, 0x6AB020F2, + 0xF3B97148, 0x84BE41DE, 0x1ADAD47D, 0x6DDDE4EB, 0xF4D4B551, 0x83D385C7, + 0x136C9856, 0x646BA8C0, 0xFD62F97A, 0x8A65C9EC, 0x14015C4F, 0x63066CD9, + 0xFA0F3D63, 0x8D080DF5, 0x3B6E20C8, 0x4C69105E, 0xD56041E4, 0xA2677172, + 0x3C03E4D1, 0x4B04D447, 0xD20D85FD, 0xA50AB56B, 0x35B5A8FA, 0x42B2986C, + 0xDBBBC9D6, 0xACBCF940, 0x32D86CE3, 0x45DF5C75, 0xDCD60DCF, 0xABD13D59, + 0x26D930AC, 0x51DE003A, 0xC8D75180, 0xBFD06116, 0x21B4F4B5, 0x56B3C423, + 0xCFBA9599, 0xB8BDA50F, 0x2802B89E, 0x5F058808, 0xC60CD9B2, 0xB10BE924, + 0x2F6F7C87, 0x58684C11, 0xC1611DAB, 0xB6662D3D, 0x76DC4190, 0x01DB7106, + 0x98D220BC, 0xEFD5102A, 0x71B18589, 0x06B6B51F, 0x9FBFE4A5, 0xE8B8D433, + 0x7807C9A2, 0x0F00F934, 0x9609A88E, 0xE10E9818, 0x7F6A0DBB, 0x086D3D2D, + 0x91646C97, 0xE6635C01, 0x6B6B51F4, 0x1C6C6162, 0x856530D8, 0xF262004E, + 0x6C0695ED, 0x1B01A57B, 0x8208F4C1, 0xF50FC457, 0x65B0D9C6, 0x12B7E950, + 0x8BBEB8EA, 0xFCB9887C, 0x62DD1DDF, 0x15DA2D49, 0x8CD37CF3, 0xFBD44C65, + 0x4DB26158, 0x3AB551CE, 0xA3BC0074, 0xD4BB30E2, 0x4ADFA541, 0x3DD895D7, + 0xA4D1C46D, 0xD3D6F4FB, 0x4369E96A, 0x346ED9FC, 0xAD678846, 0xDA60B8D0, + 0x44042D73, 0x33031DE5, 0xAA0A4C5F, 0xDD0D7CC9, 0x5005713C, 0x270241AA, + 0xBE0B1010, 0xC90C2086, 0x5768B525, 0x206F85B3, 0xB966D409, 0xCE61E49F, + 0x5EDEF90E, 0x29D9C998, 0xB0D09822, 0xC7D7A8B4, 0x59B33D17, 0x2EB40D81, + 0xB7BD5C3B, 0xC0BA6CAD, 0xEDB88320, 0x9ABFB3B6, 0x03B6E20C, 0x74B1D29A, + 0xEAD54739, 0x9DD277AF, 0x04DB2615, 0x73DC1683, 0xE3630B12, 0x94643B84, + 0x0D6D6A3E, 0x7A6A5AA8, 0xE40ECF0B, 0x9309FF9D, 0x0A00AE27, 0x7D079EB1, + 0xF00F9344, 0x8708A3D2, 0x1E01F268, 0x6906C2FE, 0xF762575D, 0x806567CB, + 0x196C3671, 0x6E6B06E7, 0xFED41B76, 0x89D32BE0, 0x10DA7A5A, 0x67DD4ACC, + 0xF9B9DF6F, 0x8EBEEFF9, 0x17B7BE43, 0x60B08ED5, 0xD6D6A3E8, 0xA1D1937E, + 0x38D8C2C4, 0x4FDFF252, 0xD1BB67F1, 0xA6BC5767, 0x3FB506DD, 0x48B2364B, + 0xD80D2BDA, 0xAF0A1B4C, 0x36034AF6, 0x41047A60, 0xDF60EFC3, 0xA867DF55, + 0x316E8EEF, 0x4669BE79, 0xCB61B38C, 0xBC66831A, 0x256FD2A0, 0x5268E236, + 0xCC0C7795, 0xBB0B4703, 0x220216B9, 0x5505262F, 0xC5BA3BBE, 0xB2BD0B28, + 0x2BB45A92, 0x5CB36A04, 0xC2D7FFA7, 0xB5D0CF31, 0x2CD99E8B, 0x5BDEAE1D, + 0x9B64C2B0, 0xEC63F226, 0x756AA39C, 0x026D930A, 0x9C0906A9, 0xEB0E363F, + 0x72076785, 0x05005713, 0x95BF4A82, 0xE2B87A14, 0x7BB12BAE, 0x0CB61B38, + 0x92D28E9B, 0xE5D5BE0D, 0x7CDCEFB7, 0x0BDBDF21, 0x86D3D2D4, 0xF1D4E242, + 0x68DDB3F8, 0x1FDA836E, 0x81BE16CD, 0xF6B9265B, 0x6FB077E1, 0x18B74777, + 0x88085AE6, 0xFF0F6A70, 0x66063BCA, 0x11010B5C, 0x8F659EFF, 0xF862AE69, + 0x616BFFD3, 0x166CCF45, 0xA00AE278, 0xD70DD2EE, 0x4E048354, 0x3903B3C2, + 0xA7672661, 0xD06016F7, 0x4969474D, 0x3E6E77DB, 0xAED16A4A, 0xD9D65ADC, + 0x40DF0B66, 0x37D83BF0, 0xA9BCAE53, 0xDEBB9EC5, 0x47B2CF7F, 0x30B5FFE9, + 0xBDBDF21C, 0xCABAC28A, 0x53B39330, 0x24B4A3A6, 0xBAD03605, 0xCDD70693, + 0x54DE5729, 0x23D967BF, 0xB3667A2E, 0xC4614AB8, 0x5D681B02, 0x2A6F2B94, + 0xB40BBE37, 0xC30C8EA1, 0x5A05DF1B, 0x2D02EF8D }; + + +static inline void CRC32_update_buf(uint32_t * const crc, + const uint8_t * const buffer, + const long size) +{ + long i; + uint32_t c = *crc; + + for (i = 0; i < size; ++i) + c = crc32[(c^buffer[i])&0xFF] ^ (c >> 8); + *crc = c; +} + + +STATIC_RW_DATA const uint8_t lzip_magic[4] = { 0x4C, 0x5A, 0x49, 0x50 }; /* "LZIP" */ + +typedef uint8_t Lzip_header[6]; /* 0-3 magic bytes */ + /* 4 version */ + /* 5 coded_dict_size */ +enum { Lh_size = 6 }; + +static inline bool Lh_verify_magic(const Lzip_header data) +{ + int i; + + for (i = 0; i < 4; ++i) + if (data[i] != lzip_magic[i]) + return false; + return true; +} + +/* detect (truncated) header */ +static inline bool Lh_verify_prefix(const Lzip_header data, const int sz) +{ + int i; + for (i = 0; i < sz && i < 4; ++i) + if (data[i] != lzip_magic[i]) + return false; + return (sz > 0); +} + +/* detect corrupt header */ +static inline bool Lh_verify_corrupt(const Lzip_header data) +{ + int matches = 0; + int i; + for (i = 0; i < 4; ++i) + if (data[i] == lzip_magic[i]) + ++matches; + return (matches > 1 && matches < 4); +} + +static inline bool Lh_verify_version(const Lzip_header data) +{ + return (data[4] == 1); +} + +static inline unsigned Lh_get_dictionary_size(const Lzip_header data) +{ + unsigned sz = (1 << (data[5] & 0x1F)); + + if (sz > min_dictionary_size) + sz -= (sz / 16) * ((data[5] >> 5) & 7); + return sz; +} + + +typedef uint8_t Lzip_trailer[20]; + /* 0-3 CRC32 of the uncompressed data */ + /* 4-11 size of the uncompressed data */ + /* 12-19 member size including header and trailer */ +enum { Lt_size = 20 }; + +static inline unsigned Lt_get_data_crc(const Lzip_trailer data) +{ + unsigned tmp = 0; + int i; + + for (i = 3; i >= 0; --i) { + tmp <<= 8; + tmp += data[i]; + } + return tmp; +} + +static inline unsigned long long Lt_get_data_size(const Lzip_trailer data) +{ + unsigned long long tmp = 0; + int i; + + for (i = 11; i >= 4; --i) { + tmp <<= 8; + tmp += data[i]; + } + return tmp; +} + +static inline unsigned long long Lt_get_member_size(const Lzip_trailer data) +{ + unsigned long long tmp = 0; + int i; + + for (i = 19; i >= 12; --i) { + tmp <<= 8; + tmp += data[i]; + } + return tmp; +} + + +struct Range_decoder { + unsigned long long partial_member_pos; + uint8_t *buffer; /* input buffer */ + long buffer_size; + long pos; /* current pos in buffer */ + long stream_pos; /* when reached, a new block must be read */ + uint32_t code; + uint32_t range; + long (*fill)(void*, unsigned long); + bool at_stream_end; + bool buffer_given; +}; + + +static bool Rd_read_block(struct Range_decoder * const rdec) +{ + if (!rdec->at_stream_end) { + rdec->stream_pos = rdec->fill ? + rdec->fill(rdec->buffer, rdec->buffer_size) : 0; + rdec->at_stream_end = (rdec->stream_pos < rdec->buffer_size); + rdec->partial_member_pos += rdec->pos; + rdec->pos = 0; + } + return rdec->pos < rdec->stream_pos; +} + + +static inline bool Rd_init(struct Range_decoder * const rdec, + uint8_t * const inbuf, const long in_len, + long (*fill)(void*, unsigned long)) +{ + rdec->partial_member_pos = 0; + rdec->buffer_given = (inbuf && in_len > 0); + rdec->buffer_size = rdec->buffer_given ? in_len : 16384; + rdec->buffer = rdec->buffer_given ? inbuf : malloc(rdec->buffer_size); + if (!rdec->buffer) + return false; + rdec->pos = 0; + rdec->stream_pos = rdec->buffer_given ? in_len : 0; + rdec->code = 0; + rdec->range = 0xFFFFFFFFU; + rdec->fill = fill; + rdec->at_stream_end = false; + return true; +} + +static inline void Rd_free(struct Range_decoder * const rdec) +{ + if (!rdec->buffer_given) + free(rdec->buffer); +} + +static inline bool Rd_finished(struct Range_decoder * const rdec) +{ + return rdec->pos >= rdec->stream_pos && !Rd_read_block(rdec); +} + +static inline unsigned long long +Rd_member_position(const struct Range_decoder * const rdec) +{ + return rdec->partial_member_pos + rdec->pos; +} + +static inline void Rd_reset_member_position(struct Range_decoder * const rdec) +{ + rdec->partial_member_pos = 0; rdec->partial_member_pos -= rdec->pos; +} + +static inline uint8_t Rd_get_byte(struct Range_decoder * const rdec) +{ + /* 0xFF avoids decoder error if member is truncated at EOS marker */ + if (Rd_finished(rdec)) + return 0xFF; + return rdec->buffer[rdec->pos++]; +} + +static inline void Rd_load(struct Range_decoder * const rdec) +{ + int i; + + rdec->code = 0; + for (i = 0; i < 5; ++i) + rdec->code = (rdec->code << 8) | Rd_get_byte(rdec); + rdec->range = 0xFFFFFFFFU; +} + +static inline void Rd_normalize(struct Range_decoder * const rdec) +{ + if (rdec->range <= 0x00FFFFFFU) { + rdec->range <<= 8; + rdec->code = (rdec->code << 8) | Rd_get_byte(rdec); + } +} + +static inline unsigned Rd_decode(struct Range_decoder * const rdec, + const int num_bits) +{ + unsigned symbol = 0; + int i; + + for (i = num_bits; i > 0; --i) { + bool bit; + + Rd_normalize(rdec); + rdec->range >>= 1; + /* symbol <<= 1; */ + /* if(rdec->code >= rdec->range) { rdec->code -= rdec->range; symbol |= 1; } */ + bit = (rdec->code >= rdec->range); + symbol = (symbol << 1) + bit; + rdec->code -= rdec->range & (0U - bit); + } + return symbol; +} + +static inline unsigned Rd_decode_bit(struct Range_decoder * const rdec, + Bit_model * const probability) +{ + uint32_t bound; + + Rd_normalize(rdec); + bound = (rdec->range >> bit_model_total_bits) * *probability; + if (rdec->code < bound) { + rdec->range = bound; + *probability += (bit_model_total - *probability) >> bit_model_move_bits; + return 0; + } else { + rdec->range -= bound; + rdec->code -= bound; + *probability -= *probability >> bit_model_move_bits; + return 1; + } +} + +static inline unsigned Rd_decode_tree3(struct Range_decoder * const rdec, + Bit_model bm[]) +{ + unsigned symbol = 1; + + symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]); + symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]); + symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]); + return symbol & 7; +} + +static inline unsigned Rd_decode_tree6(struct Range_decoder * const rdec, + Bit_model bm[]) +{ + unsigned symbol = 1; + + symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]); + symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]); + symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]); + symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]); + symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]); + symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]); + return symbol & 0x3F; +} + +static inline unsigned Rd_decode_tree8(struct Range_decoder * const rdec, + Bit_model bm[]) +{ + unsigned symbol = 1; + int i; + + for (i = 0; i < 8; ++i) + symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]); + return symbol & 0xFF; +} + +static inline unsigned +Rd_decode_tree_reversed(struct Range_decoder * const rdec, + Bit_model bm[], const int num_bits) +{ + unsigned model = 1; + unsigned symbol = 0; + int i; + + for (i = 0; i < num_bits; ++i) { + const unsigned bit = Rd_decode_bit(rdec, &bm[model]); + + model = (model << 1) + bit; + symbol |= (bit << i); + } + return symbol; +} + +static inline unsigned +Rd_decode_tree_reversed4(struct Range_decoder * const rdec, Bit_model bm[]) +{ + unsigned symbol = Rd_decode_bit(rdec, &bm[1]); + unsigned model = 2 + symbol; + unsigned bit = Rd_decode_bit(rdec, &bm[model]); + + model = (model << 1) + bit; symbol |= (bit << 1); + bit = Rd_decode_bit(rdec, &bm[model]); + model = (model << 1) + bit; symbol |= (bit << 2); + symbol |= (Rd_decode_bit(rdec, &bm[model]) << 3); + return symbol; +} + +static inline unsigned Rd_decode_matched(struct Range_decoder * const rdec, + Bit_model bm[], unsigned match_byte) +{ + unsigned symbol = 1; + unsigned mask = 0x100; + + while (true) { + const unsigned match_bit = (match_byte <<= 1) & mask; + const unsigned bit = Rd_decode_bit(rdec, &bm[symbol+match_bit+mask]); + + symbol = (symbol << 1) + bit; + if (symbol > 0xFF) + return symbol & 0xFF; + mask &= ~(match_bit ^ (bit << 8)); /* if( match_bit != bit ) mask = 0; */ + } +} + +static inline unsigned Rd_decode_len(struct Range_decoder * const rdec, + struct Len_model * const lm, + const int pos_state) +{ + if (Rd_decode_bit(rdec, &lm->choice1) == 0) + return Rd_decode_tree3(rdec, lm->bm_low[pos_state]); + if (Rd_decode_bit(rdec, &lm->choice2) == 0) + return len_low_symbols + + Rd_decode_tree3(rdec, lm->bm_mid[pos_state]); + return len_low_symbols + len_mid_symbols + + Rd_decode_tree8(rdec, lm->bm_high); +} + + +struct LZ_decoder { + unsigned long long partial_data_pos; + struct Range_decoder *rdec; + /* Don't move bm_* to LZd_decode_member; makes frame too large. */ + Bit_model bm_literal[1 << literal_context_bits][0x300]; + Bit_model bm_match[states][pos_states]; + Bit_model bm_rep[states]; + Bit_model bm_rep0[states]; + Bit_model bm_rep1[states]; + Bit_model bm_rep2[states]; + Bit_model bm_len[states][pos_states]; + Bit_model bm_dis_slot[len_states][1 << dis_slot_bits]; + Bit_model bm_dis[modeled_distances-end_dis_model+1]; + Bit_model bm_align[dis_align_size]; + struct Len_model match_len_model; + struct Len_model rep_len_model; + + unsigned long buffer_size; + unsigned dictionary_size; + uint8_t *buffer; /* output buffer */ + unsigned long pos; /* current pos in buffer */ + unsigned long stream_pos; /* first byte not yet written to file */ + uint32_t crc; + long (*flush)(void*, unsigned long); + bool pos_wrapped; + bool buffer_given; + bool write_error; +}; + +static void LZd_flush_data(struct LZ_decoder * const d) +{ + if (d->pos > d->stream_pos) { + const long size = d->pos - d->stream_pos; + + CRC32_update_buf(&d->crc, d->buffer + d->stream_pos, size); + if ((d->flush && + d->flush(d->buffer + d->stream_pos, size) != size) || + (!d->flush && d->pos_wrapped)) + d->write_error = true; + if (d->pos >= d->buffer_size) { + d->partial_data_pos += d->pos; + d->pos = 0; + d->pos_wrapped = true; + } + d->stream_pos = d->pos; + } +} + +static inline uint8_t LZd_peek_prev(const struct LZ_decoder * const d) +{ + if (d->pos > 0) + return d->buffer[d->pos-1]; + if (d->pos_wrapped) + return d->buffer[d->buffer_size-1]; + return 0; /* prev_byte of first byte */ +} + +static inline uint8_t LZd_peek(const struct LZ_decoder * const d, + const unsigned distance) +{ + const unsigned long i = ((d->pos > distance) ? 0 : d->buffer_size) + + d->pos - distance - 1; + return d->buffer[i]; +} + +static inline void LZd_put_byte(struct LZ_decoder * const d, const uint8_t b) +{ + d->buffer[d->pos] = b; + if (++d->pos >= d->buffer_size) + LZd_flush_data(d); +} + +static inline void LZd_copy_block(struct LZ_decoder * const d, + const unsigned distance, unsigned len) +{ + unsigned long lpos = d->pos, i = lpos - distance - 1; + bool fast, fast2; + + if (lpos > distance) { + fast = (len < d->buffer_size - lpos); + fast2 = (fast && len <= lpos - i); + } else { + i += d->buffer_size; + fast = (len < d->buffer_size - i); /* (i == pos) may happen */ + fast2 = (fast && len <= i - lpos); + } + if (fast) { /* no wrap */ + d->pos += len; + if (fast2) /* no wrap, no overlap */ + memcpy(d->buffer + lpos, d->buffer + i, len); + else + for (; len > 0; --len) + d->buffer[lpos++] = d->buffer[i++]; + } else + for (; len > 0; --len) { + d->buffer[d->pos] = d->buffer[i]; + if (++d->pos >= d->buffer_size) + LZd_flush_data(d); + if (++i >= d->buffer_size) + i = 0; + } +} + +static inline bool LZd_init(struct LZ_decoder * const d, + struct Range_decoder * const rde, + const unsigned dict_size, uint8_t * const outbuf, + long out_size, long (*flush)(void*, unsigned long)) +{ + d->partial_data_pos = 0; + d->rdec = rde; + Bm_array_init(d->bm_literal[0], (1 << literal_context_bits) * 0x300); + Bm_array_init(d->bm_match[0], states * pos_states); + Bm_array_init(d->bm_rep, states); + Bm_array_init(d->bm_rep0, states); + Bm_array_init(d->bm_rep1, states); + Bm_array_init(d->bm_rep2, states); + Bm_array_init(d->bm_len[0], states * pos_states); + Bm_array_init(d->bm_dis_slot[0], len_states * (1 << dis_slot_bits)); + Bm_array_init(d->bm_dis, modeled_distances - end_dis_model + 1); + Bm_array_init(d->bm_align, dis_align_size); + Lm_init(&d->match_len_model); + Lm_init(&d->rep_len_model); + + d->buffer_given = (outbuf && out_size > 0); + d->buffer_size = d->buffer_given ? out_size : dict_size; + d->dictionary_size = min_t(unsigned long, d->buffer_size, dict_size); + d->buffer = d->buffer_given ? outbuf : large_malloc(d->buffer_size); + if (!d->buffer) + return false; + d->pos = 0; + d->stream_pos = 0; + d->crc = 0xFFFFFFFFU; + d->flush = flush; + d->pos_wrapped = false; + d->write_error = false; + /* prev_byte of first byte; also for LZd_peek( 0 ) on corrupt file */ + if (!d->buffer_given) /* inbuf and outbuf may overlap */ + d->buffer[d->buffer_size-1] = 0; + return true; +} + +static inline void LZd_free(struct LZ_decoder * const d) +{ + if (!d->buffer_given) + large_free(d->buffer); +} + +static inline unsigned LZd_crc(const struct LZ_decoder * const d) +{ + return d->crc ^ 0xFFFFFFFFU; +} + +static inline unsigned long long +LZd_data_position(const struct LZ_decoder * const d) +{ + return d->partial_data_pos + d->pos; +} + + +static bool LZd_verify_trailer(struct LZ_decoder * const d) +{ + Lzip_trailer trailer; + int i = 0; + + while (i < Lt_size) + trailer[i++] = Rd_get_byte(d->rdec); + + return (Lt_get_data_crc(trailer) == LZd_crc(d) && + Lt_get_data_size(trailer) == LZd_data_position(d) && + Lt_get_member_size(trailer) == Rd_member_position(d->rdec)); +} + + +/* Return value: 0 = OK, < 0 = error (see ). */ +static int LZd_decode_member(struct LZ_decoder * const d) +{ + struct Range_decoder * const rdec = d->rdec; + unsigned rep0 = 0; /* rep[0-3] latest four distances */ + unsigned rep1 = 0; /* used for efficient coding of */ + unsigned rep2 = 0; /* repeated distances */ + unsigned rep3 = 0; + State state = 0; + + Rd_load(rdec); + while (!Rd_finished(rdec)) { + int len; + const int pos_state = LZd_data_position(d) & pos_state_mask; + + if (Rd_decode_bit(rdec, &d->bm_match[state][pos_state]) == 0) { + /* literal byte */ + Bit_model * const bm = d->bm_literal[get_lit_state(LZd_peek_prev(d))]; + + if (St_is_char(state)) { + state -= (state < 4) ? state : 3; + LZd_put_byte(d, Rd_decode_tree8(rdec, bm)); + } else { + state -= (state < 10) ? 3 : 6; + LZd_put_byte(d, Rd_decode_matched(rdec, bm, LZd_peek(d, rep0))); + } + continue; + } + /* match or repeated match */ + if (Rd_decode_bit(rdec, &d->bm_rep[state]) != 0) { + if (Rd_decode_bit(rdec, &d->bm_rep0[state]) == 0) { + if (Rd_decode_bit(rdec, &d->bm_len[state][pos_state]) == 0) { + state = St_set_short_rep(state); + LZd_put_byte(d, LZd_peek(d, rep0)); + continue; + } + } else { + unsigned distance; + + if (Rd_decode_bit(rdec, &d->bm_rep1[state]) == 0) + distance = rep1; + else { + if (Rd_decode_bit(rdec, &d->bm_rep2[state]) == 0) + distance = rep2; + else { + distance = rep3; + rep3 = rep2; + } + rep2 = rep1; + } + rep1 = rep0; + rep0 = distance; + } + state = St_set_rep(state); + len = min_match_len + Rd_decode_len(rdec, &d->rep_len_model, pos_state); + } else { /* match */ + unsigned distance; + + len = min_match_len + Rd_decode_len(rdec, &d->match_len_model, pos_state); + distance = Rd_decode_tree6(rdec, d->bm_dis_slot[get_len_state(len)]); + if (distance >= start_dis_model) { + const unsigned dis_slot = distance; + const int direct_bits = (dis_slot >> 1) - 1; + + distance = (2 | (dis_slot & 1)) << direct_bits; + if (dis_slot < end_dis_model) + distance += Rd_decode_tree_reversed(rdec, + d->bm_dis + (distance - dis_slot), direct_bits); + else { + distance += + Rd_decode(rdec, direct_bits - dis_align_bits) << dis_align_bits; + distance += Rd_decode_tree_reversed4(rdec, d->bm_align); + if (distance == 0xFFFFFFFFU) { /* marker found */ + Rd_normalize(rdec); + LZd_flush_data(d); + if (d->write_error) + return LZIP_WRITE_ERROR; + if (len == min_match_len) { /* End Of Stream marker */ + if (LZd_verify_trailer(d)) + return 0; + else + return LZIP_BAD_CRC; + } + if (len == min_match_len + 1) { /* Sync Flush marker */ + Rd_load(rdec); + continue; + } + return LZIP_BAD_DATA; /* unknown marker */ + } + } + } + rep3 = rep2; rep2 = rep1; rep1 = rep0; rep0 = distance; + state = St_set_match(state); + if (rep0 >= d->dictionary_size || + (rep0 >= d->pos && !d->pos_wrapped)) { + LZd_flush_data(d); + return LZIP_BAD_DATA; + } + } + LZd_copy_block(d, rep0, len); + } + LZd_flush_data(d); + return LZIP_DATA_EOF; +} + + +int lzip_decompress(unsigned char *inbuf, long in_len, + long (*fill)(void*, unsigned long), + long (*flush)(void*, unsigned long), + unsigned char *outbuf, long out_size, + long *in_posp, long *out_posp) +{ + unsigned char *outptr = outbuf; + struct Range_decoder rdec; + struct LZ_decoder *decoder = 0; + int retval = 0; + bool first_member; + + if (in_posp) + *in_posp = 0; + if (out_posp) + *out_posp = 0; + + if (!Rd_init(&rdec, inbuf, in_len, fill)) + return LZIP_OOM_INBUF; + + for (first_member = true;; first_member = false) { + long data_pos; + int size; + unsigned dictionary_size; + Lzip_header header; + + Rd_reset_member_position(&rdec); + for (size = 0; size < Lh_size && !Rd_finished(&rdec); ++size) + header[size] = Rd_get_byte(&rdec); + if (Rd_finished(&rdec)) { /* End Of File */ + if (first_member) + retval = LZIP_HEADER1_EOF; + else if (Lh_verify_prefix(header, size)) + retval = LZIP_HEADER2_EOF; + break; + } + if (!Lh_verify_magic(header)) { + if (first_member) + retval = LZIP_BAD_MAGIC1; + else if (Lh_verify_corrupt(header)) + retval = LZIP_BAD_MAGIC2; + break; + } + if (!Lh_verify_version(header)) { + retval = LZIP_BAD_VERSION; + break; + } + dictionary_size = Lh_get_dictionary_size(header); + if (dictionary_size < min_dictionary_size || + dictionary_size > max_dictionary_size) { + retval = LZIP_BAD_DICT_SIZE; + break; + } + + if (!decoder) + decoder = malloc(sizeof *decoder); + if (!decoder || !LZd_init(decoder, &rdec, dictionary_size, + outptr, out_size, flush)) { + retval = LZIP_OOM_OUTBUF; + break; + } + retval = LZd_decode_member(decoder); + if (in_posp) + *in_posp += Rd_member_position(&rdec); + data_pos = LZd_data_position(decoder); + if (outptr) + outptr += data_pos; + if (out_posp) + *out_posp += data_pos; + if (out_size > 0) + out_size -= data_pos; + LZd_free(decoder); + if (retval != 0) + break; + } + if (decoder) + free(decoder); + Rd_free(&rdec); + return retval; +} + +#ifndef STATIC +EXPORT_SYMBOL_GPL(lzip_decompress); +MODULE_DESCRIPTION("LZIP Decompressor"); +MODULE_AUTHOR("Antonio Diaz Diaz "); +MODULE_LICENSE("GPL"); +#endif diff --git a/lzip.h b/lzip.h new file mode 100644 index 0000000..cd00b10 --- /dev/null +++ b/lzip.h @@ -0,0 +1,173 @@ +/* Xlunzip - Test tool for the lunzip linux module + Copyright (C) 2016-2018 Antonio Diaz Diaz. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +*/ + +#ifndef max + #define max(x,y) ((x) >= (y) ? (x) : (y)) +#endif +#ifndef min + #define min(x,y) ((x) <= (y) ? (x) : (y)) +#endif + +void * resize_buffer( void * buf, const unsigned min_size ); + +enum { + min_dictionary_bits = 12, + min_dictionary_size = 1 << min_dictionary_bits, + max_dictionary_bits = 29, + max_dictionary_size = 1 << max_dictionary_bits, + min_member_size = 36 }; + + +/* defined in main.c */ +extern int verbosity; + +struct Pretty_print + { + const char * name; + char * padded_name; + const char * stdin_name; + unsigned longest_name; + bool first_post; + }; + +static inline void Pp_init( struct Pretty_print * const pp, + const char * const filenames[], + const int num_filenames ) + { + unsigned stdin_name_len; + int i; + pp->name = 0; + pp->padded_name = 0; + pp->stdin_name = "(stdin)"; + pp->longest_name = 0; + pp->first_post = false; + + if( verbosity <= 0 ) return; + stdin_name_len = strlen( pp->stdin_name ); + for( i = 0; i < num_filenames; ++i ) + { + const char * const s = filenames[i]; + const unsigned len = (strcmp( s, "-" ) == 0) ? stdin_name_len : strlen( s ); + if( len > pp->longest_name ) pp->longest_name = len; + } + if( pp->longest_name == 0 ) pp->longest_name = stdin_name_len; + } + +static inline void Pp_set_name( struct Pretty_print * const pp, + const char * const filename ) + { + unsigned name_len, padded_name_len, i = 0; + + if( filename && filename[0] && strcmp( filename, "-" ) != 0 ) + pp->name = filename; + else pp->name = pp->stdin_name; + name_len = strlen( pp->name ); + padded_name_len = max( name_len, pp->longest_name ) + 4; + pp->padded_name = resize_buffer( pp->padded_name, padded_name_len + 1 ); + while( i < 2 ) pp->padded_name[i++] = ' '; + while( i < name_len + 2 ) { pp->padded_name[i] = pp->name[i-2]; ++i; } + pp->padded_name[i++] = ':'; + while( i < padded_name_len ) pp->padded_name[i++] = ' '; + pp->padded_name[i] = 0; + pp->first_post = true; + } + +static inline void Pp_reset( struct Pretty_print * const pp ) + { if( pp->name && pp->name[0] ) pp->first_post = true; } +void Pp_show_msg( struct Pretty_print * const pp, const char * const msg ); + + +static const uint8_t lzip_magic[4] = { 0x4C, 0x5A, 0x49, 0x50 }; /* "LZIP" */ + +typedef uint8_t Lzip_header[6]; /* 0-3 magic bytes */ + /* 4 version */ + /* 5 coded_dict_size */ +enum { Lh_size = 6 }; + +static inline bool Lh_verify_magic( const Lzip_header data ) + { return ( memcmp( data, lzip_magic, 4 ) == 0 ); } + +/* detect (truncated) header */ +static inline bool Lh_verify_prefix( const Lzip_header data, const int sz ) + { + int i; for( i = 0; i < sz && i < 4; ++i ) + if( data[i] != lzip_magic[i] ) return false; + return ( sz > 0 ); + } + +/* detect corrupt header */ +static inline bool Lh_verify_corrupt( const Lzip_header data ) + { + int matches = 0; + int i; for( i = 0; i < 4; ++i ) + if( data[i] == lzip_magic[i] ) ++matches; + return ( matches > 1 && matches < 4 ); + } + +static inline uint8_t Lh_version( const Lzip_header data ) + { return data[4]; } + +static inline bool Lh_verify_version( const Lzip_header data ) + { return ( data[4] == 1 ); } + +static inline unsigned Lh_get_dictionary_size( const Lzip_header data ) + { + unsigned sz = ( 1 << ( data[5] & 0x1F ) ); + if( sz > min_dictionary_size ) + sz -= ( sz / 16 ) * ( ( data[5] >> 5 ) & 7 ); + return sz; + } + + +typedef uint8_t Lzip_trailer[20]; + /* 0-3 CRC32 of the uncompressed data */ + /* 4-11 size of the uncompressed data */ + /* 12-19 member size including header and trailer */ +enum { Lt_size = 20 }; + +static inline unsigned Lt_get_data_crc( const Lzip_trailer data ) + { + unsigned tmp = 0; + int i; for( i = 3; i >= 0; --i ) { tmp <<= 8; tmp += data[i]; } + return tmp; + } + +static inline unsigned long long Lt_get_data_size( const Lzip_trailer data ) + { + unsigned long long tmp = 0; + int i; for( i = 11; i >= 4; --i ) { tmp <<= 8; tmp += data[i]; } + return tmp; + } + +static inline unsigned long long Lt_get_member_size( const Lzip_trailer data ) + { + unsigned long long tmp = 0; + int i; for( i = 19; i >= 12; --i ) { tmp <<= 8; tmp += data[i]; } + return tmp; + } + + +/* defined in in_place.c */ +int decompress_in_place( const int infd, struct Pretty_print * const pp, + const bool testing ); + +/* defined in main.c */ +int convert_retval( const int retval ); +long flush( void * buf, unsigned long size ); +void show_error( const char * const msg, const int errcode, const bool help ); +void show_file_error( const char * const filename, const char * const msg, + const int errcode ); diff --git a/main.c b/main.c new file mode 100644 index 0000000..9f4c739 --- /dev/null +++ b/main.c @@ -0,0 +1,720 @@ +/* Xlunzip - Test tool for the lunzip linux module + Copyright (C) 2016-2018 Antonio Diaz Diaz. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +*/ +/* + Exit status: 0 for a normal exit, 1 for environmental problems + (file not found, invalid flags, I/O errors, etc), 2 to indicate a + corrupt or invalid input file, 3 for an internal consistency error + (eg, bug) which caused xlunzip to panic. +*/ + +#define _FILE_OFFSET_BITS 64 + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#if defined(__MSVCRT__) +#include +#define fchmod(x,y) 0 +#define fchown(x,y,z) 0 +#define SIGHUP SIGTERM +#define S_ISSOCK(x) 0 +#ifndef S_IRGRP +#define S_IRGRP 0 +#define S_IWGRP 0 +#define S_IROTH 0 +#define S_IWOTH 0 +#endif +#endif +#if defined(__OS2__) +#include +#endif + +#include "carg_parser.h" +#include "linux_lzip.h" +#include "linux_lunzip.h" +#include "lzip.h" + +#ifndef O_BINARY +#define O_BINARY 0 +#endif + +#if CHAR_BIT != 8 +#error "Environments where CHAR_BIT != 8 are not supported." +#endif + +int verbosity = 0; +void cleanup_and_fail( const int retval ); + +const char * const Program_name = "Xlunzip"; +const char * const program_name = "xlunzip"; +const char * const program_year = "2018"; +const char * invocation_name = 0; + +const struct { const char * from; const char * to; } known_extensions[] = { + { ".lz", "" }, + { ".tlz", ".tar" }, + { 0, 0 } }; + +char * output_filename = 0; +int infd = -1; /* needed by the fill function */ +int outfd = -1; +bool delete_output_on_interrupt = false; + + +static void show_help( void ) + { + printf( "Xlunzip is a test tool for the lzip decompression code of my lzip patch\n" + "for linux. Xlunzip is similar to lunzip, but it uses the lzip_decompress\n" + "linux module as a backend. Xlunzip tests the module for stream,\n" + "buffer-to-buffer and mixed decompression modes, including in-place\n" + "decompression (using the same buffer for input and output). You can use\n" + "xlunzip to verify that the module produces correct results when\n" + "decompressing single member files, multimember files, or the\n" + "concatenation of two or more compressed files. Xlunzip can be used with\n" + "unzcrash to test the robustness of the module to the decompression of\n" + "corrupted data.\n" + "\nNote that the in-place decompression of concatenated files can't be\n" + "guaranteed to work because an arbitrarily low compression ratio of the\n" + "last part of the data can be achieved by appending enough empty\n" + "compressed members to a file.\n" + "\nUsage: %s [options] [files]\n", invocation_name ); + printf( "\nOptions:\n" + " -h, --help display this help and exit\n" + " -V, --version output version information and exit\n" + " -c, --stdout write to standard output, keep input files\n" + " -d, --decompress decompress (this is the default)\n" + " -f, --force overwrite existing output files\n" + " -I, --in-place decompress or test using only one buffer\n" + " -k, --keep keep (don't delete) input files\n" + " -o, --output= if reading standard input, write to \n" + " -q, --quiet suppress all messages\n" + " -t, --test test compressed file integrity\n" + " -v, --verbose be verbose (a 2nd -v gives more)\n" + "These options are ignored when --in-place is in effect:\n" + " --insize[=] pre-allocate and fill inbuf [default 16 KiB]\n" + " --outsize[=] pre-allocate outbuf [default 512 MiB]\n" + " --nofill do not pass a fill function; requires --insize\n" + " --noflush do not pass a flush function; requires --outsize\n" + "If no file names are given, or if a file is '-', xlunzip decompresses\n" + "from standard input to standard output.\n" + "Numbers may be followed by a multiplier: k = kB = 10^3 = 1000,\n" + "Ki = KiB = 2^10 = 1024, M = 10^6, Mi = 2^20, G = 10^9, Gi = 2^30, etc...\n" + "\nExit status: 0 for a normal exit, 1 for environmental problems (file\n" + "not found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or\n" + "invalid input file, 3 for an internal consistency error (eg, bug) which\n" + "caused xlunzip to panic.\n" + "\nReport bugs to lzip-bug@nongnu.org\n" + "Xlunzip home page: http://www.nongnu.org/lzip/xlunzip.html\n" ); + } + + +static void show_version( void ) + { + printf( "%s %s\n", program_name, PROGVERSION ); + printf( "Copyright (C) %s Antonio Diaz Diaz.\n", program_year ); + printf( "License GPLv2+: GNU GPL version 2 or later \n" + "This is free software: you are free to change and redistribute it.\n" + "There is NO WARRANTY, to the extent permitted by law.\n" ); + } + + +/* assure at least a minimum size for buffer 'buf' */ +void * resize_buffer( void * buf, const unsigned min_size ) + { + if( buf ) buf = realloc( buf, min_size ); + else buf = malloc( min_size ); + if( !buf ) + { + show_error( "Not enough memory.", 0, false ); + cleanup_and_fail( 1 ); + } + return buf; + } + + +void Pp_show_msg( struct Pretty_print * const pp, const char * const msg ) + { + if( verbosity >= 0 ) + { + if( pp->first_post ) + { + pp->first_post = false; + fputs( pp->padded_name, stderr ); + if( !msg ) fflush( stderr ); + } + if( msg ) fprintf( stderr, "%s\n", msg ); + } + } + + +static unsigned long getnum( const char * const ptr, + const unsigned long llimit, + const unsigned long ulimit ) + { + unsigned long result; + char * tail; + errno = 0; + result = strtoul( ptr, &tail, 0 ); + if( tail == ptr ) + { + show_error( "Bad or missing numerical argument.", 0, true ); + exit( 1 ); + } + + if( !errno && tail[0] ) + { + const unsigned factor = ( tail[1] == 'i' ) ? 1024 : 1000; + int exponent = 0; /* 0 = bad multiplier */ + int i; + switch( tail[0] ) + { + case 'Y': exponent = 8; break; + case 'Z': exponent = 7; break; + case 'E': exponent = 6; break; + case 'P': exponent = 5; break; + case 'T': exponent = 4; break; + case 'G': exponent = 3; break; + case 'M': exponent = 2; break; + case 'K': if( factor == 1024 ) exponent = 1; break; + case 'k': if( factor == 1000 ) exponent = 1; break; + } + if( exponent <= 0 ) + { + show_error( "Bad multiplier in numerical argument.", 0, true ); + exit( 1 ); + } + for( i = 0; i < exponent; ++i ) + { + if( ulimit / factor >= result ) result *= factor; + else { errno = ERANGE; break; } + } + } + if( !errno && ( result < llimit || result > ulimit ) ) errno = ERANGE; + if( errno ) + { + show_error( "Numerical argument out of limits.", 0, false ); + exit( 1 ); + } + return result; + } + + +static int extension_index( const char * const name ) + { + int eindex; + for( eindex = 0; known_extensions[eindex].from; ++eindex ) + { + const char * const ext = known_extensions[eindex].from; + const unsigned name_len = strlen( name ); + const unsigned ext_len = strlen( ext ); + if( name_len > ext_len && + strncmp( name + name_len - ext_len, ext, ext_len ) == 0 ) + return eindex; + } + return -1; + } + + +static void set_d_outname( const char * const name, const int eindex ) + { + const unsigned name_len = strlen( name ); + if( eindex >= 0 ) + { + const char * const from = known_extensions[eindex].from; + const unsigned from_len = strlen( from ); + if( name_len > from_len ) + { + output_filename = resize_buffer( output_filename, name_len + + strlen( known_extensions[eindex].to ) + 1 ); + strcpy( output_filename, name ); + strcpy( output_filename + name_len - from_len, known_extensions[eindex].to ); + return; + } + } + output_filename = resize_buffer( output_filename, name_len + 4 + 1 ); + strcpy( output_filename, name ); + strcat( output_filename, ".out" ); + if( verbosity >= 1 ) + fprintf( stderr, "%s: Can't guess original name for '%s' -- using '%s'\n", + program_name, name, output_filename ); + } + + +static int open_instream( const char * const name, struct stat * const in_statsp, + const bool no_ofile ) + { + int infd = open( name, O_RDONLY | O_BINARY ); + if( infd < 0 ) + show_file_error( name, "Can't open input file", errno ); + else + { + const int i = fstat( infd, in_statsp ); + const mode_t mode = in_statsp->st_mode; + const bool can_read = ( i == 0 && + ( S_ISBLK( mode ) || S_ISCHR( mode ) || + S_ISFIFO( mode ) || S_ISSOCK( mode ) ) ); + if( i != 0 || ( !S_ISREG( mode ) && ( !can_read || !no_ofile ) ) ) + { + if( verbosity >= 0 ) + fprintf( stderr, "%s: Input file '%s' is not a regular file%s.\n", + program_name, name, + ( can_read && !no_ofile ) ? + ",\n and '--stdout' was not specified" : "" ); + close( infd ); + infd = -1; + } + } + return infd; + } + + +static bool open_outstream( const bool force, const bool from_stdin ) + { + const mode_t usr_rw = S_IRUSR | S_IWUSR; + const mode_t all_rw = usr_rw | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH; + const mode_t outfd_mode = from_stdin ? all_rw : usr_rw; + int flags = O_APPEND | O_CREAT | O_RDWR | O_BINARY; + if( force ) flags |= O_TRUNC; else flags |= O_EXCL; + + outfd = open( output_filename, flags, outfd_mode ); + if( outfd >= 0 ) delete_output_on_interrupt = true; + else if( verbosity >= 0 ) + { + if( errno == EEXIST ) + fprintf( stderr, "%s: Output file '%s' already exists, skipping.\n", + program_name, output_filename ); + else + fprintf( stderr, "%s: Can't create output file '%s': %s\n", + program_name, output_filename, strerror( errno ) ); + } + return ( outfd >= 0 ); + } + + +void cleanup_and_fail( const int retval ) + { + if( delete_output_on_interrupt ) + { + delete_output_on_interrupt = false; + if( verbosity >= 0 ) + fprintf( stderr, "%s: Deleting output file '%s', if it exists.\n", + program_name, output_filename ); + if( outfd >= 0 ) { close( outfd ); outfd = -1; } + if( remove( output_filename ) != 0 && errno != ENOENT ) + show_error( "WARNING: deletion of output file (apparently) failed.", 0, false ); + } + exit( retval ); + } + + + /* Set permissions, owner and times. */ +static void close_and_set_permissions( const struct stat * const in_statsp ) + { + bool warning = false; + if( in_statsp ) + { + const mode_t mode = in_statsp->st_mode; + /* fchown will in many cases return with EPERM, which can be safely ignored. */ + if( fchown( outfd, in_statsp->st_uid, in_statsp->st_gid ) == 0 ) + { if( fchmod( outfd, mode ) != 0 ) warning = true; } + else + if( errno != EPERM || + fchmod( outfd, mode & ~( S_ISUID | S_ISGID | S_ISVTX ) ) != 0 ) + warning = true; + } + if( close( outfd ) != 0 ) + { + show_error( "Error closing output file", errno, false ); + cleanup_and_fail( 1 ); + } + outfd = -1; + delete_output_on_interrupt = false; + if( in_statsp ) + { + struct utimbuf t; + t.actime = in_statsp->st_atime; + t.modtime = in_statsp->st_mtime; + if( utime( output_filename, &t ) != 0 ) warning = true; + } + if( warning && verbosity >= 1 ) + show_error( "Can't change output file attributes.", 0, false ); + } + + +int convert_retval( const int retval ) + { + switch( retval ) + { + case 0: return 0; + case LZIP_OOM_INBUF: + case LZIP_OOM_OUTBUF: + case LZIP_WRITE_ERROR: return 1; + case LZIP_HEADER1_EOF: + case LZIP_HEADER2_EOF: + case LZIP_BAD_MAGIC1: + case LZIP_BAD_MAGIC2: + case LZIP_BAD_VERSION: + case LZIP_BAD_DICT_SIZE: + case LZIP_BAD_DATA: + case LZIP_DATA_EOF: + case LZIP_BAD_CRC: return 2; + default: return 3; + } + } + + +static long fill( void * buf, unsigned long size ) + { + unsigned long sz = 0; + errno = 0; + while( sz < size ) + { + const int n = read( infd, (uint8_t *)buf + sz, min( 1UL << 20, size - sz ) ); + if( n > 0 ) sz += n; + else if( n == 0 ) break; /* EOF */ + else if( errno != EINTR ) break; + errno = 0; + } + return sz; + } + +long flush( void * buf, unsigned long size ) + { + unsigned long sz = ( outfd >= 0 ) ? 0 : size; + errno = 0; + while( sz < size ) + { + const int n = write( outfd, (uint8_t *)buf + sz, min( 1UL << 20, size - sz ) ); + if( n > 0 ) sz += n; + else if( n < 0 && errno != EINTR ) break; + errno = 0; + } + return sz; + } + +const char * global_name; +static void error(char *x) { show_file_error( global_name, x, 0 ); } + + +static int decompress( struct Pretty_print * const pp, const long cl_insize, + const long cl_outsize, const bool nofill, + const bool noflush, const bool testing ) + { + long in_len = cl_insize; + uint8_t * const inbuf = (in_len > 0) ? malloc( in_len ) : 0; + long out_size = cl_outsize; + uint8_t * const outbuf = (out_size > 0) ? malloc( out_size ) : 0; + long in_pos, out_pos; + int retval; + if( ( in_len > 0 && !inbuf ) || ( out_size > 0 && !outbuf ) ) + { show_error( "Not enough memory.", 0, false ); return 1; } + + if( inbuf ) + { + const long len = fill( inbuf, in_len ); + if( len < in_len ) + { if( errno ) { show_file_error( pp->name, "Read error", errno ); return 1; } + in_len = len; } + } + global_name = pp->name; + retval = convert_retval( __lunzip( inbuf, in_len, nofill ? 0 : fill, + noflush ? 0 : flush, outbuf, out_size, + &in_pos, &out_pos, error ) ); + if( retval ) return retval; + if( outbuf && noflush ) + { + const long len = flush( outbuf, out_pos ); + if( len < out_pos ) + { show_file_error( pp->name, "Write error", errno ); return 1; } + } + if( verbosity >= 1 ) Pp_show_msg( pp, 0 ); + if( verbosity >= 2 ) + { + if( out_pos <= 0 || in_pos <= 0 ) + fputs( "no data compressed. ", stderr ); + else + fprintf( stderr, "%6.3f:1, %5.2f%% ratio, %5.2f%% saved. ", + (double)out_pos / in_pos, + ( 100.0 * in_pos ) / out_pos, + 100.0 - ( ( 100.0 * in_pos ) / out_pos ) ); + if( verbosity >= 3 ) + fprintf( stderr, "decompressed %9lu, compressed %8lu. ", + out_pos, in_pos ); + } + if( verbosity >= 1 ) + fputs( testing ? "ok\n" : "done\n", stderr ); + return 0; + } + + +void signal_handler( int sig ) + { + if( sig ) {} /* keep compiler happy */ + show_error( "Control-C or similar caught, quitting.", 0, false ); + cleanup_and_fail( 1 ); + } + + +static void set_signals( void ) + { + signal( SIGHUP, signal_handler ); + signal( SIGINT, signal_handler ); + signal( SIGTERM, signal_handler ); + } + + +void show_error( const char * const msg, const int errcode, const bool help ) + { + if( verbosity < 0 ) return; + if( msg && msg[0] ) + fprintf( stderr, "%s: %s%s%s\n", program_name, msg, + ( errcode > 0 ) ? ": " : "", + ( errcode > 0 ) ? strerror( errcode ) : "" ); + if( help ) + fprintf( stderr, "Try '%s --help' for more information.\n", + invocation_name ); + } + + +void show_file_error( const char * const filename, const char * const msg, + const int errcode ) + { + if( verbosity >= 0 ) + fprintf( stderr, "%s: %s: %s%s%s\n", program_name, filename, msg, + ( errcode > 0 ) ? ": " : "", + ( errcode > 0 ) ? strerror( errcode ) : "" ); + } + + +void internal_error( const char * const msg ) + { + if( verbosity >= 0 ) + fprintf( stderr, "%s: internal error: %s\n", program_name, msg ); + exit( 3 ); + } + + +int main( const int argc, const char * const argv[] ) + { + const char * default_output_filename = ""; + const char ** filenames = 0; + long cl_insize = 0; + long cl_outsize = 0; + int num_filenames = 0; + int argind = 0; + int failed_tests = 0; + int retval = 0; + int i; + bool filenames_given = false; + bool force = false; + bool in_place = false; + bool keep_input_files = false; + bool nofill = false; + bool noflush = false; + bool stdin_used = false; + bool testing = false; + bool to_stdout = false; + struct Pretty_print pp; + + enum { opt_insize = 256, opt_outsize, opt_nofill, opt_noflush }; + const struct ap_Option options[] = + { + { 'c', "stdout", ap_no }, + { 'd', "decompress", ap_no }, + { 'f', "force", ap_no }, + { 'h', "help", ap_no }, + { 'I', "in-place", ap_no }, + { 'k', "keep", ap_no }, + { 'n', "threads", ap_yes }, + { 'o', "output", ap_yes }, + { 'q', "quiet", ap_no }, + { 't', "test", ap_no }, + { 'v', "verbose", ap_no }, + { 'V', "version", ap_no }, + { opt_insize, "insize", ap_maybe }, + { opt_outsize, "outsize", ap_maybe }, + { opt_nofill, "nofill", ap_no }, + { opt_noflush, "noflush", ap_no }, + { 0 , 0, ap_no } }; + + struct Arg_parser parser; + invocation_name = argv[0]; + + if( !ap_init( &parser, argc, argv, options, 0 ) ) + { show_error( "Not enough memory.", 0, false ); return 1; } + if( ap_error( &parser ) ) /* bad option */ + { show_error( ap_error( &parser ), 0, true ); return 1; } + + for( ; argind < ap_arguments( &parser ); ++argind ) + { + const int code = ap_code( &parser, argind ); + const char * const arg = ap_argument( &parser, argind ); + if( !code ) break; /* no more options */ + switch( code ) + { + case 'c': to_stdout = true; break; + case 'd': testing = false; break; + case 'f': force = true; break; + case 'h': show_help(); return 0; + case 'I': in_place = true; break; + case 'k': keep_input_files = true; break; + case 'n': break; + case 'o': default_output_filename = arg; break; + case 'q': verbosity = -1; break; + case 't': testing = true; break; + case 'v': if( verbosity < 4 ) ++verbosity; break; + case 'V': show_version(); return 0; + case opt_insize: + cl_insize = arg[0] ? getnum( arg, 1, LONG_MAX ) : 16384; break; + case opt_outsize: cl_outsize = arg[0] ? + getnum( arg, min_dictionary_size, LONG_MAX ) : max_dictionary_size; + break; + case opt_nofill: nofill = true; break; + case opt_noflush: noflush = true; break; + default : internal_error( "uncaught option." ); + } + } /* end process options */ + +#if defined(__MSVCRT__) || defined(__OS2__) + setmode( STDIN_FILENO, O_BINARY ); + setmode( STDOUT_FILENO, O_BINARY ); +#endif + + num_filenames = max( 1, ap_arguments( &parser ) - argind ); + filenames = resize_buffer( filenames, num_filenames * sizeof filenames[0] ); + filenames[0] = "-"; + + for( i = 0; argind + i < ap_arguments( &parser ); ++i ) + { + filenames[i] = ap_argument( &parser, argind + i ); + if( strcmp( filenames[i], "-" ) != 0 ) filenames_given = true; + } + + if( testing ) + outfd = -1; + + if( !to_stdout && !testing && + ( filenames_given || default_output_filename[0] ) ) + set_signals(); + + Pp_init( &pp, filenames, num_filenames ); + + output_filename = resize_buffer( output_filename, 1 ); + for( i = 0; i < num_filenames; ++i ) + { + const char * input_filename = ""; + int tmp; + struct stat in_stats; + const struct stat * in_statsp; + output_filename[0] = 0; + + if( !filenames[i][0] || strcmp( filenames[i], "-" ) == 0 ) + { + if( stdin_used ) continue; else stdin_used = true; + infd = STDIN_FILENO; + if( !testing ) + { + if( to_stdout || !default_output_filename[0] ) + outfd = STDOUT_FILENO; + else + { + output_filename = resize_buffer( output_filename, + strlen( default_output_filename ) + 1 ); + strcpy( output_filename, default_output_filename ); + if( !open_outstream( force, true ) ) + { + if( retval < 1 ) retval = 1; + close( infd ); infd = -1; + continue; + } + } + } + } + else + { + input_filename = filenames[i]; + infd = open_instream( input_filename, &in_stats, to_stdout || testing ); + if( infd < 0 ) { if( retval < 1 ) retval = 1; continue; } + if( !testing ) + { + if( to_stdout ) outfd = STDOUT_FILENO; + else + { + set_d_outname( input_filename, extension_index( input_filename ) ); + if( !open_outstream( force, false ) ) + { + if( retval < 1 ) retval = 1; + close( infd ); infd = -1; + continue; + } + } + } + } + + Pp_set_name( &pp, input_filename ); + if( isatty( infd ) ) + { + show_file_error( pp.name, + "I won't read compressed data from a terminal.", 0 ); + if( retval < 1 ) retval = 1; + if( testing ) { close( infd ); infd = -1; continue; } + cleanup_and_fail( retval ); + } + + in_statsp = input_filename[0] ? &in_stats : 0; + if( in_place ) + tmp = decompress_in_place( infd, &pp, testing ); + else + tmp = decompress( &pp, cl_insize, cl_outsize, nofill, noflush, testing ); + if( tmp > retval ) retval = tmp; + if( tmp ) + { if( !testing ) cleanup_and_fail( retval ); + else ++failed_tests; } + + if( delete_output_on_interrupt ) + close_and_set_permissions( in_statsp ); + if( input_filename[0] ) + { + close( infd ); infd = -1; + if( !keep_input_files && !to_stdout && !testing ) + remove( input_filename ); + } + } + if( outfd >= 0 && close( outfd ) != 0 ) + { + show_error( "Error closing stdout", errno, false ); + if( retval < 1 ) retval = 1; + } + if( failed_tests > 0 && verbosity >= 1 && num_filenames > 1 ) + fprintf( stderr, "%s: warning: %d %s failed the test.\n", + program_name, failed_tests, + ( failed_tests == 1 ) ? "file" : "files" ); + free( output_filename ); + free( filenames ); + ap_free( &parser ); + return retval; + } diff --git a/testsuite/check.sh b/testsuite/check.sh new file mode 100755 index 0000000..3fc9f12 --- /dev/null +++ b/testsuite/check.sh @@ -0,0 +1,289 @@ +#! /bin/sh +# check script for Xlunzip - Test tool for the lunzip linux module +# Copyright (C) 2016-2018 Antonio Diaz Diaz. +# +# This script is free software: you have unlimited permission +# to copy, distribute and modify it. + +LC_ALL=C +export LC_ALL +objdir=`pwd` +testdir=`cd "$1" ; pwd` +LZIP="${objdir}"/xlunzip +framework_failure() { echo "failure in testing framework" ; exit 1 ; } + +if [ ! -f "${LZIP}" ] || [ ! -x "${LZIP}" ] ; then + echo "${LZIP}: cannot execute" + exit 1 +fi + +[ -e "${LZIP}" ] 2> /dev/null || + { + echo "$0: a POSIX shell is required to run the tests" + echo "Try bash -c \"$0 $1 $2\"" + exit 1 + } + +if [ -d tmp ] ; then rm -rf tmp ; fi +mkdir tmp +cd "${objdir}"/tmp || framework_failure + +cat "${testdir}"/test.txt > in || framework_failure +in_lz="${testdir}"/test.txt.lz +zero_lz="${testdir}"/zero.lz +fail=0 +test_failed() { fail=1 ; printf " $1" ; [ -z "$2" ] || printf "($2)" ; } + +printf "testing xlunzip-%s..." "$2" + +"${LZIP}" -tq in +[ $? = 2 ] || test_failed $LINENO +"${LZIP}" -tq < in +[ $? = 2 ] || test_failed $LINENO +"${LZIP}" -cdq in +[ $? = 2 ] || test_failed $LINENO +"${LZIP}" -cdq < in +[ $? = 2 ] || test_failed $LINENO +"${LZIP}" -dq -o in < "${in_lz}" +[ $? = 1 ] || test_failed $LINENO +# these are for code coverage +"${LZIP}" -t -- nx_file 2> /dev/null +[ $? = 1 ] || test_failed $LINENO +"${LZIP}" --help > /dev/null || test_failed $LINENO +"${LZIP}" -n1 -V > /dev/null || test_failed $LINENO +"${LZIP}" -m 2> /dev/null +[ $? = 1 ] || test_failed $LINENO +"${LZIP}" -z 2> /dev/null +[ $? = 1 ] || test_failed $LINENO +"${LZIP}" --bad_option 2> /dev/null +[ $? = 1 ] || test_failed $LINENO +"${LZIP}" --t 2> /dev/null +[ $? = 1 ] || test_failed $LINENO +"${LZIP}" --test=2 2> /dev/null +[ $? = 1 ] || test_failed $LINENO +"${LZIP}" --output= 2> /dev/null +[ $? = 1 ] || test_failed $LINENO +"${LZIP}" --output 2> /dev/null +[ $? = 1 ] || test_failed $LINENO +printf "LZIP\001-.............................." | "${LZIP}" -t 2> /dev/null +printf "LZIP\002-.............................." | "${LZIP}" -t 2> /dev/null +printf "LZIP\001+.............................." | "${LZIP}" -t 2> /dev/null + +printf "\ntesting decompression..." + +"${LZIP}" -t "${in_lz}" || test_failed $LINENO +"${LZIP}" -cd "${in_lz}" > copy || test_failed $LINENO +cmp in copy || test_failed $LINENO + +rm -f copy || framework_failure +cat "${in_lz}" > copy.lz || framework_failure +"${LZIP}" -dk copy.lz || test_failed $LINENO +cmp in copy || test_failed $LINENO +printf "to be overwritten" > copy || framework_failure +"${LZIP}" -d copy.lz 2> /dev/null +[ $? = 1 ] || test_failed $LINENO +"${LZIP}" -df copy.lz || test_failed $LINENO +[ ! -e copy.lz ] || test_failed $LINENO +cmp in copy || test_failed $LINENO + +printf "to be overwritten" > copy || framework_failure +"${LZIP}" -df -o copy < "${in_lz}" || test_failed $LINENO +cmp in copy || test_failed $LINENO + +rm -f copy || framework_failure +cat "${in_lz}" > anyothername || framework_failure +"${LZIP}" -dv --output copy - anyothername - < "${in_lz}" 2> /dev/null || + test_failed $LINENO +cmp in copy || test_failed $LINENO +cmp in anyothername.out || test_failed $LINENO +rm -f copy anyothername.out || framework_failure + +"${LZIP}" -tq in "${in_lz}" +[ $? = 2 ] || test_failed $LINENO +"${LZIP}" -tq nx_file.lz "${in_lz}" +[ $? = 1 ] || test_failed $LINENO +"${LZIP}" -cdq in "${in_lz}" > copy +[ $? = 2 ] || test_failed $LINENO +cat copy in | cmp in - || test_failed $LINENO +"${LZIP}" -cdq nx_file.lz "${in_lz}" > copy +[ $? = 1 ] || test_failed $LINENO +cmp in copy || test_failed $LINENO +rm -f copy || framework_failure +cat "${in_lz}" > copy.lz || framework_failure +"${LZIP}" -dq in copy.lz +[ $? = 2 ] || test_failed $LINENO +[ -e copy.lz ] || test_failed $LINENO +[ ! -e copy ] || test_failed $LINENO +[ ! -e in.out ] || test_failed $LINENO +"${LZIP}" -dq nx_file.lz copy.lz +[ $? = 1 ] || test_failed $LINENO +[ ! -e copy.lz ] || test_failed $LINENO +[ ! -e nx_file ] || test_failed $LINENO +cmp in copy || test_failed $LINENO + +cat in in > in2 || framework_failure +cat "${in_lz}" "${in_lz}" > in2.lz || framework_failure +"${LZIP}" -t in2.lz || test_failed $LINENO +"${LZIP}" -cd in2.lz > copy2 || test_failed $LINENO +cmp in2 copy2 || test_failed $LINENO + +cat in2.lz > copy2.lz || framework_failure +printf "\ngarbage" >> copy2.lz || framework_failure +printf "to be overwritten" > copy2 || framework_failure +"${LZIP}" -df copy2.lz || test_failed $LINENO +cmp in2 copy2 || test_failed $LINENO + +printf "\ntesting bad input..." + +headers='LZIp LZiP LZip LzIP LzIp LziP lZIP lZIp lZiP lzIP' +body='\001\014\000\203\377\373\377\377\300\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000$\000\000\000\000\000\000\000' +cat "${in_lz}" > int.lz +printf "LZIP${body}" >> int.lz +if "${LZIP}" -tq int.lz ; then + for header in ${headers} ; do + printf "${header}${body}" > int.lz # first member + "${LZIP}" -tq int.lz + [ $? = 2 ] || test_failed $LINENO ${header} + cat "${in_lz}" > int.lz + printf "${header}${body}" >> int.lz # trailing data + "${LZIP}" -tq int.lz + [ $? = 2 ] || test_failed $LINENO ${header} + done +else + printf "\nwarning: skipping header test: 'printf' does not work on your system." +fi +rm -f int.lz || framework_failure + +cat "${in_lz}" "${in_lz}" "${in_lz}" > in3.lz || framework_failure +if dd if=in3.lz of=trunc.lz bs=14752 count=1 2> /dev/null && + [ -e trunc.lz ] && cmp in2.lz trunc.lz > /dev/null 2>&1 ; then + for i in 6 20 14734 14753 14754 14755 14756 14757 14758 ; do + dd if=in3.lz of=trunc.lz bs=$i count=1 2> /dev/null + "${LZIP}" -tq trunc.lz + [ $? = 2 ] || test_failed $LINENO $i + "${LZIP}" -tq < trunc.lz + [ $? = 2 ] || test_failed $LINENO $i + "${LZIP}" -cdq trunc.lz > out + [ $? = 2 ] || test_failed $LINENO $i + "${LZIP}" -dq < trunc.lz > out + [ $? = 2 ] || test_failed $LINENO $i + done +else + printf "\nwarning: skipping truncation test: 'dd' does not work on your system." +fi +rm -f trunc.lz out || framework_failure + +cat "${in_lz}" > ingin.lz || framework_failure +printf "g" >> ingin.lz || framework_failure +cat "${in_lz}" >> ingin.lz || framework_failure +"${LZIP}" -t ingin.lz || test_failed $LINENO +"${LZIP}" -cd ingin.lz > copy || test_failed $LINENO +cmp in copy || test_failed $LINENO +"${LZIP}" -t < ingin.lz || test_failed $LINENO +"${LZIP}" -d < ingin.lz > copy || test_failed $LINENO +cmp in copy || test_failed $LINENO + +printf "\ntesting linux options..." + +for i in "" =1 =2 =4 =16 =64 =256 =1024 =4096 =16384 =65536 ; do + "${LZIP}" -t --insize$i "${in_lz}" || test_failed $LINENO $i + "${LZIP}" -cd --insize$i "${in_lz}" > copy || test_failed $LINENO $i + cmp in copy || test_failed $LINENO $i +done + +for i in =1 =2 =4 =16 =64 =256 =1024 =4096 ; do + "${LZIP}" -tq --insize$i --nofill "${in_lz}" + [ $? = 2 ] || test_failed $LINENO $i + "${LZIP}" -cdq --insize$i --nofill "${in_lz}" > copy + [ $? = 2 ] || test_failed $LINENO $i + cmp -s in copy && test_failed $LINENO $i +done + +for i in "" =16384 =65536 ; do + "${LZIP}" -t --insize$i --nofill "${in_lz}" || test_failed $LINENO $i + "${LZIP}" -cd --insize$i --nofill "${in_lz}" > copy || + test_failed $LINENO $i + cmp in copy || test_failed $LINENO $i +done + +for i in =18KiB =65536 =262144 ; do + "${LZIP}" -t --outsize$i "${in_lz}" || test_failed $LINENO $i + "${LZIP}" -cd --outsize$i "${in_lz}" > copy || + test_failed $LINENO $i + cmp in copy || test_failed $LINENO $i +done + +for i in =36388 =65536 =262144 ; do + "${LZIP}" -t --outsize$i --noflush "${in_lz}" || + test_failed $LINENO $i + "${LZIP}" -cd --outsize$i --noflush "${in_lz}" > copy || + test_failed $LINENO $i + cmp in copy || test_failed $LINENO $i +done + +for i in =36387 ; do + "${LZIP}" -tq --outsize$i --noflush "${in_lz}" + [ $? = 1 ] || test_failed $LINENO $i + "${LZIP}" -cdq --outsize$i --noflush "${in_lz}" > copy + [ $? = 1 ] || test_failed $LINENO $i + cmp -s in copy && test_failed $LINENO $i +done + +for i in =4096 =16384 ; do + "${LZIP}" -tq --outsize$i --noflush "${in_lz}" + [ $? = 2 ] || test_failed $LINENO $i + "${LZIP}" -cdq --outsize$i --noflush "${in_lz}" > copy + [ $? = 2 ] || test_failed $LINENO $i + cmp -s in copy && test_failed $LINENO $i +done + +"${LZIP}" -t --insize --outsize=36388 --nofill --noflush "${in_lz}" || + test_failed $LINENO +"${LZIP}" -cd --insize --outsize=36388 --nofill --noflush "${in_lz}" > copy || + test_failed $LINENO +cmp in copy || test_failed $LINENO + +"${LZIP}" -tq --nofill "${in_lz}" +[ $? = 2 ] || test_failed $LINENO +"${LZIP}" -cdq --nofill "${in_lz}" > copy +[ $? = 2 ] || test_failed $LINENO +cmp -s in copy && test_failed $LINENO +"${LZIP}" -tq --noflush "${in_lz}" +[ $? = 1 ] || test_failed $LINENO +"${LZIP}" -cdq --noflush "${in_lz}" > copy +[ $? = 1 ] || test_failed $LINENO +cmp -s in copy && test_failed $LINENO + +"${LZIP}" -t --in-place "${in_lz}" || test_failed $LINENO +"${LZIP}" -cd --in-place "${in_lz}" > copy || test_failed $LINENO +cmp in copy || test_failed $LINENO +cat in in in > in3 || framework_failure +for i in in2 in3 ; do + "${LZIP}" -t --in-place $i.lz || test_failed $LINENO + "${LZIP}" -cd --in-place $i.lz > copy || test_failed $LINENO + cmp $i copy || test_failed $LINENO +done +"${LZIP}" -tq --in-place ingin.lz +[ $? = 2 ] || test_failed $LINENO + +cat "${in_lz}" "${zero_lz}" "${zero_lz}" "${zero_lz}" "${zero_lz}" \ + "${zero_lz}" "${zero_lz}" "${zero_lz}" > inz.lz || framework_failure +"${LZIP}" -t --in-place inz.lz || test_failed $LINENO +"${LZIP}" -cd --in-place inz.lz > copy || test_failed $LINENO +cmp in copy || test_failed $LINENO +cat "${zero_lz}" >> inz.lz || framework_failure +"${LZIP}" -tq --in-place inz.lz +[ $? = 2 ] || test_failed $LINENO +"${LZIP}" -cdq --in-place inz.lz > copy +[ $? = 2 ] || test_failed $LINENO +cmp -s in copy && test_failed $LINENO +rm -f copy ingin.lz in2 in2.lz in3 in3.lz || framework_failure + +echo +if [ ${fail} = 0 ] ; then + echo "tests completed successfully." + cd "${objdir}" && rm -r tmp +else + echo "tests failed." +fi +exit ${fail} diff --git a/testsuite/test.txt b/testsuite/test.txt new file mode 100644 index 0000000..9196a3a --- /dev/null +++ b/testsuite/test.txt @@ -0,0 +1,676 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Lesser General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Lesser General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. diff --git a/testsuite/test.txt.lz b/testsuite/test.txt.lz new file mode 100644 index 0000000..22cea6e Binary files /dev/null and b/testsuite/test.txt.lz differ diff --git a/testsuite/zero.lz b/testsuite/zero.lz new file mode 100644 index 0000000..ec60725 Binary files /dev/null and b/testsuite/zero.lz differ -- cgit v1.2.3