Thursday 8 March 2012

lrzip-0.610

I haven't been idle all this time. In all honesty I've spent a lot of time playing with my bitcoin mining code in cgminer and after a couple of hardware failures with my mining rig I was finally able to turn my attention back to lrzip which has been really fairly quiet in the bug reporting department since the last stable release.

Here is version 0.610 (when freecode updates):
http://freecode.com/projects/long-range-zip

Or directly:
http://lrzip.kolivas.org

The major feature addition in this latest version is largely thanks to the work of Michael Blumenkrantz in the form of a liblrzip library! This means you can now link in lrzip compression and decompression into any other application. There is a range of APIs available to use this capability from the simplest gzip equivalent lrzip_compress and lrzip_decompress functions to ones that expose all the low level features knobs and dials in lrzip. There are also a couple of demo examples of using these functions included in the source code, but in their simplest form they should be very easy to use. I look forward to seeing package managers, archivers and maybe even git and so on having lrzip support in the future ;) Being a large chunk of new code it is not entirely impossible that bugs or issues with using the library may show up.

WHATS-NEW summary:
The new liblrzip library allows you to add lrzip compression and decompression
to other applications with either simple lrzip_compress and lrzip_decompress
functions or fine control over all the options with low level functions.
Faster rzip stage when files are large enough to require the sliding mmap
feature (usually >1/3 of ram) and in unlimited mode.
A bug where multiple files being compressed or decompressed from the one
command line could have gotten corrupted was fixed.
Modification date of the decompressed file is now set to that of the lrzip
archive (support for storing the original file's date would require modifying
the archive format again).
Compilation warning fixes.
Make lrztar work with directories with spaces in their names.


The full changelog follows:
* Implement complete set of liblrzip libraries, documentation and example uses
with support for simple lrzip_compress() and lrzip_decompress() or complete
fine-grained control over all compression and decompression options.
* Use as much of the low buffer as possible with a single memcopy before going
fine grained byte by byte.
* Preserve the compressed time on decompression where suitable.
* Store a copy of the control struct to be reused on subsequent files to prevent
variables being modified in the control struct on the first file that corrupt
compression/decompression of the 2nd file.
* Explicitly select C99 to avoid certain warnings.
* Generic modifications to silence -Wextra warnings.
* Fix typos.
* Use an array of parameters in lrztar to allow working with directories with
spaces in their names.

Enjoy.

3 comments:

  1. Looks like there is a 5-10 % slowdown with the new release. Anyone else confirm?

    Time (s) to compress approx. 500 megs of flac files under v0.608:
    28.27
    28.65
    28.42
    avg=28.45

    Time (s) to compress approx. 500 megs of flac files under v0.610:
    32.17
    29.14
    32.23
    avg=31.18

    ===10 % slow down

    Time (s) to compress approx. 140 megs of text files under v0.608:
    54.99
    54.84
    54.89
    avg=54.91

    Time (s) to compress approx. 140 megs of text files under v0.610:
    58.06
    57.86
    57.72
    avg=57.88

    ===5 % slow down

    ReplyDelete
  2. Could be slower due to all the code being abstracted out to make it all work as a library. There aren't really any changes to the underlying code otherwise.

    ReplyDelete
  3. Thanks CK. Another problem I am experiencing is the inability to decompress lrz files I created with lrztar v0.608:

    $ lrztar -d logs.tar.lrz
    Decompressing...
    tar: This does not look like a tar archive
    tar: Exiting with failure status due to previous errors

    It does work if I do a 2 step process with 1) lrunzip and 2) tar xf but that is a pain. Any advice?

    $ lrunzip logs.tar.lrz
    Output filename is: logs.tar
    Decompressing...
    100% 141.74 / 141.74 MB
    Average DeCompression Speed: 70.500MB/s
    Output filename is: logs.tar: [OK] - 148623360 bytes
    Total time: 00:00:01.54

    $ ls
    logs.tar logs.tar.lrz

    $ tar xf logs.tar

    ReplyDelete