Results 1 to 24 of 24

Thread: QuickLZ ZIP - new zip/deflate library

  1. #1
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts
    Hope this fills a gap - fast zip compression: http://www.quicklz.com/zip.html

  2. #2
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    it's great idea!

  3. #3
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    how does it work?

  4. #4
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    because Lasse doesn't answer, i will explain how i may make this yourself:

    just use fast LZ serach engine (such as quicklz and encode found literals/matches using static huffman tree (STATIC_TREES)

  5. #5
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts
    Hi, sorry for late reply

    Yeah, Bulat is entirely right

    Unfortunatly the deflate format is not designed for speed, so it's not very fast (~58 MB/s compared to 148 MB/s for QuickLZ, for some test files on one of my pc's).

    Lasse

  6. #6
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    you can try to reuse bit i/o from tornado, may be it will be faster?

  7. #7
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts
    Cool, will look at at

    What I'm doing now is just

    void out(unsigned int data, unsigned int len)
    {
    unsigned int *dst = dst2 + (bitoffset >> 3);
    *dst |= data << (bitoffset & 0x7);
    bitoffset += len;
    }

    where dst2 is a (public) bit-offset pointer. Requires destination memory to be initial zero'ed out.

  8. #8
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts
    Oh, dst2 was the byte base pointer and bitoffset is bit offset pointer. But the solution sucks Will look at yours soon.

  9. #9
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Lasse Reinhold
    Hope this fills a gap - fast zip compression: http://www.quicklz.com/zip.html
    Thanks Lasse!

  10. #10
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    Lasse Reinhold
    yes, your way isn't fastest possible

  11. #11
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 779 Times in 486 Posts
    quicklz_zip.exe failed on enwik8. I decompressed with unzip (InfoZIP 5.52) and the output was the right size (100 MB) but differed from the original. I also get this message:

    C:
    es>unzip x8.zip
    Archive: x8.zip
    inflating: enwik8
    error: invalid compressed data to inflate

  12. #12
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts
    Yikes. I'll fix it later - currently busy with my 1.30 release.

  13. #13
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Will you be working on 'QuickLZ ZIP' now that QuickLZ v1.30 final has been released?

  14. #14
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts
    Either that, or a QuickLZ 1.40. I'm not sure yet.

  15. #15
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    I hope you get time to fix the problem that Matt reported.

  16. #16
    Member
    Join Date
    Jan 2007
    Location
    Moscow
    Posts
    239
    Thanks
    0
    Thanked 3 Times in 1 Post
    How should i compile QuickLZ 1.30 final? I've got these errors:

    g++.exe quicklz.c -oq0.exe
    quicklz.c:23: error: 'size_t' does not name a type
    quicklz.c:24: error: 'size_t' does not name a type
    quicklz.c:25: error: 'size_t' does not name a type
    quicklz.c:26: error: 'size_t' does not name a type

    cl quicklz.c
    Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3052 for 80x86
    Copyright (C) Microsoft Corporation 1984-2002. All rights reserved.

    quicklz.c
    quicklz.c(23) : error C2061: syntax error : identifier 'qlz_decompress'
    quicklz.c(23) : error C2059: syntax error : ';'
    quicklz.c(23) : error C2059: syntax error : 'type'
    quicklz.c(24) : error C2061: syntax error : identifier 'qlz_compress'
    quicklz.c(24) : error C2059: syntax error : ';'
    quicklz.c(24) : error C2059: syntax error : 'type'
    quicklz.c(25) : error C2061: syntax error : identifier 'qlz_size_decompressed'
    quicklz.c(25) : error C2059: syntax error : ';'
    quicklz.c(25) : error C2059: syntax error : 'type'
    quicklz.c(26) : error C2061: syntax error : identifier 'qlz_size_compressed'
    quicklz.c(26) : error C2059: syntax error : ';'
    quicklz.c(26) : error C2059: syntax error : 'type'

    Thanks.

  17. #17
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts
    Hi nimdansk,

    To compile the quick.exe project on the website using VC, first comment out the COMPRESSION_LEVEL flag in quicklz.c and define STREAMING_MODE to 960000 (if you want it to be compatible with the demo project on the website).

    Now go to the VCquick directory and compile with cl /Ox quick.c

    You can also compile the other C demo projects. For example, go to the C directory and compile with cl /Ox block_compress.c or cl /Ox block_decompress.c, etc.

    I'm not sure how to define the size_t type in g++, it varies between compilers.

    I'll probably add some notes to the manual about compiling soon.

  18. #18
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts
    Hmm, seems like the back slash characters got removed in my post (in the directory paths).

  19. #19
    Member
    Join Date
    Jan 2007
    Location
    Moscow
    Posts
    239
    Thanks
    0
    Thanked 3 Times in 1 Post
    Hum... I realised that there is no main() function in quicklz.c thus it couldn't be compiled

    I compiled quick.c, but is it possible to make different versions for -0, -1 etc.? Compiling quicklz.c i get the same error as before.

    I have:
    >g++ -v
    Reading specs from E:/Dev-Cpp/bin/../lib/gcc/mingw32/3.4.2/specs
    Configured with: ../gcc/configure --with-gcc --with-gnu-ld --with-gnu-as --host=mingw32 --target=mingw32 --prefix=/mingw --enab
    le-threads --disable-nls --enable-languages=c,c++,f77,ada,objc,java --disable-win32-registry --disable-shared --enable-sjlj-exc
    eptions --enable-libgcj --disable-java-awt --without-x --enable-java-gc=boehm --disable-libgcj-debug --enable-interpreter --ena
    ble-hash-synchronization --enable-libstdcxx-debug
    Thread model: win32
    gcc version 3.4.2 (mingw-special)

    C:>q
    QuickLZ 1.30. Copyright 2006-2007 Lasse Mikkel Reinhold.

    Compiled with:
    COMPRESSION_LEVEL using levels.h wrapper
    STREAMING_MODE 960000
    test_rle defined
    speedup_incompressible defined
    memory_safe not defined

  20. #20
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts
    Hi nimdansk,

    There's an older version of the demo project here which compiles with single compression level: http://www.quicklz.com/quick130beta.c

    For g++, try locating typedef.h and including it. size_t should be defined there.

  21. #21
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts
    I have fixed a bug in QuickLZ ZIP so that I can now compress enwik8 (and other files):

    http://www.quicklz.com/zip.html

  22. #22
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Lasse!

  23. #23
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts
    I have released source if anyone should care - www.quicklz.com/zip.html

    Just 355 lines for a stand alone zip/deflate library

  24. #24
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Lasse!

Similar Threads

  1. 7-Zip
    By Vacon in forum Data Compression
    Replies: 555
    Last Post: 16th October 2019, 11:55
  2. 7-Zip beta 9.11
    By Vacon in forum Data Compression
    Replies: 2
    Last Post: 23rd March 2010, 18:06
  3. 7-zip
    By squxe in forum Forum Archive
    Replies: 19
    Last Post: 9th April 2008, 23:26
  4. 7-zip 4.53
    By squxe in forum Forum Archive
    Replies: 5
    Last Post: 28th August 2007, 04:37
  5. 7-zip 4.51 beta
    By d33j4y in forum Forum Archive
    Replies: 0
    Last Post: 27th July 2007, 11:05

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •