Results 1 to 9 of 9

Thread: Fast and strong LZ only compressors

  1. #1
    Member lz77's Avatar
    Join Date
    Jan 2016
    Location
    Russia
    Posts
    80
    Thanks
    24
    Thanked 13 Times in 9 Posts

    Question Fast and strong LZ only compressors

    I did benchmark LZ only compressors (which do not squeeze after LZ-type compression) and looked at LZ4, Lizard, ulz, lzop, and quicklz. None of them can compress enwik8 on 1 core in <= 40Mb in real time (< 2 sec.) If you saw such LZ-type compressors, show me where I can download binaries for Windows, thanks.
    Where I can download snappy.exe?

  2. #2
    Member jibz's Avatar
    Join Date
    Jan 2015
    Location
    Denmark
    Posts
    124
    Thanks
    106
    Thanked 71 Times in 51 Posts
    Timing depends on the CPU and RAM of course, but I am sure some of them can reach those goals, for instance BriefLZ gets:

    Code:
    blzpack.exe -v -3 enwik8 enwik8.blz
    in 100000000 out 39508633 ratio 39% time 1.85
    Oodle and lzturbo will likely do even better.

    Edit: though BriefLZ may not fit into pure LZ as it uses a universal code.

  3. #3
    Member
    Join Date
    Sep 2018
    Location
    Philippines
    Posts
    101
    Thanks
    31
    Thanked 1 Time in 1 Post
    Quote Originally Posted by jibz View Post
    Timing depends on the CPU and RAM of course, but I am sure some of them can reach those goals, for instance BriefLZ gets:

    Code:
    blzpack.exe -v -3 enwik8 enwik8.blz
    in 100000000 out 39508633 ratio 39% time 1.85
    Oodle and lzturbo will likely do even better.

    Edit: though BriefLZ may not fit into pure LZ as it uses a universal code.
    My lzuf compressor compresses enwik8 to around 38MB maybe in 40s in my old dual-core machine in 2008, decompresses in 3s. I don't have it anymore but only lzuf2 and lzuf5.

    It is here in The Data Compression Guide's LZ77 page, plus the early LZT (or LZ-Tamayo) in 2008. The timings in LTCB for lzgt3a were somewhat different than mine:

    https://sites.google.com/site/datacompressionguide/lz77

  4. #4
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    834
    Thanks
    239
    Thanked 307 Times in 183 Posts
    What is a squeeze?
    Last edited by Jyrki Alakuijala; 25th March 2020 at 05:18.

  5. #5
    Member lz77's Avatar
    Join Date
    Jan 2016
    Location
    Russia
    Posts
    80
    Thanks
    24
    Thanked 13 Times in 9 Posts
    Quote Originally Posted by Jyrki Alakuijala View Post
    What is a squeeze?
    Pardon, I'm not an English speaker. I mean the program after LZ compression does not make Huffman/arithmetic/other compression.

    Where I can download blzpack.exe (or brieflz)?
    Last edited by lz77; 25th March 2020 at 11:01.

  6. #6
    Member CompressMaster's Avatar
    Join Date
    Jun 2018
    Location
    Lovinobana, Slovakia
    Posts
    189
    Thanks
    52
    Thanked 13 Times in 13 Posts
    Quote Originally Posted by lz77 View Post
    Where I can download blzpack.exe (or brieflz)?
    Here.

  7. Thanks:

    lz77 (26th March 2020)

  8. #7
    Member CompressMaster's Avatar
    Join Date
    Jun 2018
    Location
    Lovinobana, Slovakia
    Posts
    189
    Thanks
    52
    Thanked 13 Times in 13 Posts
    Quote Originally Posted by lz77 View Post
    Where I can download snappy.exe?
    Here.

  9. #8
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    834
    Thanks
    239
    Thanked 307 Times in 183 Posts
    Quote Originally Posted by lz77 View Post
    Pardon, I'm not an English speaker. I mean the program after LZ compression does not make Huffman/arithmetic/other compression.
    gipfeli does something that is in between of Huffman and not doing it. It might qualify to your requirements (or not)...
    Last edited by Jyrki Alakuijala; 26th March 2020 at 13:48.

  10. #9
    Member lz77's Avatar
    Join Date
    Jan 2016
    Location
    Russia
    Posts
    80
    Thanks
    24
    Thanked 13 Times in 9 Posts
    blzpack seems only LZ-type compressor (at least I see in enwik8.blz literals like 'mediawiki xmlns=' and '"http://www.').
    blzpack -2 -b1024m enwik8 gives ~ the same ratio as my program (while this is a prototype program for debugging and improving my algorithm), but my program is much faster than blzpack -1...
    I want to get time/ratio similar to zstd v1.4.4 for win64:
    timer.exe zstd.exe -1 -f --no-progress enwik8
    Both ratio & compression time will be the same but decompression time at this time 15% worse.

    I want to find a buyer(s) for my algorithms/sources. Maybe I will sell it as shareware...

Similar Threads

  1. RAZOR - strong LZ-based archiver
    By Christian in forum Data Compression
    Replies: 208
    Last Post: 4th June 2020, 22:53
  2. Benchmarking fast LZ compressors
    By dnd in forum Data Compression
    Replies: 1
    Last Post: 15th June 2015, 14:12
  3. Some of my toy compressors
    By RichSelian in forum Data Compression
    Replies: 27
    Last Post: 6th October 2011, 05:09
  4. image compressors
    By maadjordan in forum Forum Archive
    Replies: 5
    Last Post: 13th August 2007, 09:28
  5. Fastest Compressors
    By LovePimple in forum Forum Archive
    Replies: 0
    Last Post: 1st November 2006, 05:36

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •