Results 1 to 19 of 19

Thread: SLUG 1.x

  1. #1
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 153 Times in 18 Posts
    Hi everyone!

    Here's another fun-tool I wrote called SLUG. This time it's about highspeed compression. SLUG is an LZ-based compressor with an huffman entropy-stage. It has only a 128K non-sliding window because I wanted SLUG to be a flyweight on CPU *and* memory.

    I used "thor" as a reference because I really like this crazy compressor. SLUG's compression speed is comparable to "thor ef". Luckily, decompression is faster and compression is better than "thor ef" almost always.

    Have fun with SLUG,

    Christian




    Download SLUG 1.1: http://www.mytempdir.com/1309238

  2. #2
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Crrrazy thing! Very cool!

    Mirror:
    slug1.1.zip (7 KB)


  3. #3
    Member
    Join Date
    Apr 2007
    Posts
    12
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Brilliant work ! Speed is really impressive.

  4. #4
    Member
    Join Date
    May 2009
    Location
    France
    Posts
    98
    Thanks
    13
    Thanked 74 Times in 44 Posts
    Hello Christian,

    Don't want to bother you but could you generate Linux versions of your wonderful tools ? Please forgive me if you have already been asked for.

    Have a nice day,


    AiZ

  5. #5
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Chris! This is probably the fastest compressor on the planet!!!

    Mirror: Download


    Quote Originally Posted by Christian
    SLUG

  6. #6
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Quote Originally Posted by Christian
    SLUG
    I recall the Metal Slug game from SNK (Neo-Geo).

  7. #7
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 153 Times in 18 Posts
    Everyone who wondered why the decompression command e doesnt work ... its a stupid typo, sorry. It should be a d.

    So, heres a new version which displays some correct usage information: http://www.mytempdir.com/1309828

    Quote Originally Posted by AiZ
    Hello Christian,

    Dont want to bother you but could you generate Linux versions of your wonderful tools ?
    Im not planing to release the sources anytime soon. So I dont know if I can release something under linux.

    Quote Originally Posted by LovePimple
    This is probably the fastest compressor on the planet!!!
    quick0 is just too fast to catch. On some data "thor ef" is faster, too. I could still gain some more speed by dropping the huffman coder or adding some ASM *shivers* - but I wont.

  8. #8
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Chris!

    Mirror: Download

  9. #9
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 779 Times in 486 Posts

  10. #10
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    It's awesome!

  11. #11
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 153 Times in 18 Posts
    Thanks Matt! So, this lame slug seems to speed up on enwik.

  12. #12
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    Matt just don't tested quicklz in fast modes (his test includes only -3 mode)

  13. #13
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Christian
    quick0 is just too fast to catch.
    Why?

  14. #14
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    because quick1 (not even quick0) uses simplest lz77 scheme known at this moment (lzrw1 + lzss encoding)

  15. #15
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 153 Times in 18 Posts
    Quote Originally Posted by LovePimple
    Why?
    SLUG uses huffman which simply cant be faster than a byte or word-coder with some bit-flags. If Id replace the huffman-coder with a byte-coder, SLUG would become quite a bit faster in most situations, but this would go along with a loss in ratio (as always, depending on the data).
    Additionally, I compiled quick0 with GCC and compared its performance to the provided executable by Lasse. His compiler seems to create faster code than GCC (which I use), too.

    So, to sum it up in 3 words: huffman and compiler.


    Anyway, SLUG was a fun-project and mainly an experiment on writing a fast semi-dynamic huffman-coder. But I dont plan to work on SLUG anymore.

  16. #16
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Bulat Ziganshin
    because quick1 (not even quick0) uses simplest lz77 scheme known at this moment (lzrw1 + lzss encoding)
    I hope that someone will eventually write a compressor thats as fast, or faster, than QuickLZ.

    Quote Originally Posted by Christian
    SLUG uses huffman which simply cant be faster than a byte or word-coder with some bit-flags. If Id replace the huffman-coder with a byte-coder, SLUG would become quite a bit faster in most situations, but this would go along with a loss in ratio (as always, depending on the data).
    Additionally, I compiled quick0 with GCC and compared its performance to the provided executable by Lasse. His compiler seems to create faster code than GCC (which I use), too.

    So, to sum it up in 3 words: huffman and compiler.
    Thanks for the explanation!


    Quote Originally Posted by Christian
    Anyway, SLUG was a fun-project and mainly an experiment on writing a fast semi-dynamic huffman-coder. But I dont plan to work on SLUG anymore.

  17. #17
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    Quote Originally Posted by LovePimple
    hope that someone will eventually write a compressor thats as fast, or faster, than QuickLZ.
    for what? just for curiosity?

  18. #18
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    I think that any RLE-based compressor will be faster...

    The catch in compression.

  19. #19
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Bulat Ziganshin
    for what? just for curiosity?
    Yes! I would like to know if such a thing is really possible.

    Quote Originally Posted by encode
    I think that any RLE-based compressor will be faster...
    Interesting!

Similar Threads

  1. CCM, RZM , Slug and Blizzard ?
    By Nania Francesco in forum Data Compression
    Replies: 1
    Last Post: 14th August 2009, 01:57
  2. That awesome slug is back!
    By LovePimple in forum Data Compression
    Replies: 32
    Last Post: 6th March 2009, 15:47
  3. That nasty slug is back
    By Christian in forum Forum Archive
    Replies: 36
    Last Post: 2nd May 2008, 14:49

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •