Page 1 of 2 12 LastLast
Results 1 to 30 of 33

Thread: That awesome slug is back!

  1. #1
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts

    Cool That awesome slug is back!

    Slug v1.26b has been released.

    Quote Originally Posted by Christian Martelock
    I uploaded Slug 1.26b. It comes with small improvements in both, speed and ratio. So, let the hard-disks glow.
    Download


    Thanks Chris!

    Mirror: Download

  2. #2
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quick test...

    A10.jpg > 845,286
    AcroRd32.exe > 1,894,351
    english.dic > 1,072,010
    FlashMX.pdf > 3,894,215
    FP.LOG > 1,082,120
    MSO97.DLL > 2,304,324
    ohs.doc > 902,533
    rafale.bmp > 1,272,522
    vcfiu.hlp > 824,752
    world95.txt > 808,176

    Total = 14,900,289 bytes


    ENWIK8 > 35,106,390 bytes

    Compression Time: 8.314 Seconds

    Decompression Time: 8.311 Seconds

  3. #3
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 151 Times in 18 Posts
    Thanks LovePimple!

    Now, I'll wait for some benchmarks to show up.

  4. #4
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Slug 1.26c compression speed is slightly slower than 1.26b on my AMD Sempron 2400+ machine.

  5. #5
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 151 Times in 18 Posts
    Thanks for the info. Can you please check your PMs, LovePimple...

  6. #6
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts

    Thumbs up

    Here are the results you asked for. Could not send them by PM because the PM service complained about the message being 'too long'.

    Edit: <link removed>
    Last edited by LovePimple; 5th May 2008 at 17:17.

  7. #7
    Member
    Join Date
    May 2008
    Location
    Germany
    Posts
    410
    Thanks
    37
    Thanked 60 Times in 37 Posts
    First Test on ATHLON-XP 512 MB RAM
    ---------------------------------------
    compress a oracle dump-file (Text-File)
    ---------------------------------------
    RINGS13 648331264 bytes to 26356464 bytes time= 75.22 s
    -------------------------------------------------------------------------------
    RINGS15C 5 648331264 bytes to 26580793 bytes time= 77.32 s.
    -------------------------------------------------------------------------------
    THOR096A 648331264 bytes to 79322888 bytes time= 51.544s SPEED: 12.00 MB/sec
    -------------------------------------------------------------------------------
    Slug 1.26b 633136.00 kb -> 50514.93 kb (7.98%) time= 36 s
    Slug 1.26c 633136.00 kb -> 50514.93 kb (7.98%) time= 68 s
    -------------------------------------------------------------------------------
    should means
    slug 1.26b ist the fastest compressor on this system
    is a lot faster then 1.26c
    bets allways thor and the other compressors
    (in other tests the same tendency on this system)

    slug version 1.26c has no fortune on this system

    RINGS15C needs more then twice the time of slug 1.26b
    but has a really better compression too
    Last edited by joerg; 5th May 2008 at 17:32. Reason: correct spelling

  8. #8
    Member
    Join Date
    Jan 2007
    Location
    Moscow
    Posts
    239
    Thanks
    0
    Thanked 3 Times in 1 Post
    Where can i get Slug 1.26b to test?

  9. #9
    Member
    Join Date
    May 2008
    Location
    Germany
    Posts
    410
    Thanks
    37
    Thanked 60 Times in 37 Posts
    try the website from christian

    http://christian.martelock.googlepages.com/index.htm

    sorry now i see version 1.26b is no more there
    Last edited by joerg; 5th May 2008 at 17:54.

  10. #10
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 151 Times in 18 Posts
    Thanks joerg. I really do appreciate your effort, but sadly your test does not help me much. Speed test have to be done very thoroughly - multiple runs, maybe even reboots between the different runs - process times are interesting, too...

  11. #11
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 151 Times in 18 Posts
    Quote Originally Posted by nimdamsk
    Where can i get Slug 1.26b to test?
    It's still there - just take the link from 1.26c and replace the c with a b.

    Quote Originally Posted by joerg
    First Test on ATHLON-XP 512 MB RAM
    What Athlon-XP do you have and which OS are you using? The great difference between 'b' and 'c' is really strange. Is the test hard disk very fragmented? Did you run the tests more than one time?

  12. #12
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 151 Times in 18 Posts
    If someone does have some time, I'd be interested in benchmark numbers for these versions:

    [removed]

    The data-set should be at least 1GB larger than your RAM-size and NOT to nul but to a real file. Additionally, it'd be useful to do at least two runs. Wall times, process times (e.g. using timer from Igor) and your system setup (CPU,RAM,HDD,OS) would be great.

    Thanks a lot!

    EDIT: Without feedback I can only optimize for my system - and that's what I did. There'll be 1.27 tomorrow, maybe the final 1.2+ - anyway, I hope it'll work quite well for everyone.
    Last edited by Christian; 7th May 2008 at 00:14. Reason: removed download links

  13. #13
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts

    Lightbulb

    That's stopped the benchmarkers dead in their tracks!

  14. #14
    Tester
    Black_Fox's Avatar
    Join Date
    May 2008
    Location
    [CZE] Czechia
    Posts
    471
    Thanks
    26
    Thanked 9 Times in 8 Posts
    I tried compressing 3,5GB tar archive from an external disk to internal one, but it seems USB is too slow... if results can be trusted, 1.26 and 1.26b were the fastest (wall time 334 s), 1.26c the slowest (377 s) and 1.27 somewhere inbetween (353 s). Will repeat tests later on internal disk only...
    CPU: vanilla Athlon 64 X2 3800+, L1: 64+64 kB/core, L2: 512kB/core
    Last edited by Black_Fox; 8th May 2008 at 14:05.
    I am... Black_Fox... my discontinued benchmark
    "No one involved in computers would ever say that a certain amount of memory is enough for all time? I keep bumping into that silly quotation attributed to me that says 640K of memory is enough. There's never a citation; the quotation just floats like a rumor, repeated again and again." -- Bill Gates

  15. #15
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts

    Thumbs up

    Quote Originally Posted by Christian View Post
    There'll be 1.27 tomorrow, maybe the final 1.2+ - anyway, I hope it'll work quite well for everyone.
    Thanks Chris!

    Mirror: Download

  16. #16
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 151 Times in 18 Posts
    Will repeat tests later on internal disk only...
    Thanks BF - I'm looking forward to the results. *fingers crossed* But no matter what, 1.27 is the last version for the time being.

  17. #17
    Tester
    Black_Fox's Avatar
    Join Date
    May 2008
    Location
    [CZE] Czechia
    Posts
    471
    Thanks
    26
    Thanked 9 Times in 8 Posts
    Tests for compression within one disk are finished. Times were very consistent this time (wall times were +- 2 seconds):
    Code:
    SLUG 1.26
    Kernel Time  =    10.984 = 00:00:10.984 =   3%
    User Time    =   202.093 = 00:03:22.093 =  62%
    Process Time =   213.078 = 00:03:33.078 =  66%
    Global Time  =   322.016 = 00:05:22.016 = 100%
    
    SLUG 1.26b
    Kernel Time  =    12.062 = 00:00:12.062 =   3%
    User Time    =   204.312 = 00:03:24.312 =  62%
    Process Time =   216.375 = 00:03:36.375 =  66%
    Global Time  =   327.484 = 00:05:27.484 = 100%
    
    SLUG 1.26c
    Kernel Time  =    14.140 = 00:00:14.140 =   3%
    User Time    =   209.421 = 00:03:29.421 =  57%
    Process Time =   223.562 = 00:03:43.562 =  61%
    Global Time  =   365.781 = 00:06:05.781 = 100%
    
    SLUG 1.27
    Kernel Time  =    15.812 = 00:00:15.812 =   3%
    User Time    =   211.921 = 00:03:31.921 =  53%
    Process Time =   227.734 = 00:03:47.734 =  57%
    Global Time  =   396.391 = 00:06:36.391 = 100%
    I am... Black_Fox... my discontinued benchmark
    "No one involved in computers would ever say that a certain amount of memory is enough for all time? I keep bumping into that silly quotation attributed to me that says 640K of memory is enough. There's never a citation; the quotation just floats like a rumor, repeated again and again." -- Bill Gates

  18. #18
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 151 Times in 18 Posts
    Thanks a lot for testing, BlackFox! And many thanks to Matt for testing Slug and RZM!

    But no matter what, 1.27 is the last version for the time being.
    Surprise, there's now Slug 1.27b and it's not a bugfix.
    It comes with a small compression improvement. Good news is, the ratio improvement is for free - at least on my system and over at Metacompressor's live-benchmarking.

    For now I won't tweak IO anymore - it's boring and without enough feedback (only 4 people reported proper feedback) it's just like tapping in the dark. Btw., I tried IO with a reader- and a writer-thread, but no luck. Results did not improve at all on my system.
    Last edited by Christian; 12th May 2008 at 21:38. Reason: added download link

  19. #19
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Christian View Post
    Surprise, there's now Slug 1.27b and it's not a bugfix.
    Thanks Chris!

    Mirror: Download

  20. #20
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    BUG:
    Mistake in a command line:
    Code:
    slug f ".\projects.tar" ".\a.b"
    Slug tries to create infinite archive.

  21. #21
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 151 Times in 18 Posts
    There's a new experimental version of slug available - actually, it's a rewrite because I plan to open the source in the future. Some things have changed:

    -a tiny bit slower on my C2D, maybe ~10%
    -no extra handling of already compressed data
    -better compression on most files

    If you have some time spare, feel free to check for any errors (except wrong commandline usage ).
    Attached Files Attached Files

  22. #22
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by Christian View Post
    There's a new experimental version of slug available - actually, it's a rewrite because I plan to open the source in the future. Some things have changed:

    -a tiny bit slower on my C2D, maybe ~10%
    -no extra handling of already compressed data
    -better compression on most files

    If you have some time spare, feel free to check for any errors (except wrong commandline usage ).
    That's a news!
    Thanks a lot, I'm on it.

  23. #23
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quick results:
    Code:
    TCUP:
    slug	  120218388 13.235s
    slug X 116258883 ~19s
    
    Bookstar:
    slug   14320218 1.468s
    slug X 13881376 ~1.6s

  24. #24
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 151 Times in 18 Posts
    Quote Originally Posted by m^2 View Post
    Quick results:
    Code:
    TCUP:
    slug	  120218388 13.235s
    slug X 116258883 ~19s
    
    Bookstar:
    slug   14320218 1.468s
    slug X 13881376 ~1.6s
    Thanks m^2! Interesting results. Probably, I'll have to add handling for hardly compressible data parts at a later time. Damn it, this will bloat the code by another couple of lines.

    Is the file TCUP available somewhere?

  25. #25
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by Christian View Post
    Thanks m^2! Interesting results. Probably, I'll have to add handling for hardly compressible data parts at a later time. Damn it, this will bloat the code by another couple of lines.

    Is the file TCUP available somewhere?
    Normally no.
    Really, I haven't decided whether I do better by testing by publicly available data or private one - because of the rule to tweak on one data and verify on another.
    But you can get the installer from www.tcup.eu. I use version 4.1.
    Last edited by m^2; 13th January 2009 at 00:44.

  26. #26
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 151 Times in 18 Posts
    Small update: I reintroduced compressed data handling. There are no other changes.

    Slug should ratio-wise be on par with "thor e4" while being a lot faster.
    Attached Files Attached Files
    Last edited by Christian; 13th January 2009 at 17:58. Reason: spelling

  27. #27
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    408
    Thanks
    0
    Thanked 5 Times in 5 Posts
    How is the compressed data handling working? Lets say we have a tar file with a mix of files compressed with your ccm and very good compressable text files.

  28. #28
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    TCUP:
    115933595 ~15
    Bookstar:
    13881376 ~1.6
    And one of my not yet ready test sets, XSOS, virtual machine image, 442515968 B:
    slug 69986283 10.141
    thor e4 67513532 38.312
    slug X 67232526 ~12

  29. #29
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 151 Times in 18 Posts
    Quote Originally Posted by Simon Berger View Post
    How is the compressed data handling working? Lets say we have a tar file with a mix of files compressed with your ccm and very good compressable text files.
    Well, the algo detects hardly compressible parts in the tar file on the fly. When such a part is detected, a faster compression mode is selected. While in control and processing the data, the faster mode decides at which point it makes sense to switch back.

    The faster mode is just huffman coding plus some rolz book keeping.

  30. #30
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    408
    Thanks
    0
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Christian
    Well, the algo detects hardly compressible parts in the tar file on the fly. When such a part is detected, a faster compression mode is selected. While in control and processing the data, the faster mode decides at which point it makes sense to switch back.
    Thanks. Yes the switching to faster compression mode isn't this hard (except a good system to mark these points) but I wondered about if it switch back again.

Page 1 of 2 12 LastLast

Similar Threads

  1. IzArc is back
    By Vacon in forum Data Compression
    Replies: 4
    Last Post: 1st March 2011, 23:50
  2. -Delete back in srep
    By SvenBent in forum Data Compression
    Replies: 0
    Last Post: 13th December 2009, 20:36
  3. CCM, RZM , Slug and Blizzard ?
    By Nania Francesco in forum Data Compression
    Replies: 1
    Last Post: 14th August 2009, 00:57
  4. That nasty slug is back
    By Christian in forum Forum Archive
    Replies: 36
    Last Post: 2nd May 2008, 13:49
  5. SLUG 1.x
    By Christian in forum Forum Archive
    Replies: 18
    Last Post: 28th April 2007, 15:14

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •