Results 1 to 19 of 19

Thread: LZTURBO 0.9 parallel compressor

  1. #1
    Member
    Join Date
    Aug 2007
    Posts
    30
    Thanks
    0
    Thanked 0 Times in 0 Posts
    New version : lzturbo 0.9 parallel compressor
    at http://consultant-berater.de/lzturbo

    - improved & faster compression.
    - compress/decompress multiple files in parallel.
    - compress/decompress directories recursively.
    - compress/decompress as easy as the copy command. No archive files needed.

    Please compare lzturbo to thor, slug, quicklz. Specially on newer hardware (like core 2 duo).
    Use switches as recommended in lzturbo.txt file.

    Please read lzturbo.txt

    Enjoy!

  2. #2
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Hamid!

  3. #3
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quick test...

    Compression settings: -11

    A10.jpg > 842,504
    AcroRd32.exe > 2,095,707
    english.dic > 1,648,598
    FlashMX.pdf > 4,341,107
    FP.LOG > 2,032,838
    MSO97.DLL > 2,649,965
    ohs.doc > 1,107,412
    rafale.bmp > 1,912,757
    vcfiu.hlp > 1,142,726
    world95.txt > 1,243,243

    Total = 19,016,857 bytes


    Compression settings: -59

    A10.jpg > 839,071
    AcroRd32.exe > 1,417,442
    english.dic > 687,644
    FlashMX.pdf > 3,736,621
    FP.LOG > 789,021
    MSO97.DLL > 1,810,448
    ohs.doc > 833,243
    rafale.bmp > 1,006,476
    vcfiu.hlp > 677,425
    world95.txt > 576,470

    Total = 12,373,861 bytes

  4. #4
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Another quick test...

    Test machine: Intel PIII (Coppermine) @750 MHz, 512 MB RAM, Windows 2000 Pro SP4

    Test File: ENWIK9 (1,000,000,000 bytes)

    Timed with AcuTimer v1.2


    Compression settings: -11

    Compressed size: 430,453,227 bytes
    Elapsed Time: 00:02:05.970 (125.970 Seconds)


    Compression settings: -12

    Compressed size: 386,091,908 bytes
    Elapsed Time: 00:02:49.249 (169.249 Seconds)

  5. #5
    Member
    Join Date
    Aug 2007
    Posts
    30
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Thanks LovePimple!

  6. #6
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 779 Times in 486 Posts

  7. #7
    Member
    Join Date
    Jun 2008
    Location
    G
    Posts
    372
    Thanks
    26
    Thanked 22 Times in 15 Posts
    Quick Test:

    Eve Online installation:

    uncompressed 3.037.826.415 Bytes

    7z -md=128m -mfb=273 -ms=on -mx=9
    compressed 1.204.469.019 Bytes
    Kernel Time = 20.546 = 00:00:20.546 = 0%
    User Time = 4742.406 = 01:19:02.406 = 160%
    Process Time = 4762.953 = 01:19:22.953 = 161%
    Global Time = 2946.204 = 00:49:06.204 = 100%

    LZTurbo -59 -r
    compressed 1.427.313.134 Bytes
    Kernel Time = 64.968 = 00:01:04.968 = 0%
    User Time = 13569.484 = 03:46:09.484 = 198%
    Process Time = 13634.453 = 03:47:14.453 = 199%
    Global Time = 6832.703 = 01:53:52.703 = 100%

  8. #8
    Member
    Join Date
    Aug 2007
    Posts
    30
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Thanks Matt!
    Thanks thometal !

    thometal
    Can you report decompression time, please.

  9. #9
    Member
    Join Date
    Jun 2008
    Location
    G
    Posts
    372
    Thanks
    26
    Thanked 22 Times in 15 Posts
    7z decompressions time:
    Kernel Time = 8.843 = 00:00:08.843 = 2%
    User Time = 194.953 = 00:03:14.953 = 62%
    Process Time = 203.796 = 00:03:23.796 = 65%
    Global Time = 312.359 = 00:05:12.359 = 100%

    LZTurbo will i post later

  10. #10
    Member
    Join Date
    Jun 2008
    Location
    G
    Posts
    372
    Thanks
    26
    Thanked 22 Times in 15 Posts
    LZTurbo decompressions time:

    Kernel Time = 22.640 = 00:00:22.640 = 5%
    User Time = 586.765 = 00:09:46.765 = 143%
    Process Time = 609.406 = 00:10:09.406 = 148%
    Global Time = 409.657 = 00:06:49.657 = 100%

  11. #11
    Member
    Join Date
    Aug 2007
    Posts
    30
    Thanks
    0
    Thanked 0 Times in 0 Posts
    enwik9: AMD X2-3800+ (Dual core)
    __________________________size____Global comp._time in sec.
    7z_-md=128m_-mfb=273___206938137__32m13
    ___-ms=on_-mx=9
    lzturbo_______-59________232701587___9m33

    thor_0.96a_e4___________314092324___1m05
    thor_0.96a_e5___________315611172___1m27
    lzturbo_______-43________318502927_____35
    gzip_-9_________________322591995___1m41

    lzturbo_______-41________361432918_____24.82
    quicklz1.4b6_3___________397439311___1m20

    lzturbo_______-21________401221587_____25.37
    slug1.1_________________404250979_____26.0
    quicklz1.4b6_2___________406053574_____30.38
    lzturbo_______-11________430453227_____30.52
    quicklz1.4b6_1___________466309381_____32.4
    thor_0.96a_e1___________488397982_____32.6

  12. #12
    Member
    Join Date
    Jun 2009
    Location
    Kraków, Poland
    Posts
    1,475
    Thanks
    26
    Thanked 121 Times in 95 Posts
    donotdisturb
    use CODE tag

  13. #13
    Member
    Join Date
    Aug 2007
    Posts
    30
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Code:
     
    enwik9: AMD X2-3800+ (Dual core) 
                              size    Global comp. time in sec. 
    7z -md=128m -mfb=273   206938137  32m13 
       -ms=on -mx=9 
    lzturbo       -59      232701587   9m33 
    lzturbo	      -44      297641351   1m21 
     
    thor 0.96a e4          314092324   1m05 
    thor 0.96a e5          315611172   1m27 
    lzturbo       -43      318502927     35 
    gzip -9                322591995   1m41 
     
    lzturbo       -41      361432918     24.82 
    quicklz1.4b6 3         397439311   1m20 
     
    lzturbo       -21      401221587     25.37 
    slug1.1                404250979     26.0 
    quicklz1.4b6 2         406053574     30.38 
    lzturbo       -11      430453227     30.52 
    quicklz1.4b6 1         466309381     32.4 
    thor 0.96a e1          488397982     32.6

  14. #14
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    donotdisturb
    By the way, can you describe the -59 algorithm? How do you search matches (hash chains, binary tree, ...), how do you encode LZ output, and so on.


  15. #15
    Member
    Join Date
    Jun 2009
    Location
    Kraków, Poland
    Posts
    1,475
    Thanks
    26
    Thanked 121 Times in 95 Posts
    i'm curious about 7- zip resluts in lzma mode with 1 gb dictionary

  16. #16
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    The MC SFC summary shows a total of 12,381,859 bytes rather than the 12,373,861 bytes I got using the -59 setting.

    http://www.maximumcompression.com/data/summary_sf.php

  17. #17
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    Quote Originally Posted by encode
    how do you encode LZ output, and so on
    it seems that he used coders from tornado for -1*/-2*/-4* modes

  18. #18
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Quote Originally Posted by Bulat Ziganshin
    it seems that he used coders from tornado for -1*/-2*/-4* modes
    Indeed suspicious compressor - too many mirrored features from TORNADO...

  19. #19
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    i don't think that there are many people who can develop good huf/ari encoder, so it's not surprising

Similar Threads

  1. Replies: 23
    Last Post: 24th March 2018, 18:57
  2. Parallel BZIP2 (PBZIP2)
    By joerg in forum Data Compression
    Replies: 21
    Last Post: 18th January 2012, 23:33
  3. parallel compression with batches
    By evg in forum Data Compression
    Replies: 4
    Last Post: 17th September 2009, 19:14
  4. LZTURBO 0.91 Parallel Compressor (Win32/Linux)
    By donotdisturb in forum Forum Archive
    Replies: 26
    Last Post: 19th April 2008, 21:15
  5. LZTURBO 0.1 parallel compressor
    By donotdisturb in forum Forum Archive
    Replies: 5
    Last Post: 7th October 2007, 23:44

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •