Results 1 to 30 of 30

Thread: BALZ v1.07 is here!

  1. #1
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts

    PIM Logo BALZ v1.07 is here!

    OK, as a fastest author on the planet, let me introduce the brand new BALZ v1.07. For sure, this new version defines a new level of all my data compression software. Many changes inside compared to the previous version:
    • Enlarged ROLZ model to 128 MB
    • Dictionary size is 64 MB (mem.use 2N!)
    • Improved match finder - 1.5-2X compression speed gain (compared to the BALZ with same model size - NOT to the previous version)
    • Improved LZ-output coding
    • Improved parsing
    • Improved EXE transformer
    • Some minor improvements/optimizations
    Enjoy! I think I will make some break in BALZ development - at least to finally see its results on benchmarks - since new versions are out faster than benchmarks update...

    http://encode.su/balz/index.htm


  2. #2
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,239
    Thanks
    192
    Thanked 968 Times in 501 Posts
    E2160 @ 9x360=3.24, DDR2-800 5-5-5-18 @ 900
    http://shelwien.googlepages.com/balz107.htm
    Somehow it seems that 1.06 was better
    Well, decoding surely became slower.

  3. #3
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts
    Quote Originally Posted by Shelwien View Post
    E2160 @ 9x360=3.24, DDR2-800 5-5-5-18 @ 900
    http://shelwien.googlepages.com/balz107.htm
    Somehow it seems that 1.06 was better
    Well, decoding surely became slower.
    Well, since compression ratio is higher, at your test compression speed IS also higher, I think it's OK!
    EDIT: Decompression is still faster than RAR's one at higher compression levels.

  4. #4
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts

    Thumbs up

    Thanks Ilia!

    Mirror: Download

  5. #5
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,239
    Thanks
    192
    Thanked 968 Times in 501 Posts
    > Decompression is still faster than RAR's one at higher compression levels.

    Well, I added a couple of rar entries to the table at http://shelwien.googlepages.com/balz107.htm

  6. #6
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts
    Thanks! Yes rar 3.71 -m5 -mdg -mct- has a faster decompression, but RAR's compression is notable weaker in this case...

  7. #7
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,239
    Thanks
    192
    Thanked 968 Times in 501 Posts
    Why don't you think about that too?
    I mean, segmentation + LZ/CM switching?
    Imho rar shows that its quite useful.
    Actually, 7-zip has both better LZ and newer ppmd version,
    but still doesn't win over rar just because of that scheme.

  8. #8
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts
    Quote Originally Posted by Shelwien View Post
    Why don't you think about that too?
    I mean, segmentation + LZ/CM switching?
    I saw RAR's source code... Crazy thing - RARVM...
    BALZ is very simple, and performs not that worser... Anyway, maybe I should at least try to implement/test such idea...

  9. #9
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quick test...

    Test files: MC SFC files.

    BALZ [e]

    A10.jpg > 836,375
    AcroRd32.exe > 1,465,509
    english.dic > 861,103
    FlashMX.pdf > 3,763,083
    FP.LOG > 675,283
    MSO97.DLL > 1,887,647
    ohs.doc > 831,132
    rafale.bmp > 1,046,912
    vcfiu.hlp > 699,712
    world95.txt > 605,779

    Total = 12,672,535 bytes


    BALZ [ex]

    A10.jpg > 836,375
    AcroRd32.exe > 1,452,713
    english.dic > 756,409
    FlashMX.pdf > 3,754,817
    FP.LOG > 570,238
    MSO97.DLL > 1,878,006
    ohs.doc > 824,780
    rafale.bmp > 1,009,411
    vcfiu.hlp > 668,656
    world95.txt > 576,364

    Total = 12,327,769 bytes



    Test files: All ten MC SFC files tarred. (mcsfc.tar 53,143,552 bytes)

    BALZ [e]

    Compression

    Compressed Size: 12,664,201 bytes

    Elapsed Time: 96.94 Seconds

    00 Days 00 Hours 01 Minutes 36.94 Seconds


    Decompression

    Elapsed Time: 16.75 Seconds



    BALZ [ex]

    Compression

    Compressed Size: 12,317,196 bytes

    Elapsed Time: 310.41 Seconds

    00 Days 00 Hours 05 Minutes 10.41 Seconds


    Decompression

    Elapsed Time: 16.55 Seconds

  10. #10
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts
    BTW, this is my very first asymmetric compressor that benefits from tarring SFC... Also, if files will be correctly sorted inside an archive... Try to add all SFC files to 7z archive with Store method - 7-Zip should correctly sort files inside! Thanks for testing!

  11. #11
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts
    sfc.7z (53,135,131 bytes)
    BALZ v1.07, ex: 12,302,816 bytes


  12. #12
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 151 Times in 18 Posts

    Thumbs up

    Good work, Ilia! If you're keeping it up at this rate you're hitting the 12.000.000 on SFC sooner or later. I'm curious about results on Squeezechart/MFC/...

  13. #13
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts

    Wink

    Don't forget about my PIMPLE, TC, ... Here the catch in efficiency - i.e. nicely balanced:
    • compression ratio
    • compression speed
    • decompression speed
    • memory usage

  14. #14
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 151 Times in 18 Posts

    Thumbs up

    I meant that you're hitting 12.000.000 with BALZ (and with much better efficiency), of course. And of course, I do remember TC and PIMPLE.

  15. #15
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts

  16. #16
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts
    Thank you!

  17. #17
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts

    Talking

    Code:
    gimp-2.0.0.tar (78,745,600 bytes)
    
    BALZ 1.07, ex: 9,399,976 bytes
    CABARC, -m lzx:21: 10,906,957 bytes
    TOR 0.4, -12: 10,911,945 bytes
    PKZIP 2.50, -exx: 18,146,686 bytes
    Having said that only PKZIP was faster than BALZ...

  18. #18
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts
    www.maximumcompression.com updated!

    BALZ v1.07 showed nice results at both SFC and MFC!

  19. #19
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    874
    Thanks
    464
    Thanked 175 Times in 85 Posts

    Thumbs up Hi Ilia

    On my testsets BALZ 1.07 reached 150th place (BALZ 1.06 = 168th place)
    and it is 4,000 sec faster (vs the 15.000 sec of BALZ 1.06).

    I'll put the results online tomorrow but just wanted to let you know I did not forget you

  20. #20
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts
    Thank you!

  21. #21
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts

    Thumbs up

    Speed improvement is excellent!

  22. #22
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts

    Talking

    OK, tested BALZ for further improvements. Well, a larger ROLZ model gives no serious compression gain at the cost of 2X slower compression and slightly slower decompression. So, not worth it. Another improvement I tested is more interesting - I mirrored some ideas from QUAD - a higher order literal coder, the compression gain sometimes is truly amazing:

    butterfly4.bmp (18,048,054 bytes)
    BALZ v1.07, ex: 7,202,002 bytes
    BALZ v1.08, ex: 6,764,199 bytes

    Bliss.bmp (1,440,054 bytes)
    BALZ v1.07, ex: 649,147 bytes
    BALZ v1.08, ex: 605,810 bytes

    rafale.bmp (4,149,414 bytes)
    BALZ v1.07, ex: 1,009,411 bytes
    BALZ v1.08, ex: 1,002,393 bytes

    fp.log (20,617,071 bytes)
    BALZ v1.07, ex: 570,238 bytes
    BALZ v1.08, ex: 563,092 bytes

    world95.txt (2,988,578 bytes)
    BALZ v1.07, ex: 576,364 bytes
    BALZ v1.08, ex: 570,012 bytes

    ENWIK9 (1,000,000,000 bytes)
    BALZ v1.07, ex: 237,492,151 bytes
    BALZ v1.08, ex: 234,814,711 bytes

    pak0.pak (183,997,730 bytes)
    BALZ v1.07, ex: 86,324,729 bytes
    BALZ v1.08, ex: 85,123,570 bytes

    Note that the compression/decompression speed of both versions is nearly the same.
    Looking forward for your feedback...

  23. #23
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts
    However, the only downside of this approach is some compression loss on binary and small files, sometimes this loss is notable. At the same time larger ROLZ model in most cases does not hurt compression. Check out some additional results:

    Legend:
    BALZ v1.07 - BALZ as is
    BALZ v1.08a - BALZ with higher order literal coder
    BALZ v1.08b - BALZ with a larger ROLZ model (192 MB)

    ENWIK9 (1,000,000,000 bytes)
    BALZ v1.07: 237,492,151 bytes
    BALZ v1.08a: 234,814,711 bytes
    BALZ v1.08b: 232,315,251 bytes <-

    pak0.pak (183,997,730 bytes)
    BALZ v1.07: 86,324,729 bytes
    BALZ v1.08a: 85,123,570 bytes <-
    BALZ v1.08b: 86,215,548 bytes

    calgary.tar (3,152,896 bytes)
    BALZ v1.07: 863,463 bytes
    BALZ v1.08a: 866,779 bytes
    BALZ v1.08b: 860,597 bytes <-

    sfc.tar (53,144,064 bytes)
    BALZ v1.07: 12,317,318 bytes
    BALZ v1.08a: 12,330,883 bytes
    BALZ v1.08b: 12,297,705 bytes <-

    sfc.7z (53,135,131 bytes)
    BALZ v1.07: 12,302,816 bytes
    BALZ v1.08a: 12,307,801 bytes
    BALZ v1.08b: 12,285,990 bytes <-

    rafale.bmp (4,149,414 bytes)
    BALZ v1.07: 1,009,411 bytes
    BALZ v1.08a: 1,002,393 bytes <-
    BALZ v1.08b: 1,007,532 bytes

    fp.log (20,617,071 bytes)
    BALZ v1.07: 570,238 bytes
    BALZ v1.08a: 563,092 bytes <-
    BALZ v1.08b: 567,437 bytes

    world95.txt (2,988,578 bytes)
    BALZ v1.07: 576,364 bytes
    BALZ v1.08a: 570,012 bytes
    BALZ v1.08b: 569,264 bytes <-



    Can we spend some memory and processing time in favor higher compression? I think we can. I just need more compression at MFC - at least 69%... And at Squeeze Chart I need more compression as well, maybe not at 7-Zip class but at least as RAR-one...

  24. #24
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts

    Cool

    Playing with BALZ, invented some interesting modification - a low memory ROLZ, which needs only 20 MB or less for compression and even smaller amount for decompression. At the same time compression results in some cases even better compared to the current BALZ. However, compression is about a few times slower, but decompression is faster... Interesting indeed!

  25. #25
    Member Zonder's Avatar
    Join Date
    May 2008
    Location
    Home
    Posts
    55
    Thanks
    20
    Thanked 6 Times in 5 Posts
    I thought ratio will be worse than WinRar, but... here it is:

    Code:
    Testset1: ~4Gb (45653 files)
    Machine:  Core2 T5500, DDR2-666 2Gb
    
    35.700% //   547kb/s //  5512kb/s       // RZM v0.7e (stored-split 7z)
    37.541% //   919kb/s //  2115kb/s(no/o) // WinRK 3.0.3 Rolz3 Fastest
    39.407% //   993kb/s //  2563kb/s       // Sbc -m3 -b62
    40.423% //   670kb/s //  7171kb/s       // Balz v1.7 ex (stored 7z)
    41.060% //  1966kb/s // 15327kb/s(no/o) // WinRar v3.71 Best
    btw, decompression is not optimized, because "Process Time" is only 79% , so it can be increased by 20% easily for big files.
    Last edited by Zonder; 20th May 2008 at 00:19.

  26. #26
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts

    Smile

    Quote Originally Posted by encode View Post
    Playing with BALZ, invented some interesting modification - a low memory ROLZ, which needs only 20 MB or less for compression and even smaller amount for decompression. At the same time compression results in some cases even better compared to the current BALZ. However, compression is about a few times slower, but decompression is faster... Interesting indeed!
    It seems promising, but I don't like the idea of BALZ slowing down again.

  27. #27
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts
    Quote Originally Posted by Zonder View Post
    btw, decompression is not optimized, because "Process Time" is only 79% , so it can be increased by 20% easily for big files.
    Actually, the decompression is extremely optimized and may not be faster. From all my compressors BALZ is the most optimized so far. The catch in intense memory access...

    Quote Originally Posted by LovePimple
    It seems promising, but I don't like the idea of BALZ slowing down again.
    At least tested the idea.

    Well, I made some experiments with BALZ and my brand new order-1 Fast CM. Amazing results in both speed and compression. BALZ with a larger ROLZ model and this CM for literal and match index encoding rocks... It's heavy duty! Results will be posted later...

  28. #28
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts
    Quick testing results:

    sfc.7z: 12,137,241 bytes

    fp.log: 557,064 bytes
    world95.txt: 559,698 bytes
    rafale.bmp: 990,556 bytes


  29. #29
    Member Zonder's Avatar
    Join Date
    May 2008
    Location
    Home
    Posts
    55
    Thanks
    20
    Thanked 6 Times in 5 Posts
    Quote Originally Posted by encode View Post
    Actually, the decompression is extremely optimized ...
    i meant something with i/o because processor usage drops to 0% too often!?

  30. #30
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,979
    Thanks
    376
    Thanked 347 Times in 137 Posts
    Quote Originally Posted by Zonder View Post
    i meant something with i/o because processor usage drops to 0% too often!?
    I/O is OK... Like I said, random memory access is the key. Decoder randomly accesses to 64 MB (in upcoming version to 128 MB) + random buffer access...

Similar Threads

  1. BALZ v1.12 is here!
    By encode in forum Data Compression
    Replies: 23
    Last Post: 10th June 2008, 16:02
  2. BALZ v1.11 is here!
    By encode in forum Data Compression
    Replies: 16
    Last Post: 30th May 2008, 16:48
  3. BALZ v1.05 is here!
    By encode in forum Data Compression
    Replies: 6
    Last Post: 8th May 2008, 23:34
  4. balz v1.04 is here!
    By encode in forum Forum Archive
    Replies: 28
    Last Post: 1st May 2008, 22:41
  5. balz v1.03 is here!
    By encode in forum Forum Archive
    Replies: 43
    Last Post: 24th April 2008, 14:53

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •