Page 1 of 2 12 LastLast
Results 1 to 30 of 44

Thread: balz v1.03 is here!

  1. #1
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    OK, new version has been released. This version introduces new greedy encoder which is *FAST*. I hope you enjoy it. Also I hope that BALZ become a fast LZ77 coder. So, it's interesting to compare it with TORNADO, and others.

    http://encode.su/balz/index.htm


  2. #2
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Ilia!

  3. #3
    Member Vacon's Avatar
    Join Date
    May 2008
    Location
    Germany
    Posts
    523
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello everyone,

    see my comment here:
    URL

    Best regards!

  4. #4
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    INTEL Core 2 duo E6600

    SFC 13.436.454 B COMP.=16,797 sec. DEC.= 2,781 sec.

    MOC test 163.570.844 COMP.=111,434 sec. DEC.=25,429

  5. #5
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Nania Francesco Antonio
    Measuring compression ratio, don't forget about ex mode which is still exists (balz ex in out)!


    However, in your test you may use the most efficient one!

  6. #6
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    MOC Efficiency 411,79 [Normal Mode] (Very good for LZ77!)

  7. #7
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    how about sending it to the http://www.metacompressor.com/submit.aspx ?

  8. #8
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Quote Originally Posted by Bulat Ziganshin
    how about sending it to the http://www.metacompressor.com/submit.aspx ?
    Throws a error message!

  9. #9
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    what you mean? i'm successfully test 4x4 there

  10. #10
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Quote Originally Posted by Bulat Ziganshin
    what you mean? im successfully test 4x4 there
    It throws:
    You have not uploaded a zip file but application/zip!

    I tried many times with differently packed ZIP archives - nothing works!

  11. #11
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    are you use IE? this form does't work with Opera at least

  12. #12
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    at least, 1.02 was tested there by someone

  13. #13
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Quote Originally Posted by Bulat Ziganshin
    are you use IE?
    Firefox!

  14. #14
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    >timer balz.exe e dll100.dll balz
    User Time = 394.837 = 00:06:34.837 = 78%
    >timer balz.exe d balz nul
    User Time = 20.609 = 00:00:20.609 = 82%
    36,720,996 bytes


    >tor -5 dll100.dll -otor
    User Time = 25.797 = 00:00:25.797 = 78%
    >timer tor -d tor -o
    User Time = 5.708 = 00:00:05.708 = 82%
    32,634,030 bytes

    although my computer is by no way modern - 64k+64k cach size

  15. #15
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Quote Originally Posted by Bulat Ziganshin
    are you use IE?
    With IE it throws:

    Your code is wrong!

    What is CODE??

    Quote Originally Posted by Bulat Ziganshin
    dll100.dll
    Try others...

  16. #16
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    code is LongNightsDebugging

  17. #17
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    ex mode tested

    for "e" mode resulting total was 16 859 360, comp speed 836 kBps and decompression speed little bit smaller than for ex mode

  18. #18
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quick test...

    Test file: ENWIK8

    Test Machine: AMD Sempron 2400+, Windows XP SP2

    mode: ex

    Compression Time: 10814.922s

    Compressed Size: 30,604,477 bytes

    Decompression Time: 17.408s


    mode: e

    Compression Time: 1381.077s

    Compressed Size: 32,406,406 bytes

    Decompression Time: 18.962s

  19. #19
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    By the way, in future versions I may increase dictionary size to say 1..4 MB. Also, ex may represent a Lazy Parsing, not SS, which may be completely removed. In addition, I will reduce memory usage to ~70 MB, even if we deal with 4 MB and larger dictionaries.



    I think the future is in such *FAST* modes, since SS' compression speed is unacceptable. What do you think?

  20. #20
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by encode
    I think the future is in such *FAST* modes, since SS compression speed is unacceptable. What do you think?
    I agree!

  21. #21
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Furthermore, in most cases the difference in compression is really small, even if we compare to greedy (unoptimized) parsing, as currently BALZ make use. If we deal with Lazy Matching we even further close the gap, being just slightly slower.


  22. #22
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    What I've already done with BALZ v1.04:
    + Removed SS parsing
    + Reduced memory usage to ~80 MB
    + Changed dictionary size to 1 MB
    + Mode "e" uses greedy parsing
    + Mode "ex" uses lazy matching with 2-byte lookahead
    Having said that with a larger dictionary and a new, simpler parsing, new BALZ often outperforms an old one (current version), being incomparable faster and with smaller memory footprint. Very cool!

  23. #23
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    Ilya, what's your email??

  24. #24
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    ilia_muraviev # yahoo . com

    Hope that bots will not extract my box...

  25. #25
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Continue improving BALZ:
    + Changed dictionary size to 2 MB. Looks like 2 MB is some kind of standard value for modern LZ77 coders (CABARC,QUANTUM,etc.)
    + Tested more deeply parsing with 2-byte lookahead. In some cases such thing may slightly hurt compression, comapred to simple 1-byte lookahead lazy matching. But overall it helps, especially on text files.
    + Just stuck in a middle with formula - len/offset limits. i.e. which offset should be the max for each length. With 2 MB dictionary I've found that I should restrict offsets for 3,4,5-byte matches.
    3 - ~256
    4 - ~4k
    5 - ~512k
    Continue digging...
    P.S.
    This new beast, due to a larger dictionary, has a higher compression, in some cases notable higher, even with such simpler parsing scheme. At the same time it's faster...

  26. #26
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,503
    Thanks
    741
    Thanked 665 Times in 359 Posts
    Quote Originally Posted by encode
    Looks like 2 MB is some kind of standard value for modern LZ77 coders (CABARC,QUANTUM,etc.)
    yes, if you live in 90s rar/ace already had 4mb dicts

    Quote Originally Posted by encode
    This new beast, due to a larger dictionary, has a higher compression, in some cases notable higher, even with such simpler parsing scheme. At the same time its faster...
    you have a lot of room for improvement

  27. #27
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Quote Originally Posted by Bulat Ziganshin
    rar/ace already had 4mb dicts
    Actually, I can set ANY dictionary size. Larger dictionary = slower compression. Well, Ill test the BALZ with 4 MB dictionary. Maybe I should keep 4 MB, at least for "pht.psd" file.

  28. #28
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    OK, tested BALZ with various window sizes (1..4 MB). Well, maybe 4 MB is too heavy - Hash Chain based match finder is not so efficient on such large dictionaries. The main question is - what is BALZ - fast and efficient LZ77 or LZMA competitor. Well, even with 4 MB BALZ may not compete with LZMA - we need Optimal Parsing and Binary Tree based match finder. Therefore, I think I should keep something in middle between Deflate and LZMA - fast LZ77. Concluding, 1 MB dictionary is enough, although I will make additional tests with 2 MB one. In new BALZ I also improved parsing, new "ex" mode uses an advanced Lazy Matching with 2-byte lookahead, also during decision of dropping a match it looks for offset of a current match, is it closer/good match, also is current offset in a rep state, and so on. Another parameter I tested is a hash chain length – 4k, 8k, 16k; 16k is too large value and with a large dictionaries, say 4 MB, may heavily affect compression speed. 8k is very deep search, 4k is OK with 1 MB dictionary but not really enough on a larger ones. Anyway, you may post your own thought about what BALZ do you really want to see – i.e. fast or not, favor compression or speed, etc.


  29. #29
    Programmer
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    420
    Thanks
    28
    Thanked 153 Times in 18 Posts
    Quote Originally Posted by encode
    Anyway, you may post your own thought about what BALZ do you really want to see – i.e. fast or not, favor compression or speed, etc.
    Id like to see stronger or much faster compression. In case you go for stronger compression an increase in dictonary size would be great. But frankly speaking, Id love to see an improved version of TC combining your know-how from quad and lzpm. I first found this forum because I was looking for some info on TC.

  30. #30
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Yep, TC is one of the craziest things I've ever made. Another cool compressor is one closed source version of QUAD, which represents an order-2 fast CM+LZP layer. The performance of this compression is crazy! However, starting with QUAD idea I'm looking more carefully at asymmetric things like ROLZ and LZ77. Indeed, new BALZ in some cases has identical or greater compression than my old PIMPLE, being incomparable faster. In addition, my new LZ77 easily outperforms LZPM on binary files. But the coolest part of BALZ is its simplicity – I think it's one of the simplest compressors ever made – BALZ v1.04 has ~7 KB source code (encoder/decoder/interface, all stuff), at the same time, things like TC have high complexity – large sources, lots of classes, etc. - hard to work on such large projects. Anyway, things like fast CM and LZP is well known to me and to release a new compressor I may just Copy+Paste my own code. Just currently, I more interested in relatively new area to me – pure LZ77. OK, will look at MFC's results and will decide what stuff and tricks to insert to BALZ, maybe again I'll skip back to CM+LZP.

Page 1 of 2 12 LastLast

Similar Threads

  1. BALZ v1.12 is here!
    By encode in forum Data Compression
    Replies: 23
    Last Post: 10th June 2008, 17:02
  2. BALZ v1.11 is here!
    By encode in forum Data Compression
    Replies: 16
    Last Post: 30th May 2008, 17:48
  3. BALZ v1.05 is here!
    By encode in forum Data Compression
    Replies: 6
    Last Post: 9th May 2008, 00:34
  4. balz v1.05 is here!
    By encode in forum Forum Archive
    Replies: 1
    Last Post: 3rd May 2008, 01:34
  5. balz v1.04 is here!
    By encode in forum Forum Archive
    Replies: 28
    Last Post: 1st May 2008, 23:41

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •