Page 2 of 3 FirstFirst 123 LastLast
Results 31 to 60 of 61

Thread: BALZ - An Open-Source ROLZ-based compressor

  1. #31
    Programmer toffer's Avatar
    Join Date
    May 2008
    Location
    Erfurt, Germany
    Posts
    587
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Nice to have some new open source compressor. The source is tiny, only 11.8 kB BTW very readable.

    One remark:

    I thought you use a sliding window? Blockwise processing like this doesn't make it faster (you always clear your offset table) and looses compression.

  2. #32
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,965
    Thanks
    367
    Thanked 343 Times in 134 Posts
    BTW, I want to add BALZ-file decompression to PIM archiver...

  3. #33
    Member
    Join Date
    May 2008
    Location
    Antwerp , country:Belgium , W.Europe
    Posts
    487
    Thanks
    1
    Thanked 3 Times in 3 Posts
    Quote Originally Posted by encode View Post
    BTW, I want to add BALZ-file decompression to PIM archiver...
    Can the current PIM work in CLI mode ? (for scripts, .bat's)
    No matter what switches I used, I always get the GUI firing up...
    Are there any "hidden" options to do this ?
    It might be interesting to add CLI support though.

  4. #34
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,965
    Thanks
    367
    Thanked 343 Times in 134 Posts
    Quote Originally Posted by pat357 View Post
    Can the current PIM work in CLI mode ? (for scripts, .bat's)
    No matter what switches I used, I always get the GUI firing up...
    Are there any "hidden" options to do this ?
    It might be interesting to add CLI support though.
    PIM, IIRC, has no command-line switches. PIMPLE - has. With PIM I did main accent on WinRAR/WinZip-like interface - very few freeware archivers have such familiar interface...

  5. #35
    Programmer
    Join Date
    Jul 2008
    Location
    Finland
    Posts
    102
    Thanks
    0
    Thanked 1 Time in 1 Post
    Quote Originally Posted by encode View Post
    Hooray! I did it!

    Welcome people:
    http://balz.sourceforge.net/

    Nice and tidy as is all your open source. I glanced the source and added a very quick 1min hack to to replace the inefficient table update.

    balz_ref e acrord32.exe ref -> 3.7sec
    balz_hack e acrord32.exe hack -> 3.1sec

    Less than I expected, but no time right now for more.
    Attached Files Attached Files
    Last edited by Sami; 8th July 2008 at 19:05.

  6. #36
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,965
    Thanks
    367
    Thanked 343 Times in 134 Posts
    Quote Originally Posted by Sami View Post
    Nice and tidy as is all your open source. I glanced the source and added a very quick 1min hack to to replace the inefficient table update.

    balz_ref e acrord32.exe ref -> 3.7sec
    balz_hack e acrord32.exe hack -> 3.1sec

    Less than I expected, but no time right now for more.
    Well, don't be so sure. I tried many variants, some of them, like yours, is really might be more efficient with "e" mode. However, all stuff is tuned for "ex" mode and with other approaches we may seriously loose too much compression speed, believe me. All this stuff developed by years, years of testing and tuning... Anyway, I'm happy to see some user attempts and suggestions. Thank you!

  7. #37
    Programmer
    Join Date
    Jul 2008
    Location
    Finland
    Posts
    102
    Thanks
    0
    Thanked 1 Time in 1 Post
    Quote Originally Posted by encode View Post
    Well, don't be so sure. I tried many variants, ...
    So you cannot reproduce the results? Did you check my attachment?

  8. #38
    Programmer
    Join Date
    Jul 2008
    Location
    Finland
    Posts
    102
    Thanks
    0
    Thanked 1 Time in 1 Post
    Quote Originally Posted by encode View Post
    Well, don't be so sure. I tried many variants, some of them, like yours, is really might be more efficient with "e" mode. However, all stuff is tuned for "ex" mode and with other approaches we may seriously loose too much compression speed, believe me.
    I tested ex mode with acrord32.exe gets 5.5sec and the hacked version is 5.0sec.

  9. #39
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,965
    Thanks
    367
    Thanked 343 Times in 134 Posts
    Quote Originally Posted by Sami View Post
    So you cannot reproduce the results? Did you check my attachment?
    Check this out:
    Code:
    TraktorDJStudio3.exe (29,124,024 bytes)
    
    balz_ref e - 5.2 sec
    balz_hack e - 5.0 sec
    balz e - 4.7 sec
    
    balz_ref ex - 16.7 sec
    balz_hack ex - 16.9 sec
    balz ex - 12.5 sec
    The end of discussion...

  10. #40
    Programmer
    Join Date
    Jul 2008
    Location
    Finland
    Posts
    102
    Thanks
    0
    Thanked 1 Time in 1 Post
    Quote Originally Posted by encode View Post
    Check this out:
    The end of discussion...
    The ref is a reference, compiled with same settings. Try compiling the hack source with the same compiler as your balz.

    --edit--
    TraktorDJStudio3.exe (ex) ref=39.3s hack=38.2sec
    Last edited by Sami; 8th July 2008 at 19:26.

  11. #41
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,965
    Thanks
    367
    Thanked 343 Times in 134 Posts
    Quote Originally Posted by Sami View Post
    The ref is a reference, compiled with same settings. Try compiling the hack source with the same compiler as your balz.
    Timings for balz_hack compiled with the same compiler as original:
    Code:
    The same TraktorDJStudio3.exe...
    balz_hack e - 5.0 sec
    balz_hack ex - 15.4 sec

  12. #42
    Programmer
    Join Date
    Jul 2008
    Location
    Finland
    Posts
    102
    Thanks
    0
    Thanked 1 Time in 1 Post
    Quote Originally Posted by encode View Post
    Code:
    The same TraktorDJStudio3.exe...
    balz_hack e - 5.0 sec
    balz_hack ex - 15.4 sec
    Ok, so we got an improvment.

    --edit---
    no we don't, I read you incorrectly. Now I'm puzzled why the results are like that.
    --edit--

    TPPM coder looks suspicious to me speedwise and ratiowise, I think whoever continues now should replace it.

  13. #43
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,965
    Thanks
    367
    Thanked 343 Times in 134 Posts
    Quote Originally Posted by Sami View Post
    Ok, so we got an improvment.

    --edit---
    no we don't, I read you incorrectly. Now I'm puzzled why the results are like that.
    --edit--

    TPPM coder looks suspicious to me speedwise and ratiowise, I think whoever continues now should replace it.
    Don't forget about that we should work in conjunction with the compiler...

  14. #44
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,965
    Thanks
    367
    Thanked 343 Times in 134 Posts
    Forget about BALZ - it's not so easy as might be at the first sight, better continue working on your marvelous NanoZip!

  15. #45
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,965
    Thanks
    367
    Thanked 343 Times in 134 Posts
    BTW, can somebody test the IA64 compile of BALZ? I'm very curious about it's timings...

  16. #46
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    I've looked at the source in a short time. It's very readable. I just surprised when I see LZMA fast mode like match-length/index based optimal parsing (of course it's more simpler) in this tiny source. Is there a chance for adding bit optimal parsing for exx mode? I wonder the results.

    Also, LZH like 9 bits symbols are ok. But, using flags is more stronger (I have tested in my compressor: BIT). Also, note that I have collected some statistics on well used test files. Most of files have an interesting statistics about match lengths. <8 and <16 most probable cases. So, I divided the match length into three parts: [2,8], (8-16], (16, MaxMatchLength]. In my experiments, I also confirm that order-2 match context for ROLZ is well-suited for fast compressing modes (it's a balancing point for efficiency: fast and strong compression for me).

    Overall, good job Ilia!
    Last edited by osmanturan; 9th July 2008 at 11:57.

  17. #47
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,965
    Thanks
    367
    Thanked 343 Times in 134 Posts
    Quote Originally Posted by osmanturan View Post
    Is there a chance for adding bit optimal parsing for exx mode?
    It is still possible, though I have no time currently to do it myself. Someday, somehow...

  18. #48
    Member
    Join Date
    May 2008
    Location
    CN
    Posts
    45
    Thanks
    0
    Thanked 0 Times in 0 Posts
    thx for LovePimple's work
    I am in China where can access http://balz.sourceforge.net/ but not
    http://sourceforge.net/

  19. #49
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Do you mean that you found Ilia's BALZ project because of the mirror at my web site?

  20. #50
    Member
    Join Date
    May 2008
    Location
    CN
    Posts
    45
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by LovePimple View Post
    Do you mean that you found Ilia's BALZ project because of the mirror at my web site?
    thank you and mike supply the download address

  21. #51
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts

    Thumbs up

    Here's a few of my own speed optimised compiles of BALZ v1.15. A Pentium 3 or later processor is required.

    ENJOY!
    Attached Files Attached Files

  22. #52
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by encode View Post
    BTW, can somebody test the IA64 compile of BALZ? I'm very curious about it's timings...
    I nearly forgot this. Here is the test result with ENWIK8 on my laptop (Core2 Duo 2.2GHz, 2 GB RAM, Vista Business x64 SP1). I run the test for two times for ensuring the results.

    BALZ
    Compression:
    54.869 seconds
    51.856 seconds

    Decompression:
    11.319 seconds
    11.506 seconds

    BALZ64
    Compression:
    70.912 Seconds
    70.815 Seconds

    Decompression:
    10.931 seconds
    10.316 seconds
    Suprisingly, 64-bit version much worser in speed term on compression. My experiments was opposite on ROLZ (I had 20% speed gain on compression/decompression by using x64 compile rather than x86). Note that, I had used data alignment on dictionary and ROLZ offset table which increases cache efficiency. I wonder why speed was dropped on compression.

  23. #53
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,965
    Thanks
    367
    Thanked 343 Times in 134 Posts
    Quote Originally Posted by osmanturan View Post
    Suprisingly, 64-bit version much worser in speed term on compression. My experiments was opposite on ROLZ (I had 20% speed gain on compression/decompression by using x64 compile rather than x86). Note that, I had used data alignment on dictionary and ROLZ offset table which increases cache efficiency. I wonder why speed was dropped on compression.
    32-bit version is more optimized. I did many tests, tunings, etc. 64-bit version was just blindly compiled...

  24. #54
    Member
    Join Date
    Jul 2008
    Posts
    54
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Mini test, balz vs freearc 0.50a
    Source : xampp-linux-1.6.7 from http://sourceforge.net/project/showf...ease_id=611649
    Used only GNU/linux versions
    Original File size:
    61209332 bytes (gzip compression)
    Ungziped:
    207175680 bytes

    I ungziped and recompressed the tar..

    Balz:
    35679161 bytes (using -ex) (35mb's)

    Freearc 0.50a
    28629187 bytes (using -mx only) (28mb's)

  25. #55
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,965
    Thanks
    367
    Thanked 343 Times in 134 Posts
    Compression time?

  26. #56
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 660 Times in 354 Posts
    freearc = lzma/ppmd + preprocessors

    are you really expected that balz or anything else will easily outperform two the best compression algorithms ATM?

    (i don't even say about preprocessors effect)

  27. #57
    Member
    Join Date
    Jul 2008
    Posts
    54
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Cpu it's 2.6Ghz HT with 512 Mb's of Ram
    Kernel linux used 2.6.26

    Packing times:
    Balz:
    --
    real 3m55.129s
    user 3m40.806s
    sys 0m2.468s


    Freearc 0.50a
    ---
    real 4m42.682s
    user 6m21.600s
    sys 0m6.700s


    Unpacking times:
    Balz:
    real 0m33.908s
    user 0m30.270s
    sys 0m2.028s

    Freearc:
    real 0m15.976s
    user 0m8.525s
    sys 0m2.152s
    Last edited by John; 30th July 2008 at 22:18.

  28. #58
    Member
    Join Date
    Jun 2008
    Location
    USA
    Posts
    111
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by encode View Post
    BTW, can somebody test the IA64 compile of BALZ? I'm very curious about it's timings...
    Did you mean Itanium (VLIW) or AMD64/EM64T (long mode)? Because the latter is ubiquitous but the former is probably rare for home users!

  29. #59
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,965
    Thanks
    367
    Thanked 343 Times in 134 Posts
    Decided to update the BALZ project:

    Uploaded optimized binaries for "Classic" BALZ v1.15 - From year 2008

    Released "Optimized" BALZ v1.20 - fully compatible with the v1.15, but notable faster. Also, Normal compression mode now is much faster at the cost of compression ratio. Plus, now it has a proper command-line interface, at last!

    http://sourceforge.net/projects/balz/

    Enjoy!

  30. The Following 7 Users Say Thank You to encode For This Useful Post:

    Fu Siyuan (6th March 2015),jibz (6th March 2015),Matt Mahoney (6th March 2015),Mike (6th March 2015),moisesmcardona (6th January 2019),Nania Francesco (6th March 2015),WayneD (6th March 2015)

  31. #60
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,965
    Thanks
    367
    Thanked 343 Times in 134 Posts
    Code:
    C:\Test>balz c enwik8 enwik8.z
    Compressing enwik8...
    100000000 -> 30056097 in 6.047s
    
    C:\Test>balz cx enwik8 enwik8.z
    Compressing enwik8...
    100000000 -> 28232824 in 19.345s
    
    C:\Test>balz d enwik8.z e8
    Decompressing enwik8.z...
    28232824 -> 100000000 in 2.562s
    
    C:\Test>balz c enwik9 enwik9.z
    Compressing enwik9...
    1000000000 -> 261416611 in 52.988s
    
    C:\Test>balz cx enwik9 enwik9.z
    Compressing enwik9...
    1000000000 -> 245218274 in 193.654s
    
    C:\Test>balz d enwik9.z e9
    Decompressing enwik9.z...
    245218274 -> 1000000000 in 22.407s

Page 2 of 3 FirstFirst 123 LastLast

Similar Threads

  1. Replies: 23
    Last Post: 24th March 2018, 17:57
  2. PeaZip - open source archiver
    By squxe in forum Data Compression
    Replies: 1
    Last Post: 3rd December 2009, 21:01
  3. fcm1 - open source order-1 cm encoder
    By encode in forum Data Compression
    Replies: 34
    Last Post: 4th June 2008, 23:16
  4. New fast open-source paq-based jpeg compressor
    By Bulat Ziganshin in forum Forum Archive
    Replies: 14
    Last Post: 13th September 2007, 13:57
  5. Replies: 0
    Last Post: 26th July 2007, 18:47

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •