Results 1 to 29 of 29

Thread: Test set: bitmap

  1. #1
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts

    Test set: bitmap

    Data: A bitmap with a photo of the earth, 8192x4096, 24BPP, http://visibleearth.nasa.gov/view_rec.php?id=2433

    100663350B

    Code:
    Archiver    Size          Time 
    XCOPY       100663350     0.328
    quick -1    40214520      0.703
    thor e1     34053360      0.797
    4x4  2      31224627      1.141
    4x4  3      30909051      1.203
    LZTurbo -41 26693056      1.578
    BZP         24705186      7.656
    4x4 1t      21936482      9.765
    4x4 3t      21918458     12.406
    rings 4     20435852     12.829
    rings 5     20214963     12.860
    rings 6     20097063     12.875
    rings 7     20043717     12.968
    rings 9     19917961     13.219
    PAQ9a 8     19701951    247.157
    NanoZip -cc 19528902    309.203
    No slug...no FreeArc...only 1 Nanozip mode at the top...strange.

    You can find Tor 0.5a next to 0.4 in this test.
    Attached Files Attached Files
    Last edited by m^2; 2nd January 2009 at 18:25.

  2. #2
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    No wonder, why some of programs are bad at this ranking. It's obvious that filtering or specialized codecs. But, some of bit-wise context mixing compressor have chance without filtering (I mean they might be better than bit-wise + without filters).

    I have just tested this sample with BIT 0.7 on my laptop (Core2Duo 2.2 GHz, 2 GB RAM, Vista Business x64).

    Compressed size: 20,148,127 bytes @ 827 KB (118.810 seconds)

    Timing is not measured with an external program. Just written what BIT reported.

    Note: AFAIK, Rings has BMP filter and BIT is a competitor without filtering in this test
    BIT Archiver homepage: www.osmanturan.com

  3. #3
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Yeah, I'll test BIT too. It just has lower priority because it's slow.

  4. #4
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Yep it's slow. But not slower than LPAQ9k and PAQ9a which have listed in your chart BTW, I have some idea for further improvements. You can expect more better compression and/or speed in next releases
    BIT Archiver homepage: www.osmanturan.com

  5. #5
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by osmanturan View Post
    Yep it's slow. But not slower than LPAQ9k and PAQ9a which have listed in your chart
    Yep. But they reached the top in some tests, which affects priority too.

    Quote Originally Posted by osmanturan View Post
    BTW, I have some idea for further improvements. You can expect more better compression and/or speed in next releases
    I'm looking forward to see it.

  6. #6
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts

    Wink

    What about PIM? Although PIM uses its image compression not for huge files (<= 16 MB). For huge images or for weird ones it uses not so advanced technique. Check for yourself...

  7. #7
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by encode View Post
    What about PIM? Although PIM uses its image compression not for huge files (<= 16 MB). For huge images or for weird ones it uses not so advanced technique. Check for yourself...
    I downloaded the 1st 2.8 version, tried to find some command line reference, failed to do so, concluded that it's useless and deleted it.

    I can check the size, but won't put it in the ranking w/out CLI.
    Add CLI and I'll make it a regular contender.

    ADDED: BTW, when choosing to use a large bitmap I was afraid that some programs might not treat it the best they can, but taring several smaller ones is more likely to break some optimizations in single file compressors.
    I've been thinking about doing a similar test, just with icons instead of bmps to make things slightly harder.
    I think I'll do it after I finished the 3 GB of tests that I have pending and updated 7zip / Tornado in the already released ones.
    Last edited by m^2; 18th December 2008 at 13:32.

  8. #8
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    PIM 2.82 beta:
    PPMd: 20140703
    BZIP2: 21478527
    BALZ: 25130647

    BIT 0.7:
    3/4/5: 20308313...different from yours. EDIT: While doing the quick test I've had to little free memory and BIT used mode 3 in all tests.

    BTW how not to hate Core 2 owners with 15% lower CPU clock and double performance?
    Last edited by m^2; 18th December 2008 at 15:41.

  9. #9
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by encode View Post
    What about PIM? Although PIM uses its image compression not for huge files (<= 16 MB). For huge images or for weird ones it uses not so advanced technique. Check for yourself...
    Possibly, most of compressors which have filters or specialized codecs failed to handle such that huge bitmap. AFAIK, NanoZip has a specialized codec for both image and audio and also it's detection based on file signatures too (it also try to classify data on-the-fly for unknown data). So, that can be answer why it's top ranked. Also, note that CCMX has been beaten by BIT in this test while CCMX has good filters. Probably, it's filters couldn't handle this image too.
    BIT Archiver homepage: www.osmanturan.com

  10. #10
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by m^2 View Post
    BIT 0.7:
    3/4/5: 20308313...different from yours.
    Let me tell what I did. I have downloaded the image with given link and converted it to BMP file with ACDSee by choosing "not to preserve metadata". The produced file size exactly equal yours (100663350 bytes). I also renamed the file "earth8k.bmp" and put it in the same directory with "bit.exe" (putting same directory will remove extra file path metadata in BIT archive). And last, I compressed it with following commandline:

    bit a -p=5 earth8k.bmp

    I recompressed it again and got the same file size: 20,148,127 bytes

    Quote Originally Posted by m^2
    BTW how not to hate Core 2 owners with 15% lower CPU clock and double performance?
    No wonder at least for BIT Because, my development machine is same. So, I tweak it Core2Duo already. Note that, Core2Duo has a really good architecture to handle my computations (multiplication, caching etc). BTW, did I mention that BIT 0.7 reached ~1200 KB/s on some part of this file?
    BIT Archiver homepage: www.osmanturan.com

  11. #11
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    MD5: bb7a828355cfcdac5cf18d877ce75165
    If your's different, I'll upload my bitmap somewhere.

    I used TotalRSZ Total Commander plugin.
    And renamed it to bmp.tar
    Archivers that can compress whole directories received b.bmp in a bmp directory.

    EDIT: corrected checksum format.
    Last edited by m^2; 18th December 2008 at 14:36.

  12. #12
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by m^2 View Post
    CRC32: bb7a828355cfcdac5cf18d877ce75165
    How did you produced this CRC32?
    BIT Archiver homepage: www.osmanturan.com

  13. #13
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by osmanturan View Post
    How did you produced this CRC32?
    Sorry, it was MD5.

  14. #14
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by m^2 View Post
    Sorry, it was MD5.
    Don't mind
    BTW, the hash is different from yours for "earth8k.bmp":
    Code:
    # MD5 checksums generated by MD5summer (http://www.md5summer.org)
    # Generated 18.12.2008 13:36:29
    
    67b2911e7c462f280aed1439d05b2989 *earth8k.bmp
    a64c3b4c2c31482c06c12bb821376381 *earth8k.bit
    I have uploaded compressed file:
    http://www.osmanturan.com/earth8k.zip
    You can extract it with following command:
    bit e earth8k.bit

    P.S.: I had to put it in the ZIP file. Because, it seems my hosting do not allow to download unusual file extension such as bit, 7zip etc.
    BIT Archiver homepage: www.osmanturan.com

  15. #15
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    My bitmap.

    Only 4 bytes in the header are different.

    Now as I manually checked your command line I found a warning that BIT automatically reduced amount of memory. Good. I'll probably get a result very close to yours when I'll be doing tests on a clean machine.

    BTW next time you send a file to sb. with 1 GB RAM better choose a mode that doesn't require over 660 MB for extraction...

  16. #16
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by m^2
    Only 4 bytes in the header are different.
    So, this is good then.

    Quote Originally Posted by m^2
    Now as I manually checked your command line I found a warning that BIT automatically reduced amount of memory.
    I wonder, why some people tend to not read any information which a compressor displayed

    Quote Originally Posted by m^2
    Good. I'll probably get a result very close to yours when I'll be doing tests on a clean machine.
    Ok!

    Quote Originally Posted by m^2
    BTW next time you send a file to sb. with 1 GB RAM better choose a mode that doesn't require over 660 MB for extraction...
    Err...sorry. I thought, if you could test p=5 mode, that means you have already enough free memory to extract it. Sorry for that

    BTW, I'll download and test your bitmap right now.
    BIT Archiver homepage: www.osmanturan.com

  17. #17
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by osmanturan View Post
    I wonder, why some people tend to not read any information which a compressor displayed
    It ended up deep in debug logs that got deleted as soon as I started another test.

    Quote Originally Posted by osmanturan View Post
    Err...sorry. I thought, if you could test p=5 mode, that means you have already enough free memory to extract it. Sorry for that
    OK, it's not that big problem.

  18. #18
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    @m^2:
    I compressed the sample which you provided. The compressed file size is same (20,148,127 bytes). So, that means if you can free your memory up to 650 MB, BIT 0.7 can squeeze it down at that level.

    BTW, it seems I have to add "-forcemem" option which disables automatic memory usage reduction in BIT 0.8
    BIT Archiver homepage: www.osmanturan.com

  19. #19
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by osmanturan View Post
    @m^2:
    I compressed the sample which you provided. The compressed file size is same (20,148,127 bytes). So, that means if you can free your memory up to 650 MB, BIT 0.7 can squeeze it down at that level.

    BTW, it seems I have to add "-forcemem" option which disables automatic memory usage reduction in BIT 0.8
    I don't think it's useful. If I used it, I would surely notice that something's wrong and just kill BIT.
    But I won't add it to my testing scripts because in case that with some release you increased the ma memreqs I could end up like I did with Tornado twice already - with machine thrashing hdd for a whole weekend.
    If I compress w/out scripts, I notice warnings, so it's unnecessary.

    -quitifnomem could have some use though, but I don't think it's worth your time. Next time I'll remember how is it.

  20. #20
    Member
    Join Date
    May 2008
    Location
    Earth
    Posts
    115
    Thanks
    0
    Thanked 0 Times in 0 Posts
    m^2, can you test arc like at metacompressor.com:
    Code:
    arc a -m=mm:3*8+grzip:l

  21. #21
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by IsName View Post
    m^2, can you test arc like at metacompressor.com:
    Code:
    arc a -m=mm:3*8+grzip:l
    OK. I won't see my computer until January, so it has to wait though.

  22. #22
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by IsName View Post
    m^2, can you test arc like at metacompressor.com:
    Code:
    arc a -m=mm:3*8+grzip:l
    Is 3*8 a generic switch that should work well with all bitmaps or tuning for the metacompressor one?

    I made quick tests with this parameter and w/out any...works very bad.

    Also, I'd like to include some specialized image compressors to see how png and others perform. Can anyone suggest me a good command line image converter? Preferably with wide support for lossless formats and switches that affect compression speed.

  23. #23
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    I found a bug in my testing script that resulted in greatly increased FreeArc compression times.
    It seems that only this data set was affected, but I'll check it.

    EDIT: Size is affected too because FreeArc took 2 copies of the same file and with my early mm+grzip tests it meant doubling output size. Now it's fine.

    EDIT2: All published results are correct. The problem was only with my quick tests.
    Last edited by m^2; 2nd January 2009 at 18:27.

  24. #24
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,507
    Thanks
    742
    Thanked 665 Times in 359 Posts
    >Is 3*8 a generic switch that should work well with all bitmaps or tuning for the metacompressor one?

    3*8 means "3 colors, each 8 bit". you can use just "mm" for auto-detection, or specify exact type of your bitmap, such as 1*16, 3*32 and so on..

  25. #25
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    OK. So 3*8 is good here too. And works better than automatic detection.
    I'll test both settings.

    Thanks for clarification.

  26. #26
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    958
    Thanks
    570
    Thanked 396 Times in 294 Posts

    Post Some unpractical scores of Paq8k and Paq8p

    Hi!
    I've made some tests on earth8k file using paq8k, paq8p, RKim106, and WinRar 3.50. Of course compression times are noncomparable to your times becouse my computer have diffrerent parameters.
    But compression sizes are proper. Interesting (from my point of view) is especially paq8p -7 score, which provides to compression ratio =0.166 and is the best for this benchmark at this moment. Efficiency score for this test (=494) is of course worse than Nanozip (score=309). There are the scores:

    Archiver Size Time KB/s Efficiency(maximumcompression.com)
    Paq8k -8 18350188 1045.860 94.0 688.3
    Paq8k -7 18349400 1070.500 91.8 704.3
    Paq8k -6 18348197 1043.060 94.2 686.0
    Paq8k -5 18347237 1047.300 93.9 688.5
    Paq8k -4 18364190 1055.090 93.2 697.8
    Rkim106 -cx 20546295 61.172 1607.0 87.8
    WinRar 3.50 22538044 15.783 6228.5 45.9
    Paq8p -7 16721137 1338.550 73.4 494.1
    Paq8p -6 16748096 1324.910 74.2 493.8

    Regards, Darek

  27. #27
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by Darek View Post
    Hi!
    I've made some tests on earth8k file using paq8k, paq8p, RKim106, and WinRar 3.50. Of course compression times are noncomparable to your times becouse my computer have diffrerent parameters.
    But compression sizes are proper. Interesting (from my point of view) is especially paq8p -7 score, which provides to compression ratio =0.166 and is the best for this benchmark at this moment. Efficiency score for this test (=494) is of course worse than Nanozip (score=309). There are the scores:

    Archiver Size Time KB/s Efficiency(maximumcompression.com)
    Paq8k -8 18350188 1045.860 94.0 688.3
    Paq8k -7 18349400 1070.500 91.8 704.3
    Paq8k -6 18348197 1043.060 94.2 686.0
    Paq8k -5 18347237 1047.300 93.9 688.5
    Paq8k -4 18364190 1055.090 93.2 697.8
    Rkim106 -cx 20546295 61.172 1607.0 87.8
    WinRar 3.50 22538044 15.783 6228.5 45.9
    Paq8p -7 16721137 1338.550 73.4 494.1
    Paq8p -6 16748096 1324.910 74.2 493.8

    Regards, Darek
    Thanks.
    I'd like to test it on my PC to have correct times.
    However I don't have rar license, so it's out.
    PAQ8 - just no. It's too slow, I can find better use for my CPU power.
    So I'll test only RKIM. And it's incompatible with my testing script. I can fix it easily, but this is quite a lot of work, I'll finish the huge batch of other tests that I started first.

    From a quick test of RKIM I can tell you that it's gonna be ~90s, so you can just add 50% to all your times.

    Regards, Maciek.

  28. #28
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    958
    Thanks
    570
    Thanked 396 Times in 294 Posts
    Thanks.

    I've used rkim archiver for comparison because in some my testbed BMP files rkim1.06 have still best scores (highest compression ratios) until now. Unfortunatelly it's not a earth8k case.

    Paq8 - it's my fafourite compressor due to compression ratios (especially paq8k version). Time isn't an important factor in my testset, but if you are looking for practical compressor, paq8k won't be usuable at all .

    Regards, Darek

  29. #29
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Update. A week later than promised, but slightly bigger.
    New tests:
    BIT
    Flashzip (enters "the best" list)
    LPAQ8
    PAQ9a (modes 1-7)
    UHARC (enters "the best" list)
    SBC
    PPMX
    MIX
    LZPM
    FreeArc mm+* (enters "the best" list)
    m1 0.3
    Blizzard 140000000 (enters "the best" list)
    Slug X


    Code:
    XCOPY                      100663350    0.328
    quick -1                    40214520    0.703
    thor e1                     34053360    0.797
    4x4  2                      31224627    1.141
    4x4  3                      30909051    1.203
    LZTurbo -41                 26693056    1.578
    FreeArc -mmm:3*8+1xb        24006543    3.172
    FreeArc -mmm:3*8+2xb        23873047    8.859
    Flashzip m1 s1 b5           23708773    8.937
    4x4 1t                      21936482    9.765
    FreeArc -mmm:3*8+grzip:m4   21011576   10.140
    FreeArc -mmm:3*8+grzip:m4:l 20937361   10.219
    rings 4                     20435852   12.829
    rings 5                     20214963   12.860
    rings 6                     20097063   12.875
    rings 7                     20043717   12.968
    rings 9                     19917961   13.219
    bliz c 140000000            19867057   26.625
    UHARC m1 md32768            19085954  132.500
    UHARC m2 md32768            19066402  144.172
    UHARC m3 md32768            19013699  163.891
    UHARC mx md32768            18840805  186.078
    Attached Files Attached Files
    Last edited by m^2; 13th January 2009 at 17:48.

Similar Threads

  1. Test set: bookstar
    By m^2 in forum Data Compression
    Replies: 5
    Last Post: 11th February 2009, 17:49
  2. Test set: installer
    By m^2 in forum Data Compression
    Replies: 10
    Last Post: 11th February 2009, 14:47
  3. Test set: music
    By m^2 in forum Data Compression
    Replies: 0
    Last Post: 10th December 2008, 00:25
  4. Test set: backup
    By m^2 in forum Data Compression
    Replies: 1
    Last Post: 23rd October 2008, 23:16
  5. A lil bitmap test...
    By jethro in forum Forum Archive
    Replies: 6
    Last Post: 25th June 2007, 00:25

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •