Page 1 of 2 12 LastLast
Results 1 to 30 of 44

Thread: BIM (a new lossless image compressor) is here!

  1. #1
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts

    Cool BIM (a new lossless image compressor) is here!

    http://encode.narod.ru/

    Please welcome the very first version of my new lossless image compressor!

    Also note that this version is kind of a "core" or "bare-bone" of my new image codec - BIM can compress faster with a slightly worse compression, also BIM can compress notable better at the cost of processing speed.

    Anyway, please enjoy new release!




    UPDATE:
    BIM v0.02 supports 24-bit BMP, TGA and PNM/PPM images and has a higher compression

  2. #2
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,423
    Thanks
    223
    Thanked 1,052 Times in 565 Posts
    Code:
    CRW_3473                 CRW_3475                 CRW_3480
    9843646  1.578s  1.734s  9259860  1.531s  1.687s  10806540  1.672s  1.828s // bim 01
    9633421  0.297s  0.328s  8942837  0.281s  0.312s  10629911  0.297s  0.328s // qlic v1.demo
    9606709  5.453s  0.938s  8810824  5.187s  0.937s  10642841  5.313s  0.969s // bcif 1.0 beta
    9262732  2.344s  1.532s  8514152  2.328s  1.500s  10127448  2.437s  1.594s // bmf 2.01
    9258959  1.844s  1.953s  8450343  1.813s  1.937s   9908162  1.938s  2.047s // flic v1.4.demo
    9198483  1.438s  1.609s  8586752  1.406s  1.594s  10072920  1.437s  1.640s // flic v2.1.demo
    8682920 37.281s 32.688s  7899272 33.234s 29.203s   9490224 36.172s 32.437s // bmf 2.01 -S 
    8681236 91.593s 33.781s  7898968 84.766s 29.219s   9490224 96.031s 31.547s // bmf 2.01 -S -Q9
    images are 2840x2128x24bit files converted from canon raws.

  3. #3
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    237
    Thanks
    39
    Thanked 92 Times in 48 Posts
    1 204 847 739 bytes on LPCB images, all decompressed files were bit-exact.

    Why flic v1.4.demo instead of 2.1.demo? There is no doubt the latter must be used.

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  4. #4
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Thanks for testing! I hope you'll add it to the LPCB page too.

    2Shelwien
    Thanks for good examples of "bad" images for BIM. I will keep these for testing!

  5. #5
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,423
    Thanks
    223
    Thanked 1,052 Times in 565 Posts
    added flic 2.1 - 1.4 was the first link in http://www.imagecompression.info/gralic/
    Looks like it was better though.

  6. #6
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Currently working on BIM v0.02. This new version will feature BMP image support, support for PPM images with comments, it will be slightly faster and 100% compatible with current version.
    As an example, please download this packed BMP file and unpack it with the current BIM v0.01! BIM image format is extremely flexible!
    Attached Files Attached Files

  7. #7
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Well, new version will have a higher compression at higher speed - i.e. with same complexity + some optimizations. And it will be not compatible with the current one!

  8. #8
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Some testing results:
    Code:
    cow.ppm
    bim001: 18361655->7207617
    bim002: 18361655->6956958
    
    canon24.ppm
    bim001: 53747729->22567517
    bim002: 53747729->21844975
    
    DSC_8341.ppm
    bim001: 36636689->15934249
    bim002: 36636689->14886643
    
    CRW_3480.ppm
    bim001: 18130577->10806497
    bim002: 18130577->10426843
    
    But, in some cases:
    sony24.ppm
    bim001: 48098321->19324061
    bim002: 48098321->19689776
    The most important thing to note is that the new BIM v0.02 IS FASTER than BIM v0.01! Overall it has notable better compression (Except sony24.ppm and some weird artificial files)

  9. #9
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Added TGA image support. Please note that it's better to test BIM with PPM/PNM or TGA files, instead of BMPs. BMP files contain extra bytes for row alignment, latter may contain noise. Since BIM is TRULY lossless (all header info and comments are kept untouched) it will compress that noise/redundant bytes as well. With a fake PPM header, as example, you may compress and decompress any file with no loss, even a text file!

    Code:
    C:\bim\Release>bim c cow.ppm cow.bim
    Compressing cow.ppm: 18361655->6956962 in 0.624s
    
    C:\bim\Release>bim c cow.tga cow.bim
    Compressing cow.tga: 18361656->6956572 in 0.624s
    
    C:\bim\Release>bim c cow.bmp cow.bim
    Compressing cow.bmp: 18367770->7236460 in 0.624s

  10. #10
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 779 Times in 486 Posts
    That doesn't look right. A normal BMP header is only 54 bytes, and end of row padding should be predictable.

  11. #11
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    The file shown is the worst case scenario. Looks like these padding bytes add noise.

  12. #12
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    An opposite example:
    Code:
    C:\bim\Release>bim c PIA12203.ppm PIA12203.bim
    Compressing PIA12203.ppm: 353296335->95418720 in 10.873s
    
    C:\bim\Release>bim c PIA12203.bmp PIA12203.bim
    Compressing PIA12203.bmp: 353311650->95378096 in 10.966s

  13. #13
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    237
    Thanks
    39
    Thanked 92 Times in 48 Posts
    Code:
    C:\bim\Release>bim c cow.ppm cow.bim
    Compressing cow.ppm: 18361655->6956962 in 0.624s
    ...
    C:\bim\Release>bim c cow.bmp cow.bim
    Compressing cow.bmp: 18367770->7236460 in 0.624s
    in PPM files the lines are in top-to-bottom order,
    in BMP they are bottom-to-top,
    this may be the reason.

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  14. #14
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    BIM v0.02 has been released!

    http://encode.narod.ru/

    Please enjoy new release!



    Just a few notes. As I said, new version is slightly faster, supports 24-bit BMP, TGA and PNM/PPM images. It also has a slightly more complex internals than BIM v0.01, but I'm aiming on speed, so I'll continue to keep it as simple as possible. As example, changing CM Counters to more advanced ones will notable improve compression at the cost of 1.5-2X slowdown - it's not the case for a fast image compressor.

  15. #15
    Member Bloax's Avatar
    Join Date
    Feb 2013
    Location
    Dreamland
    Posts
    52
    Thanks
    11
    Thanked 2 Times in 2 Posts
    After long years of lurking, I couldn't really miss out to post that BIM performs pretty badly on some images.

    Which could of course be explained by the primary focus being photographic images, but that's far from the only use for a lossless image format.
    So yeah, uh, hi?

  16. #16
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    BIM is designed for large (12+ MP), true-color, photographic images. It is just not intended for such artificial images.

    However, about such "bad" performance. Check out how some other image compressors perform on one of your test files:

    No code has to be inserted here.

    Yes, for artificial images BIM v0.01 is just better. (Better pattern recognition blah blah blah)


  17. #17
    Member Bloax's Avatar
    Join Date
    Feb 2013
    Location
    Dreamland
    Posts
    52
    Thanks
    11
    Thanked 2 Times in 2 Posts
    Oh noes!
    Code:
    *** LZ4 v1.3b, by Yann Collet (Sep  3 2012) ***
    Compressed Filename is : H:\Test.ppm.lz4
    Detected : 2 cores
    Compressing H:\Test.ppm using 1 threads (compression level = 0)
    Compression completed : 3230KB --> 317KB  (9.82%) (324960 Bytes)
    Total Time : 0.02s ==> 194.6MB/s
    (CPU : 0.02s = 92%)


    Though this is definitely as artificial as images get, it's literally one huge scene composed from a set of little images. (Which is why it's so compressible.)

    Also oh dear, BCIF sure did bad on it. :s
    Last edited by Bloax; 6th February 2013 at 02:13.

  18. #18
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts

  19. #19
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Code:
    C:\bcm\x64\Release>bcm test.ppm
    BCM 0.12 Experimental BWT compressor
    Copyright (C) 2010 Ilia Muraviev
    
    Compressing...
    3307588 -> 54477 in 0 sec

  20. #20
    Member Bloax's Avatar
    Join Date
    Feb 2013
    Location
    Dreamland
    Posts
    52
    Thanks
    11
    Thanked 2 Times in 2 Posts
    The point really is that it's kind of silly that lz4 in maximum Speed Racer mode is that much better.

  21. #21
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 779 Times in 486 Posts
    Code:
       29,493 test1.paq8px v69 -7
       54,461 test1-3.zpaq
       54,884 test1-8.zpaq
       54,886 test1-7.zpaq
       54,894 test1-6.zpaq
       54,903 test1-5.zpaq
       64,877 test1.dmc 1000000000
       67,690 test1-4.zpaq
       73,067 test1.bz2 -9
       75,602 test1.7z -mx
       80,261 test1.pmd -o8 -m256 -r1
      108,830 test1-2.zpaq
      125,027 test1.gz -9
      152,642 test1.ctw -d6 -n16M -f16M
      158,299 test1-1.zpaq
      419,393 test1.Z
    2,407,842 test1.fpaq0
    3,307,588 test.ppm
    zpaq -method 3 is BWT. 4..8 are CM. 1..2 are LZ77.

  22. #22
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    237
    Thanks
    39
    Thanked 92 Times in 48 Posts
    1 161 296 397 bytes on LPCB images, all decompressed files were bit-exact.

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  23. #23
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Thanks for testing!

  24. #24
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Results with different counters - BIM v0.02 - Fast counters, BIM v0.02s - Slow counters (1.5X slower)

    No code has to be inserted here.

    This means only one thing - current counters are crap. Will search for a faster version of these "slow" counters.

  25. #25
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    237
    Thanks
    39
    Thanked 92 Times in 48 Posts
    Quote Originally Posted by encode View Post
    This means only one thing - current counters are crap.
    Not necessarily. Look at them from the other side: when you switch from slow counters to fast, compression quality is ~1% worse, but speed is 1.5 times higher!
    Actually, even less than 1% on average most likely,
    e.g. PIA12203.ppm: 95418720/94794028 = 1.00659

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  26. #26
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Yep, in terms of efficiency - with no doubt. But my current goal with BIM - a higher compression! BIM must beat RKIM and a couple of others. At the same time, I have no room for an additional complexity that will slow it down, easily. FLIC defined a bargain.
    Like I said, I have a bunch of tricks that will significantly improve BIM's compression performance - at the cost of 2X-10X speed penalty - no reason to do that since we have PAQ8(im) and such. So currently I'm trying to squeeze-in some of the ideas, in simplified and optimized forms.
    Like these slow counters might be simulated with much faster FSM-based ones. Just currently, I had no luck - FSM building and optimization is not the easiest task.

  27. #27
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    GARTEST - re-rendered with a newer ViewNX software + NEF(RAW)
    http://compressme.net/gartest.rar

    Currently, I'm working on BIM v0.03

    Some results on DSC_8341.ppm:
    BIM v0.02 -> 14,871,253 bytes
    BIM v0.03 -> 14,363,389 bytes

  28. #28
    Member kampaster's Avatar
    Join Date
    Apr 2010
    Location
    ->
    Posts
    55
    Thanks
    4
    Thanked 6 Times in 6 Posts
    Hi Ilya.
    No code has to be inserted here.
    Attached Files Attached Files

  29. #29
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 779 Times in 486 Posts
    I used ImageMagick convert to convert .tga to .bmp and .pmm. Tests with zpaq were on the .tga file.

    Code:
       544,707 AIMP_free-m1.zpaq
       494,449 AIMP_free-m2.zpaq
       518,582 AIMP_free-m3.zpaq
       355,052 AIMP_free-m4.zpaq
       308,943 AIMP_free-m5.zpaq
       279,088 AIMP_free-m6.zpaq
    16,439,334 AIMP_free.bmp
     2,050,086 AIMP_free.bmp.bim
    16,437,860 AIMP_free.ppm
     2,064,893 AIMP_free.ppm.bim
    16,437,887 AIMP_free.tga
     2,049,953 AIMP_free.tga.bim
       731,038 AIMP_free.zip

  30. #30
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Once again - BIM was not designed for such images. BIM is for true color pictures from your camera...

Page 1 of 2 12 LastLast

Similar Threads

  1. FLIC - a new fast lossless image compressor
    By Alexander Rhatushnyak in forum Data Compression
    Replies: 25
    Last Post: 10th January 2013, 19:46
  2. New lossless image compressor
    By encode in forum Data Compression
    Replies: 105
    Last Post: 10th January 2013, 10:36
  3. Lossless image coders
    By Madgeniy in forum Data Compression
    Replies: 26
    Last Post: 11th July 2011, 10:06
  4. GraLIC - new lossless image compressor
    By Alexander Rhatushnyak in forum Data Compression
    Replies: 17
    Last Post: 29th November 2010, 21:27
  5. New lossless compressor for 24-bit images (3 channels, 8 bits per channel)
    By Alexander Rhatushnyak in forum Data Compression
    Replies: 28
    Last Post: 23rd September 2010, 02:43

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •