Page 3 of 4 FirstFirst 1234 LastLast
Results 61 to 90 of 106

Thread: New lossless image compressor

  1. #61
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,023
    Thanks
    415
    Thanked 416 Times in 158 Posts
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	4837912076_736d9c514d_b.jpg 
Views:	237 
Size:	405.9 KB 
ID:	2025  

  2. #62
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,023
    Thanks
    415
    Thanked 416 Times in 158 Posts
    Intel build, and some compression time comparison (PIA13912.pnm)

    No code has to be inserted here.

    FLIC is yet better but yet slower...



    The log:
    Code:
    C:\Test>timer gar PIA13912.pnm
    
    Timer 3.01  Copyright (c) 2002-2003 Igor Pavlov  2003-07-10
    BIM/GAR v0.01-ALPHA2 by Ilia Muraviev
    Compressing...
    Dimensions: 6330 x -1
    111642227 -> 33386430 in 4.35 sec
    
    Kernel Time  =     0.093 = 00:00:00.093 =   2%
    User Time    =     4.274 = 00:00:04.274 =  97%
    Process Time =     4.368 = 00:00:04.368 = 100%
    Global Time  =     4.368 = 00:00:04.368 = 100%
    
    C:\Test>timer bmf PIA13912.pnm
    
    Timer 3.01  Copyright (c) 2002-2003 Igor Pavlov  2003-07-10
    BMF lossless image compressor, v.2.01 (C) 1998-1999, 2009 by Dmitry Shkarin
    File     PIA13912.pnm, image 6330x5879x24, size - 111642210: 7.140 bpp
    
    Kernel Time  =     0.234 = 00:00:00.234 =   3%
    User Time    =     6.068 = 00:00:06.068 =  93%
    Process Time =     6.302 = 00:00:06.302 =  97%
    Global Time  =     6.474 = 00:00:06.474 = 100%
    
    
    C:\Test>timer flic c PIA13912.flic PIA13912.pnm
    
    Timer 3.01  Copyright (c) 2002-2003 Igor Pavlov  2003-07-10
    Encoding PIA13912.pnm           111642227->27031158, 1.9370 bpc, 5.19 sec. 21511
     kb/s
    
    Kernel Time  =     0.265 = 00:00:00.265 =   5%
    User Time    =     4.929 = 00:00:04.929 =  94%
    Process Time =     5.194 = 00:00:05.194 =  99%
    Global Time  =     5.195 = 00:00:05.195 = 100%
    
    C:\Test>timer packpnm PIA13912.pnm
    
    Timer 3.01  Copyright (c) 2002-2003 Igor Pavlov  2003-07-10
    
    --> packPNM v1.0e (11/11/2011) by Matthias Stirner <--
    
    Processed  1 of  1 files [■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■]
    
    -> 1 file(s) processed, 0 error(s), 0 warning(s)
     ---------------------------------
     time taken        :    35020 msec
     avrg. byte per ms :     3187 byte
     avrg. comp. ratio :    34.99 %
     ---------------------------------
    
    Kernel Time  =     0.171 = 00:00:00.171 =   0%
    User Time    =    33.540 = 00:00:33.540 =  95%
    Process Time =    33.711 = 00:00:33.711 =  96%
    Global Time  =    35.022 = 00:00:35.022 = 100%

  3. #63
    Tester
    Black_Fox's Avatar
    Join Date
    May 2008
    Location
    [CZE] Czechia
    Posts
    471
    Thanks
    26
    Thanked 9 Times in 8 Posts
    This is getting interesting
    I am... Black_Fox... my discontinued benchmark
    "No one involved in computers would ever say that a certain amount of memory is enough for all time? I keep bumping into that silly quotation attributed to me that says 640K of memory is enough. There's never a citation; the quotation just floats like a rumor, repeated again and again." -- Bill Gates

  4. #64
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,023
    Thanks
    415
    Thanked 416 Times in 158 Posts
    The most important part, testing GAR on bunch of files shows about the same picture - its compression is about the same as with BMF or even better, but GAR is notable faster. GAR can have a higher compression, but in this case it will be slower than FLIC.

  5. #65
    Member
    Join Date
    May 2012
    Location
    UK
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by encode View Post
    FLIC 1.4.demo
    lena.pnm -> 419,012 bytes
    cow.pnm -> 6,785,997 bytes
    butterfly.pnm -> 4,414,260 bytes

    Test files:
    http://narod.ru/disk/61965621001.24e...rtest.rar.html
    I just tested version 2.1 Demo of Flic and got slightly different results
    Code:
    flic c lena.flic lena.pnm
    Encoding lena.pnm                786447-> 424930, 4.3225 bpc, 0.25 sec. 3146 kb/s
    
    flic c cow.flic cow.pnm
    Encoding cow.pnm                18361655->6483119, 2.8246 bpc, 1.88 sec. 9793 kb/s
    
    flic c butterly.flic butterfly.pnm
    Encoding butterfly.pnm          18048017->4364968, 1.9348 bpc, 1.66 sec. 10899 kb/s
    Looking at the results it appears lena.pnm compresses better in 1.4 but the others compress better in 2.1

  6. #66
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    876
    Thanks
    474
    Thanked 175 Times in 85 Posts
    How does GAR compress on my image set?

  7. #67
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,023
    Thanks
    415
    Thanked 416 Times in 158 Posts
    Overlooked the latest FLIC...

    C:\Test>gar PHIL_2942.ppm
    Compressing PHIL_2942.ppm: 17952785 -> 4306462 in 0.71 sec

    C:\Test>gar sigma24.ppm
    Compressing sigma24.ppm: 13939217 -> 4507953 in 0.57 sec

    C:\Test>gar sony24.ppm
    Compressing sony24.ppm: 48098321 -> 20057481 in 2.10 sec


    The development goes on...

  8. #68
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,023
    Thanks
    415
    Thanked 416 Times in 158 Posts
    C:\Test>gar hs-2005-30-a-full_tif.ppm
    Compressing hs-2005-30-a-full_tif.ppm: 35979389 -> 23517817 in 1.88 sec

  9. #69
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,023
    Thanks
    415
    Thanked 416 Times in 158 Posts
    One of the features of this new codec is a small memory footprint (actual mem use is less than 1 MB). The image width or size is virtually not restricted. i.e. you can compress 24-bit images of up to 2 GB in size, and 40000+ px in wide.
    I will release 32-bit or 64-bit version only.
    The 64-bit version is notable faster, even with precise range coder.
    32-bit one is notable slower, with precise range coder it is at least 2X time slower than 64-bit version. With a faster range coder 32-bit version is still slower than 64-bit, but not that much.
    Personally, I'd release 64-bit version only. But not all users have 64-bit Windows, even under 64-bit CPUs. And for Linux users, can WINE run 64-bit executables?
    What do you think?

  10. #70
    Member
    Join Date
    May 2008
    Location
    Antwerp , country:Belgium , W.Europe
    Posts
    487
    Thanks
    1
    Thanked 3 Times in 3 Posts
    I vote for a Win 64-bit version !
    Why would you not release Win32 + Win64 version ? What is the problem with releasing both ?

  11. #71
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,023
    Thanks
    415
    Thanked 416 Times in 158 Posts
    I prefer to keep one reference compile. In any case the main version must be either 64 or 32-bit. Like I said, there are some design differences - mainly in less precise range coder - since this makes a huge difference in 32-bit version, making it two times faster! Extra registers in 64-bit compile make a notable difference too!

  12. #72
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 798 Times in 489 Posts
    Why would 32 and 64 bit versions produce different outputs? You can do 64 bit arithmetic on a 32 bit processor, just not as fast.

  13. #73
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,023
    Thanks
    415
    Thanked 416 Times in 158 Posts
    Quote Originally Posted by Matt Mahoney View Post
    Why would 32 and 64 bit versions produce different outputs? You can do 64 bit arithmetic on a 32 bit processor, just not as fast.
    Yes, of course. Just FLIC set a definitely new level in terms of compression ratio vs compression time. My new compressor, currently, has lower compression ratio compared to FLIC, so the speed is a must!

  14. #74
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    876
    Thanks
    474
    Thanked 175 Times in 85 Posts
    I guess a 64-bit build is more future-proof but may not be used on 32-bit operating systems. Nevertheless I would prefer a 64-bit build.

  15. #75
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 798 Times in 489 Posts
    Most computers are 64 bits now, but about 40% of computers are still running Windows XP. Without a 32 bit version, you lose half your users.

  16. #76
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,610
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by Matt Mahoney View Post
    Most computers are 64 bits now, but about 40% of computers are still running Windows XP. Without a 32 bit version, you lose half your users.
    Do you mean "32-bit version of Windows XP"?
    I run XP too, but don't count me among 32-bit users.

  17. #77
    Member
    Join Date
    Jun 2008
    Location
    G
    Posts
    377
    Thanks
    26
    Thanked 23 Times in 16 Posts
    i think encode should provide both 32bit and 64bit thats just of course normal, the algorithm should at least perform very well in 64bit mode. The algorithm should be independent from build, but of course needs to optimized I think 64bit is importanter than 32bit at least in the early test days. I think this discussion doesn't need to discuss further lets go back to the interesting topics like will the algorithm be able to compress 8,16,32 bit images its possible to decode via more than 2 threads etc.

  18. #78
    Member
    Join Date
    May 2012
    Location
    UK
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts
    If its open source then just build the 64bit binary and let anyone on a 32bit system build their own binary, but if it isn't open source and you just want to make a fast compressor then just make a 64bit binary if that's faster (i personally am on a 32bit version of XP so i will have to use it in a virtual machine to try it out).

  19. #79
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 798 Times in 489 Posts
    Here are a couple of surveys:
    http://store.steampowered.com/hwsurvey/
    http://www.netmarketshare.com/operat...10&qpcustomd=0

    The first is for gamers, who tend to have powerful hardware. About 2/3 have 64 bits.
    The second gives OS versions but doesn't distinguish 32 and 64 bits. 41% have XP, 44% have Win7.
    Note that in the first survey, 12% have XP, 74% have Win7. Among XP, only 4% are 64 bit. Among Win7, 80% are 64 bit.

  20. #80
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,593
    Thanks
    801
    Thanked 698 Times in 378 Posts
    you can also look at the end of http://freearc.org/Statistics.aspx

  21. #81
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,610
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by Bulat Ziganshin View Post
    you can also look at the end of http://freearc.org/Statistics.aspx
    This counts only GUI users. :P

  22. #82
    Member Fallon's Avatar
    Join Date
    May 2008
    Location
    Europe - The Netherlands
    Posts
    164
    Thanks
    15
    Thanked 12 Times in 6 Posts
    It's nice to see you have something up your sleeve!
    64-bit. Anything else is old.
    I just updated to 16 GB DDR3 RAM, (Corsair Vengeance) because it's dirt cheap right now. (Impossible on 32-bit).

  23. #83
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,023
    Thanks
    415
    Thanked 416 Times in 158 Posts
    Okay, I just made an accent on this, since I was quite surprised about how faster 64-bit version is, compared to 32-bit one - 2X to 3X times with current model structure. Just before an actual release I will post detailed performance on each build. Actually the difference in compression between fast and precise range coders is really small - less than 1 KB on compressing 35 MB image.
    I have more serious questions to deal with. Currently I'm working really hard on compression ratio and efficiency (ratio vs speed)
    Some statements:
    Currently my compressor supports only PPM files. Why not BMP? BMP has such stupid thing as row alignment - it adds from 0 to 3 bytes per each row. These bytes may contain anything, and this can hurt compression, a little in most cases. As a note far from all BMP files needs to be aligned. The padding size can be computed as follows:
    RowSize=ImageWidth*3; // (for 24-bit images)
    RowPad=(RowSize+3)&0xfffc;
    Taking this into account, I keep rowsize instead of image width in compressed stream. So, in future I may add compression of BMP files and others.
    Naturally my codec can support 8-bit Grayscale, 24-bit RGB, 32-bit RGBA. But the main focus will be on 24-bit, since my compressor intended for high resolution 24-bit images, untouched by lossy JPEG. As example images you convert from RAF/NEF (RAW) format from you camera to TIFF or something. 48-bit images can be supported in future too.
    In addition, my compressor will preserve all data to be 100% lossless. You can even compress an EXE file with it and correctly decompress it back.
    I think I'll include CRC checking even in early releases!
    One thing, I still cannot invent the name for my compressor...
    The good thing is that it really good in terms of compression and speed. Only the best codecs may compete with it. JPEG-LS, WIC and others are far far away. It's good!

  24. #84
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,023
    Thanks
    415
    Thanked 416 Times in 158 Posts
    New results:

    Old gartest (for reference)

    No code has to be inserted here.

    Squeeze Chart

    No code has to be inserted here.

    GARTEST

    No code has to be inserted here.
    All pictures are taken by me, recently, on my Nikon D90. Converted directly from in-camera RAW(NEF): NEF->TIFF->PPM
    First eight pictures was taken in Thailand, the last two in my summer residence.

    DOWNLOAD LINK:
    http://narod.ru/disk/62402017001.19d...rtest.rar.html


  25. #85
    Member
    Join Date
    Jun 2008
    Location
    G
    Posts
    377
    Thanks
    26
    Thanked 23 Times in 16 Posts
    How about FLPC for Free Lossless Picture Compressor? Will older compressors decompress newer compressors files?

  26. #86
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,023
    Thanks
    415
    Thanked 416 Times in 158 Posts
    It's CM-based, so any new compression improvements incompatible with an old decoder. I'll take into account your suggested name, although it's like a FLIC. Unfortunately, Alex was first...

  27. #87
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 798 Times in 489 Posts
    You could write it as a ZPAQ model and all versions would be compatible.

  28. #88
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,023
    Thanks
    415
    Thanked 416 Times in 158 Posts
    Yep! And actually my image compressor has some predictors a la HBB that PAQ8* missing...

  29. #89
    Member
    Join Date
    Jun 2008
    Location
    G
    Posts
    377
    Thanks
    26
    Thanked 23 Times in 16 Posts
    How about EIC - Encodes Image Compressor but then it should handle all types of images well otherwise EPC may better.

  30. #90
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,023
    Thanks
    415
    Thanked 416 Times in 158 Posts
    Quote Originally Posted by thometal View Post
    How about EIC - Encodes Image Compressor but then it should handle all types of images well otherwise EPC may better.
    EPIC

Page 3 of 4 FirstFirst 1234 LastLast

Similar Threads

  1. FLIC - a new fast lossless image compressor
    By Alexander Rhatushnyak in forum Data Compression
    Replies: 25
    Last Post: 10th January 2013, 19:46
  2. Lossless image coders
    By Madgeniy in forum Data Compression
    Replies: 26
    Last Post: 11th July 2011, 10:06
  3. GraLIC - new lossless image compressor
    By Alexander Rhatushnyak in forum Data Compression
    Replies: 17
    Last Post: 29th November 2010, 21:27
  4. New lossless compressor for 24-bit images (3 channels, 8 bits per channel)
    By Alexander Rhatushnyak in forum Data Compression
    Replies: 28
    Last Post: 23rd September 2010, 02:43
  5. image compressors
    By maadjordan in forum Forum Archive
    Replies: 5
    Last Post: 13th August 2007, 10:28

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •