Results 1 to 8 of 8

Thread: lightning-fast lossless image compression (heads up)

Threaded View

Previous Post Previous Post   Next Post Next Post
  1. #1
    Member
    Join Date
    Oct 2013
    Location
    Filling a much-needed gap in the literature
    Posts
    350
    Thanks
    177
    Thanked 49 Times in 35 Posts

    lightning-fast lossless image compression (heads up)

    I've been talking to some graphics guys, and it seems that you can losslessly compress or decompress images by about a factor of two at hundreds of megabytes per second, single core, using SIMD instructions (e.g., SSE4). (8-bit grayscale images from the imagecompression.info gray8 benchmark.)

    Unfortunately, the people who know how to do this are not publishing how or releasing the code because it's not a standard compression format and their companies are pursuing standardized compression formats like JPEG-LS instead.

    On the other hand, it seems that the techniques are not patented and it doesn't appear that they will be, so this might be a good thing for a free software person who's good at images and SIMD to look into, if they're so inclined.

    This posting at Jan Wassenberg's blog is particularly intriguing:

    http://wassenberg.dreamhosters.com/b...e-compression/

    Jan did some very cool work for his dissertation on symmetrical compression of 16-bit images, but that code did not work well for 8-bit color channels; the newer variant he describes does, at a cost in compression ratio of a bit under 20 percent.

    On his blog he says:

    The most interesting part is fixed-to-variable coding for entire vector registers of bits. For the image residuals above, it compresses to within 5-20% of the entropy at speeds comparable to or exceeding state of the art integer coders (http://arxiv.org/abs/1209.2137). A single core can compress/decompress between 4600 MB/s (sparse bit arrays with 0.05% ones) and 1000 MB/s (relatively dense, 12% ones). Of course this scales to multiple cores, as there is no shared dictionary to update.
    Although the instruction set improved with SSE4, writing vector code is still an exercise in improvisation. Finding 1-bits is accomplished by a hash function based on de Bruijn sequences (http://supertech.csail.mit.edu/papers/debruijn.pdf) and using the FPU normalization hardware. Vectorized shifts are emulated with the PSHUFB universal shuffle instruction.
    I'm not an expert on either images or SIMDfying algorithms, but that sounded very promising.

  2. Thanks:

    avitar (15th May 2014)

Similar Threads

  1. BIM (a new lossless image compressor) is here!
    By encode in forum Data Compression
    Replies: 43
    Last Post: 17th September 2013, 16:00
  2. FLIC - a new fast lossless image compressor
    By Alexander Rhatushnyak in forum Data Compression
    Replies: 25
    Last Post: 10th January 2013, 19:46
  3. New lossless image compressor
    By encode in forum Data Compression
    Replies: 105
    Last Post: 10th January 2013, 10:36
  4. iz: New fast lossless RGB photo compression
    By cfeck in forum Data Compression
    Replies: 63
    Last Post: 4th December 2012, 12:21
  5. Lossless image coders
    By Madgeniy in forum Data Compression
    Replies: 26
    Last Post: 11th July 2011, 10:06

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •