Quo Vadis JPEG - an update

[thorfdbg]

Hi folks,

just came back from the JPEG meeting in Paris (actually, Saint Denis near Paris), and it is probably worth an update on what is going to happen. Actually, quite interesting as we have now three open calls for proposals. But one after another:

* There is a call for proposals for a backwards compatible JPEG codec for coding of HDR images. That is, a codec that encodes (and decodes) > 8bpp images with a 8bpp base layer in such a way that any JPEG (10918-1) decoder can decode it directly. This is of course in line what you already have with the libjpeg codec I posted here a while ago, except that you want to have a smarter tone mapping than just the downshift implemented in the 0.2. The 0.3 will be rolled out in the near future and will contain a smarter base layer computation, so stay tuned here. The call also requests for lossless coding of such images - that's already available.

I should state that the "lossless one block mode" of the IJG code is not 10918-1 compliant and hence does not address such needs (note to all: This is *not* a JPEG codestream it generates in such a case - avoid! - and it is unlikely that we will standardize anything *not* backwards compatible to 10918. There are already many lossless standards such as LS, and nobody sees the need of creating another lossless one.)

* We requested liason with MPEG to cooperate on an image codec based on HEVC (the second thing that was already mentioned earlier). HEVC requires a couple of extension to be suitable for that, for example higher bitdepth and larger gammut, but its I-frame compression is a very neat image codec already, so there is likely only a couple of minor extensions to be discussed with the committee "next door". Whether MPEG picks this up or not is still to see.

* We got one input from VESA for an extremely light-weight image compression for computer-display interfaces with limited bandwidth. Basically, VESA is asking for a "lossy compressed" version of HDMI to address the needs of larger screens while keeping the bandwidth limited. There is no frame buffer assumed here, and all one has for granted is a two-lines image buffer at the decoder. Of course, this will be implemented in hardware.

Thus, yes, we're moving foreward, and - I am happy to announce - quite in the direction where I want to move. Stay tuned, there will be updates for the software soon.

[m^2]

Thank you.
Lossy compression is in general severely underrepresented on this forum and you posts are very helpful in filling the holes.

[caveman]

Still no sign of transparency?

[thorfdbg]

Still no sign of transparency?

Not as of the 0.3, and we haven't the feature list ready yet. The challenge is to find a good balance between the needed features and the complexity of the standard. For example, JPEG 2000 went completely overboard, and even JPEG had stuff in it nobody needs or cares. (Hierarchical mode - anyone?). So the idea is to keep this thing really simple, and probably made some careful extensions later. Thus, I don't think we need anything more than sequential and progressive with Huffman coding. Arithmetic coding in JPEG never made it to the market, so it will likely go onto the trashdump. The lossless predictive mode of 10918 will go on the trashdump (use 10918, or JPEG LS, no need for a new standard). I'm unclear on the DNL marker. As far as alpha-channels are concerned, the immediate response I got from Dolby was that transparency is not needed since the application domain is digital photography. Maybe. Maybe this should go into an extensions part, maybe not at all. It's probably not the right tool for the job, who needs alpha in digital photos? If you need images on the web, take PNG. The real question is: What would be the use case for alpha? If I get a smart answer for Dolby, I'll go ahead. Anyhow, I don't know yet. The code includes some provisions to support it later, but I don't see any immediate pressure.

[toi007]

Well I dont know nothing about lossy compression except for the home made lossy format (a try out of jpg compression) but it does not uses DCT just quantetization and a gradient overlay of 8x8 pixels its similar to jpg compression if you see my post but quality its a very subjective word in my tests (im not a programmer just a curious guy that trys to keep up 10% of what you say)
I can add transparancy thow in a new layer but never tryed the 0-255 degrees of transparency

[caveman]

Not as of the 0.3, and we haven't the feature list ready yet. The challenge is to find a good balance between the needed features and the complexity of the standard. For example, JPEG 2000 went completely overboard, and even JPEG had stuff in it nobody needs or cares. (Hierarchical mode - anyone?)...

Hierarchical mode could be part of the answer to address responsive web design problems and the quest for a "one file fits all screen sizes" image format.
When you read proposals such as this one: Responsive image format by Yoav Weiss (He suggests to use progressive JPEG and stop downloading the file after only a few passes when rendering on small screens) should you laugh or beg for hierarchical JPEG?
"Retina" (HiDPI) iOS or Mac OS X apps have to embed duplicate resources (one image for standard definition screens and an other one for "Retina" displays) that's obviously a waste of space that could also be addressed by a hierarchical image format.

[caveman]

As far as alpha-channels are concerned, the immediate response I got from Dolby was that transparency is not needed since the application domain is digital photography. Maybe. Maybe this should go into an extensions part, maybe not at all. It's probably not the right tool for the job, who needs alpha in digital photos? If you need images on the web, take PNG. The real question is: What would be the use case for alpha? If I get a smart answer for Dolby, I'll go ahead. Anyhow, I don't know yet. The code includes some provisions to support it later, but I don't see any immediate pressure.

The problem with RGB+Alpha PNG files is that they are huge, WebP lossy+alpha is an answer to this (5 or 6 times smaller files are quite common in my own tests) but as you wrote WebP lossy is more complicated than JPEG.
JPEG+alpha has been tried before in the form of JNG (JPEG streams inside a PNG lookalike file) unfortunately it did not thrive and is practically dead nowadays, it could be interesting to revive it for comparison against WebP lossy+alpha.

[thorfdbg]

Hierarchical mode could be part of the answer to address responsive web design problems and the quest for a "one file fits all screen sizes" image format. When you read proposals such as this one: Responsive image format by Yoav Weiss (He suggests to use progressive JPEG and stop downloading the file after only a few passes when rendering on small screens) should you laugh or beg for hierarchical JPEG? "Retina" (HiDPI) iOS or Mac OS X apps have to embed duplicate resources (one image for standard definition screens and an other one for "Retina" displays) that's obviously a waste of space that could also be addressed by a hierarchical image format.

Let's play a little bit the devil's advocate (yup, I like this role): Why do you want to use JPEG in the web in first place? Or, to argue even different: There is already a format that does that: It is called JPEG 2000. Don't get me wrong. I know what you're saying. But I'm trying to find a convincing argument - and what I say above would probably the answer from some people.

[Matt Mahoney]

Because JPEG is everywhere. Not just the web, but almost every piece of software that has to display images.