Hi folks,
ICIP was last month, and as advertised earlier on this forum, we run a challenge on lossy and lossless image compression there. Amongst the lossy candidates we had JPEG, JPEG 2000, JPEG XR, HEVC, Daala and WebP in the test plan. For lossless, we had many variants of JPEG (including hierarchical), JPEG (predictive), JPEG XR, JPEG 2000, JPEG LS, PNG and FLIF.
By resolution of the committee, we are happy to make the ICIP presentation publicly available here:
https://jpeg.org/downloads/aic/wg1n7..._challenge.pdf
Some reviews:
For lossy, HEVC is the winner, followed closely (and sometimes even left behind by) Daala, which performs very well on face images. Daala also improved from the previous test on the bike test due to smarter filtering.
Surprisingly, "JPEG on Steroids" can even compete with these methods for higher bitrates. It is a JPEG variant with uses some advanced coding tricks similar to mozjpeg. Luckely, I got the permission from the owner to make it available on my github pages under GPLv3, I just have to do it. It will appear there in a couple of days, I will post again as soon as I uploaded it.
WebP is not really any better than existing technology.
JPEG 2000 was unfortunately misconfigured due to an oversight on my side - I run it with the 5/3 wavelet rather than the 9/7 wavelet. My fault. Sorry about that, so please disregard its graphs for now.
Worst performing is JPEG. So what can we learn? It depends a lot on what precisely you do.
For lossless, the winner is FLIF. However, the distance to JPEG-LS part-2 is not very big. FLIF is rather high-complexity, JPEG-LS 2 a low-complexity codec. I guess it's just the usual with lossless compression: If you try something simple, you get 2:1. If you try very hard, you get 2:1. Or something that is a little bit better. After all, you are mostly compressing noise.
I want to thank all participants for submitting their software, and for the test labs for running the subjective tests.
Greetings,
Thomas