Psychovisual measurements on modern image codecs

[khavish]

Hello ,
I am performing a study on image compression where am trying different encoders and compare their quality using tools that determine the psychovisual similarity between images such as butteraugli and ssimulacra. The issue that i am having is that when i use a still image as a source , i get really bad image quality.However when i use a series of images (25 identical frames) as a source to create the mp4 video and then extract a frame from it to compare qualities , the quality is way better and the compression is less aggressive.

1.Is it because x264 and x265 compress better across frames ?
2.Is there an issue with my script? (Check attachment)

I would really appreciate if someone could enlighten me on this...

Kind Regards ,
Khavish

[Jyrki Alakuijala]

Digging out some data from the test results to support the question:

x264 233464 bytes, butteraugli = 0.948613, ssimulacra = 0.00761919
x265 241125 bytes, butteraugli = 1.689265, ssimulacra = 0.01749584

x265 gives significantly worse results than x264. Perhaps someone who knows ffmpeg flags can help.


Also, here jpeg for reference (also digged from the result_corpus.zip). In this test x264 is a lot better than jpeg, and x265 worse?!:

libjpeg/q93/yuv444 235122, butteraugli = 1.467834, ssimulacra = 0.01439446

[Jyrki Alakuijala]

Could you write out the ffmpeg command lines that you used?

[zubzer0]

Video compression used IPB frames https://en.wikipedia.org/wiki/Video_..._picture_types

When you squeeze 1 frame, used I-type frames. Which is basically an improved DCT model of JPEG.
Video codecs are not designed for the compression of single pictures, because create a lot of unnecessary information for subsequent frames, which are not.

But there are certain practices that are optimized for the compression of pictures. For example "BPG" - https://bellard.org/bpg/ (x265-fast, JCTVC-quality)

At the moment, as practice shows. There is no tests besides a visual assessment to identify quality! Since compressing images and especially video uses psychovisual methods.
This is a very complicated methods that spoil the image for better visual perception. For example emulate the noise, but not such as was in initially, and such that it was compressed.
Another point is the blur. Who likes to cheat HEVC all sorts of tests on quality, but at the same time would look fuzzy and not clear, with damage to small parts.
Only visual assessment!

Only with lossless compression, you can evaluate the quality of the numbers. According to file size.

[zubzer0]

I have some experience on how to improve test quality. But they require improvement.

( SSIM(db) + MSU_Blurring(db*) ) / 2
* log(1-(encode_blur / source_blur))×(-10)

if encode_blur < source_blur then (encode_blur / source_blur) - [be blurred image]
if encode_blur > source_blur then (source_blur / encode_blur) - [be sharp image]
if encode_blur = source_blur then 0.9999..9 - [depending on the number of decimal places at calculation of the SSIM metric]

ps. "MSU Blurring" metric -> MSU VQMT

[Jyrki Alakuijala]

"MSU Blurring" metric

Current version of Butteraugli measures four different kinds of blurring metrics:

1) It measures a local L2 norm over 9-pixel long line segments to 32 directions. These catch blurring that is removing thin lines (ridges). Think of a telephone line handing against the sky or cables of a sailing boat against the sky. In the same L2 norm it adds 8 directions of edge detectors.

2) It measures a gaussian blur of absolute values of high frequencies (after DoG filtering), and uses a local L2 norm for this.

3) It measures four different directions of edge activity, compares the gaussian blurred version of absolute values of the edge activity (and again local L2 norm of such measurements)

4) It compares the frequency bands separately, i.e., the image is divided into four frequency bands by DoG filtering and these are compared separate with a local L2 norm. This is more to include the differences in color perception at different levels of detail, but also participates in ensuring that the high frequency information is maintained sufficiently.

[khavish]

Digging out some data from the test results to support the question:

x264 233464 bytes, butteraugli = 0.948613, ssimulacra = 0.00761919
x265 241125 bytes, butteraugli = 1.689265, ssimulacra = 0.01749584

x265 gives significantly worse results than x264. Perhaps someone who knows ffmpeg flags can help.


Also, here jpeg for reference (also digged from the result_corpus.zip). In this test x264 is a lot better than jpeg, and x265 worse?!:

libjpeg/q93/yuv444 235122, butteraugli = 1.467834, ssimulacra = 0.01439446

BGP implementation of x265 encoder give worst results

x265(BGP) 247445 ​
bytes, butteraugli = 3.456237 , ssimulacra = 0.02138388

[Jyrki Alakuijala]

BGP implementation of x265 encoder give worst results

Very interesting!

What if you actually look at the heatmaps of butteraugli -- i.e., run: butteraugli input-for-x265.png output-from-x265.png heatmap.pnm

Then look at the heatmap.pnm and check if your eyes agree that there is degradation where butteraugli claims it is.

If no degradation there, file an issue against butteraugli.

It seems to me that there is very likely degradation somewhere, because both butteraugli and ssimulacra report a degradation.

[khavish]

Very interesting!

What if you actually look at the heatmaps of butteraugli -- i.e., run: butteraugli input-for-x265.png output-from-x265.png heatmap.pnm

Then look at the heatmap.pnm and check if your eyes agree that there is degradation where butteraugli claims it is.

If no degradation there, file an issue against butteraugli.

It seems to me that there is very likely degradation somewhere, because both butteraugli and ssimulacra report a degradation.

Yup butteraugli is right...P.S the heatmap was super helpful

[Jyrki Alakuijala]

Yup butteraugli is right...P.S the heatmap was super helpful

Wow. That is an unimpressive result from BPG. Looks like old formats are way more precise at the qualities 90+?!?! Perhaps some BPG expert could check this result.

FYI: http://xooyoozoo.github.io/yolo-octo...um&jpg=l&png=l uses bpgenc with these flags:

bpgenc-0.9.1 -m 8, with x265-1.4+189 --aq-mode 1 --deblock -1:-1 --crf $Q

[Intrinsic]

The version of x265 BPG uses is ancient(ver 1.4, 31 Oct 2014) so that could explain some of the issues, it would be interesting to see what it's like compiled with a more recent version. If you use the default settings for BPG i believe at actually falls back to the reference h265 encoder as well, from the readme "by default bpgenc uses the JCTVC encoder"

[Jyrki Alakuijala]

The version of x265 BPG uses is ancient(ver 1.4, 31 Oct 2014) so that could explain some of the issues, it would be interesting to see what it's like compiled with a more recent version. If you use the default settings for BPG i believe at actually falls back to the reference h265 encoder as well, from the readme "by default bpgenc uses the JCTVC encoder"

Are there decent opensource tools or free web services for using x265 for high end photograph compression (like jpeg yuv444 quality 93+)?

[zubzer0]

0.9.5 - last true version, JCTVC default
0.9.6 - add x265 and default
0.9.7 - cosmetic
http://www.mediafire.com/download/89dokq3uecaqkb3/libbpg-0.9.5_20150112.7z

if you update the bpg, it is better to upgrade JCTVC library

[Jyrki Alakuijala]

bpg + JCTVC
http://www.mediafire.com/download/89dokq3uecaqkb3/libbpg-0.9.5_20150112.7z

What are the correct flags for getting the most out of it for high quality image compression?

[load]

a while ago i made a BPG v0.9.7 compile with a newer version of JCTVC (v16.12 instead of v16.2, current is 16.16) and x265 v2.1

but i don't think it's of any help, i've just thrown the source code together and compiled it.

[khavish]

a while ago i made a BPG v0.9.7 compile with a newer version of JCTVC (v16.12 instead of v16.2, current is 16.16) and x265 v2.1

but i don't think it's of any help, i've just thrown the source code together and compiled it.

How was the compression compared to mine ?

[Shelwien]

Btw, do you include paq8px+jpeg in your tests?
If paq is too slow, there's still packjpg/lepton (its better to transform jpegs to progressive format when testing with these).

[Jyrki Alakuijala]

Btw, do you include paq8px+jpeg in your tests?
If paq is too slow, there's still packjpg/lepton (its better to transform jpegs to progressive format when testing with these).

It is a good idea, but might not be necessary in this context: Pik is that and more. It is basically a jpeg + recompression + format and encoding changes for further gains.

[Shelwien]

http://maximumcompression.com/data/files/jpg-test.zip

842,468 A10.jpg  // original
780,872 A10p.jpg // jpegtran -optimize -progressive -copy none -outfile A10p.jpg A10.jpg
688,273 A10.lep  // lepton20160715.exe A10.jpg
686,974 A10p.lep // lepton20160715.exe -unjailed -allowprogressive -multithread -skipvalidation A10p.jpg
673,197 A10.pjg  // packJPG.exe A10.jpg 
673,150 A10p.pjg // packJPG.exe A10p.jpg
630,296 A10.jpg.paq8px  // paq8px_v100.exe -7 A10.jpg 
772,785 A10p.jpg.paq8px // paq8px_v100.exe -7 A10p.jpg

Yes, but "result_corpus.zip" includes compression rates.
I think its very important to take as a reference not just a normal jpeg, but recompressed jpeg.
Also I kinda doubt that pik is better than paq8px.

[Jyrki Alakuijala]

Also I kinda doubt that pik is better than paq8px.

I don't know paq8px, but my silly prediction is that pik is 44 % more dense for the same visual user experience, decodes 30x faster and encodes 20x slower. Being in control of the loss and particularly creating a format where the quantization is psychovisually inspired and adaptive is quite powerful.

[Jyrki Alakuijala]

Khavish, please consider publishing your results so far as a graph of all codecs on a single metric. Also, would be very nice to have your scripts on github, so that it is possible to repeat and extend the tests.

[load]

How was the compression compared to mine ?

no idea. honestly, i wasn't really impressed by the bpg quality, so i never updated to the latest versions of x265 and JCTVC.
JCTVC may be better, but for practical use it is way too slow.

but i can do that if there is some interest...

[MegaByte]

842,468 A10.jpg  // original
780,872 A10p.jpg // jpegtran -optimize -progressive -copy none -outfile A10p.jpg A10.jpg
688,273 A10.lep  // lepton20160715.exe A10.jpg
686,974 A10p.lep // lepton20160715.exe -unjailed -allowprogressive -multithread -skipvalidation A10p.jpg
673,197 A10.pjg  // packJPG.exe A10.jpg 
673,150 A10p.pjg // packJPG.exe A10p.jpg
630,296 A10.jpg.paq8px  // paq8px_v100.exe -7 A10.jpg 
772,785 A10p.jpg.paq8px // paq8px_v100.exe -7 A10p.jpg

FWIW:

764,325 A10u.jpg  // jpegultrascan.pl -i -s -j -t -o mozjpegtran A10.jpg A10u.jpg     // standard JPEG
736,479 A10ua.jpg // jpegultrascan.pl -a -i -s -j -t -o mozjpegtran A10.jpg A10ua.jpg // arithmetic coding

[Shelwien]

There're no recompressors for arithmetic-coded jpegs, so not much sense.
paq8 also can't recompress progressive jpegs, but you can try 764k one with http://nishi.dreamhosters.com/u/lepton20170214.exe

[zubzer0]

Shelwien, from lepton there are two versions - normal and "slow-best-ratio"

672,242 A10p.lep // lepton-slow-best-ratio.exe -unjailed -allowprogressive A10p.jpg

The lepton.exe binary makes .lep files that are fast to decode by using multithreading
the lepton-slow-best-ratio.exe binary makes .lep files that are slightly smaller but may not be decoded in parallel

But there is one problem, the binary works only with SSE4, and to run on processors below requires an Assembly for SSE2.
Or opening through sde (sde.exe -- lepton-slow-best-ratio.exe ...) which greatly reduces the speed of execution

[Shelwien]

Just use packjpg then, lepton uses the same parser anyway.

[Shelwien]

built "lepton-slow-best-ratio" for SSE2, its simply solid compression, with MT disabled
http://nishi.dreamhosters.com/u/lepton64_20170827.exe

[zubzer0]

Shelwien Fatal error. To test SSE2 on new processors , it is possible to simulate P4 Prescott (SSE3, EM64T)
sde.exe -p4p -- lepton64_20170827.exe test.jpg

[Shelwien]

Reuploaded, seems to work with -p4p now.