New butteraugli version

[Jyrki Alakuijala]

Substantially improved butteraugli version available at https://github.com/google/butteraugli/

[lorents17]

Please, compile butteraugli under Windows. Thanks!

[necros]

+1

[load]

no idea if it works... don't know what it is supposed to do.

hm... tells me always about invalid or malformed input files.

/edit: removed.

[MegaByte]

@Jyrki, does this metric obviate the need to convert to Y'CbCr 4:2:0 as in the following benchmark since it already takes into account color perception? https://web.archive.org/web/20160604...udy_july_2014/ https://github.com/bdaehlie/lossy-compression-test

[Jyrki Alakuijala]

@Jyrki, does this metric obviate the need to convert to Y'CbCr 4:2:0 as in the following benchmark since it already takes into account color perception? https://web.archive.org/web/20160604...udy_july_2014/ https://github.com/bdaehlie/lossy-compression-test

Yes, butteraugli does not benefit from nor need Y'CbCr 4:2:0. YUV 420 is typically a harmful representation according to butteraugli. YUV 420 modeling introduces artefacts that are visually clearly detectable, and butteraugli is sensitive to these artefacts. It is easy to create a YUV444 images that passes the butteraugli tests, and often impossible to do the same in YUV 420.

Butteraugli does not use any of the previously known colorspaces, but introduces its own XYB color space. It is relatively complicated, and I never tried to describe it in a single sentence yet. Gamma correction happens together with imperfect color filtering and eye jitter modeling in a complex and non-linear fashion, and the non-existence of the blue receptors in fovea is taken into account. XYB refers to a space where:

X = reddish yellow - greenish yellow
Y = greenish yellow
B = blue

I will soon write something that is more complete than this email and easier to read than the code.

[Jyrki Alakuijala]

hm... tells me always about invalid or malformed input files.

Which operating system are you using? Were you able to use any of your files?

[load]

i'm using windows 7. and no, no matter what files i try, none work.

if i use PNG, i get the following error:

libpng error: Read Error
File test.png is neither a valid JPEG nor a valid PNG.

for JPG, another one:

Premature end of JPEG file
JPEG datastream contains no image
File test.jpg is a malformed JPEG.

i used mingw to compile it, without any errors.

$ make
x86_64-w64-mingw32-g++ -std=c++11 -I. -c -o butteraugli.o butteraugli.cc
x86_64-w64-mingw32-g++ -std=c++11 -I. -c -o butteraugli_main.o butteraugli_main.cc
x86_64-w64-mingw32-g++ -std=c++11 -I. butteraugli.o butteraugli_main.o -lpng -ljpeg -lz -o butteraugli

libs are up to date, libpng-1.6.25 and libjpeg-turbo-1.5.1

[Jyrki Alakuijala]

i'm using windows 7. and no, no matter what files i try, none work.

It is because Windows open files in a strange text mode by default. Please, try if it works after changing fopen in butteraugli_main.cc from "r" to "rb". I'll fix it tomorrow in the repository.

[load]

ah, that did the trick. thanks a lot.

[Jyrki Alakuijala]

Now I have fixed it in github.com/google/butteraugli/ (Windows file I/O compatibility).

[pothos2]

Is this also useful for the development of AV1 or not does the advantage not really apply there?

[Jyrki Alakuijala]

Is this also useful for the development of AV1 or not does the advantage not really apply there?

It is an interesting question.

I think many butteraugli concepts could be used in all image and video codecs, but difficult to say if it is always worth the additional calculation and complexity. Butteraugli hasn't been validated with many eyes, and we are still actively changing it, including fundamentals like the integral transform, visual masking and separation of frequencies.

If you get some results with butteraugli/guetzli please share the experiences. User experiences could matter when the av1 engineers decide where to put their time.

[Jyrki Alakuijala]

Again, a new more accurate version. Similarities with the previous version: LMS color space with the gamma correction done in LMS, and many parts of visual masking are similar to the previous version. What is different: the FFT has been abandoned and a Difference-of-Gaussians scale space with ridge and edge detectors replaces it.

It is slightly worse on detecting metablock boundary artefacts being introduced than the previous version. This is likely coming from the symmetry requirement that we have imposed on butteraugli: butteraugli(image1, image2) == butteraugli(image2, image1), so a tiny sharp edge disappearing gets the same penalty as a tiny sharp edge emerging, even when the latter is quite a lot more annoying.

In our tests butteraugli (including the new version) is the only method (of psnr, psnrhvs-m, ssim, ssimulacra) in addition to humans that can detect that guetzli is actually better than libjpeg.

New version is available at: http://github.com/google/butteraugli/

[Jyrki Alakuijala]

Again, a new butteraugli version is available at: http://github.com/google/butteraugli/

We plan to release new versions of guetzli and PIK soon -- to match this version of butteraugli.

On the coming new guetzli: we found a few tricks to get better results at guetzli at lower qualities (particularly less block artefacts on slow gradients), but still not sure if it is useful and competitive at lower qualities. The hard limit of quality 84 is still there. Bad news that it is twice as slow as the previous version of guetzli. Probably previous version is a better fit for practical use and this a better fit for extreme use and research. With new guetzli, at 1.0 butteraugli score I get 7 % less bytes, some artefacts are gone, but the images tend to look just slightly more blurry (it's subtle, you will probably not notice it).

[SerGen]

Most of the time I found that results of old version of butteraugli are reliable.
On last weekends I have some fun with av1, compared it with guetzli and got unexpected result.

Some preparations.

> identify -format '%Q' input.jpg
99

Note small difference between guetzli and av1.

> butteraugli input.jpg guetzli-q92.jpg
1.651097
> butteraugli input.jpg av1-ec50d6b-cq5.png
1.602598

Latest version performs much better and in accordance with my eyes :)

> butteraugli input.jpg guetzli-q92.jpg
1.995533
> butteraugli input.jpg av1-ec50d6b-cq5.png
1.376524

[Jyrki Alakuijala]

Heatmap with the latest butteraugli between av1 and input:

[Jyrki Alakuijala]

original guetzli



latest guetzli (will appear on github this week)



and pik at butteraugli score 1.0 (about quality 95 on jpeg scale, 44797 bytes, here as much larger png):

[SerGen]

AV1: 96251 B, 1.376524 BA


Guetzli current: 57936 B, 1.995533 BA


Guetzli latest: 54303 B, 1.294250 BA


PIK: 44797 B, 0.958885 BA


Guetzli: distribution of errors more flat now, color errors of current version will gone in new, great!
PIK is really impressive!

One thing is bother me.
I can see artefacts on red button on the right of PIK result.
And can't say that color change of AV1 on chalkboard is worse than PIK artefacts on the button. For me they are at least the same.

[Jyrki Alakuijala]

Guetzli: distribution of errors more flat now, color errors of current version will gone in new, great!

Good to hear!

PIK is really impressive!

Thanks!! We are working hard on it...

One thing is bother me.
I can see artefacts on red button on the right of PIK result.
And can't say that color change of AV1 on chalkboard is worse than PIK artefacts on the button. For me they are at least the same.

Do you mean the DCT artefacts from the white "kupit" text projected on the surrounding red pixels?

[SerGen]

I mean long horizontal line surrounded by lighter pixels on the bottom of button (ringing?).
DCT artefacts around text are less noticeable for me.

Just for illustration, button on 4x

[Jyrki Alakuijala]

I mean long horizontal line surrounded by lighter pixels on the bottom of button (ringing?).
DCT artefacts around text are less noticeable for me.

Just for illustration, button on 4x

Cool! I didn't even notice it before you showed it. I'll try to improve on it in the next butteraugli version.

[Biozynotiker]

Can anyone please provide Windows binaries, from the latest butteraugli version?
Thanks in advance.

[SerGen]

I have try mingw to build, but have no luck.
Already answered in the Download area: https://encode.su/threads/2499-butte...ll=1#post54019

Maybe someone more skilled in building can help us.

[arminvb]

Can anyone please provide Windows binaries, from the latest butteraugli version?
Thanks in advance.

+1

[SerGen]

I have find interesting image, other metrics are rate it noticeable different.
First one was compressed by mozjpeg, second — guetzli.

butteraugli
2.897616 src_part.mj.png
1.644465 src_part.g.png

dssim
0.00242834 src_part.mj.png
0.00262183 src_part.g.png

ssimulacra
0.02160381 src_part.mj.png
0.02244841 src_part.g.png

Why butteraugli rates an image like this? Because of blur detection?

[Jyrki Alakuijala]

Why butteraugli rates an image like this?

Thank you for asking. Ringing noise around the letters seems to be the most visible artefact on these two images. I will take a deeper look next week into these images.