Recommended readings for perceptual vision

[khavish]

Perceptually aware image compression make lossy image compression more acceptable and give rise to visually lossless images.Tools such as butteraugli , ssimulacra and other modern image quality metrics are able to make pretty accurate predictions concerning human vision.What are the recommended papers or reading to learn more about human vision perception ?.
Here are some content that i have found useful

Videos



Papers
Image Quality Assessment: From Error Visibility to Structural Similarity
Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures

Books
The Physiology of Human Vision

[Jyrki Alakuijala]

My personal experience is that using sRGB MSE/PSNR is not that bad when there is a lot of degradation in the image. However, SSIM works better there. There are still a few specific kinds of problems with SSIM, PSNRHVS-M and PSNR, many of which are solved in butteraugli:

1. They don't notice when a long thin line disappears.
2. They don't measure the impact of banding errors appearing into smooth gradients.
3. They don't punish enough for blurring the image.
4. They don't work for red and magenta backgrounds (red is not measured directly in retina but only computed from L-M, both L and M containing biological non-linearities) -- thus red backgrounds are more complex to model in color distance.
5. They don't work very well at the just-noticeable-difference border, they only give good values when the errors are already pretty big -- larger than you'd like to have your photographs compressed with.
6. In compression adding artefacts (such as jpeg 8x8 ringing and blocking) is a more significant error than doing the opposite, i.e., blurring some image features. None of the SSIM, PSNRHVS-M or PSNR are asymmetric.
7. When you add a gray border to an image, and match it perfectly in the other image, the PSNR/SSIM/PSNRHVS-M scores get better, even though the image has the same horrible mistakes as it used to. "Look I got that blue wall perfectly right, now I can spoil the face of that person to reach the goal quality score..."
8. PSNR/SSIM doesn't understand direction of textures, and I have seen an SSIM-optimized neural codec to apply a few vertical waves on a lake (likely gives a better SSIM score than the more common horizontal waves). PSNRHVS-M may be slightly better in this.
9. High frequency signals in blue channel are often overemphasized -- but also underemphasized in low red/green situations.
10. PSNR and SSIM do not deal with visual masking.

[porneL]

Great insights! Can you recommend any papers/sources about this? I've especially struggled to find anything about color similarity (every similarity paper starts with "color is hard, so we'll just use luma") and visual masking between luma and chroma.

[Jyrki Alakuijala]

Great insights! Can you recommend any papers/sources about this? I've especially struggled to find anything about color similarity (every similarity paper starts with "color is hard, so we'll just use luma") and visual masking between luma and chroma.

I looked into it with my own experiments inspired by textbook vision as well as some milestone papers. I wasn't careful with sources since I was going to replicate all tests anyway.

If you have any specific questions, I'd be happy to talk about it.