
Originally Posted by
Matt Mahoney
Judging image quality is an unsolved AI problem. You have to model the human visual perception system, which we don't know how to do at a high level. For example, it is more important to get faces right than the background, so your test software has to be able to recognize faces.
If we knew how to solve it, then it would be possible to compress images to less than 1 KB by converting it to a text description and compressing it (assuming a picture is worth 1000 words). The decompresser would read the text and draw the picture. The test software would compress it again and see if it generates the same text.
In fact, we know from experiments with human long term memory, like those done by Standing and Landauer, that we only store about 30 bits after looking at a picture for 5 seconds. That's enough to pass a recall test where you look at 10,000 pictures, then are shown new photos and are asked if you have seen them before.
So until this problem is solved, you will have to judge image quality by looking at it.