Results 1 to 29 of 29

Thread: Psychovisual measurements on modern image codecs

  1. #1
    Member
    Join Date
    Aug 2017
    Location
    Mauritius
    Posts
    59
    Thanks
    67
    Thanked 22 Times in 16 Posts

    Psychovisual measurements on modern image codecs

    Hello ,
    I am performing a study on image compression where am trying different encoders and compare their quality using tools that determine the psychovisual similarity between images such as butteraugli and ssimulacra. The issue that i am having is that when i use a still image as a source , i get really bad image quality.However when i use a series of images (25 identical frames) as a source to create the mp4 video and then extract a frame from it to compare qualities , the quality is way better and the compression is less aggressive.

    1.Is it because x264 and x265 compress better across frames ?
    2.Is there an issue with my script? (Check attachment)

    I would really appreciate if someone could enlighten me on this...

    Kind Regards ,
    Khavish
    Attached Files Attached Files
    Last edited by khavish; 19th August 2017 at 20:35.

  2. Thanks:

    Jyrki Alakuijala (19th August 2017)

  3. #2
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    744
    Thanks
    215
    Thanked 280 Times in 163 Posts
    Digging out some data from the test results to support the question:

    x264 233464 bytes, butteraugli = 0.948613, ssimulacra = 0.00761919
    x265 241125 bytes, butteraugli = 1.689265, ssimulacra = 0.01749584

    x265 gives significantly worse results than x264. Perhaps someone who knows ffmpeg flags can help.


    Also, here jpeg for reference (also digged from the result_corpus.zip). In this test x264 is a lot better than jpeg, and x265 worse?!:

    libjpeg/q93/yuv444 235122, butteraugli = 1.467834, ssimulacra = 0.01439446

  4. Thanks:

    khavish (19th August 2017)

  5. #3
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    744
    Thanks
    215
    Thanked 280 Times in 163 Posts
    Could you write out the ffmpeg command lines that you used?

  6. #4
    Member
    Join Date
    Jul 2016
    Location
    Russia
    Posts
    22
    Thanks
    13
    Thanked 9 Times in 7 Posts
    Video compression used IPB frames https://en.wikipedia.org/wiki/Video_..._picture_types
    Click image for larger version. 

Name:	700px-IPB_images_sequence.png 
Views:	82 
Size:	49.1 KB 
ID:	5129
    When you squeeze 1 frame, used I-type frames. Which is basically an improved DCT model of JPEG.
    Video codecs are not designed for the compression of single pictures, because create a lot of unnecessary information for subsequent frames, which are not.

    But there are certain practices that are optimized for the compression of pictures. For example "BPG" - https://bellard.org/bpg/ (x265-fast, JCTVC-quality)

    At the moment, as practice shows. There is no tests besides a visual assessment to identify quality! Since compressing images and especially video uses psychovisual methods.
    This is a very complicated methods that spoil the image for better visual perception. For example emulate the noise, but not such as was in initially, and such that it was compressed.
    Another point is the blur. Who likes to cheat HEVC all sorts of tests on quality, but at the same time would look fuzzy and not clear, with damage to small parts.
    Only visual assessment!

    Only with lossless compression, you can evaluate the quality of the numbers. According to file size.

  7. Thanks (2):

    Jyrki Alakuijala (19th August 2017),khavish (19th August 2017)

  8. #5
    Member
    Join Date
    Jul 2016
    Location
    Russia
    Posts
    22
    Thanks
    13
    Thanked 9 Times in 7 Posts
    I have some experience on how to improve test quality. But they require improvement.

    ( SSIM(db) + MSU_Blurring(db*) ) / 2
    * log(1-(encode_blur / source_blur))×(-10)

    if encode_blur < source_blur then (encode_blur / source_blur) - [be blurred image]
    if encode_blur > source_blur then (source_blur / encode_blur) - [be sharp image]
    if encode_blur = source_blur then 0.9999..9 - [depending on the number of decimal places at calculation of the SSIM metric]

    ps. "MSU Blurring" metric -> MSU VQMT

  9. Thanks:

    khavish (20th August 2017)

  10. #6
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    744
    Thanks
    215
    Thanked 280 Times in 163 Posts
    Quote Originally Posted by zubzer0 View Post
    "MSU Blurring" metric
    Current version of Butteraugli measures four different kinds of blurring metrics:

    1) It measures a local L2 norm over 9-pixel long line segments to 32 directions. These catch blurring that is removing thin lines (ridges). Think of a telephone line handing against the sky or cables of a sailing boat against the sky. In the same L2 norm it adds 8 directions of edge detectors.

    2) It measures a gaussian blur of absolute values of high frequencies (after DoG filtering), and uses a local L2 norm for this.

    3) It measures four different directions of edge activity, compares the gaussian blurred version of absolute values of the edge activity (and again local L2 norm of such measurements)

    4) It compares the frequency bands separately, i.e., the image is divided into four frequency bands by DoG filtering and these are compared separate with a local L2 norm. This is more to include the differences in color perception at different levels of detail, but also participates in ensuring that the high frequency information is maintained sufficiently.
    Last edited by Jyrki Alakuijala; 20th August 2017 at 19:11.

  11. Thanks (2):

    khavish (20th August 2017),zubzer0 (21st August 2017)

  12. #7
    Member
    Join Date
    Aug 2017
    Location
    Mauritius
    Posts
    59
    Thanks
    67
    Thanked 22 Times in 16 Posts
    Quote Originally Posted by Jyrki Alakuijala View Post
    Digging out some data from the test results to support the question:

    x264 233464 bytes, butteraugli = 0.948613, ssimulacra = 0.00761919
    x265 241125 bytes, butteraugli = 1.689265, ssimulacra = 0.01749584

    x265 gives significantly worse results than x264. Perhaps someone who knows ffmpeg flags can help.


    Also, here jpeg for reference (also digged from the result_corpus.zip). In this test x264 is a lot better than jpeg, and x265 worse?!:

    libjpeg/q93/yuv444 235122, butteraugli = 1.467834, ssimulacra = 0.01439446
    BGP implementation of x265 encoder give worst results

    Code:
    x265(BGP) 247445 ​
    bytes, butteraugli = 3.456237 , ssimulacra = 0.02138388
    Attached Files Attached Files
    Last edited by khavish; 20th August 2017 at 19:14. Reason: added bpg results

  13. #8
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    744
    Thanks
    215
    Thanked 280 Times in 163 Posts
    Quote Originally Posted by khavish View Post
    BGP implementation of x265 encoder give worst results
    Very interesting!

    What if you actually look at the heatmaps of butteraugli -- i.e., run: butteraugli input-for-x265.png output-from-x265.png heatmap.pnm

    Then look at the heatmap.pnm and check if your eyes agree that there is degradation where butteraugli claims it is.

    If no degradation there, file an issue against butteraugli.

    It seems to me that there is very likely degradation somewhere, because both butteraugli and ssimulacra report a degradation.

  14. #9
    Member
    Join Date
    Aug 2017
    Location
    Mauritius
    Posts
    59
    Thanks
    67
    Thanked 22 Times in 16 Posts

    Smile

    Quote Originally Posted by Jyrki Alakuijala View Post
    Very interesting!

    What if you actually look at the heatmaps of butteraugli -- i.e., run: butteraugli input-for-x265.png output-from-x265.png heatmap.pnm

    Then look at the heatmap.pnm and check if your eyes agree that there is degradation where butteraugli claims it is.

    If no degradation there, file an issue against butteraugli.

    It seems to me that there is very likely degradation somewhere, because both butteraugli and ssimulacra report a degradation.
    Yup butteraugli is right...P.S the heatmap was super helpful
    Attached Files Attached Files

  15. #10
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    744
    Thanks
    215
    Thanked 280 Times in 163 Posts
    Quote Originally Posted by khavish View Post
    Yup butteraugli is right...P.S the heatmap was super helpful
    Wow. That is an unimpressive result from BPG. Looks like old formats are way more precise at the qualities 90+?!?! Perhaps some BPG expert could check this result.

    FYI: http://xooyoozoo.github.io/yolo-octo...um&jpg=l&png=l uses bpgenc with these flags:

    bpgenc-0.9.1 -m 8, with x265-1.4+189 --aq-mode 1 --deblock -1:-1 --crf $Q
    Last edited by Jyrki Alakuijala; 22nd August 2017 at 00:24.

  16. #11
    Member
    Join Date
    May 2008
    Location
    England
    Posts
    325
    Thanks
    18
    Thanked 6 Times in 5 Posts
    The version of x265 BPG uses is ancient(ver 1.4, 31 Oct 2014) so that could explain some of the issues, it would be interesting to see what it's like compiled with a more recent version. If you use the default settings for BPG i believe at actually falls back to the reference h265 encoder as well, from the readme "by default bpgenc uses the JCTVC encoder"

  17. Thanks (2):

    Jyrki Alakuijala (24th August 2017),khavish (24th August 2017)

  18. #12
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    744
    Thanks
    215
    Thanked 280 Times in 163 Posts
    Quote Originally Posted by Intrinsic View Post
    The version of x265 BPG uses is ancient(ver 1.4, 31 Oct 2014) so that could explain some of the issues, it would be interesting to see what it's like compiled with a more recent version. If you use the default settings for BPG i believe at actually falls back to the reference h265 encoder as well, from the readme "by default bpgenc uses the JCTVC encoder"
    Are there decent opensource tools or free web services for using x265 for high end photograph compression (like jpeg yuv444 quality 93+)?

  19. Thanks:

    khavish (24th August 2017)

  20. #13
    Member
    Join Date
    Jul 2016
    Location
    Russia
    Posts
    22
    Thanks
    13
    Thanked 9 Times in 7 Posts
    0.9.5 - last true version, JCTVC default
    0.9.6 - add x265 and default
    0.9.7 - cosmetic
    http://www.mediafire.com/download/89dokq3uecaqkb3/libbpg-0.9.5_20150112.7z

    if you update the bpg, it is better to upgrade JCTVC library

  21. Thanks:

    khavish (24th August 2017)

  22. #14
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    744
    Thanks
    215
    Thanked 280 Times in 163 Posts
    Quote Originally Posted by zubzer0 View Post
    bpg + JCTVC
    http://www.mediafire.com/download/89dokq3uecaqkb3/libbpg-0.9.5_20150112.7z
    What are the correct flags for getting the most out of it for high quality image compression?

  23. Thanks:

    khavish (24th August 2017)

  24. #15
    Member
    Join Date
    Apr 2009
    Location
    here
    Posts
    204
    Thanks
    170
    Thanked 109 Times in 65 Posts
    a while ago i made a BPG v0.9.7 compile with a newer version of JCTVC (v16.12 instead of v16.2, current is 16.16) and x265 v2.1

    but i don't think it's of any help, i've just thrown the source code together and compiled it.

  25. Thanks:

    khavish (24th August 2017)

  26. #16
    Member
    Join Date
    Aug 2017
    Location
    Mauritius
    Posts
    59
    Thanks
    67
    Thanked 22 Times in 16 Posts
    Quote Originally Posted by load View Post
    a while ago i made a BPG v0.9.7 compile with a newer version of JCTVC (v16.12 instead of v16.2, current is 16.16) and x265 v2.1

    but i don't think it's of any help, i've just thrown the source code together and compiled it.
    How was the compression compared to mine ?

  27. #17
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,688
    Thanks
    264
    Thanked 1,180 Times in 651 Posts
    Btw, do you include paq8px+jpeg in your tests?
    If paq is too slow, there's still packjpg/lepton (its better to transform jpegs to progressive format when testing with these).

  28. #18
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    744
    Thanks
    215
    Thanked 280 Times in 163 Posts
    Quote Originally Posted by Shelwien View Post
    Btw, do you include paq8px+jpeg in your tests?
    If paq is too slow, there's still packjpg/lepton (its better to transform jpegs to progressive format when testing with these).
    It is a good idea, but might not be necessary in this context: Pik is that and more. It is basically a jpeg + recompression + format and encoding changes for further gains.

  29. #19
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,688
    Thanks
    264
    Thanked 1,180 Times in 651 Posts
    http://maximumcompression.com/data/files/jpg-test.zip
    Code:
    842,468 A10.jpg  // original
    780,872 A10p.jpg // jpegtran -optimize -progressive -copy none -outfile A10p.jpg A10.jpg
    688,273 A10.lep  // lepton20160715.exe A10.jpg
    686,974 A10p.lep // lepton20160715.exe -unjailed -allowprogressive -multithread -skipvalidation A10p.jpg
    673,197 A10.pjg  // packJPG.exe A10.jpg 
    673,150 A10p.pjg // packJPG.exe A10p.jpg
    630,296 A10.jpg.paq8px  // paq8px_v100.exe -7 A10.jpg 
    772,785 A10p.jpg.paq8px // paq8px_v100.exe -7 A10p.jpg
    Yes, but "result_corpus.zip" includes compression rates.
    I think its very important to take as a reference not just a normal jpeg, but recompressed jpeg.
    Also I kinda doubt that pik is better than paq8px.

  30. Thanks:

    Jyrki Alakuijala (25th August 2017)

  31. #20
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    744
    Thanks
    215
    Thanked 280 Times in 163 Posts
    Quote Originally Posted by Shelwien View Post
    Also I kinda doubt that pik is better than paq8px.
    I don't know paq8px, but my silly prediction is that pik is 44 % more dense for the same visual user experience, decodes 30x faster and encodes 20x slower. Being in control of the loss and particularly creating a format where the quantization is psychovisually inspired and adaptive is quite powerful.

  32. #21
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    744
    Thanks
    215
    Thanked 280 Times in 163 Posts
    Khavish, please consider publishing your results so far as a graph of all codecs on a single metric. Also, would be very nice to have your scripts on github, so that it is possible to repeat and extend the tests.

  33. #22
    Member
    Join Date
    Apr 2009
    Location
    here
    Posts
    204
    Thanks
    170
    Thanked 109 Times in 65 Posts
    Quote Originally Posted by khavish View Post
    How was the compression compared to mine ?
    no idea. honestly, i wasn't really impressed by the bpg quality, so i never updated to the latest versions of x265 and JCTVC.
    JCTVC may be better, but for practical use it is way too slow.

    but i can do that if there is some interest...

  34. #23
    Member
    Join Date
    Mar 2016
    Location
    USA
    Posts
    49
    Thanks
    7
    Thanked 23 Times in 15 Posts
    Quote Originally Posted by Shelwien View Post
    Code:
    842,468 A10.jpg  // original
    780,872 A10p.jpg // jpegtran -optimize -progressive -copy none -outfile A10p.jpg A10.jpg
    688,273 A10.lep  // lepton20160715.exe A10.jpg
    686,974 A10p.lep // lepton20160715.exe -unjailed -allowprogressive -multithread -skipvalidation A10p.jpg
    673,197 A10.pjg  // packJPG.exe A10.jpg 
    673,150 A10p.pjg // packJPG.exe A10p.jpg
    630,296 A10.jpg.paq8px  // paq8px_v100.exe -7 A10.jpg 
    772,785 A10p.jpg.paq8px // paq8px_v100.exe -7 A10p.jpg
    FWIW:
    Code:
    764,325 A10u.jpg  // jpegultrascan.pl -i -s -j -t -o mozjpegtran A10.jpg A10u.jpg     // standard JPEG
    736,479 A10ua.jpg // jpegultrascan.pl -a -i -s -j -t -o mozjpegtran A10.jpg A10ua.jpg // arithmetic coding

  35. #24
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,688
    Thanks
    264
    Thanked 1,180 Times in 651 Posts
    There're no recompressors for arithmetic-coded jpegs, so not much sense.
    paq8 also can't recompress progressive jpegs, but you can try 764k one with http://nishi.dreamhosters.com/u/lepton20170214.exe

  36. #25
    Member
    Join Date
    Jul 2016
    Location
    Russia
    Posts
    22
    Thanks
    13
    Thanked 9 Times in 7 Posts
    Shelwien, from lepton there are two versions - normal and "slow-best-ratio"

    672,242 A10p.lep // lepton-slow-best-ratio.exe -unjailed -allowprogressive A10p.jpg

    The lepton.exe binary makes .lep files that are fast to decode by using multithreading
    the lepton-slow-best-ratio.exe binary makes .lep files that are slightly smaller but may not be decoded in parallel
    But there is one problem, the binary works only with SSE4, and to run on processors below requires an Assembly for SSE2.
    Or opening through sde (sde.exe -- lepton-slow-best-ratio.exe ...) which greatly reduces the speed of execution

  37. #26
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,688
    Thanks
    264
    Thanked 1,180 Times in 651 Posts
    Just use packjpg then, lepton uses the same parser anyway.

  38. #27
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,688
    Thanks
    264
    Thanked 1,180 Times in 651 Posts
    built "lepton-slow-best-ratio" for SSE2, its simply solid compression, with MT disabled
    http://nishi.dreamhosters.com/u/lepton64_20170827.exe

  39. Thanks:

    zubzer0 (27th August 2017)

  40. #28
    Member
    Join Date
    Jul 2016
    Location
    Russia
    Posts
    22
    Thanks
    13
    Thanked 9 Times in 7 Posts
    Shelwien Fatal error. To test SSE2 on new processors , it is possible to simulate P4 Prescott (SSE3, EM64T)
    sde.exe -p4p -- lepton64_20170827.exe test.jpg

  41. Thanks:

    Shelwien (28th August 2017)

  42. #29
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,688
    Thanks
    264
    Thanked 1,180 Times in 651 Posts
    Reuploaded, seems to work with -p4p now.

  43. Thanks:

    zubzer0 (28th August 2017)

Similar Threads

  1. Large-scale Netflix test of x264, x265, and VP9
    By SolidComp in forum Data Compression
    Replies: 8
    Last Post: 25th October 2016, 18:58
  2. Why does BWT work on images?
    By m^2 in forum Data Compression
    Replies: 6
    Last Post: 21st September 2012, 04:46
  3. Detector for ex-JPEG images
    By Alexander Rhatushnyak in forum Data Compression
    Replies: 22
    Last Post: 13th September 2012, 04:18
  4. ISO images compression
    By Surfer in forum Data Compression
    Replies: 17
    Last Post: 24th March 2011, 23:16
  5. how the brain compresses images
    By willvarfar in forum Data Compression
    Replies: 12
    Last Post: 13th February 2011, 17:32

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •