Results 1 to 5 of 5

Thread: Exogenous decompression

  1. #1
    Member SolidComp's Avatar
    Join Date
    Jun 2015
    Location
    USA
    Posts
    353
    Thanks
    131
    Thanked 54 Times in 38 Posts

    Exogenous decompression

    Hi all – I've been thinking about exogenous decompression lately. I created this term to describe a scenario where a decompressor can better restore lossily compressed data by using exogenous information, such as characteristics of the encoder/compressor.

    For example, with a video stream maybe the decompressor knows which encoder was used and can use this information to achieve a better result than standard decompression. Maybe there are specific characteristics and quirks of the encoder that would be fruitful to know at decomp time.

    Does this sound feasible? This is slightly related to my idea to use exogenous metadata to better compress audio/music, but this is purely about decomp.

    I don't know enough about video codecs to know specifically what sort of information might be helpful.

    Relatedly, a Google project gave me an idea for video streams. When there's footage from, say, a security camera, perhaps it could be enhanced by knowing specifics about the camera model and its encoder. Even more interesting would be to:


    1. Take known objects, like employees, get them on the camera
    2. Relate that camera footage to high res imagery (video or photo) of those same employee faces
    3. Figure out how the camera encodes those known objects
    4. Use that to do the reverse – enhance camera footage of unknown targets (e.g. criminals) by understanding how the camera encodes known examples


    Thoughts?

  2. #2
    Member
    Join Date
    Dec 2011
    Location
    Cambridge, UK
    Posts
    506
    Thanks
    187
    Thanked 177 Times in 120 Posts
    There are various video players with options that attempt to correct for mpeg artifacts, eg ringing and blocking. That's essentially doing the same trick, but it's more of a format correction than an encoder specific correction.

    Edit: allegedly the Huawei P30 Pro moon pics are corrected for using predefined moons pics! That's a rather exceptional case of a rendering algorithm correcting for loss if so, but the company disputes it.

    https://www.androidauthority.com/huawei-p30-pro-moon-mode-controversy-978486/

    I have to laugh though: "To test out this theory, Yue photographed the moon from several different angles as well as photographed non-moon objects using Moon Mode." Different angles! Is he an astronaught? I guess he could have meant different phases and maybe it's just a bad translation.

  3. #3
    Member
    Join Date
    Aug 2014
    Location
    Argentina
    Posts
    538
    Thanks
    238
    Thanked 92 Times in 72 Posts
    This seems to be akin to the AI field. If I understand you correctly, you're not talking about some kind of off-line dictionary like BARF and its built-in Calgary Corpus but actual reconstruction AKA imagination. This off course can only be applied to lossy formats. There are several attempts to do so right now. Just google 'deep learning image reconstruction' or similar and you'll find quite a few.
    I suppose a properly trained network could learn exactly what artifacts are introduced by a particular camera model or codec and find a way to compensate for it. Here is one example of image reconstruction using neural networks, extracted from the article 30 AMAZING APPLICATIONS OF DEEP LEARNING:


    "

    3. Pixel restoration CSI style


    In the show CSI they often zoom into videos beyond the resolution of the actual video. This seemed completely unreliable and there are even a few videos on YouTube like the one below where people explain they don't watch CSI because that is unrealistic.



    Well, it was unrealistic until Deep Learning. Early in 2017, Google Brain researchers trained a Deep Learning network to take very low resolution images of faces and predict what each face most likely looks like. They call the method Pixel Recursive Super Resolution which enhances resolution of photos significantly. In the image below you can see the original 8x8 photos, the ground truth (which was the real face originally in the photos) and in the middle the guess of the computer. Obviously it is not perfect, as it cannot be, but it is pretty unbelievable that the computer can guesstimate so well many of the features of the person in the photo.
    8x8 pixel photos were inputted into a Deep Learning network which tried to guess what the original face looked like. As you can see it was fairly close (the correct answer is under "ground truth").


    "
    Last edited by Gonzalo; 12th August 2019 at 20:26. Reason: example

  4. #4
    Member
    Join Date
    Nov 2013
    Location
    Kraków, Poland
    Posts
    803
    Thanks
    244
    Thanked 255 Times in 159 Posts
    There are lots of options here, e.g.:
    1) rendering - decoders have models of among others actors and render entire movie - ultimate video compression,
    2) ML based upscaling, especially using GANs: upscaler trains to improve image such that discriminator cannot distinguish it from training dataset,
    3) lots of information theory-based possibilities, especially Wyner-Ziv ( https://en.wikipedia.org/wiki/Distributed_source_coding ) - encoder can build on information known only to decoder, for example:
    - https://en.wikipedia.org/wiki/Free_viewpoint_television - only decoder known in which direction the person is actually looking,
    - universal updates - encoder doesn't need to know versions of individual decoders,
    - entropy coding where only decoder knows statistical model.

    ps. Gonzalo, it is probably GAN-based trained on CelebA detaset - will generate nicely looking celebrity faces, but usually far from real face of common person. Like searching dataset for closest face.

  5. Thanks:

    SolidComp (15th August 2019)

  6. #5
    Member SolidComp's Avatar
    Join Date
    Jun 2015
    Location
    USA
    Posts
    353
    Thanks
    131
    Thanked 54 Times in 38 Posts
    Quote Originally Posted by JamesB View Post
    There are various video players with options that attempt to correct for mpeg artifacts, eg ringing and blocking. That's essentially doing the same trick, but it's more of a format correction than an encoder specific correction.

    Edit: allegedly the Huawei P30 Pro moon pics are corrected for using predefined moons pics! That's a rather exceptional case of a rendering algorithm correcting for loss if so, but the company disputes it.

    https://www.androidauthority.com/huawei-p30-pro-moon-mode-controversy-978486/

    I have to laugh though: "To test out this theory, Yue photographed the moon from several different angles as well as photographed non-moon objects using Moon Mode." Different angles! Is he an astronaught? I guess he could have meant different phases and maybe it's just a bad translation.
    Interesting, I had heard about the P30 having some unique features but not that.

Similar Threads

  1. JPG decompression
    By CompressMaster in forum Data Compression
    Replies: 6
    Last Post: 14th May 2019, 19:41
  2. May be this will accelerate LZ4 decompression?
    By lz77 in forum Data Compression
    Replies: 4
    Last Post: 14th November 2017, 10:26
  3. Decompression Help. lzna?
    By Ginsor in forum Data Compression
    Replies: 2
    Last Post: 1st September 2017, 19:14
  4. Decompression speed test
    By m^2 in forum Data Compression
    Replies: 8
    Last Post: 17th August 2010, 23:42
  5. Best decompression on embedded device
    By ikes in forum Data Compression
    Replies: 7
    Last Post: 16th February 2009, 21:05

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •