Results 1 to 13 of 13

Thread: how the brain compresses images

  1. #1
    Member
    Join Date
    Feb 2010
    Location
    Nordic
    Posts
    200
    Thanks
    41
    Thanked 36 Times in 12 Posts

    how the brain compresses images

    a popular-science intro to some newly published research: http://www.theregister.co.uk/2011/02/11/brain_jpegs/
    what would be interesting is lossy compression that favoured this, or as input into object recognition problems. But perhaps this is all so obvious everyone is already on top of it.

  2. #2
    Member
    Join Date
    Feb 2011
    Location
    St. Albans, England
    Posts
    20
    Thanks
    0
    Thanked 0 Times in 0 Posts
    The brain uses a "modelling" method of remembering things. We all know what a woman likes without clothing, but we "forget" about the little details such as moles, variants in skin tones & such - unless it's one of them hairy moles, which the brain tends to record. We know the shape (model), we just don't recall ALL the details in precision. Likewise, we remember what the ball we throw at the dog looks like, but we tend to ignore a lot of the detail (exact teeth marks, imperfections in the colouring and such). However, we don't eliminate ALL the imperfections.

    I agree that an encoding method would be nice that could break down the "basic" shapes found in an image, eg: squares, circles, triangles etc. and record a certain amount of the "imperfections" of the image.

    I deal a lot with 3D renderings. The biggest challenge in that is creating imperfections in the image. A good example being the skin, which is NEVER perfect. So an image of someone with great skin looks like what we want people to see, but most of the time it's "hmm, there's something about this image that's not quite right" - we get that "niggling" feeling something is missing. So we spend a LOT of time creating tonnes of files with "imperfect" skin, and use that texture on the final passes. It takes me hours on perfecting the imperfections, and i've only had a few "good" results so far. I ramble again.

    An engine that not only breaks down the shapes, but can recall certain degrees of imperfection (the lossy part) would be a GREAT boon to the art world in general. To my knowledge there's no such beast as yet. No doubt many have thought of it though.

    Hope that made some kind of sense.

  3. #3
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 779 Times in 486 Posts
    Very interesting. Landauer's studies of long term memory ( http://csjarchive.cogsci.rpi.edu/198...p0493/MAIN.PDF ) suggest we remember at a rate of around 2 bits per second. So in theory, you could compress video to this rate using a model of your brain (which of course we don't have). One way to do this would be to encode a movie as a script and compress it. The decompresser would have to be able to read the script and construct a movie close enough to the original that you wouldn't notice the difference. It's an AI problem.

    We do have models of the simpler or lower layers of perception, which lossy compressors already exploit. For example, we can quantize audio samples on a logarithmic scale to match the response of our ears.

  4. #4
    Member
    Join Date
    Feb 2011
    Location
    St. Albans, England
    Posts
    20
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Actually - i'll continue my rambling for a bit as I think this is "kind of" relevant.

    I use Poser for 3D rendering. The file sizes it uses is hideously over bloated for various reasons (mostly badly encoded JPEG/PNGs). A single model can contain up to 3 x 4000x4000 JPEG files which are the main texture itself, a bump map (greyscale map showing where the detail is raised in the texture), and a transparency map (greyscale showing where opacity is used). Sometimes there's a template file too. All these JPEGs are USUALLY similar in shape - so an engine could recognise the similar models in the files, and create a better compressed file - very much similar to encoding a film.

    Another sore point in these images is "eyes" for example. Usually the only difference is the colouring of the iris - however each one creates a file that is over 1Mb for such a tiny bit of change.

    Maybe I should be thinking along the lines of "motion-picture" type encoding on these though ?

    [quick edit]
    Seems I posted this just before I saw Matts post above. For a single file, not sure if it's worthwhile, but in a "collection" of images it might be VERY useful indeed. Certainly worth thinking about.
    Last edited by EwenG; 11th February 2011 at 21:29.

  5. #5
    Member
    Join Date
    Jun 2009
    Location
    Kraków, Poland
    Posts
    1,474
    Thanks
    26
    Thanked 121 Times in 95 Posts
    Matt:
    I think the problem is that human focuses on some very small set of details. And every time on a different set. So using brain compression we can store something like memories, not actual sensations.

  6. #6
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 779 Times in 486 Posts
    True. To get 2 bps video the compressor would need to know what part of the video you are likely to pay attention to and remember. Each person would need a custom compressor based on a model of their brain. Not likely anytime soon.

    Still, I think there is much room for improvement using AI techniques. Speech could be compressed to 10 bps by converting it to text if we had better speech recognition and speech synthesis. Both are hard problems.

  7. #7
    Member
    Join Date
    Feb 2010
    Location
    Nordic
    Posts
    200
    Thanks
    41
    Thanked 36 Times in 12 Posts
    random thought:
    1. record audio
    2. speech to text conversion and timing data
    3. then text to speech conversion
    4. then store the text, the timing data and a lossy difference between the text-to-speech conversion and the recorded audio.

    Is this approach been tried? Any bad results to dissect?

  8. #8
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,322
    Thanks
    209
    Thanked 1,001 Times in 526 Posts
    @willvarfar: So what you're expecting to get like that? If its perfect, maybe the listener won't notice that speech is synthesized,
    and would acquire the same information as with original record.
    Current implementations are far from that, though.
    But its also kinda obvious that its not a matter of "lossy difference" - you won't be able to align the text-to-speech signal
    with original so easily, and the difference would be likely less compressible than original.

    Well, it may be possible to use some kind of phonetic transcription as a context for compression of speech signal,
    which is somewhat similar to what you suggest, but imho syntesized speech alone would be much less useful
    as a context than transcription.

    And as to differences, I think you can test it with a simpler framework first - get a good modern speech codec
    to encode/decode your recording, then subtract it from original and try compressing it.
    (there're console codecs like that in Intel IPP if you don't know what to use).

  9. #9
    Member
    Join Date
    May 2008
    Location
    brazil
    Posts
    163
    Thanks
    0
    Thanked 3 Times in 3 Posts
    Quote Originally Posted by Matt Mahoney View Post
    True. To get 2 bps video the compressor would need to know what part of the video you are likely to pay attention to and remember. Each person would need a custom compressor based on a model of their brain. Not likely anytime soon.

    Still, I think there is much room for improvement using AI techniques. Speech could be compressed to 10 bps by converting it to text if we had better speech recognition and speech synthesis. Both are hard problems.
    I think this is not a good way to follow. I believe the language is a simple subset what we call "image" . "A" is just a image.

    The good way is use more image communication than trying to convert everything for language.

    OFF:Extraterrestrials use image-like languages!

    Someday I will se a very easy computer language which uses only images to program.

  10. #10
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,322
    Thanks
    209
    Thanked 1,001 Times in 526 Posts
    > Someday I will se a very easy computer language which uses only images to program.

    http://en.wikipedia.org/wiki/Piet_(programming_language)

    The whole idea with images for communication is really bothersome though, imho.
    Like when people make GUIs with colorful icons, but there's no way to find out what they do without clicking them all.

  11. #11
    Member
    Join Date
    May 2008
    Location
    brazil
    Posts
    163
    Thanks
    0
    Thanked 3 Times in 3 Posts
    Quote Originally Posted by Shelwien View Post
    > Someday I will se a very easy computer language which uses only images to program.

    http://en.wikipedia.org/wiki/Piet_(programming_language)

    The whole idea with images for communication is really bothersome though, imho.
    Like when people make GUIs with colorful icons, but there's no way to find out what they do without clicking them all.
    Yes,but images is a LOT more expressive and produces a easy way to program things very quickly .

    Piet is very simplistic .It's a 2d language. But it's a good progress because it includes color . And does not use lot of geometry like cubes, spheres and all 3d forms.

    Textual Programming languages should include color in syntax.

  12. #12
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,322
    Thanks
    209
    Thanked 1,001 Times in 526 Posts
    > Yes,but images is a LOT more expressive and produces a easy way to program things very quickly .

    1. google defines expressive as "Effectively conveying thought or feeling".
    And in compression context "effective" is supposed to mean shorter description length.
    Now, 16x16 RGB icon is 768 bytes.
    Do you really think that 768 bytes of english plaintext is less expressive than a 16x16 icon?

    2. There're too many compatibility issues. People use XML instead of binary container formats
    (eg. RIFF is flexible enough to replace XML, and would be much more compact) because there're
    issues with supported charsets in viewers, editors, and even communication protocols.
    And there're even worse "compatibility issues" - http://en.wikipedia.org/wiki/Color_b...s#Epidemiology

  13. #13
    Member
    Join Date
    May 2008
    Location
    brazil
    Posts
    163
    Thanks
    0
    Thanked 3 Times in 3 Posts
    Quote Originally Posted by Shelwien View Post
    > Yes,but images is a LOT more expressive and produces a easy way to program things very quickly .

    1. google defines expressive as "Effectively conveying thought or feeling".
    And in compression context "effective" is supposed to mean shorter description length.
    Now, 16x16 RGB icon is 768 bytes.
    Do you really think that 768 bytes of english plaintext is less expressive than a 16x16 icon?

    2. There're too many compatibility issues. People use XML instead of binary container formats
    (eg. RIFF is flexible enough to replace XML, and would be much more compact) because there're
    issues with supported charsets in viewers, editors, and even communication protocols.
    And there're even worse "compatibility issues" - http://en.wikipedia.org/wiki/Color_b...s#Epidemiology


    There are problems of course,but using colors and forms to program computers helps designers create programs easily.

    SVG for example is a lot more easy to draw a circle using a illustrator than writing in a xml file.

Similar Threads

  1. New lossless compressor for 24-bit images (3 channels, 8 bits per channel)
    By Alexander Rhatushnyak in forum Data Compression
    Replies: 28
    Last Post: 23rd September 2010, 02:43
  2. Images PreProcessor - PrePNG
    By PAQer in forum Data Compression
    Replies: 3
    Last Post: 21st May 2010, 13:21
  3. Idea for raising compression efficiency on disk images
    By Mexxi in forum Data Compression
    Replies: 10
    Last Post: 18th February 2010, 05:56
  4. Precompression of Tiff Images
    By Simon Berger in forum Data Compression
    Replies: 52
    Last Post: 8th May 2009, 01:14
  5. Compression used in .wim (Vista) images
    By jaclaz in forum Data Compression
    Replies: 9
    Last Post: 29th August 2008, 19:05

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •