Results 1 to 4 of 4

Thread: Optimizing image compression codecs for object recognition / a 21st century format

  1. #1
    Member SolidComp's Avatar
    Join Date
    Jun 2015
    Location
    USA
    Posts
    238
    Thanks
    95
    Thanked 47 Times in 31 Posts

    Optimizing image compression codecs for object recognition / a 21st century format

    Hi all,

    Here's an interesting article
    about moving object recognition neural networks directly into a camera's optics.

    This reminded me that our next-generation image format should be optimized for object recognition, machine vision, etc. Among other things, that is. We still need better compression, fast encode and decode, etc.

    I think that if you're working on a new image format in 2018, you should think about how machine vision and object recognition work, and the various ways an image format could make those workloads easier.

    I also think that it's time for a new acquisition format. It's time to replace JPEG. Recent formats like webp have been focused on the web, but it would be better if a new image format was designed as an acquisition format – the format the camera would encode when a photo is taken. It will be easier to replace JPEG and get wide adoption if a new image format were carefully and rigorously designed to replace JPEG at every stage, from acquisition, professional work, the web, etc.

    And a new format should be GPU-only or GPU-accelerated. The platforms we're going to care about will all have GPUs. We don't need to worry about really old devices – they can continue to use JPEG or webp. GPU formats will be much more efficient than CPU-driven formats. It's such a waste of energy to use CPUs for imaging. (The cameras themselves use ASICs, basically, to encode. And if there's any way to design an image format such that it's easier or cheaper to build ASICs for, we should do that.)

    If all these goals are compatible, great. If not, oh well. But I think Google, Apple, Microsoft, and others (Dropbox?) should form a team that is committed to excellence, a team that is driven to develop a great new image format for the 21st century, a team that will start with a clean sheet and kick ass. We need this because we're wasting too much bandwidth and storage with JPEG. The small, half-hearted efforts we've seen so far are unlikely to result in a format that replaces JPEG, and therefore will leave us with lots of wasted storage space and bandwidth.

  2. #2
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    704
    Thanks
    210
    Thanked 267 Times in 157 Posts
    Quote Originally Posted by SolidComp View Post
    If not, oh well. But I think Google, Apple, Microsoft, and others ...
    This team exists. Their work is pretty good and ready. It is called av1. It is complex enough that in practice it can be run only with special hardware, but not with gpus as far as I know. Special hardware can be more energy efficient than either gpu or cpu.

    The deep learning research that you can read about usually comes with a small model of 777 MB to 3.33 GB that is necessary for decoding, and covers only one domain area such as cityscapes from Stockholm in spring 2017.
    Last edited by Jyrki Alakuijala; 23rd August 2018 at 21:28.

  3. #3
    Member SolidComp's Avatar
    Join Date
    Jun 2015
    Location
    USA
    Posts
    238
    Thanks
    95
    Thanked 47 Times in 31 Posts
    Quote Originally Posted by Jyrki Alakuijala View Post
    This team exists. Their work is pretty good and ready. It is called av1. It is complex enough that in practice it can be run only with special hardware, but not with gpus as far as I know. Special hardware can be more energy efficient than either gpu or cpu.

    The deep learning research that you can read about usually comes with a small model of 777 MB to 3.33 GB that is necessary for decoding, and covers only one domain area such as cityscapes from Stockholm in spring 2017.
    Didn't you say something about how still images from AV1 are not that good? Or was that HEVC?

    What about PIK? Is it based on AV1?

  4. #4
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    704
    Thanks
    210
    Thanked 267 Times in 157 Posts
    Quote Originally Posted by SolidComp View Post
    Didn't you say something about how still images from AV1 are not that good? Or was that HEVC?

    What about PIK? Is it based on AV1?
    AV1 and HEVC are pretty good on low BPP (around ~0.5 BPP).

    PIK seems to work better than AV1 and HEVC at high BPP (~1.0 BPP and above).

    PIK is much faster to decode on software.

  5. Thanks:

    SolidComp (25th August 2018)

Similar Threads

  1. PIK image format
    By Jyrki Alakuijala in forum Data Compression
    Replies: 72
    Last Post: 30th May 2019, 00:39
  2. Psychovisual analysis on modern lossy image codecs
    By khavish in forum Data Compression
    Replies: 94
    Last Post: 17th August 2018, 08:17
  3. New video, image and audio codecs you might not be aware of
    By spaceship9876 in forum Data Compression
    Replies: 15
    Last Post: 26th February 2018, 14:36
  4. OLE Object optimizing: looking for cmd tools
    By Psykauze in forum Data Compression
    Replies: 5
    Last Post: 26th November 2017, 12:41
  5. Psychovisual measurements on modern image codecs
    By khavish in forum Data Compression
    Replies: 28
    Last Post: 28th August 2017, 03:41

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •