The space of let say images in everyday applications is much smaller than of all bitmaps – restricting to such subspace allows to define one of them with a much smaller number of bits – allows for efficient data compression.
However, this “space of everyday images” is extremely complex, in practice defined by huge datasets. There are now machine learning approaches allowing to work on such complex subspaces defined by datasets like GAN or VAE.
Recently, data compressors based on them have starting appearing – often claiming huge gains, what might be extremely important in near future (unless patents will make them impossible to use :/ ) as e.g. video is about half of internet traffic.
There is needed a thread to generally gather and discuss them – please post your thoughts here, papers, compressors, approaches etc. I will try to update this top post.
Probably the most promising are GANs (generative adversarial network), especially closed-source (patents?) WaveOne: https://arxiv.org/pdf/1705.05823.pdf
Here is its open-source alternative: https://arxiv.org/pdf/1804.02958.pdf
https://github.com/Justin-Tan/generative-compression
It is kind of wavelet transform learned on the dataset: there is pyramidal decomposition of blocks into their downscaled versions, in each scale there are extracted features using a convolution. Their parameters are optimized (on dataset) to minimize not only error of such final process: encoding-quantization-decoding, but additionally that adversary (separate NN) cannot distinguish them from the learned dataset.
Another approach is VAE (variational autoencoder) – recent paper claims its superiority for MNIST: https://openreview.net/pdf?id=ryE98iR5tm
https://github.com/bits-back/bits-back
It wants to directly represent the “space of everyday images” as a lower dimensional latent space – assuming e.g. Gaussian distribution there. Encoder can use a pseudorandom value of latent variable by traveling in both directions on LIFO entropy coder.
What are other interesting ML-based approaches trying to encode within learned subspace of everyday data?