University of Cambridge > > Artificial Intelligence Research Group Talks (Computer Laboratory) > Relative Entropy Coding for Learned Data Compression

Relative Entropy Coding for Learned Data Compression

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Mateja Jamnik.

In recent years, machine learning (ML) ignited a revolution in data compression as researchers and engineers can now design codecs that learn how to encode information optimally from large datasets. These ML-based methods use deep generative models (DGM), such as variational autoencoders or diffusion models, to build a distribution over the data. DGMs generate data by simulating a sample from a simple latent distribution, such as a Gaussian, which they transform into a sample from the data distribution using a deep neural network. Hence, we can encode data by encoding the latent sample that generated it and using the DGM to reconstruct it. However, a surprising fact is that traditional methods for encoding the latent sample are suboptimal, and a far more efficient approach exists called relative entropy coding (REC).

In this talk, I will first give an overview of learned data compression and some issues it faces and use it to motivate REC . Then, I will present a simple REC algorithm, revealing a surprising equivalence between sampling and search. Finally, I will discuss the main limitations of current REC algorithms, which prevent their practical application so far and lay out some potential ways to resolve these limitations.

You can also join us on Zoom

This talk is part of the Artificial Intelligence Research Group Talks (Computer Laboratory) series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.


© 2006-2023, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity