Relative Entropy Coding for Learned Data Compression

Add to your list(s) Download to your calendar using vCal

• Greg Flamich, Engineering Department (Cambridge)
• Tuesday 23 May 2023, 13:00-14:00
• Lecture Theatre 2.

If you have a question about this talk, please contact Mateja Jamnik.

In recent years, machine learning (ML) ignited a revolution in data compression as researchers and engineers can now design codecs that learn how to encode information optimally from large datasets. These ML-based methods use deep generative models (DGM), such as variational autoencoders or diffusion models, to build a distribution over the data. DGMs generate data by simulating a sample from a simple latent distribution, such as a Gaussian, which they transform into a sample from the data distribution using a deep neural network. Hence, we can encode data by encoding the latent sample that generated it and using the DGM to reconstruct it. However, a surprising fact is that traditional methods for encoding the latent sample are suboptimal, and a far more efficient approach exists called relative entropy coding (REC).

In this talk, I will first give an overview of learned data compression and some issues it faces and use it to motivate REC . Then, I will present a simple REC algorithm, revealing a surprising equivalence between sampling and search. Finally, I will discuss the main limitations of current REC algorithms, which prevent their practical application so far and lay out some potential ways to resolve these limitations.

You can also join us on Zoom

This talk is part of the Artificial Intelligence Research Group Talks (Computer Laboratory) series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Artificial Intelligence Research Group Talks (Computer Laboratory)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge talks
• Chris Davis' list
• Department of Computer Science and Technology talks and seminars
• Guy Emerson's list
• Hanchen DaDaDash
• Interested Talks
• Lecture Theatre 2
• School of Technology
• Speech Seminars
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• rp587
• yk373's list

Note that ex-directory lists are not shown.