A Mixture-based framework for guiding Diffusion models

Add to your list(s) Download to your calendar using vCal

• Alain Oliviero-Durmus - Ecole Polytechnique
• Wednesday 21 May 2025, 15:00-16:00
• CBL Seminar Room.

If you have a question about this talk, please contact .

Inverse problems—such as reconstructing images from partial or noisy measurements, or separating individual sources from mixed signals—are inherently challenging due to their ill-posed nature. In such settings, Bayesian inference, when combined with generative modelling, provides a systematic and principled approach. By using generative models trained on representative data distributions, these methods incorporate meaningful prior knowledge, which can then be integrated with the likelihood function describing the observed data. This leads to a posterior distribution, whose samples represent plausible solutions that harmonize both the observed data and prior assumptions. In recent developments, diffusion models have emerged as state-of-the-art generative models, demonstrating exceptional capabilities in image and audio generation tasks. Diffusion models function by first progressively adding noise to data samples through a forward diffusion process, ultimately converting them into pure Gaussian noise. The generative model is then trained to reverse this noising process, effectively learning to reconstruct original data from noise. While diffusion models provide powerful priors, directly using them for inverse problems typically requires constructing a posterior denoiser that blends this learned prior with the gradient of the log-likelihood function derived from the observations. However, existing posterior sampling methods for diffusion models often rely on crude approximations of the likelihood gradient and require significant heuristic tuning and adjustments specific to each task. In this talk, I will introduce a novel principled approach specifically designed to overcome these limitations. The core contribution of this approach is the construction of a mixture approximation of intermediate posterior distributions defined by the diffusion model. The sampling is carried out sequentially via Gibbs sampling, a Markov Chain Monte Carlo method, using a careful data augmentation scheme. Gibbs sampling is employed here due to its simplicity and theoretical guarantees, allowing for exact conditional updates at each iteration, thus ensuring stability and efficiency. One key advantage of the presented algorithm is its flexibility: it adapts to varying levels of computational resources by adjusting the number of Gibbs iterations. Consequently, substantial performance gains can be achieved by increasing inference-time computational effort. I will present extensive experimental results demonstrating strong empirical performance across ten diverse image restoration tasks, involving both pixel-space and latent-space diffusion models, and showcase its successful application in musical source separation.

This talk is part of the Cambridge Ellis Unit series.

This talk is included in these lists:

• CBL Seminar Room
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Ellis Unit
• Cambridge talks
• Chris Davis' list
• Hanchen DaDaDash
• Information Engineering Division seminar list
• Interested Talks
• Trust & Technology Initiative - interesting events
• bld31
• my List
• ndk22's list
• ob366-ai4er
• rp587

Note that ex-directory lists are not shown.