Non-asymptotic bounds for forward processes in denoising diffusions: Ornstein-Uhlenbeck is optimal

Add to your list(s) Download to your calendar using vCal

• Aleksandar Mijatovic (University of Warwick), Miha Bresar (University of Warwick)
• Monday 01 July 2024, 10:30-12:00
• External.

If you have a question about this talk, please contact nobody.

DML - Diffusions in machine learning: Foundations, generative models and non-convex optimisation

Denoising diffusion probabilistic models (DDPMs) represent a recent advance in generative modelling that has delivered state-of-the-art results across many domains of applications. Despite their success, a rigorous theoretical understanding of the error within these generative models, particularly the non-asymptotic bounds for the forward diffusion processes, remain scarce. Making minimal assumptions on the initial data distribution, allowing for example the manifold hypothesis, this paper presents explicit non-asymptotic bounds on the forward diffusion error in total variation (TV), expressed as a function of  the terminal time $T$. We parametrize an arbitrary data distribution in terms of the maximal distance $R$ between its modes and consider forward diffusions with additive and multiplicative noise. Our analysis rigorously proves that, under mild assumptions, the canonical choice of the Ornstein-Uhlenbeck (OU) process cannot be significantly improved in terms of reducing the terminal time $T$ as a function $R$ and  error tolerance $\eps>0$. We also establish a cut-off like phenomenon (as $R\to\infty$) for the convergence of an OU process to its invariant measure in TV, initialized at a multi-modal distribution with maximal mode distance $R$, typically present in DDP Ms.

This talk is part of the Isaac Newton Institute Seminar Series series.

This talk is included in these lists:

• All CMS events
• External
• Featured lists
• INI info aggregator
• Isaac Newton Institute Seminar Series
• School of Physical Sciences
• bld31

Note that ex-directory lists are not shown.