Abstract

Denoising diffusion probabilistic models (DDPMs) represent a recent advance in generative modelling that has delivered state-of-the-art results across many domains of applications. Despite their success, a rigorous theoretical understanding of the error within these generative models, particularly the non-asymptotic bounds for the forward diffusion processes, remain scarce. Making minimal assumptions on the initial data distribution, allowing for example the manifold hypothesis, this paper presents explicit non-asymptotic bounds on the forward diffusion error in total variation (TV), expressed as a function of  the terminal time $T$. We parametrize an arbitrary data distribution in terms of the maximal distance $R$ between its modes and consider forward diffusions with additive and multiplicative noise. Our analysis rigorously proves that, under mild assumptions, the canonical choice of the Ornstein-Uhlenbeck (OU) process cannot be significantly improved in terms of reducing the terminal time $T$ as a function $R$ and  error tolerance $\eps>0$. We also establish a cut-off like phenomenon (as $R\to\infty$) for the convergence of an OU process to its invariant measure in TV, initialized at a multi-modal distribution with maximal mode distance $R$, typically present in DDP Ms.