BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Memorization and Regularization in Generative Diffusion Models - A
 ndrew Stuart (CALTECH (California Institute of Technology))
DTSTART:20250515T093000Z
DTEND:20250515T103000Z
UID:TALK230545@talks.cam.ac.uk
DESCRIPTION:Ricardo Baptista\, Agnimitra Dasgupta\, Nikola B. Kovachki\, A
 ssad Oberai\, Andrew M. Stuart\nDiffusion models have emerged as a powerfu
 l framework for generative modeling. At the heart of the methodology is sc
 ore matching: learning gradients of families of log-densities for noisy ve
 rsions of the data distribution at different scales. When the loss functio
 n adopted in score matching is evaluated using empirical data\, rather tha
 n the population loss\, the minimizer corresponds to the score of a time-d
 ependent Gaussian mixture. However\, use of this analytically tractable mi
 nimizer leads to data memorization: in both unconditioned and conditioned 
 settings\, the generative model returns the training samples. This talk ex
 plains the dynamical mechanism underlying memorization. The analysis highl
 ights the need for regularization to avoid reproducing the analytically tr
 actable minimizer\; and\, in so doing\, lays the foundations for a princip
 led understanding of how to regularize. Numerical experiments investigate 
 the properties of: (i) Tikhonov regularization\; (ii) regularization desig
 ned to promote asymptotic consistency\; and (iii) regularizations induced 
 by under-parameterization of a neural network or by early stopping when tr
 aining a neural network. These experiments are evaluated in the context of
  memorization\, and directions for future development of regularization ar
 e highlighted.
LOCATION:Seminar Room 2\, Newton Institute
END:VEVENT
END:VCALENDAR