BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:From Mixing Time to Sample Complexity: Elucidating the Design Spac
 e of Score-Based Losses - Andrej Risteski (Carnegie Mellon University)
DTSTART:20240705T093000Z
DTEND:20240705T110000Z
UID:TALK217954@talks.cam.ac.uk
DESCRIPTION:Score-based losses have emerged as a more computationally appe
 aling alternative to maximum likelihood for fitting (probabilistic) genera
 tive models with an intractable likelihood (for example\, energy-based mod
 els and diffusion models). What is gained by foregoing maximum likelihood 
 is a tractable gradient-based training algorithm. What is lost is less cle
 ar: in particular\, since maximum likelihood is asymptotically optimal in 
 terms of statistical efficiency\, how suboptimal are score-based losses?\n
 I will survey a recently developing connection relating the&nbsp\;statisti
 cal efficiency&nbsp\;of broad families of generalized score losses\, to th
 e&nbsp\;algorithmic efficiency&nbsp\;of a natural inference-time algorithm
 : namely\, the mixing time of a suitable diffusion using the score that ca
 n be used to draw samples from the model. This &ldquo\;dictionary&rdquo\; 
 allows us to elucidate the design space for score losses with good statist
 ical behavior\, by &ldquo\;translating&rdquo\; techniques for speeding up 
 Markov chain convergence (e.g.\, preconditioning and lifting). I will also
  touch upon a parallel story for learning discrete probability distributio
 ns\, in which the "analogue" of score-based losses is played by masked pre
 diction-like losses.&nbsp\;\nBased in part on https://arxiv.org/abs/2210.0
 0726\,&nbsp\;https://arxiv.org/abs/2306.09332\,&nbsp\;https://arxiv.org/ab
 s/2306.01993.&nbsp\; &nbsp\;
LOCATION:External
END:VEVENT
END:VCALENDAR