COOKIES: By using this website you agree that we can place Google Analytics Cookies on your device for performance monitoring. |

University of Cambridge > Talks.cam > Isaac Newton Institute Seminar Series > From Mixing Time to Sample Complexity: Elucidating the Design Space of Score-Based Losses

## From Mixing Time to Sample Complexity: Elucidating the Design Space of Score-Based LossesAdd to your list(s) Download to your calendar using vCal - Andrej Risteski (Carnegie Mellon University)
- Friday 05 July 2024, 10:30-12:00
- External.
If you have a question about this talk, please contact nobody. DML - Diffusions in machine learning: Foundations, generative models and non-convex optimisation Score-based losses have emerged as a more computationally appealing alternative to maximum likelihood for fitting (probabilistic) generative models with an intractable likelihood (for example, energy-based models and diffusion models). What is gained by foregoing maximum likelihood is a tractable gradient-based training algorithm. What is lost is less clear: in particular, since maximum likelihood is asymptotically optimal in terms of statistical efficiency, how suboptimal are score-based losses? I will survey a recently developing connection relating the statistical efficiency of broad families of generalized score losses, to the algorithmic efficiency of a natural inference-time algorithm: namely, the mixing time of a suitable diffusion using the score that can be used to draw samples from the model. This “dictionary” allows us to elucidate the design space for score losses with good statistical behavior, by “translating” techniques for speeding up Markov chain convergence (e.g., preconditioning and lifting). I will also touch upon a parallel story for learning discrete probability distributions, in which the “analogue” of score-based losses is played by masked prediction-like losses. Based in part on https://arxiv.org/abs/2210.00726, https://arxiv.org/abs/2306.09332, https://arxiv.org/abs/2306.01993. This talk is part of the Isaac Newton Institute Seminar Series series. ## This talk is included in these lists:- All CMS events
- External
- Featured lists
- INI info aggregator
- Isaac Newton Institute Seminar Series
- School of Physical Sciences
- bld31
Note that ex-directory lists are not shown. |
## Other listsCambridge University Raja Yoga Meditation Society Developmental Biology Seminar Series Cambridge Risk and Uncertainty Series## Other talksRandom Walk on the symmetric Exclusion process Dimensionless Approaches to Identifying Anomalous Streams with Rough Path Theory: with Applications Riemann-Hilbert Approach to the Elastodynamic Equation. Quarter Plane. Connection formulae between the Rayleigh waves along vertical and horizontal parts of the border. A scientists guide to the art of radiation therapy and Neurosurgery for the oncologist Approximate solutions to Wiener-Hopf equations via the implicit quadrature scheme ECFP: At the Interface of Academia and Industry |