Mean-field dynamics and training of deep transformers

Add to your list(s) Download to your calendar using vCal

• Christoph Reisinger (University of Oxford)
• Monday 10 November 2025, 15:50-16:30
• Seminar Room 1, Newton Institute.

If you have a question about this talk, please contact nobody.

SCLW01 - Bridging Stochastic Control And Reinforcement Learning: Theories and Applications

In this talk, we will examine continuous limits of transformer architectures, which form the basis of common generative models. There is rich literature on the limiting behaviour of neural networks, including for large width of single layer neural networks (mean-field analysis) and large depth of residual neural networks (neural ODE and SDE analysis). Here, we consider limits of transformers with attention and scaling for a large number of layers, tokens, and attention heads. The analysis reveals that for plausible training outputs, a McKean—Vlasov limit with or without diffusive common noise results. Joint work with William Gibson.

This talk is part of the Isaac Newton Institute Seminar Series series.

This talk is included in these lists:

• All CMS events
• Featured lists
• INI info aggregator
• Isaac Newton Institute Seminar Series
• School of Physical Sciences
• Seminar Room 1, Newton Institute
• bld31

Note that ex-directory lists are not shown.