Geometry and Generalization: The case of Grokking

Add to your list(s) Download to your calendar using vCal

• Matthieu Tehenan, Department of Computer Sciences and Technology
• Tuesday 28 April 2026, 12:00-13:00
• S3.05, Simon Sainsbury Centre, Cambridge Judge Business School.

If you have a question about this talk, please contact Luning Sun.

Please contact the organiser if you'd like to attend the talk online.

Generalization in deep learning remains poorly understood, particularly in the feature-learning regime where representations are learned jointly with predictors. Classical statistical learning theory provides only limited explanations for the strong generalization observed in overparameterized models. The phenomenon of grokking, where models achieve identical training performance yet generalize at different times, offers a controlled setting in which to study the mechanisms underlying generalization. In this paper, we propose that generalization can be explained through the geometry of learned representations. For tasks with known algebraic structure, we show that the geometry of the ideal representation can be specified analytically prior to training, yielding what we call the geometry of the task. In modular arithmetic, this structure corresponds to the regular representation of the cyclic group, whose irreducible decomposition defines a hierarchy of low-dimensional subspaces. We hypothesize that generalization occurs when a model’s internal representations align with this universal geometry. We formalize this connection and derive bounds on the generalization gap in terms of the geometric deviation between learned and ideal representations. Experiments support this view, showing a critical threshold of geometric alignment beyond which generalization emerges and demonstrating how regularization encourages convergence toward the universal geometry.

This talk is part of the Cambridge Psychometrics Centre Seminars series.

This talk is included in these lists:

• Biology
• Cambridge Neuroscience Seminars
• Cambridge Psychometrics Centre Seminars
• Cambridge talks
• Chris Davis' list
• Department of Psychiatry talks stream
• Featured lists
• Life Science
• Life Sciences
• Neuroscience
• Neuroscience Seminars
• Neuroscience Seminars
• Psychology talks and events
• S3.05, Simon Sainsbury Centre, Cambridge Judge Business School
• Stem Cells & Regenerative Medicine
• Yishu's list
• dh539
• dh539

Note that ex-directory lists are not shown.