BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Using A Function-Centric Lens to Re-consider Regularisation\, Repr
 esentation Transfer and Geometric Properties of Neural Networks    - Israe
 l Mason-Williams (Imperial/KCL)
DTSTART:20260522T110000Z
DTEND:20260522T120000Z
UID:TALK246697@talks.cam.ac.uk
CONTACT:Suchir Salhan
DESCRIPTION:Abstract: Neural networks have shown remarkable performance ac
 ross data domains\, especially in regimes of increasing compute budgets. H
 owever\, fundamental insights into how neural networks process information
 \, share representations and traverse loss landscapes remain uncertain. In
  this work\, we quantify the functional impact of distribution matching\, 
 facilitated by knowledge sharing mechanisms such as knowledge distillation
 \, under student-teacher optimisation strategies. Our empirical evaluation
  across modalities\, architectures and extensive hyperparameter settings s
 hows that the functional impact of distribution matching is far more nuanc
 ed than current literature would suggest. We unveil a fundamental property
  of negative asymmetric transfer\, which underpins logit matching optimisa
 tion\, calling for a reappraisal of logit matching as primarily a form of 
 regularisation rather than a beneficial or consistent knowledge transfer m
 echanism. Following this\, we explore geometric properties of neural netwo
 rks and how regularisation strategies modify internal representations of m
 odels and minima found at the end of training. From our function-centric l
 ens\, we provide empirical evidence from synthetic tasks to high-dimension
 al datasets that minima geometry represents decision boundary structure an
 d that generalised preferences for flat minima need to be reconsidered. As
  a result\, we can decouple the relationship between minima geometry\, gen
 eralisation\, and memorisation to understand how different inductive biase
 s and regularisation strategies improve performance on different data dist
 ributions.
LOCATION:SS03 Hybrid (In-Person + Online). Here is the Google Meet Link: h
 ttps://meet.google.com/cru-hcuo-rhu
END:VEVENT
END:VCALENDAR