BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//talks.cam.ac.uk//v3//EN
BEGIN:VTIMEZONE
TZID:Europe/London
BEGIN:DAYLIGHT
TZOFFSETFROM:+0000
TZOFFSETTO:+0100
TZNAME:BST
DTSTART:19700329T010000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0100
TZOFFSETTO:+0000
TZNAME:GMT
DTSTART:19701025T020000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CATEGORIES:Isaac Newton Institute Seminar Series
SUMMARY:Emergence of heavy tails in homogenised stochastic
  gradient descent - Martin Keller-ressel (Technisc
 he Universität Dresden)
DTSTART;TZID=Europe/London:20240425T151500
DTEND;TZID=Europe/London:20240425T160000
UID:TALK214234AThttp://talks.cam.ac.uk
URL:http://talks.cam.ac.uk/talk/index/214234
DESCRIPTION:We analyze a continuous diffusion approximation of
  SGD\, called homogenized stochastic gradient desc
 ent\, show that it behaves asymptotically heavy-ta
 iled\, and give explicit upper and lower bounds on
  its tail-index. We validate these bounds in numer
 ical experiments and show that they are typically 
 close approximations to the empirical tail-index o
 f SGD iterates. In addition\, their explicit form 
 enables us to quantify the interplay between optim
 ization hyperparameters and the tail-index. Our re
 sults show that also continuous diffusions\, not o
 nly L&eacute\;vy-driven SDEs\, can accurately repr
 esent the emergence of heavy tails in SGD. In addi
 tion\, our results suggest skew Student-t-distribu
 tions\, not alpha-stable distributions\, as surrog
 ates of parameter distributions under SGD.\n&nbsp\
 ;\n&nbsp\;
LOCATION:External
CONTACT:
END:VEVENT
END:VCALENDAR