BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//talks.cam.ac.uk//v3//EN
BEGIN:VTIMEZONE
TZID:Europe/London
BEGIN:DAYLIGHT
TZOFFSETFROM:+0000
TZOFFSETTO:+0100
TZNAME:BST
DTSTART:19700329T010000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0100
TZOFFSETTO:+0000
TZNAME:GMT
DTSTART:19701025T020000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CATEGORIES:Signal Processing and Communications Lab Seminars
SUMMARY:Information-theoretic perspectives on learning alg
orithms - Dr Varun Jog\, University of Wisconsin-M
adison
DTSTART;TZID=Europe/London:20180305T143000
DTEND;TZID=Europe/London:20180305T153000
UID:TALK102100AThttp://talks.cam.ac.uk
URL:http://talks.cam.ac.uk/talk/index/102100
DESCRIPTION:In statistical learning theory\, generalization er
ror is used to quantify the degree to which a supe
rvised machine learning algorithm may overfit to t
raining data. We overview some recent work [Xu and
Raginsky (2017)] that bounds generalization error
of empirical risk minimization based on the mutua
l information between the algorithm input and the
algorithm output. We leverage these results to der
ive generalization error bounds for a broad class
of iterative algorithms that are characterized by
bounded\, noisy updates with Markovian structure\,
such as stochastic gradient Langevin dynamics (SG
LD). We describe certain shortcomings of mutual in
formation-based bounds\, and propose alternate bou
nds that employ the Wasserstein metric from optima
l transport theory. We compare the Wasserstein met
ric-based bounds with the mutual information-based
bounds and show that for a class of data generati
ng distributions\, the former leads to stronger bo
unds on the generalization error.\n\nThis is joint
work with Adrian Tovar-Lopez\, Ankit Pensia\, and
Po-Ling Loh
LOCATION:LT6\, Baker Building\, CUED
CONTACT:Prof. Ramji Venkataramanan
END:VEVENT
END:VCALENDAR