BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Information-theoretic perspectives on learning algorithms - Varun 
 Jog (University of Wisconsin-Madison)
DTSTART:20180222T110000Z
DTEND:20180222T120000Z
UID:TALK101302@talks.cam.ac.uk
CONTACT:INI IT
DESCRIPTION:In statistical learning theory\, generalization error is used 
 to quantify the degree to which a supervised machine learning algorithm ma
 y overfit to training data. We overview some recent work [Xu and Raginsky 
 (2017)] that bounds generalization error of empirical risk minimization ba
 sed on the mutual information I(S\;W) between the algorithm input S and th
 e algorithm output W. We leverage these results to derive generalization e
 rror bounds for a broad class of iterative algorithms that are characteriz
 ed by bounded\, noisy updates with Markovian structure\, such as stochasti
 c gradient Langevin dynamics (SGLD). We describe certain shortcomings of m
 utual information-based bounds\, and propose alternate bounds that employ 
 the Wasserstein metric from optimal transport theory. We compare the Wasse
 rstein metric-based bounds with the mutual information-based bounds and sh
 ow that for a class of data generating distributions\, the former leads to
  stronger bounds on the generalization error.  <br><br><br><br>
LOCATION:Seminar Room 2\, Newton Institute
END:VEVENT
END:VCALENDAR