BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//talks.cam.ac.uk//v3//EN
BEGIN:VTIMEZONE
TZID:Europe/London
BEGIN:DAYLIGHT
TZOFFSETFROM:+0000
TZOFFSETTO:+0100
TZNAME:BST
DTSTART:19700329T010000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0100
TZOFFSETTO:+0000
TZNAME:GMT
DTSTART:19701025T020000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CATEGORIES:Statistics
SUMMARY:Minimum L1-norm interpolators: Precise asymptotics
and multiple descent - Yuting Wei (University of
Pennsylvania)
DTSTART;TZID=Europe/London:20220506T140000
DTEND;TZID=Europe/London:20220506T150000
UID:TALK173333AThttp://talks.cam.ac.uk
URL:http://talks.cam.ac.uk/talk/index/173333
DESCRIPTION:An evolving line of machine learning works observe
empirical evidence that suggests interpolating es
timators --- the ones that achieve zero training e
rror --- may not necessarily be harmful. In this t
alk\, we pursue theoretical understanding for an i
mportant type of interpolators: the minimum L1-nor
m interpolator\, which is motivated by the observa
tion that several learning algorithms favor low L1
-norm solutions in the over-parameterized regime.
Concretely\, we consider the noisy sparse regressi
on model under Gaussian design\, focusing on linea
r sparsity and high-dimensional asymptotics (so th
at both the number of features and the sparsity le
vel scale proportionally with the sample size).\n\
nWe observe\, and provide rigorous theoretical jus
tification for\, a curious multi-descent phenomeno
n\; that is\, the generalization risk of the minim
um L1-norm interpolator undergoes multiple (and po
ssibly more than two) phases of descent and ascent
as one increases the model capacity. This phenome
non stems from the special structure of the minimu
m L1-norm interpolator as well as the delicate int
erplay between the over-parameterized ratio and th
e sparsity\, thus unveiling a fundamental distinc
tion in geometry from the minimum L2-norm interpol
ator. Our finding is built upon an exact character
ization of the risk behavior\, which is governed b
y a system of two non-linear equations with two un
knowns.
LOCATION:MR12\, Centre for Mathematical Sciences
CONTACT:Qingyuan Zhao
END:VEVENT
END:VCALENDAR