University of Cambridge > Talks.cam > Statistics > Minimum L1-norm interpolators: Precise asymptotics and multiple descent

  Browse A-Z

Log in

Cambridge users (raven)details Other usersdetails No account?details

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

Minimum L1-norm interpolators: Precise asymptotics and multiple descent

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Qingyuan Zhao.

An evolving line of machine learning works observe empirical evidence that suggests interpolating estimators— the ones that achieve zero training error— may not necessarily be harmful. In this talk, we pursue theoretical understanding for an important type of interpolators: the minimum L1-norm interpolator, which is motivated by the observation that several learning algorithms favor low L1-norm solutions in the over-parameterized regime. Concretely, we consider the noisy sparse regression model under Gaussian design, focusing on linear sparsity and high-dimensional asymptotics (so that both the number of features and the sparsity level scale proportionally with the sample size).

We observe, and provide rigorous theoretical justification for, a curious multi-descent phenomenon; that is, the generalization risk of the minimum L1-norm interpolator undergoes multiple (and possibly more than two) phases of descent and ascent as one increases the model capacity. This phenomenon stems from the special structure of the minimum L1-norm interpolator as well as the delicate interplay between the over-parameterized ratio and the sparsity, thus unveiling a fundamental distinction in geometry from the minimum L2-norm interpolator. Our finding is built upon an exact characterization of the risk behavior, which is governed by a system of two non-linear equations with two unknowns.

This talk is part of the Statistics series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

Graduate Union talks PDN Talks Quastel Midgen LLP Presentation

Other talks

Host Age-dependent Evolution of a Plant RNA Virus Understanding Myc Transcription ‘The decolonisation of African states. Taxation and expenditure in former French Africa, 1900-2020’ Designing the BEARS (Both Ears) Virtual Reality Training Package to Improve Spatial Hearing in Young People with Bilateral Cochlear Implant Multivalency and Selective Binding Uniqueness of shock waves under small perturbations for the Isentropic Euler Equations
 

© 2006-2024 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity