Double-descent curves in neural networks: A new perspective using Gaussian processes.

Add to your list(s) Download to your calendar using vCal

• Ouns El Harzli
• Tuesday 01 June 2021, 15:00-16:00
• https://cl-cam-ac-uk.zoom.us/j/96785077311?pwd=akljc1JQVS81R1FxelZCMUdQR3I2dz09.

If you have a question about this talk, please contact .

pwd: akljc1JQVS81R1FxelZCMUdQR3I2dz09

Double-descent curves in neural networks describe the phenomenon that the generalisation error initially descends with increasing parameters, then grows after reaching an optimal number of parameters which is less than the number of data points, but then descends again in the overparameterised regime. Here we use a neural network Gaussian process (NNGP) which maps exactly to a fully connected network (FCN) in the infinite width limit, combined with techniques from random matrix theory, to calculate this generalisation behaviour, with a particular focus on the overparameterised regime. We verify our predictions with numerical simulations of the corresponding Gaussian process regressions. An advantage of our NNGP approach is that the analytical calculations are easier to interpret. We argue that neural network generalization performance improves in the overparameterised regime precisely because that is where they converge to their equivalent Gaussian process.

This talk is part of the ML@CL Seminar Series series.

This talk is included in these lists:

• Hanchen DaDaDash
• ML@CL Seminar Series
• https://cl-cam-ac-uk.zoom.us/j/96785077311?pwd=akljc1JQVS81R1FxelZCMUdQR3I2dz09

Note that ex-directory lists are not shown.