Compositional mathematics and automatic gradient descent

Add to your list(s) Download to your calendar using vCal

• Jeremy Bernstein, MIT
• Friday 14 July 2023, 11:00-12:00
• Cambridge University Engineering Department, CBL Seminar room BE4-38..

If you have a question about this talk, please contact Dr R.E. Turner.

At its heart, deep learning involves composing operators and iteratively perturbing their weights. But we lack the mathematical tools needed to understand how compound operators behave under perturbation. As a consequence, our state-of-the-art training algorithms can be brittle and require manual tuning to work well. In this talk, we propose a new suite of mathematical tools for dealing with compound operators. This includes a new chain rule for understanding how “smoothness” or “linearisation error” behaves under composition, and also perturbation bounds for compound operators. We assemble these tools and apply the majorise-minimise principle to derive “automatic gradient descent”. AGD is a hyperparameter-free training algorithm for deep neural networks that has been validated at ImageNet scale and has the potential to further automate machine learning workflows.

This talk is part of the Machine Learning @ CUED series.

This talk is included in these lists:

• Seminar
• All Talks (aka the CURE list)
• Biology
• CBL important
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge Neuroscience Seminars
• Cambridge University Engineering Department, CBL Seminar room BE4-38.
• Cambridge talks
• Chris Davis' list
• Creating transparent intact animal organs for high-resolution 3D deep-tissue imaging
• Featured lists
• Guy Emerson's list
• Hanchen DaDaDash
• Inference Group Summary
• Information Engineering Division seminar list
• Interested Talks
• Joint Machine Learning Seminars
• Life Science
• Life Sciences
• ML
• Machine Learning @ CUED
• Machine Learning Summary
• Neuroscience
• Neuroscience Seminars
• Neuroscience Seminars
• Required lists for MLG
• Simon Baker's List
• Stem Cells & Regenerative Medicine
• Trust & Technology Initiative - interesting events
• bld31
• dh539
• kt532's list
• ndk22's list
• ob366-ai4er
• rp587
• yk373's list

Note that ex-directory lists are not shown.