COOKIES: By using this website you agree that we can place Google Analytics Cookies on your device for performance monitoring. |
Online Meta-LearningAdd to your list(s) Download to your calendar using vCal
If you have a question about this talk, please contact Carl Edward Rasmussen. We study the problem in which a series of learning tasks are observed sequentially and the goal is to incrementally adapt a learning algorithm in order to improve its performance on future tasks. The tasks are sampled from a meta-distribution, called the environment in the learning-to-learn literature (Baxter 2000). We focus on linear learning algorithms based on regularized empirical risk minimization such as ridge regression or support vector machines. The algorithms are parametrized by either a (representation) matrix applied to the raw inputs or by a bias vector used in the regularizer. In both settings, we develop a computational efficient meta-algorithm to incrementally adapt the learning algorithm after a task dataset is observed. The meta-algorithm performs stochastic gradient descent on a proxy objective of the risk of the learning algorithm. We derive bounds on the performance of the meta-algorithm, measured by the average risk of the learning algorithm on random tasks from the environment. Our analysis leverages ideas from multitask learning and learning-to-learn with tools from online learning and stochastic optimization. In the last part of the talk, we discuss extensions of the framework to nonlinear models such a deep neural nets and draw links between meta-learning, bilevel optimization and gradient-based hyperparameter optimization. This talk is part of the Machine Learning @ CUED series. This talk is included in these lists:
Note that ex-directory lists are not shown. |
Other listsImagined Civities J M Keynes Fellowship Fund Lectures Machine Learning Journal ClubOther talksSelf-Assembly in Deep Eutectic Solvents Imaging Clinic Cafe Synthetique: Sampling Biodiversity from the Botanic Garden to the Cam Former UN Assistant Secretary-General Simulating higher spatial dimensions with atoms and photons Understanding Source Code using Natural Language and Graph Neural Networks |