University of Cambridge > > Machine Learning @ CUED > Online Meta-Learning

Online Meta-Learning

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Carl Edward Rasmussen.

We study the problem in which a series of learning tasks are observed sequentially and the goal is to incrementally adapt a learning algorithm in order to improve its performance on future tasks. The tasks are sampled from a meta-distribution, called the environment in the learning-to-learn literature (Baxter 2000). We focus on linear learning algorithms based on regularized empirical risk minimization such as ridge regression or support vector machines. The algorithms are parametrized by either a (representation) matrix applied to the raw inputs or by a bias vector used in the regularizer. In both settings, we develop a computational efficient meta-algorithm to incrementally adapt the learning algorithm after a task dataset is observed. The meta-algorithm performs stochastic gradient descent on a proxy objective of the risk of the learning algorithm. We derive bounds on the performance of the meta-algorithm, measured by the average risk of the learning algorithm on random tasks from the environment. Our analysis leverages ideas from multitask learning and learning-to-learn with tools from online learning and stochastic optimization. In the last part of the talk, we discuss extensions of the framework to nonlinear models such a deep neural nets and draw links between meta-learning, bilevel optimization and gradient-based hyperparameter optimization.

This talk is part of the Machine Learning @ CUED series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.


© 2006-2024, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity