The second generation of meta-learning methods

Add to your list(s) Download to your calendar using vCal

• Massimiliano (Max) Patacchiola, University of Cambridge
• Thursday 03 November 2022, 13:00-14:00
• Small lecture room, Microsoft Research Ltd, 21 Station Road, Cambridge, CB1 2FB.

If you have a question about this talk, please contact Microsoft Research Cambridge Talks Admins.

In the last few years, several methods have been developed to tackle the few-shot setting that is, learning from a limited amount of data belonging to a specific task. Most approaches have followed a meta-learning paradigm that is, learning-to-learn through exposure to large numbers of training tasks (episodic training) with the aim of generalizing to new unseen tasks at test time. Meta-learning has been accomplished in different ways, via rapid adaption in a few gradient steps (e.g. MAML ), exploiting learned metrics (e.g. ProtoNets), or through probabilistic approaches (e.g. Deep Kernel Transfer, CNA Ps). However, most meta-learning methods are affected by severe issues such as training instability and poor scaling that hinder their performance. In addition, there has been a growing body of empirical research showing that simpler fine-tuning routines are very effective at few-shot image classification, while being much easier to train and deploy. In this talk I will start from these empirical findings, comparing strengths and weaknesses of meta-learners and fine-tuners. I will then introduce technical solutions that could be used as building blocks for a second generation of meta-learners. In particular, I will describe our recent method called LITE (Bronskill et al., NeurIPS 2021) that allows meta-training efficiently on large images, and a new adaptive block called CaSE (Patacchiola et al., NeurIPS 2022) that allows fast adaptation of pretrained models on a context set. I will provide strong empirical evidence showing that methods based on LITE and CaSE are able to achieve state-of-the-art performance on a variety of tasks, including real-world personalization benchmarks such as the recently proposed ORBIT .

This talk is part of the Microsoft Research Cambridge, public talks series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge talks
• Chris Davis' list
• Guy Emerson's list
• Interested Talks
• Microsoft Research Cambridge, public talks
• Optics for the Cloud
• PMRFPS's
• School of Technology
• Small lecture room, Microsoft Research Ltd, 21 Station Road, Cambridge, CB1 2FB
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• personal list
• rp587

Note that ex-directory lists are not shown.