Provable representation learning in deep learning

Add to your list(s) Download to your calendar using vCal

• Jason Lee (Princeton University)
• Friday 13 November 2020, 16:00-17:00
• https://maths-cam-ac-uk.zoom.us/j/92821218455?pwd=aHFOZWw5bzVReUNYR2d5OWc1Tk15Zz09.

If you have a question about this talk, please contact Dr Sergio Bacallado.

Deep representation learning seeks to learn a data representation that transfers to downstream tasks. In this talk, we study two forms of representation learning: supervised pre-training and self-supervised learning.

Supervised pre-training uses a large labeled source dataset to learn a representation, then trains a classifier on top of the representation. We prove that supervised pre-training can pool the data from all source tasks to learn a good representation which transfers to downstream tasks with few labeled examples.

Self-supervised learning creates auxiliary pretext tasks that do not require labeled data to learn representations. These pretext tasks are created solely using input features, such as predicting a missing image patch, recovering the colour channels of an image, or predicting missing words. Surprisingly, predicting this known information helps in learning a representation effective for downstream tasks. We prove that under a conditional independence assumption, self-supervised learning provably learns representations.

This talk is part of the Statistics series.

This talk is included in these lists:

• https://maths-cam-ac-uk.zoom.us/j/92821218455?pwd=aHFOZWw5bzVReUNYR2d5OWc1Tk15Zz09
• All CMS events
• All Talks (aka the CURE list)
• CMS Events
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge talks
• Chris Davis' list
• DPMMS Lists
• DPMMS info aggregator
• DPMMS lists
• Guy Emerson's list
• Hanchen DaDaDash
• Interested Talks
• Machine Learning
• School of Physical Sciences
• Statistical Laboratory info aggregator
• Statistics
• Statistics Group
• bld31
• custom
• rp587

Note that ex-directory lists are not shown.