Self-Supervised Representation Learning

Add to your list(s) Download to your calendar using vCal

• Javier Antoran (University of Cambridge)
• Wednesday 26 February 2020, 11:00-12:30
• Engineering Department, CBL Room BE-438.

If you have a question about this talk, please contact jg801.

Self-Supervised Learning (SSL), combined with deep neural network models, has seen great success recently. Representations learnt with SSL have been used to obtain state-of-the-art results on imagenet as well as text and speech classification tasks. We begin by introducing SSL , focusing on two of the most successful approaches: contrastive predictive coding (CPC) and deep InfoMax (DIM). We provide information theoretic interpretations of these approaches, and discuss the role of mutual information as a principal motivator for this framework. Next, we consider the role of identifiability in representation learning with generative models. We discuss recent results demonstrating that seemingly heuristic SSL approaches can be used to guarantee identifiability in nonlinear-ICA. These results imply an interesting link between identifiable generative models and SSL , potentially providing a principled foundation for SSL approaches.

This talk is part of the Machine Learning Reading Group @ CUED series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge University Engineering Department Talks
• Cambridge talks
• Centre for Smart Infrastructure & Construction
• Chris Davis' list
• Computational Continuum Mechanics Group Seminars
• Engineering Department, CBL Room BE-438
• Featured lists
• Guy Emerson's list
• Hanchen DaDaDash
• Inference Group Journal Clubs
• Inference Group Summary
• Information Engineering Division seminar list
• Interested Talks
• ML
• Machine Learning Reading Group
• Machine Learning Reading Group @ CUED
• Machine Learning Summary
• Quantum Matter Journal Club
• Required lists for MLG
• School of Technology
• Simon Baker's List
• TQS Journal Clubs
• Trust & Technology Initiative - interesting events
• bld31
• custom
• ndk22's list
• ob366-ai4er
• rp587
• yk373's list

Note that ex-directory lists are not shown.