Neural Attention

Add to your list(s) Download to your calendar using vCal

• Elre Oldewage, George Hron
• Wednesday 20 November 2019, 14:00-15:30
• Engineering Department, CBL Room BE-438.

If you have a question about this talk, please contact Robert Pinsler.

Sequence-to-sequence models have been very successful in natural language processing tasks such as language translation. Self-attention – an attention mechanism that relates different positions of a single sequence – has proved to be an important technique that significantly improves the quality of model outputs. We will discuss attention in the context of neural machine translation and introduce the transformer model. Transformers rely entirely on self-attention to capture long-range dependencies, rather than recurrence or convolutions. By dispensing with recurrence, transformers are also more parallelizable than recurrent models, which are inherently sequential. We consider a number of case studies, most notably BERT , which is the major success story for self-attention. We also consider applications of attention to images and graph neural networks.

This talk is part of the Machine Learning Reading Group @ CUED series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge University Engineering Department Talks
• Cambridge talks
• Centre for Smart Infrastructure & Construction
• Chris Davis' list
• Computational Continuum Mechanics Group Seminars
• Engineering Department, CBL Room BE-438
• Featured lists
• Guy Emerson's list
• Hanchen DaDaDash
• Inference Group Journal Clubs
• Inference Group Summary
• Information Engineering Division seminar list
• Interested Talks
• ML
• Machine Learning Reading Group
• Machine Learning Reading Group @ CUED
• Machine Learning Summary
• Quantum Matter Journal Club
• Required lists for MLG
• School of Technology
• Simon Baker's List
• TQS Journal Clubs
• Trust & Technology Initiative - interesting events
• bld31
• custom
• ndk22's list
• ob366-ai4er
• rp587
• yk373's list

Note that ex-directory lists are not shown.