Zero-shot Language Learning through Bayesian Neural Models

Add to your list(s) Download to your calendar using vCal

• Edoardo Maria Ponti (University of Cambridge)
• Friday 07 February 2020, 12:00-13:00
• FW26, Computer Laboratory.

If you have a question about this talk, please contact James Thorne.

A key feature of general linguistic intelligence is the ability to generalise to new domains with sample efficiency, i.e. based on zero or few examples. This goal is of great practical importance, as most combinations of tasks and languages lack in-domain examples for supervised training. A possible solution consists in biasing the induction of neural models towards unseen languages by constructing an informative prior over parameters (imbued with universal linguistic knowledge) from seen languages. Subsequently, MAP inference allows for learning efficiently from few in-domain examples by leveraging the prior information. Another solution resorts to estimating parameters for unseen task-language combinations based on seen combinations. In particular, the space of neural parameters can be factorized into its constituent latent variables, one for each task and language. The posteriors over these latent variables can be learned through stochastic Variational Inference. In this talk, I will argue that these approaches yield comparable or better performance than state-of-the-art zero-shot transfer methods for character-level language modelling, POS tagging and Named Entity Recognition over a wide and typologically diverse sample of languages. What is more, the Bayesian treatment of the inference problem allows for quantifying the uncertainty of each prediction, which is especially valuable in settings characterised by distribution shifts such as zero-shot transfer.

This talk is part of the NLIP Seminar Series series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge talks
• Chris Davis' list
• Department of Computer Science and Technology talks and seminars
• FW26, Computer Laboratory
• Graduate-Seminars
• Guy Emerson's list
• Interested Talks
• Language Sciences for Graduate Students
• NLIP Seminar Series
• PMRFPS's
• Simon Baker's List
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• rp587

Note that ex-directory lists are not shown.