Predicting Rich Linguistic Structure with Neural Networks

Add to your list(s) Download to your calendar using vCal

• Jan Buys, University of Oxford
• Tuesday 13 June 2017, 14:00-15:00
• FW11, Computer Laboratory.

If you have a question about this talk, please contact Amandla Mabona.

The recent success of neural networks has raised questions about the role of linguistic structure in NLP models, but also opens up new opportunities. In this talk I’ll show how neural networks enable us to predict richer linguistic representations than previously feasible, and explore how to incorporate linguistically-informed structural biases in language generation models. First, I’ll present a robust end-to-end parser for Minimal Recursion Semantics (MRS), a framework for compositional semantics implemented in high-precision computational grammars. This task presents several challenges for structure prediction models; I’ll show how they can be addressed through generalizing transition-based parsing in the framework of encoder-decoder recurrent neural networks, and using pointer networks. Results show that incorporating structure into the neural architecture improves performance over attention-based baselines by a large margin on both MRS and Abstract Meaning Representation parsing. Second, I’ll present a generative dependency parser which also serves as a neural syntactic language model. A feed-forward architecture and a decoding algorithm based on particle filtering enable efficient and accurate inference, while unsupervised fine-tuning improves language modelling perplexity. Finally, I’ll present a sequence-to-sequence model in which the alignment between the input and output is a latent variable marginalized through dynamic programming. The model is structured to learn mostly monotone alignments, which makes it applicable to many transduction tasks while enabling online inference.

This talk is part of the NLIP Seminar Series series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge talks
• Chris Davis' list
• Department of Computer Science and Technology talks and seminars
• FW11, Computer Laboratory
• Graduate-Seminars
• Guy Emerson's list
• Interested Talks
• Language Sciences for Graduate Students
• NLIP Seminar Series
• PMRFPS's
• Simon Baker's List
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• rp587

Note that ex-directory lists are not shown.