University of Cambridge > > NLIP Seminar Series > Predicting Rich Linguistic Structure with Neural Networks

Predicting Rich Linguistic Structure with Neural Networks

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Amandla Mabona.

The recent success of neural networks has raised questions about the role of linguistic structure in NLP models, but also opens up new opportunities. In this talk I’ll show how neural networks enable us to predict richer linguistic representations than previously feasible, and explore how to incorporate linguistically-informed structural biases in language generation models. First, I’ll present a robust end-to-end parser for Minimal Recursion Semantics (MRS), a framework for compositional semantics implemented in high-precision computational grammars. This task presents several challenges for structure prediction models; I’ll show how they can be addressed through generalizing transition-based parsing in the framework of encoder-decoder recurrent neural networks, and using pointer networks. Results show that incorporating structure into the neural architecture improves performance over attention-based baselines by a large margin on both MRS and Abstract Meaning Representation parsing. Second, I’ll present a generative dependency parser which also serves as a neural syntactic language model. A feed-forward architecture and a decoding algorithm based on particle filtering enable efficient and accurate inference, while unsupervised fine-tuning improves language modelling perplexity. Finally, I’ll present a sequence-to-sequence model in which the alignment between the input and output is a latent variable marginalized through dynamic programming. The model is structured to learn mostly monotone alignments, which makes it applicable to many transduction tasks while enabling online inference.

This talk is part of the NLIP Seminar Series series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.


© 2006-2024, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity