Towards More Robust and Interpretable Models for Structured Prediction and Language Generation

Add to your list(s) Download to your calendar using vCal

• Adhiguna Kuncoro, DeepMind
• Friday 20 October 2017, 12:00-13:00
• FW26, Computer Laboratory.

If you have a question about this talk, please contact Amandla Mabona.

Many important tasks in natural language processing involve problems with structured output spaces, such as machine translation and syntactic parsing. In line with recent advances in representation learning, I will describe two ways to improve state of the art neural methods in language modeling and syntactic parsing. Despite the expressive power of recurrent neural networks, I explore the hypothesis that such models can still benefit from linguistically-motivated inductive biases, thereby facilitating better generalization given a limited amount of labeled data. I will also argue for the importance of cost functions in learning, where a refined notion of costs, based on the idea of ensemble distillation, improves the performance of a strong neural dependency parser.

In the opposite direction, neural representation learners can be thought of as “mini-linguists”, in the sense that they often need to come up with generalizations and theories about certain latent aspects of language. The learners’ findings can then be used as empirical means to confirm or refute certain linguistic hypotheses. To this end, I demonstrate that the findings of Recurrent Neural Network Grammars (RNNG), a state of the art model for parsing and language modeling, mostly align with certain linguistic theories of syntax, while also discovering some syntactic phenomena that are different from our intuition, but are interesting nevertheless.

BIOGRAPHY : Adhiguna Kuncoro is a DPhil student in computer science at the University of Oxford and research scientist at DeepMind. His primary research interest lies at the intersection of natural language processing and machine learning, particularly on designing statistical models of natural language that are: i.) robust, ii.) interpretable, and iii.) able to learn more with less amount of annotated data. His co-authored work received an Outstanding Long Paper award at EACL 2017 . He holds a Master’s degree in language technologies from CMU LTI , where he worked on low-resource language processing under the DARPA LORELEI project, and another Master’s degree in computer science from the University of Oxford. His DPhil study at Oxford is jointly funded by an EPSRC studentship under the flexibility to support the very best students scheme, the Oxford Department of Computer Science scholarship, and a Balliol Mark Sadler scholarship award. He completed his undergraduate degree in Informatics Engineering from Institut Teknologi Bandung, Indonesia.

This talk is part of the NLIP Seminar Series series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge talks
• Chris Davis' list
• Department of Computer Science and Technology talks and seminars
• FW26, Computer Laboratory
• Graduate-Seminars
• Guy Emerson's list
• Interested Talks
• Language Sciences for Graduate Students
• NLIP Seminar Series
• PMRFPS's
• Simon Baker's List
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• rp587

Note that ex-directory lists are not shown.