Efficient Structured Prediction on Long Texts

Add to your list(s) Download to your calendar using vCal

• Mrinmaya Sachan (ETH Zurich)
• Friday 28 October 2022, 12:00-13:00
• Virtual (Zoom).

If you have a question about this talk, please contact Michael Schlichtkrull.

Abstract:

Vast majority of past NLP research has focused on domains such as tweets, blogs, Wikipedia and news articles. However, documents in several other domains of interest such as scientific articles, legal proceedings or novels and textbooks are substantially longer. Long documents pose a significant computational challenge to typical NLP models. In this talk, I will focus on structured prediction over long texts, particularly motivated by the problem of scaling coreference resolution models to long documents. State of the art end-to-end coreference models use expensive span representations and antecedent prediction mechanisms. These approaches are expensive both in terms of their memory requirements as well as compute time, and are particularly ill-suited for long documents. I would describe a succession of recent efforts from our group in scaling these models using a) efficient structured span selection which relies on the intuition that most spans of interest in typical span selection tasks are syntactic constituents, b) token level span representations and nearest neighbor sparsification for more efficient antecedent prediction, and c) autoregressive structured prediction which models structures as a sequence of actions in a dynamic action space using large language models. This is joint work with Tianyu Liu, Yuchen (Eleanor) Jiang, Raghuveer Thirukovalluru, Kumar Shridhar, Nicholas Monath and Ryan Cotterell.

Bio:

Mrinmaya Sachan is an Assistant Professor of Computer Science at ETH Zurich. His research is in the area of Natural language processing and the interface of Machine learning and Education. Prior to this position, Mrinmaya was a Research Assistant Professor at TTI Chicago. Before that, he received a Ph.D. from the Machine Learning Department at CMU and a B.Tech. in Computer Science from IIT Kanpur where he received an Academic Excellence Award. He has received several awards for his work, including an outstanding paper award at ACL 2015 , an IBM PhD fellowship, the Siebel scholarship and the CMU CMLH fellowship. His current research is funded by grants from the Swiss National Science Foundation, the ETH Zurich foundation and Haslerstiftung.

Topic: NLIP Seminar Time: Oct 28, 2022 12:00 PM London

Join Zoom Meeting https://cl-cam-ac-uk.zoom.us/j/91073515866?pwd=UnJmTER6dmZLeWpPOUo0VUJBOGxYQT09

Meeting ID: 910 7351 5866 Passcode: 646960

This talk is part of the NLIP Seminar Series series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge talks
• Chris Davis' list
• Department of Computer Science and Technology talks and seminars
• Graduate-Seminars
• Guy Emerson's list
• Interested Talks
• Language Sciences for Graduate Students
• NLIP Seminar Series
• PMRFPS's
• Simon Baker's List
• Trust & Technology Initiative - interesting events
• Virtual (Zoom)
• bld31
• ndk22's list
• ob366-ai4er
• rp587

Note that ex-directory lists are not shown.