What’s not an Autoregressive LLM?

Add to your list(s) Download to your calendar using vCal

• Prof. Lingpeng Kong, HKU
• Thursday 20 February 2025, 11:00-12:00
• https://cam-ac-uk.zoom.us/j/97599459216?pwd=QTRsOWZCOXRTREVnbTJBdXVpOXFvdz09.

If you have a question about this talk, please contact shun shao.

Abstract:

This talk explores alternatives to autoregressive Large Language Models (LLMs), with a particular focus on discrete diffusion models. The presentation covers recent advances in non-autoregressive approaches to text generation, reasoning, and planning tasks. Key developments discussed include Reparameterized Discrete Diffusion Models (RDMs), which show promising results in machine translation and error correction, and applications of discrete diffusion to complex reasoning tasks like countdown games, Sudoku, and chess. The talk also examines sequence-to-sequence text diffusion models, as well as the novel Diffusion of Thoughts (DoTs) framework for chain-of-thought reasoning. These non-autoregressive approaches demonstrate competitive performance while offering potential advantages in terms of parallel processing and flexible generation patterns compared to traditional autoregressive models.

A short CV:

https://ikekonglp.github.io/lingpenk_cv.pdf

Bio:

Lingpeng Kong is an assistant professor in the Department of Computer Science, the University of Hong Kong. From 2017-2020, he was a (senior) research scientist at Google DeepMind. He received his Ph.D. in Artificial Intelligence from the School of Computer Science, Carnegie Mellon University. His work lies at the intersection of natural language processing (NLP) and machine learning (ML), with a focus on representation learning, structured prediction, and generative models. He has served as the (senior) area chairs for top NLP and ML conferences such as ICLR , NeurIPS, ICML , EMNLP, and NAACL . He has also served as an action editor for Transactions on Machine Learning Research (TMLR) and ACL rolling review (ARR). His research was recognized with outstanding paper awards from ACL and EACL .

A photo for the event web page: https://ikekonglp.github.io/me_new.jpeg

This talk is part of the Language Technology Lab Seminars series.

This talk is included in these lists:

• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge talks
• Chris Davis' list
• Guy Emerson's list
• Interested Talks
• Language Sciences for Graduate Students
• Language Technology Lab Seminars
• Simon Baker's List
• Trust & Technology Initiative - interesting events
• bld31
• https://cam-ac-uk.zoom.us/j/97599459216?pwd=QTRsOWZCOXRTREVnbTJBdXVpOXFvdz09
• ndk22's list
• ob366-ai4er
• rp587

Note that ex-directory lists are not shown.