COOKIES: By using this website you agree that we can place Google Analytics Cookies on your device for performance monitoring. |
University of Cambridge > Talks.cam > Language Technology Lab Seminars > Mechanistic Understanding of Language Models in Arithmetic Reasoning and Code Generation
Mechanistic Understanding of Language Models in Arithmetic Reasoning and Code GenerationAdd to your list(s) Download to your calendar using vCal
If you have a question about this talk, please contact Tiancheng Hu. Abstract: Transformer-based language models (LMs) have demonstrated the promise in solving more and more complicated tasks, yet alongside their advancements are growing concerns on their safety and reliability. These concerns primarily stem from our limited understanding of these LMs and the difficulty in interpreting their behaviors. In this talk, I will present our two recent projects towards forming a mechanistic understanding of LMs. In the first project (published at ACL ’24), we explain how Chain-of-Thoughts (CoT) elicit the arithmetic reasoning of LMs by looking into the neuron activation inside the models; in the second project (ongoing), we generalize the analysis to understand the mechanism of how LMs solve the structured code generation problem. Finally, I will conclude the talk by briefly sharing our other effort along the line of LM planning and interpretability. Bio: Ziyu Yao (https://ziyuyao.org/) is an Assistant Professor in the Department of Computer Science at George Mason University, where she co-leads the George Mason NLP group (https://nlp.cs.gmu.edu/). Her research topics cover LLMs, semantic parsing/code generation, model interpretability, and human-AI interaction. Her work has been funded by National Science Foundation, Virginia Commonwealth Cyber Initiative, Microsoft’s Accelerating Foundation Models Research Program, among others. She has regularly served as an area chair at top-tier NLP /AI conferences and was the Diversity & Inclusion Co-Chair at NAACL 2024 . Prior to George Mason, she graduated with a Ph.D. degree in Computer Science and Engineering from the Ohio State University in 2021, where she was awarded the prestigious Presidential Fellowship. She was also selected as a rising star in EECS by UC Berkeley in 2021. This talk is part of the Language Technology Lab Seminars series. This talk is included in these lists:
Note that ex-directory lists are not shown. |
Other listsToys are children's words | Choose the first-rate toys to shop for your children Naked mole-rats Pharmacology Lunch ClubOther talksOverlooked wildlife, or, Why are there so many species? Grand Rounds - QICG/Vetsafe Sai Shruthi Murali on Prebiotic Chemical Kinetics Early Cancer Institute Seminar: Prof Utkan Demirci, Department of Radiology at Stanford University School of Medicine and the Canary Center at Stanford for Cancer Early Detection Electrical Engineering (by courtesy). Designer alloys for tailored deformation-induced phase transformations: Revealing per-grain behaviour “Memory output from the Germinal Centre. A second level of selection?” |