Pretraining, Instruction Tuning, Alignment: Towards Building Large Language Models from First Principles

Add to your list(s) Download to your calendar using vCal

• Yao Fu, University of Edinburgh
• Thursday 25 May 2023, 11:00-12:00
• https://cam-ac-uk.zoom.us/j/97599459216?pwd=QTRsOWZCOXRTREVnbTJBdXVpOXFvdz09.

If you have a question about this talk, please contact Panagiotis Fytas.

Recently, the field has been greatly impressed and inspired by Large Language Models (LLMs). LLMs’ multi-dimensional abilities are significantly beyond many AI researchers’ and practitioners’ expectations and thus reshaping the AI research paradigm. A natural question is how LLMs get there, and where these fantastic abilities come from. In this talk, we try to dissect the strong LLMs’ capabilities and trace them to their sources. We first review the generic recipe for building large language models from first principles. Then we discuss recipes for improving language models’ reasoning capabilities. Finally, we consider further improvements by complexity-based prompting, distilling chain-of-thought, and learning from AI feedback.

This talk is part of the Language Technology Lab Seminars series.

This talk is included in these lists:

• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge talks
• Chris Davis' list
• Guy Emerson's list
• Interested Talks
• Language Sciences for Graduate Students
• Language Technology Lab Seminars
• Simon Baker's List
• Trust & Technology Initiative - interesting events
• bld31
• https://cam-ac-uk.zoom.us/j/97599459216?pwd=QTRsOWZCOXRTREVnbTJBdXVpOXFvdz09
• ndk22's list
• ob366-ai4er
• rp587

Note that ex-directory lists are not shown.