University of Cambridge > Talks.cam > Foundation AI > How deep can LLMs reason? Benchmarking GPT and Llama models on nested mathematical formulas

  Browse A-Z

Log in

University Account External (via Google)

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹī¸ Information

How deep can LLMs reason? Benchmarking GPT and Llama models on nested mathematical formulas

Download to your calendar using vCal

If you have a question about this talk, please contact Pietro Lio .

While LLMs have shown proficiency in a diverse range of language-related tasks, their performance in mathematical problem-solving remains an area of exploration. Through empirical evidence, I challenge the prevailing notion that scaling LLMs and employing advanced prompting strategies inherently enhance their ability to solve complex mathematical formulas. The findings I will present indicate that while LLMs can handle certain mathematical tasks effectively, they exhibit limitations when faced with nested formulas which require to reason systematically to be solved, for example by applying simple algorithms. I will present results from both GPT and Llama models, highlighting the similarities and differences in their performance. Additionally, I will discuss potential underlying factors, consider the remaining limitations of this work, and present promising avenues for future research to enhance LLMs’ mathematical problem-solving capabilities.

This talk is part of the Foundation AI series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

cu palestine society FW26, William Gates Building. Zoom link: https://cl-cam-ac-uk.zoom.us/j/4361570789?pwd=Nkl2T3ZLaTZwRm05bzRTOUUxY3Q4QT09&from=addon https://ukri.zoom.us/j/98387914302; BAS seminar room 1

Other talks

Title to be confirmed Welcome Reception and Poster Session Morning Tea and Coffee Polar Oceans Seminar Talk TBC Professor Philip Rosenstiel, W3 Professor of Clinical Molecular Biology, Director, Institute of Clinical Molecular Biology (IKMB), Kiel University
 

Š 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity