COOKIES: By using this website you agree that we can place Google Analytics Cookies on your device for performance monitoring. |
University of Cambridge > Talks.cam > NLIP Seminar Series > Collaborative Pretraining on Evolving Pretraining and Small Manageable Tasks
Collaborative Pretraining on Evolving Pretraining and Small Manageable TasksAdd to your list(s) Download to your calendar using vCal
If you have a question about this talk, please contact Richard Diehl Martinez. Pretraining is monolithic. In this talk, I will discuss a collaborative approach to pertaining, by iterative model merging (originally fusing). We will then discuss making evaluation reliable and efficient, to allow anyone to evaluate. We might mention the BabyLM challenge, of pretraining models with human feasible amount of data as well (If interested in more, contact me, babyLM would be CoNLL’s shared task next year as well). Leshem Choshen is a postdoctoral researcher at MIT -IBM, aiming to collaboratively pretrain through model recycling, efficient evaluation, and efficient pretraining research (e.g., babyLM). He received the postdoctoral Rothschild and Fulbright fellowship as well as IAAI and Blavatnik best Ph.D. awards. With broad NLP and ML interests, he also worked on Reinforcement Learning, Evaluation and Understanding of how neural networks learn. In parallel, he participated in Project Debater, creating a machine that could hold a formal debate, ending in a Nature cover and live debate. He is also a dancer and runs tei.ma, a food and science blog (NisuiVeTeima on Instagram, Facebook and Tiktok). This talk is part of the NLIP Seminar Series series. This talk is included in these lists:
Note that ex-directory lists are not shown. |
Other lists.. DELETED .. Beyond Profit Careers Stream Type the title of a new list hereOther talksSharp convergence guarantees for iterative (non)-convex empirical risk minimization with random data Quantum Mechanics and Many Body Games Turning biological morphology into numbers Discovery of potent broadly neutralizing antibodies against dengue virus using a targeted E dimer probe Fortran 77: It's really C with none of the safeguards On the Disappointment of Revolutions |