University of Cambridge > Talks.cam > Machine learning in Physics, Chemistry and Materials discussion group (MLDG) > Language based Pre-training for Drug Discovery

Language based Pre-training for Drug Discovery

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Bingqing Cheng .

Pretraining has taken the NLP world by storm as ever larger language models have broken successive benchmarks. In this talk, I’ll review some recent work applying pretraining to scientific challenges, and in particular will discuss the challenges of pretraining for molecular machine learning. I’ll introduce our new architecture, ChemBERTa, which explores the use of BERT -style pretraining for machine learning problems inspired by drug discovery applications.

This talk is part of the Machine learning in Physics, Chemistry and Materials discussion group (MLDG) series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

 

© 2006-2021 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity