University of Cambridge > > Lennard-Jones Centre > Language based Pre-training for Drug Discovery

Language based Pre-training for Drug Discovery

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Dr Christoph Schran.

Pretraining has taken the NLP world by storm as ever larger language models have broken successive benchmarks. In this talk, I’ll review some recent work applying pretraining to scientific challenges, and in particular will discuss the challenges of pretraining for molecular machine learning. I’ll introduce our new architecture, ChemBERTa, which explores the use of BERT -style pretraining for machine learning problems inspired by drug discovery applications.

This talk is part of the Lennard-Jones Centre series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.


© 2006-2024, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity