BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Fairness Evaluation in Generative NLP - Seraphina Goldfarb-Tarrant
  (Cohere)
DTSTART:20231201T120000Z
DTEND:20231201T130000Z
UID:TALK206206@talks.cam.ac.uk
CONTACT:Michael Schlichtkrull
DESCRIPTION:The largest shifts in NLP over the past five years have been t
 he shift to reliance on large pre-trained models (with the advent of the T
 ransformer)\, followed by the shift to using generative rather than discri
 minative language models. These shifts each come with serious challenges f
 or ensuring fairness in an NLP system. For the first\, the relationship be
 tween fairness during pretraining and downstream applications is tenuous a
 nd understudied. This causes challenges for where to apply mitigations\, a
 nd also causes logistical challenges because a different set of engineers 
 creates the pretrained model and the application. For the second shift\, g
 enerative systems are notoriously hard to evaluate for anything\, with poo
 r correlation between automatic metrics and humans\, and low agreement sco
 res even among humans. In this talk\, I'll present my own research into bo
 th of these areas\, discuss an overview of current challenges\, and make s
 ome suggestions for future promising directions of research.\n\n \n\nBio: 
 \n\nSeraphina Goldfarb-Tarrant is the Head of Safety at Cohere\, where she
  works on both the practice and the theory of evaluating and mitigating ha
 rms from LLMs. She did her PhD under Adam Lopez in Fairness in Tranfer Lea
 rning for NLP\, at the Institute for Language\, Cognition\, and Computatio
 n (ILCC) in the Informatics department at the University of Edinburgh. She
  did her MSc in NLP\, with a focus on Natural Language Generation\, at the
  University of Washington under Fei Xia in collaboration with Nanyun Peng.
  Her research interests include the intersection of fairness with robustne
 ss and generalisation\, cross-lingual transfer\, and causal analysis. She 
 had an industry career before her PhD\, where she worked at Google in Toky
 o\, NYC\, and Shanghai. She also spent two years as a sailor in the North 
 Sea.
LOCATION:Computer Lab\, SS03
END:VEVENT
END:VCALENDAR