BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:You Cannot Feed Two Birds with One Score: the Accuracy-Naturalness
  Tradeoff in Translation - Gergely Flamich (Imperial College London)
DTSTART:20250807T090000Z
DTEND:20250807T100000Z
UID:TALK235006@talks.cam.ac.uk
DESCRIPTION:The goal of translation\, be it by human or by machine\, is\, 
 given some textin a source language\, to produce text in a target language
  that simultaneously 1) preserves the meaning of the source text and 2) ac
 hieves natural expression in the target language. However\, researchers in
  the machine translation community usually assess translations using a sin
 gle score in-tended to capture semantic accuracy and the naturalness of th
 e output simultaneously.\nIn this talk\, I build on Michaeli and Blau&rsqu
 o\;s the seminal work on distortion-perception theory to propose a general
  notion of accuracy and naturalness\, and show that such single-score summ
 aries do not and cannot give the complete picture of a translation system&
 rsquo\;s true performance. I also demonstrate the accuracy-naturalness tra
 deoff by evaluating the performance of the state-of-the-art translation sy
 stems on a popular benchmark. These findings help explain well-known empir
 ical phenomena\, such as the observation that optimizing translation syste
 ms for a specific accuracy metric initially improves the system&rsquo\;s n
 aturalness\, while &ldquo\;overfitting&rdquo\; the system to the metric ca
 n significantly degrade its naturalness. Finally\, I propose a new statist
 ical distance in the spirit of integral probability metrics to measure nat
 uralness in practice and discuss some of its properties.
LOCATION:Seminar Room 2\, Newton Institute
END:VEVENT
END:VCALENDAR