BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Benefits of Multi-modality: alignment\, unpaired data and cross-mo
 del correspondences - Virtual presentation - Stefanie Jegelka (Technical U
 niversity of Munich)
DTSTART:20260210T090000Z
DTEND:20260210T100000Z
UID:TALK242308@talks.cam.ac.uk
DESCRIPTION:In many applications\, data is observed in multiple modalities
 . In this talk\, we will explore some benefits and properties of multi-mod
 al learned representations.&nbsp\;\nFirst\, it has been shown that alignme
 nt with a learned representation can help guide diffusion models for bette
 r representations and hence generation quality. Here\, we derive strategie
 s to enhance representation alignment in diffusion models\; for instance\,
  they can profit from guidance even from different modalities.\nSecond\, w
 hile alignment assumes paired data\, i.e.\, "views" of the same data point
  in different modalities\, even more data is available in unpaired form. I
 s it possible to leverage unpaired multimodal data to enhance a model in t
 he target modality? Indeed\, it is possible to exploit such data\, by buil
 ding on the assumption that different modalities are projections of a shar
 ed underlying reality\, yielding gains in practice and theory.\nLast\, tak
 ing a general view on representation geometry\, we explore correspondences
  between independently learned multimodal representations across different
  models. Typically\, they learn similar notions of "similarity". We show t
 hat it is possible to align representations across models with a single li
 near map. In fact\, a map that is fit to align one modality across models 
 transfers and automatically aligns the other modality across the same mode
 ls. This finding enables\, for instance\, backward-compatible model upgrad
 es without costly re-embedding.\nThis talk is based on joint work with Sha
 rut Gupta\, Chenyu Wang\, Cai Zhou\, Zongyu Lin\, Shobhita Sundaram\, Sany
 am Kansal\, Stephen Bates\, Tommi Jaakola\, Phillip Isola and Vikas Garg&n
 bsp\;
LOCATION:Seminar Room 1\, Newton Institute
END:VEVENT
END:VCALENDAR