BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//talks.cam.ac.uk//v3//EN
BEGIN:VTIMEZONE
TZID:Europe/London
BEGIN:DAYLIGHT
TZOFFSETFROM:+0000
TZOFFSETTO:+0100
TZNAME:BST
DTSTART:19700329T010000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0100
TZOFFSETTO:+0000
TZNAME:GMT
DTSTART:19701025T020000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CATEGORIES:CCIMI Seminars
SUMMARY:Fundamental limits of generative AI - Helmut Bölcs
kei - ETH Zurich
DTSTART;TZID=Europe/London:20230428T140000
DTEND;TZID=Europe/London:20230428T150000
UID:TALK189104AThttp://talks.cam.ac.uk
URL:http://talks.cam.ac.uk/talk/index/189104
DESCRIPTION:Generative AI has seen tremendous successes recent
ly\, most notably the chatbot ChatGPT and the DALL
E2 software creating realistic images and artwork
from text descriptions. Underlying these and other
generative AI systems are usually neural networks
trained to produce text\, images\, audio\, or vid
eo from text inputs. The aim of this talk is to de
velop an understanding of the fundamental capabili
ties of generative neural networks. Specifically a
nd mathematically speaking\, we consider the reali
zation of high-dimensional random vectors from one
-dimensional random variables through deep neural
networks. The resulting random vectors follow pres
cribed conditional probability distributions\, whe
re the conditioning represents the text input of t
he generative system and its output can be text\,
images\, audio\, or video. It is shown that every
d-dimensional probability distribution can be gene
rated through deep ReLU networks out of a 1-dimens
ional uniform input distribution. What is more\, t
his is possible without incurring a cost—in terms
of approximation error as measured in Wasserstein-
distance—relative to generating the d-dimensional
target distribution from d independent random vari
ables. This is enabled by a space-filling approach
which realizes a Wasserstein-optimal transport ma
p and elicits the importance of network depth in d
riving the Wasserstein distance between the target
distribution and its neural network approximation
to zero. Finally\, we show that the number of bit
s needed to encode the corresponding generative ne
tworks equals the fundamental limit for encoding p
robability distributions (by any method) as dictat
ed by quantization theory according to Graf and Lu
schgy. This result also characterizes the minimum
amount of information that needs to be extracted f
rom training data so as to be able to generate a d
esired output at a prescribed accuracy and establi
shes that generative ReLU networks can attain this
minimum.\n\nThis is joint work with D. Perekreste
nko and L. Eberhard
LOCATION: Centre for Mathematical Sciences MR12\, CMS
CONTACT:Randolf Altmeyer
END:VEVENT
END:VCALENDAR