BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//talks.cam.ac.uk//v3//EN
BEGIN:VTIMEZONE
TZID:Europe/London
BEGIN:DAYLIGHT
TZOFFSETFROM:+0000
TZOFFSETTO:+0100
TZNAME:BST
DTSTART:19700329T010000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0100
TZOFFSETTO:+0000
TZNAME:GMT
DTSTART:19701025T020000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CATEGORIES:Machine Learning @ CUED
SUMMARY:Variational Bayes In Private Settings - Mijung Par
k\, University of Amsterdam
DTSTART;TZID=Europe/London:20170125T150000
DTEND;TZID=Europe/London:20170125T160000
UID:TALK70334AThttp://talks.cam.ac.uk
URL:http://talks.cam.ac.uk/talk/index/70334
DESCRIPTION:Bayesian methods are frequently used for analysing
privacy-sensitive datasets\, including medical re
cords\, emails\, and educational data\, and there
is a growing need for practical Bayesian inference
algorithms that protect the privacy of individual
s' data. To this end\, we provide a general framew
ork for privacy-preserving variational Bayes (VB)
for a large class of probabilistic models\, called
the conjugate exponential (CE) family. Our primar
y observation is that when models are in the CE fa
mily\, we can privatise the variational posterior
distributions simply by perturbing the expected su
fficient statistics of the complete-data likelihoo
d. For widely used non-CE models with binomial lik
elihoods (e.g.\, logistic regression)\, we exploit
the Polya-Gamma data augmentation scheme to bring
such models into the CE family\, such that infere
nces in the modified model resemble the original (
private) variational Bayes algorithm as closely as
possible. The iterative nature of variational Bay
es presents a further challenge for privacy preser
vation\, as each iteration increases the amount of
noise needed. We overcome this challenge by combi
ning: (1) a relaxed notion of differential privacy
\, called concentrated differential privacy\, whic
h provides a tight bound on the privacy cost of mu
ltiple VB iterations and thus significantly decrea
ses the amount of additive noise\; and (2) the pri
vacy amplification effect of subsampling mini-batc
hes from large-scale data in stochastic learning.
We empirically demonstrate the effectiveness of ou
r method in CE and non-CE models including latent
Dirichlet allocation (LDA)\, Bayesian logistic reg
ression\, and Sigmoid Belief Networks (SBNs)\, eva
luated on real-world datasets.\n\nSpeaker Bio:\n\n
Mijung Park completed her Ph.D. in the department
of Electrical and Computer Engineering under the s
upervision of Prof. Jonathan Pillow (now at Prince
ton University) and Prof. Alan Bovik at The Univer
sity of Texas at Austin. She worked with Prof. Man
eesh Sahani as a postdoc at the Gatsby computation
al neuroscience unit at University College London.
Currently\, she works with Prof. Max Welling as a
postdoc in the informatics institute at Universit
y of Amsterdam. Her research focuses on developing
practical algorithms for privacy preserving data
analysis. Previously\, she worked on a broad range
of topics including approximate Bayesian computat
ion (ABC)\, probabilistic manifold learning\, acti
ve learning for drug combinations and neurophysiol
ogy experiments\, and Bayesian structure learning
for sparse and smooth high dimensional parameters.
LOCATION:CBL Room BE-438\, Department of Engineering
CONTACT:Alessandro Davide Ialongo
END:VEVENT
END:VCALENDAR