BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:AI + Pizza September 2018 - Adria Garriga Alonso\, University of C
 ambridge\, Alex Gaunt\, Microsoft Research Cambridge
DTSTART:20180928T163000Z
DTEND:20180928T180000Z
UID:TALK111097@talks.cam.ac.uk
CONTACT:Microsoft Research Cambridge Talks Admins
DESCRIPTION:*Speaker one* - Adria Garriga Alonso\n \n*Title* - Deep Convol
 utional Networks as shallow Gaussian Processes \n\n*Abstract* - We show th
 at the output of a (residual) convolutional neural network (CNN) with an a
 ppropriate prior over the weights and biases is a Gaussian process (GP) in
  the limit of infinitely many convolutional filters. The result is an exte
 nsion of the theorem for dense networks due to Alex Matthews et al. (2018)
 \, also presented in this AI+Pizza series.\n\nFor a CNN\, the equivalent k
 ernel can be computed exactly and\, unlike "deep kernels"\, has very few p
 arameters: only the hyperparameters of the original CNN. Further\, we show
  that this kernel has two properties that allow it to be computed efficien
 tly\; the cost of evaluating the kernel for a pair of images is similar to
  a single forward pass through the original CNN with only one filter per l
 ayer.The kernel equivalent to a 32-layer ResNet obtains 0.84% classificati
 on error on MNIST\, a new record for GPs with a comparable number of param
 eters.\n\nThis is joint work with Laurence Aitchison and Carl Rasmussen. \
 n\n*Speaker Two* - Alexander Gaunt\n\n*Title* - Fixing Variational Bayes: 
 Deterministic Variational Inference for Bayesian Neural Networks \n\n*Abst
 ract* - Bayesian neural networks hold great promise as flexible and princi
 pled solution to deal with uncertainty when learning from finite data. Amo
 ng approaches to realize probabilistic inference in deep neural networks\,
  variational Bayes (VB) is principled\, generally applicable\, and computa
 tionally efficient. With wide recognition of potential advantages\, why is
  it that variational Bayes has seen very limited practical use for neural 
 networks in real applications? We argue that variational inference in neur
 al networks is fragile: to get the approach to work requires careful initi
 alization and tuning of prior variances as well as controlling the varianc
 e of stochastic gradient estimates. We fix VB and turn it into a robust in
 ference tool for Bayesian neural networks. We achieve this by two innovati
 ons: first\, we introduce a novel deterministic method to approximate mome
 nts in neural networks\, reducing gradient variance to zero\; second\, we 
 introduce a hierarchical prior for parameters and a novel Empirical Bayes 
 procedure for automatically selecting prior variances. Combining these two
  innovations\, the resulting method is highly efficient and robust. On the
  application of heteroscedastic regression we demonstrate strong predictiv
 e performance over alternative approaches. \n
LOCATION:Auditorium\, Microsoft Research Ltd\, 21 Station Road\, Cambridge
 \, CB1 2FB
END:VEVENT
END:VCALENDAR