University of Cambridge > Talks.cam > Inference Group > A Fast Learning Algorithm for Deep Belief Nets

  Browse A-Z

Log in

Cambridge users (raven)details Other usersdetails No account?details

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

A Fast Learning Algorithm for Deep Belief Nets

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Phil Cowans.

I will show how ``complementary priors’’ can be used to eliminate the explaining away effects that make inference difficult in densely-connected belief nets that have many hidden layers. Using complementary priors, I will derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. The fast, greedy algorithm is used to initialize a slower learning procedure that fine-tunes the weights using a contrastive version of the wake-sleep algorithm. After fine-tuning, a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels. This generative model gives better classification performance than discriminative learning algorithms. The low-dimensional manifolds on which the digits lie are modeled by long ravines in the free-energy landscape of the top-level associative memory and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind. (Joint work with Simon Osindero and Yee-Whye Teh)

This talk is part of the Inference Group series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

Part III Seminar Series Lent 2008 Talks at Centre of Molecular Materials for Photonics and Electronics (CMMPE)

Other talks

Towards a whole brain model of perceptual learning South American Opuntioids Mechanical properties of cells or cell components on the micro- and nanometer scale Exhibiting Ice Age Cambridge Borel Local Lemma Group covariance functions for Gaussian process metamodels with categorical inputs Mathematical applications of little string theory Horizontal transfer of antimicrobial resistance drives multi-species population level epidemics Alzheimer's talks 'Honouring Giulio Regeni: a plea for research in risky environments' Towards bulk extension of near-horizon geometries Developing an optimisation algorithm to supervise active learning in drug discovery
 

© 2006-2024 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity