Abstract

Latent variable modeling and Bayesian inference are appealing in theory—- they provide a unified mathematical framework for solving a wide range of machine learning problems—- but are often difficult to apply effectively in practice. Accurate inference in even simple models can seem computationally intractable, while more realistic models are difficult to even write down precisely.

In this talk, I will introduce new probabilistic programming technology that aims to alleviave these difficulties. Unlike graphical models, which marries statistics with graph theory, probabilistic programming marries Bayesian inference with universal computation. Probabilistic programming can make it easier to build useful, fast machine learning software that goes significantly beyond graphical models in flexibility and power. I will illustrate probabilistic programming using page-long probabilistic programs that break simple CAPTCH As— by running randomized CAPTCHA generators backwards— interpret noisy time-series data from clinical medicine, and estimate good predictive models for arbitrary structured data tables without any parameter tuning or pre-processing.

I will also describe stochastic digital circuit architectures that carry these principles down to the physical layer and yield 1000x speed and 10-100x power improvements over deterministic designs on problems of optical flow, clustering, and inference in discrete graphical models.

Throughout, I will highlight the ways probabilistic programming points the way to a new model of computation, based on universal inference over distributions rather than universal calculation of functions, and exposes the mathematical and algorithmic structure needed to engineer efficient, distributed machine learning systems.