University of Cambridge > Talks.cam > NLIP Seminar Series > Evaluating Deep Generative Models on Out-of-Distribution Inputs

  Browse A-Z

Log in

University Account External (via Google)

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹ️ Information

Evaluating Deep Generative Models on Out-of-Distribution Inputs

Download to your calendar using vCal

If you have a question about this talk, please contact James Thorne .

Generative models are widely believed to be more robust to out-of-training-distribution inputs than conditional (i.e. predictive) models. In this talk, I challenge this assumption. We find that the density learned by flow-based models, VAEs, and PixelCNNs cannot distinguish images of common objects such as dogs, trucks, and horses from those of house numbers, assigning a higher likelihood to the latter when the model is trained on the former. We posit that this phenomenon is caused by a mismatch between the model’s typical set and its areas of high probability density. In-distribution inputs should reside in the former but not necessarily in the latter. To determine whether or not inputs reside in the typical set, we propose a computationally efficient hypothesis test using the empirical distribution of model likelihoods. Experiments show that this test succeeds in detecting out-of-distribution inputs in many cases in which previously proposed threshold-based techniques fail.

This talk is part of the NLIP Seminar Series series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

Forum for Youth Participation and Democracy https://ukri.zoom.us/j/91769463955; BAS seminar room 1 Seminar Room G, Second Floor, 17 Mill Lane, Cambridge, CB2 1RX

Other talks

New insights on the turbulent energy cascade from non-equilibrium statistical mechanics The uplift and erosion history of the Gamburtsev Subglacial Mountains and their role in the formation of the East Antarctic Ice Sheet Title TBC Organiser Welcome Title TBC Title to be confirmed
 

© 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity