On Estimation of Unnormalized Density Models

Add to your list(s) Download to your calendar using vCal

• Song Liu — University of Bristol
• Friday 25 October 2019, 14:00-15:00
• MR12.

If you have a question about this talk, please contact Dr Sergio Bacallado.

In many machine learning applications, complicated parametric density models such as Deep Neural Networks are favoured due to their flexibility and expressiveness. However, unlike classic density models, these density functions cannot be easily normalised. Thus, Maximum Likelihood Estimation (MLE) cannot be easily applied for model parameter estimation. If the dimensionality of the input variable is high, MCMC based MLE may also fail. In the machine learning community, some Stein Identity based methods have risen in popularity due to their ability to measure the “goodness of fit” of unnormalisable models. In the first part of the talk [1], we study the performance of a parameter estimator using Stein’s identity. Particularly, we construct a Stein density ratio estimator, which estimates the ratio function between a data distribution and a model distribution. Then we minimise the fitted likelihood ratio to estimate model parameters. In the second part of the talk [2], I discuss a specific type of unnormalised density model: Truncated densities. We show how an augmented score matching estimator can be applied to estimate parameters of density models with a complex truncation domain (such as a polytope in R^2).

[1] Liu, S., Kanamori, T., Jitkrittum, W., Chen, Y; Fisher Efficient Inference of Intractable Models; arxiv:1805.07454, 2019. [2] Liu, S., Kanamori, T., Estimating Density Models with Complex Truncation Boundaries; arXiv:1910.03834, 2019.

This talk is part of the Statistics series.

This talk is included in these lists:

• All CMS events
• All Talks (aka the CURE list)
• CMS Events
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge talks
• Chris Davis' list
• DPMMS Lists
• DPMMS info aggregator
• DPMMS lists
• Guy Emerson's list
• Hanchen DaDaDash
• Interested Talks
• MR12
• Machine Learning
• School of Physical Sciences
• Statistical Laboratory info aggregator
• Statistics
• Statistics Group
• bld31
• custom
• rp587

Note that ex-directory lists are not shown.