University of Cambridge > Talks.cam > Statistics > On Estimation of Unnormalized Density Models

On Estimation of Unnormalized Density Models

Add to your list(s) Download to your calendar using vCal

  • UserSong Liu — University of Bristol
  • ClockFriday 25 October 2019, 14:00-15:00
  • HouseMR12.

If you have a question about this talk, please contact Dr Sergio Bacallado.

In many machine learning applications, complicated parametric density models such as Deep Neural Networks are favoured due to their flexibility and expressiveness. However, unlike classic density models, these density functions cannot be easily normalised. Thus, Maximum Likelihood Estimation (MLE) cannot be easily applied for model parameter estimation. If the dimensionality of the input variable is high, MCMC based MLE may also fail. In the machine learning community, some Stein Identity based methods have risen in popularity due to their ability to measure the “goodness of fit” of unnormalisable models. In the first part of the talk [1], we study the performance of a parameter estimator using Stein’s identity. Particularly, we construct a Stein density ratio estimator, which estimates the ratio function between a data distribution and a model distribution. Then we minimise the fitted likelihood ratio to estimate model parameters. In the second part of the talk [2], I discuss a specific type of unnormalised density model: Truncated densities. We show how an augmented score matching estimator can be applied to estimate parameters of density models with a complex truncation domain (such as a polytope in R^2).

[1] Liu, S., Kanamori, T., Jitkrittum, W., Chen, Y; Fisher Efficient Inference of Intractable Models; arxiv:1805.07454, 2019. [2] Liu, S., Kanamori, T., Estimating Density Models with Complex Truncation Boundaries; arXiv:1910.03834, 2019.

This talk is part of the Statistics series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

 

© 2006-2024 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity