Hybrid of Generative and Discriminative Models

Add to your list(s) Download to your calendar using vCal

• Simon Lacoste-Julien (University of Cambridge)
• Thursday 12 February 2009, 14:00-15:30
• Engineering Department, CBL Room 438.

If you have a question about this talk, please contact Shakir Mohamed.

In this RCC , I will cover the topic of combining generative and discriminative learning approaches for classification.

I will start by covering a classic:

• On Discriminative vs. Generative Classifiers: A comparison of logistic regression and Naive Bayes, Andrew Y. Ng and Michael Jordan. In NIPS 14 , 2002. pdf

This paper is unfortunately not an easy read, but it is one of the most cited on the topic of comparing generative and discriminative learning. The analysis is somewhat specific, but still provides interesting insight.

I will then compare the two hybrid generative / discriminative approaches proposed in the following two papers (the first one being more Bayesian; the second one being more Frequentist):

• Generative or Discriminative? getting the best of both worlds , Bishop, C. M. and Lasserre, J. (2007) In Bayesian Statistics 8,  Bernardo, J. M. et al. (Eds), Oxford University Press. 3–23. With discussion. pdf
• Bias-variance tradeoff in hybrid generative-discriminative models, G. Bouchard. In proc. of the Sixth International conference on Machine Learning and Applications (ICMLA 07), Cincinnati, Ohio, USA , 13-15 Dec.2007., 2007. pdf

If you have time to read only one paper, read the Bishop’s one (I will focus more on it) and skim through the other two. A slightly shorter conference version is available here .

If you are quite interested on the topic and you want more details, have a look at the PhD thesis of Julia Lassere and chapter 5 of the PhD thesis of Guillaume Bouchard (chapter 5 is in English even though the several other chapters are in French).

This talk is part of the Machine Learning Reading Group @ CUED series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge University Engineering Department Talks
• Centre for Smart Infrastructure & Construction
• Engineering Department, CBL Room 438
• Featured lists
• Inference Group Journal Clubs
• Inference Group Summary
• Machine Learning Reading Group @ CUED
• Machine Learning Summary
• Quantum Matter Journal Club
• Required lists for MLG
• School of Technology
• TQS Journal Clubs
• engineering

Note that ex-directory lists are not shown.