Frank-Wolfe optimization insights in machine learning

Add to your list(s) Download to your calendar using vCal

• Simon Lacoste-Julien (INRIA, ENS, Paris)
• Friday 24 August 2012, 11:00-12:00
• Engineering Department, CBL Room BE-438.

If you have a question about this talk, please contact Zoubin Ghahramani.

The Frank-Wolfe optimization algorithm (also called conditional gradient) is a very simple and intuitive optimization algorithm proposed in the 1950s by Marguerite Frank and Phil Wolfe. It was partly forgotten as it became superseded by faster algorithms, but it is making a recent revival in machine learning, thanks to its ability to exploit well the structure of the machine learning optimization problems. In this talk, I will mention two recent advances making use of Frank-Wolfe. In the first part, I will describe how it can be efficiently applied to large margin learning for structured prediction. I will show how several previous algorithms were special cases of Frank-Wolfe, and I will present a new block-coordinate version of Frank-Wolfe which yields a simple algorithm which outperforms the state-of-the-art. In the second part, I will describe how the herding algorithm recently proposed by Max Welling is actually equivalent to the Frank-Wolfe optimization of a quadratic moment discrepancy. This link enables us to obtain a weighted version of herding which converge faster for the task of approximating integrals (obtaining adaptive quadrature rules). On the other hand, our experiments indicate that herding could still be better for the task learning, shedding more light on the properties of the herding algorithm.

This is joint work with Francis Bach, Martin Jaggi, Guillaume Obozinski, Mark Schmidt and Patrick Pletscher.

This talk is part of the Machine Learning @ CUED series.

This talk is included in these lists:

• Seminar
• All Talks (aka the CURE list)
• Biology
• CBL important
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge Neuroscience Seminars
• Cambridge talks
• Chris Davis' list
• Creating transparent intact animal organs for high-resolution 3D deep-tissue imaging
• Engineering Department, CBL Room BE-438
• Featured lists
• Guy Emerson's list
• Hanchen DaDaDash
• Inference Group Summary
• Information Engineering Division seminar list
• Interested Talks
• Joint Machine Learning Seminars
• Life Science
• Life Sciences
• ML
• Machine Learning @ CUED
• Machine Learning Summary
• Neuroscience
• Neuroscience Seminars
• Neuroscience Seminars
• Required lists for MLG
• Simon Baker's List
• Stem Cells & Regenerative Medicine
• Trust & Technology Initiative - interesting events
• bld31
• dh539
• ndk22's list
• ob366-ai4er
• rp587
• yk373's list

Note that ex-directory lists are not shown.