Mean Field Approaches to Two Sequential Decision Making Problems.

Add to your list(s) Download to your calendar using vCal

• Ramki Gummadi
• Monday 08 June 2015, 17:00-18:00
• Engineering Department, CBL Room BE-438 - Skype.

If you have a question about this talk, please contact dsu21.

This talk will be given via Skype

In this presentation, I will present some of my past research on mean field approaches to sequential decision making problems under uncertainty. The talk has two parts.

In the first part, we’ll consider multiarmed bandit games, where much of the classical work focuses on arm rewards that are stationary over time. By contrast, I will present a bandit model with multiple agents where the rewards obtained by an agent can also depend on how many other agents choose the same arm (as might be the case in many competitive or cooperative scenarios). Such a dynamical system can be non-stationary due to the interdependent evolution of agents, which makes an analysis using typical equilibrium concepts intractable. I will introduce a general model of multiarmed bandit games, and define a notion of equilibrium inspired mean field methods, called the mean field stationary state (MFSS). I will discuss three key results— first, we establish existence of an MFSS under quite general conditions. Second, we show under a contraction condition that the MFE is unique, and that the population profile converges to it from any initial state. Finally, we show that under the contraction condition, MFE is a good approximation to the behavior of finite systems with many agents. The contraction condition requires that the agent population is sufficiently mixing and that the sensitivity of the reward function to the population profile is low enough.

The second part is motivated by dynamic ad auction markets (e.g Facebook, Google, Bing) where ad-slots are auctioned off dynamically at large scale. Using a mean field approximation, I will formulate the sequential optimization problem facing a bidder participating in such repeated auctions in the form of a very general stochastic control model, and present an approximate solution to a limiting MDP that allows for efficient computation of the optimal bids. I will also discuss how the optimal bid at any given time in a dynamic setting reduces to that of an equivalent static auction with a shaded private valuation, for a very general class of auction formats that includes the second price auction as a special case.

This is based on joint work with Ramesh Johari (Stanford), Peter Key (MSR Cambridge), Jia Yuan Yu (IBM Research) and Alexandre Proutiere (KTH Stockholm)

This talk is part of the Machine Learning @ CUED series.

This talk is included in these lists:

• Seminar
• All Talks (aka the CURE list)
• Biology
• CBL important
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge Neuroscience Seminars
• Cambridge talks
• Chris Davis' list
• Creating transparent intact animal organs for high-resolution 3D deep-tissue imaging
• Engineering Department, CBL Room BE-438 - Skype
• Featured lists
• Guy Emerson's list
• Hanchen DaDaDash
• Inference Group Summary
• Information Engineering Division seminar list
• Interested Talks
• Joint Machine Learning Seminars
• Life Science
• Life Sciences
• ML
• Machine Learning @ CUED
• Machine Learning Summary
• Neuroscience
• Neuroscience Seminars
• Neuroscience Seminars
• Required lists for MLG
• Simon Baker's List
• Stem Cells & Regenerative Medicine
• Trust & Technology Initiative - interesting events
• bld31
• dh539
• kt532's list
• ndk22's list
• ob366-ai4er
• rp587
• yk373's list

Note that ex-directory lists are not shown.