BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//talks.cam.ac.uk//v3//EN
BEGIN:VTIMEZONE
TZID:Europe/London
BEGIN:DAYLIGHT
TZOFFSETFROM:+0000
TZOFFSETTO:+0100
TZNAME:BST
DTSTART:19700329T010000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0100
TZOFFSETTO:+0000
TZNAME:GMT
DTSTART:19701025T020000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CATEGORIES:CUED Control Group Seminars
SUMMARY:Q-learning and Pontryagin's Minimum Principle - Pr
ofessor Sean Meyn (Director\, Decision &\; Cont
rol Lab\, CSL ECE UIUC)
DTSTART;TZID=Europe/London:20100113T140000
DTEND;TZID=Europe/London:20100113T150000
UID:TALK21985AThttp://talks.cam.ac.uk
URL:http://talks.cam.ac.uk/talk/index/21985
DESCRIPTION:"*Q-learning and Pontryagin's Minimum Principle*":
https://netfiles.uiuc.edu/meyn/www/spm_files/Q2009
/Q09.html\n\nQ-learning is a technique used to com
pute an optimal policy for a controlled Markov cha
in based on observations of the system controlled
using a non-optimal policy. It has proven to be ef
fective for models with finite state and action sp
ace. This paper establishes connections between Q-
learning and nonlinear control of continuous-time
models with general state space and general action
space. The main contributions are summarized as f
ollows.\n\n * The starting point is the observa
tion that the "Q-function" appearing in Q-learning
algorithms is an extension of the Hamiltonian tha
t appears in the Minimum Principle. Based on this
observation we introduce the steepest descent Q-le
arning (SDQ-learning) algorithm to obtain the opti
mal approximation of the Hamiltonian within a pres
cribed finite-dimensional function class.\n * A
transformation of the optimality equations is per
formed based on the adjoint of a resolvent operato
r. This is used to construct a consistent algorith
m based on stochastic approximation that requires
only causal filtering of the time-series data.\n
* Several examples are presented to illustrate t
he application of these techniques\, including app
lication to distributed control of multi-agent sys
tems.\n
LOCATION:Cambridge University Engineering Department\, Lect
ure Theatre 6
CONTACT:Dr Ioannis Lestas
END:VEVENT
END:VCALENDAR