BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Stochastic Gradient Descent with Adaptive Data - Jing Dong (Columb
 ia University)
DTSTART:20240425T100000Z
DTEND:20240425T104500Z
UID:TALK214237@talks.cam.ac.uk
DESCRIPTION:Stochastic gradient descent (SGD) is a powerful optimization t
 echnique\, particularly useful in online learning scenarios. Its convergen
 ce analysis/effectiveness is relatively well understood under the assumpti
 on that the data samples are independent and identically distributed (iid)
 . However\, applying online learning to policy optimization problems in op
 erations research involves a distinct challenge: the policy changes the en
 vironment and thereby affects the data used to update the policy. The adap
 tively generated data stream involves samples that are non-stationary\, no
  longer independent from each other\, and are affected by previous decisio
 ns. The influence of previous decisions on the environment introduces esti
 mation bias in the gradients\, which presents a potential source of instab
 ility for online learning. In this paper\, we introduce simple criteria fo
 r the adaptively generated data stream to guarantee the convergence of SGD
 . We show that the convergence speed of SGD with adaptive data is largely 
 similar to the classical iid setting\, as long as the mixing time of the p
 olicy-induced dynamics is factored in. Our Lyapunov-function analysis allo
 ws one to translate existing stability analysis of systems studied in oper
 ations research into convergence rates for SGD\, and we demonstrate this f
 or queuing and inventory management problems. We also showcase how our res
 ult can be applied to study an actor-critic policy gradient algorithm.\nTh
 is is joint work with Ethan Che and Xin Tong.\n&nbsp\;
LOCATION:External
END:VEVENT
END:VCALENDAR