Reinforcement Learning with Exogenous States and Rewards

Add to your list(s) Download to your calendar using vCal

• Professor Thomas G. Dietterich, School of EECS, Oregon State University
• Wednesday 01 April 2026, 13:30-14:30
• Department of Engineering - LT1.

If you have a question about this talk, please contact Kimberly Cole.

Exogenous state variables and rewards can slow reinforcement learning by injecting uncontrolled variation into the reward signal. In this talk, I’ll describe our work on formalizing exogenous state variables and rewards. Then I’ll discuss our main result: if the reward function decomposes additively into endogenous and exogenous components, the MDP can be decomposed into an exogenous Markov Reward Process (based on the exogenous reward) and an endogenous Markov Decision Process (optimizing the endogenous reward). Any optimal policy for the endogenous MDP is also an optimal policy for the original MDP , but because the endogenous reward typically has reduced variance, the endogenous MDP is easier to solve. The second half of the talk will introduce two algorithms for causal discovery of the exogenous subspace of the state space. Once discovered, we can model the exogenous reward function and remove it from the MDP so that RL can focus on the endogenous reward only. Experiments on a variety of challenging synthetic MDPs show that these methods, applied online, discover large exogenous state spaces and produce substantial speedups in reinforcement learning. (Joint work with George Trimponias (Intercom.io))

This will be followed by a discussion from 2.30pm to 3pm about the future of research in the presence of automated AI/ML research:

How should we choose research topics to study (either for automation or topics that are not amenable to automation)? How should the research be reported? What are the work products? How should we evaluate automated research? How can we separate real research from fake imitations of research? How can we assimilate an exponentially-exploding number of research results?

This talk is part of the Information Engineering Distinguished Lecture Series series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge University Engineering Department Talks
• Cambridge talks
• Centre for Smart Infrastructure & Construction
• Chris Davis' list
• Computational Continuum Mechanics Group Seminars
• Department of Engineering - LT1
• Featured lists
• Information Engineering Distinguished Lecture Series
• Information Engineering Division seminar list
• Interested Talks
• Probabilistic Systems, Information, and Inference Group Seminars
• School of Technology
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• rp587
• yk449

Note that ex-directory lists are not shown.