SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning

Add to your list(s) Download to your calendar using vCal

• Nicholas Zolman (University of Washington)
• Wednesday 14 February 2024, 14:00-15:00
• Centre for Mathematical Sciences, MR14.

If you have a question about this talk, please contact Matthew Colbrook.

Deep Reinforcement Learning (DRL) has shown significant promise for uncovering sophisticated control policies that interact in environments with complicated dynamics, such as stabilizing the magnetohydrodynamics of a tokamak reactor and minimizing the drag force exerted on an object in a fluid flow. However, these algorithms require many training examples and can become prohibitively expensive for many applications. In addition, the reliance on deep neural networks results in an uninterpretable, black-box policy that may be too computationally challenging to use with certain embedded systems. Recent advances in sparse dictionary learning, such as the Sparse Identification of Nonlinear Dynamics (SINDy), have shown to be a promising method for creating efficient and interpretable data-driven models in the low-data regime. In this work, we extend ideas from the SIN Dy literature to introduce a unifying framework for combining sparse dictionary learning and DRL to create efficient, interpretable, and trustworthy representations of the dynamics model, reward function, and control policy. We demonstrate the effectiveness of our approaches on benchmark control environments and challenging fluids problems, achieving comparable performance to state-of-the-art DRL algorithms using significantly fewer interactions in the environment and an interpretable control policy orders of magnitude smaller than a deep neural network policy.

This talk is part of the Applied and Computational Analysis series.

This talk is included in these lists:

• All CMS events
• All Talks (aka the CURE list)
• Applied and Computational Analysis
• CMS Events
• Centre for Mathematical Sciences, MR14
• DAMTP info aggregator
• Featured lists
• Interested Talks
• My seminars
• Type the title of a new list here
• bld31

Note that ex-directory lists are not shown.