Causal Reasoning from Meta-reinforcement Learning

Add to your list(s) Download to your calendar using vCal

• Jakub Perlin (University of Cambridge)
• Wednesday 13 March 2019, 17:00-19:00
• Engineering Department, CBL Seminar room BE4-38.

If you have a question about this talk, please contact Adrià Garriga Alonso.

This week we read “Causal Reasoning from Meta-reinforcement Learning” by Ishita Dasgupta et al. (https://arxiv.org/abs/1901.08162). Please read the paper before coming.

As usual, there will be free pizza. The first half hour is for stragglers to finish reading.

Abstract:

Discovering and exploiting the causal structure in the environment is a crucial challenge for intelligent agents. Here we explore whether causal reasoning can emerge via meta-reinforcement learning. We train a recurrent network with model-free reinforcement learning to solve a range of problems that each contain causal structure. We find that the trained agent can perform causal reasoning in novel situations in order to obtain rewards. The agent can select informative interventions, draw causal inferences from observational data, and make counterfactual predictions. Although established formal causal reasoning algorithms also exist, in this paper we show that such reasoning can arise from model-free reinforcement learning, and suggest that causal reasoning in complex settings may benefit from the more end-to-end learning-based approaches presented here. This work also offers new strategies for structured exploration in reinforcement learning, by providing agents with the ability to perform—and interpret—experiments.

Invite your friends to join the mailing list (https://lists.cam.ac.uk/mailman/listinfo/eng-safe-ai), the Facebook group (https://www.facebook.com/groups/1070763633063871) or the talks.cam page (https://talks.cam.ac.uk/show/index/80932). Details about the next meeting, the week’s topic and other events will be advertised in these places.

This talk is part of the Engineering Safe AI series.

This talk is included in these lists:

• Cambridge talks
• Chris Davis' list
• Engineering Department, CBL Seminar room BE4-38
• Engineering Safe AI
• Trust & Technology Initiative - interesting events

Note that ex-directory lists are not shown.