Computational Neuroscience Journal Club

Add to your list(s) Download to your calendar using vCal

• Dr Theoklitos Amvrosiadis (University of Cambridge)
• Wednesday 21 February 2024, 14:00-16:00
• CBL Seminar Room, Engineering Department, 4th floor Baker building.

If you have a question about this talk, please contact Puria Radmard.

Please join us for our Computational Neuroscience journal club on Wednesday 21st February at 2pm UK time in the CBL seminar roo

The title is “Meta Reinforcement Learning”, presented by Luke Johnston and Theoklitos Amvrosiadis.

Summary:

Following on from last week’s journal club on distributional reinforcement learning, this week we examine another widely discussed topic within RL - that of meta reinforcement learning. Meta RL has long been of interest to neuroscientists for the extent to which it better reflects the macroscopic behaviour of animals, where the ability to generalise learning experiences across contexts (ie ‘learning to learn’) forms a vital part of higher-order cognition. However unlike traditional single task learning – where, for example, the mechanisms of classical conditioning are well characterised – the neural basis of meta-learning remains a source of debate.

We begin by taking a look back at the seminal 2018 paper from Botvinick et al. [1], which proposes meta learning to occur within a ‘prefrontal network’ centred largely in the PFC . Specifically, the authors postulate a two-part meta RL system that includes initial dopamine based learning by way of the classical cortico–basal ganglia–thalamo–cortical loop to shape the recurrent connectivity of the PFC network, followed by a second PFC -centred algorithm which is dynamically tailored to the task at hand.

In the second part of the journal club, we will look at a recent paper from the Komiyama lab [2], in which through 2-photon calcium imaging experiments in mice and computational models, the researchers argue that synaptic plasticity within orbitofrontal cortex (OFC) is necessary for learning across sessions, but not for within-session learning in already trained subjects.

[1] Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., ...Botvinick, M. (2018). Prefrontal cortex as a meta-reinforcement learning system. Nat. Neurosci., 21, 860–868. doi: 10.1038/s41593-018-0147-8 [2] Hattori, R., Hedrick, N.G., Jain, A. et al. Meta-reinforcement learning via orbitofrontal cortex. Nat Neurosci 26, 2182–2191 (2023). https://doi.org/10.1038/s41593-023-01485-3 (

This talk is part of the Computational Neuroscience series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Biology
• Biology
• CBL Seminar Room, Engineering Department, 4th floor Baker building
• CBL important
• CamBridgeSens
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Neuroscience Seminars
• Cambridge talks
• Chris Davis' list
• Computational Neuroscience
• Computational and Biological Learning Seminar Series
• Featured lists
• Guy Emerson's list
• Hanchen DaDaDash
• Inference Group Journal Clubs
• Inference Group Summary
• Information Engineering Division seminar list
• Interested Talks
• Life Science
• Life Science Interface Seminars
• Life Sciences
• Life Sciences
• ME Seminar
• Neuroscience
• Neuroscience Seminars
• Neuroscience Seminars
• Quantum Matter Journal Club
• Required lists for MLG
• Stem Cells & Regenerative Medicine
• TQS Journal Clubs
• Trust & Technology Initiative - interesting events
• bld31
• custom
• dh539
• kt532's list
• my_list
• ndk22's list
• ob366-ai4er
• other talks
• rp587
• se456's list
• yk373's list

Note that ex-directory lists are not shown.