University of Cambridge > Talks.cam > Computational Neuroscience > Computational Neuroscience Journal Club

  Browse A-Z

Log in

Cambridge users (raven)details Other usersdetails No account?details

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

Computational Neuroscience Journal Club

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Rodrigo Echeveste.

Michael Rule will be presenting:

• Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization

• Nicolas Y. Masse, Gregory D. Grant, and David J. Freedman

• PNAS (2018)

• http://www.pnas.org/content/115/44/E10467

Abstract: Humans and most animals can learn new tasks without forgetting old ones. However, training artificial neural networks (ANNs) on new tasks typically causes them to forget previously learned tasks. This phenomenon is the result of “catastrophic forgetting,” in which training an ANN disrupts connection weights that were important for solving previous tasks, degrading task performance. Several recent studies have proposed methods to stabilize connection weights of ANNs that are deemed most important for solving a task, which helps alleviate catastrophic forgetting. Here, drawing inspiration from algorithms that are believed to be implemented in vivo, we propose a complementary method: adding a context-dependent gating signal, such that only sparse, mostly nonoverlapping patterns of units are active for any one task. This method is easy to implement, requires little computational overhead, and allows ANNs to maintain high performance across large numbers of sequentially presented tasks, particularly when combined with weight stabilization. We show that this method works for both feedforward and recurrent network architectures, trained using either supervised or reinforcement-based learning. This suggests that using multiple, complementary methods, akin to what is believed to occur in the brain, can be a highly effective strategy to support continual learning.

This talk is part of the Computational Neuroscience series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

Japanese Society in Cambridge ケンブリッジ日本人会 Continuity in Education and Cultural links between countries: Brexit Talk and Q&A Earthwatch Lecture

Other talks

Coercion in a Subjective World A tale of Terror and Erebus: international collaboration and competition in the search for Franklin’s lost expedition EXOMARS - A MARTIAN SURVIVAL STORY From nuclear transfer to prospects for cell replacement therapy: The origins and future of pluripotency and cellular reprogramming
 

© 2006-2024 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity