Observation and Intervention Incentives in Causal Influence Diagrams: Towards an Understanding of Powerful Machine Learning Systems

Add to your list(s) Download to your calendar using vCal

• Tom Everitt (DeepMind)
• Friday 25 January 2019, 11:00-12:00
• Engineering Department, LR5 (in front of library, 1st floor).

If you have a question about this talk, please contact Adrià Garriga Alonso.

As machine learning systems gain in capability and complexity, understanding their incentives will become increasingly important. In this paper, we model their objectives and environment interaction in graphical models called influence diagrams. This allows us to answer two fundamental questions about the incentives of a machine learning system directly from the graphical representation: (1) which nodes would the system like to observe in addition to its observations, and (2) which nodes would the system like to control in addition to its actions? The answers tell us which information and influence points need extra protection, and have applications to fairness and reward tampering. For example, we may want a classifier for job applications to not use the race of the candidate, and a reinforcement learning agent not to take direct control of its reward mechanism. Different algorithms and training paradigms can lead to different influence diagrams, so our results can help designing algorithms with less problematic observation and intervention incentives.

This talk is part of the Machine Learning @ CUED series.

This talk is included in these lists:

• Seminar
• All Talks (aka the CURE list)
• Biology
• CBL important
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge Neuroscience Seminars
• Cambridge talks
• Chris Davis' list
• Creating transparent intact animal organs for high-resolution 3D deep-tissue imaging
• Engineering Department, LR5 (in front of library, 1st floor)
• Featured lists
• Guy Emerson's list
• Hanchen DaDaDash
• Inference Group Summary
• Information Engineering Division seminar list
• Interested Talks
• Joint Machine Learning Seminars
• Life Science
• Life Sciences
• ML
• Machine Learning @ CUED
• Machine Learning Summary
• Neuroscience
• Neuroscience Seminars
• Neuroscience Seminars
• Required lists for MLG
• Simon Baker's List
• Stem Cells & Regenerative Medicine
• Trust & Technology Initiative - interesting events
• bld31
• dh539
• ndk22's list
• ob366-ai4er
• rp587
• yk373's list

Note that ex-directory lists are not shown.