Marissa McBride (Imperial) - Lessons from structured expert elicitation using the IDEA protocol

Add to your list(s) Download to your calendar using vCal

• 
• Wednesday 10 June 2020, 15:35-16:05
• Seminar Room 1, Newton Institute.

If you have a question about this talk, please contact INI IT.

IDP - Infectious Dynamics of Pandemics: Mathematical and statistical challenges in understanding the dynamics of infectious disease pandemics

Abstract: Expert
judgment can support policy decision-making in situations where data are
scarce, knowledge is incomplete and decisions are imminent. Research has demonstrated
both the potential value and the potential dangers of expert judgments, and
established the benefit of using systematic, structured procedures to elicit
judgments from experts which anticipate and correct for the most severe and
predictable limitations of expert opinion. In this presentation I outline the
origins and development of the IDEA protocol, a Delphi-style structured
elicitation procedure that combines psychologically robust interactions among
experts with mathematical aggregation of individual estimates and structured
question formats to improve the accuracy of elicited judgments. I discuss
examples and emerging insights from its use to support decision-making across a
range of applications including biosecurity, natural resource management and
real-time geopolitical forecasting. Drawing from these experiences I reflect on
the challenges presented by COVID -19 and how structured elicitation procedures
might best support epidemiological modelling and simulation efforts in
providing timely public health policy advice.



Resources:

IDEA protocol for structured expert elicitation
– key reference

1. Hemming,
Victoria, et al. “A practical guide to structured expert elicitation using
the IDEA protocol.”Methods in Ecology and Evolution9.1
(2018): 169-180.

[This article provides a practical step-by-step
guide to carrying out a structured elicitation using the IDEA protocol. The
supplementary information includes ready-made templates and resources for
planning and implementing a structured elicitation using IDEA ]

An excellent introductory blog post discussing
this article and the need for structured expert elicitation is available here: https://methodsblog.com/2018/03/27/idea-protocol-2/

Expert elicitation references – introductory
& overview

2. Morgan, M.
Granger. “Use (and abuse) of expert elicitation in support of decision
making for public policy.”Proceedings of the National academy of
Sciences111.20 (2014): 7176-7184.

[Excellent overview article for using expert
judgment that is accessible for most audiences, and with an excellent reference
list for further reading]

3. Alahmadi, Amani, et al. “Influencing
public health policy with data-informed mathematical models of infectious
diseases: Recent developments and new challenges.”Epidemics(2020):
100393.

[Includes coverage of the role and challenges for
expert elicitation from the perspective of modelling infectious diseases]



Cognitive Biases

4. Cognitive biases infographic. Structured elicitation
protocols include steps that attempt to reduce the impacts of many of these
biases on the judgments elicited from experts.

https://www.visualcapitalist.com/every-single-cognitive-bias/


A
more complete version of the cognitive biases infographic that includes
definitions of each of the biases is also available here

This talk is part of the Isaac Newton Institute Seminar Series series.

This talk is included in these lists:

• All CMS events
• Featured lists
• INI info aggregator
• Isaac Newton Institute Seminar Series
• School of Physical Sciences
• Seminar Room 1, Newton Institute
• bld31

Note that ex-directory lists are not shown.