Statistical biases in peer review

Add to your list(s) Download to your calendar using vCal

• Remco Heesen (Faculty of Philosophy)
• Thursday 26 April 2018, 15:30-17:00
• Seminar Room 2, Department of History and Philosophy of Science.

If you have a question about this talk, please contact Agnes Bolinska.

Various biases are known to affect the peer review system, which is used to judge journal articles for their suitability for publication and grant proposals for their suitability for funding. These biases are generally attributed to cognitive biases held by individual peer reviewers. For example, gender bias in peer review is explained by the (explicit or implicit) gender bias of individual peer reviewers, as evidenced by the generally lower scores given to submissions authored by women. Here I introduce the notion of ‘purely statistical biases’: biases in peer review that arise even when individual peer reviewers are unbiased. This notion suggests that certain social groups or research programs may be disadvantaged by the peer review system even in the absence of cognitive biases. I use formal models to identify three possible mechanisms for purely statistical biases. The first mechanism relies on differences in information about authors available to decision makers. The second mechanism relies on differences in the underlying distributions of the ‘quality’ of submissions. Finally, the third mechanism comes into play when reviewers judge submissions on multiple criteria: aggregating these judgments into a final decision leads to a third possible source of bias.

This talk is part of the Departmental Seminars in History and Philosophy of Science series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge talks
• Department of History and Philosophy of Science
• Departmental Seminars in History and Philosophy of Science
• Featured lists
• History and Philosophy of Science long list
• Philosophy Events
• Seminar Room 2, Department of History and Philosophy of Science
• Trust & Technology Initiative - interesting events
• hc446
• jer64's list

Note that ex-directory lists are not shown.