“Extreme reviewing”: Use of text-mining to reduce impractical screening workload in extremely large scoping reviews

Add to your list(s) Download to your calendar using vCal

• Ian Shemilt, Evidence Synthesis Programme, Behaviour and Health Research Unit
• Friday 02 November 2012, 13:00-14:00
• Large Seminar Room, 1st Floor, Institute of Public Health, University Forvie Site, Robinson Way, Cambridge.

If you have a question about this talk, please contact Dr Simon Richard White.

Background: In scoping reviews of broad evidence bases, boundaries of relevant evidence may be initially fuzzy, with a refined conceptual understanding of interventions and related phenomena of interest an intended output of the process rather than its starting point. Searches are therefore sensitive, retrieving large record sets that can be impractical to screen using conventional methods. Objectives: To evaluate use of text-mining to reduce impractical screening workload in two large-scale scoping reviews of evidence for impacts of (i) choice architecture interventions (CA) and (ii) changes in the economic environment (EE) on health-related behaviours and corollary outcomes. Methods: Baseline inclusion rates (BIRs) were estimated by screening random samples of records drawn from retrieved sets comprising over 800K (CA) and 1 million (EE) de-duplicated records. Text-mining technologies were applied to prioritise records for manual screening. 47,541 (CA) and 46,099 (EE) prioritised records were manually screened and observed inclusion rates (OIRs) recorded. Text-mining performance was measured in terms of OIRs relative to BIRs. Eligible records prioritised using text-mining were compared with those located using parallel snowball searches to assess unique yields and potential biases of each approach. Results: Overall unadjusted OIRs were 10.1 (CA) and 8.3 (EE) times higher than BIRs. Text-mining reduced manual screening workload by 90% (CA) and 88% (EE) compared with conventional methods (absolute reductions of approximately 430,000 (CA) and 378,000 (EE) records), to identify 85% (CA) and 38% (EE) of remaining eligible records. Conclusions: This study expands an emerging corpus of empirical evidence for use of text-mining to support screening, by demonstrating its feasibility, strengths and limitations in extremely large-scale scoping reviews. By reducing screening workload, text-mining made it possible to assemble, describe and delimit large and complex evidence-bases that crossed research-disciplinary boundaries. Findings are transferable to other large-scale scoping and systematic reviews that incorporate conceptual development or explanatory dimensions.

This talk is part of the Bradford Hill Seminars series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• BHRU Annual Lecture 2015
• BHRU Annual Lecture 2016
• Biology
• Cambridge Global Food Security
• Cambridge Infectious Disease
• Cambridge Infectious Diseases
• Cardiovascular Epidemiology Unit Special Seminars
• Department of Psychiatry talks stream
• Department of Public Health and Primary Care
• Large Seminar Room, 1st Floor, Institute of Public Health, University Forvie Site, Robinson Way, Cambridge
• Life Sciences
• Life Sciences
• ME Seminar
• Primary Care
• PublicHealth@Cambridge
• my_list
• ndk22's list
• other talks
• primary care

Note that ex-directory lists are not shown.