University of Cambridge > > Bradford Hill Seminars > “Extreme reviewing”: Use of text-mining to reduce impractical screening workload in extremely large scoping reviews

“Extreme reviewing”: Use of text-mining to reduce impractical screening workload in extremely large scoping reviews

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Dr Simon Richard White.

Background: In scoping reviews of broad evidence bases, boundaries of relevant evidence may be initially fuzzy, with a refined conceptual understanding of interventions and related phenomena of interest an intended output of the process rather than its starting point. Searches are therefore sensitive, retrieving large record sets that can be impractical to screen using conventional methods. Objectives: To evaluate use of text-mining to reduce impractical screening workload in two large-scale scoping reviews of evidence for impacts of (i) choice architecture interventions (CA) and (ii) changes in the economic environment (EE) on health-related behaviours and corollary outcomes. Methods: Baseline inclusion rates (BIRs) were estimated by screening random samples of records drawn from retrieved sets comprising over 800K (CA) and 1 million (EE) de-duplicated records. Text-mining technologies were applied to prioritise records for manual screening. 47,541 (CA) and 46,099 (EE) prioritised records were manually screened and observed inclusion rates (OIRs) recorded. Text-mining performance was measured in terms of OIRs relative to BIRs. Eligible records prioritised using text-mining were compared with those located using parallel snowball searches to assess unique yields and potential biases of each approach. Results: Overall unadjusted OIRs were 10.1 (CA) and 8.3 (EE) times higher than BIRs. Text-mining reduced manual screening workload by 90% (CA) and 88% (EE) compared with conventional methods (absolute reductions of approximately 430,000 (CA) and 378,000 (EE) records), to identify 85% (CA) and 38% (EE) of remaining eligible records. Conclusions: This study expands an emerging corpus of empirical evidence for use of text-mining to support screening, by demonstrating its feasibility, strengths and limitations in extremely large-scale scoping reviews. By reducing screening workload, text-mining made it possible to assemble, describe and delimit large and complex evidence-bases that crossed research-disciplinary boundaries. Findings are transferable to other large-scale scoping and systematic reviews that incorporate conceptual development or explanatory dimensions.

This talk is part of the Bradford Hill Seminars series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.


© 2006-2024, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity