![]() |
COOKIES: By using this website you agree that we can place Google Analytics Cookies on your device for performance monitoring. | ![]() |
University of Cambridge > Talks.cam > Cambridge Psychometrics Centre Seminars > Crowd IQ: Weighting Votes in Crowdsourcing and Multi-Agent Systems using Item Response Theory
![]() Crowd IQ: Weighting Votes in Crowdsourcing and Multi-Agent Systems using Item Response TheoryAdd to your list(s) Download to your calendar using vCal
If you have a question about this talk, please contact Chan Yin Wah Fiona. Crowd IQ: Weighting Votes in Crowdsourcing and Multi-Agent Systems using Item Response Theory This work introduces a Weighted Majority Voting (WMV) approach for boosting the performance of crowds and multi-agent systems. Our approach relies on the estimation of performance of individual agents based on their ability modelled using Item Response Theory (IRT), a core theory in modern psychological assessment. We provide a brief introduction to IRT and derive the formula defining the weight of individual agents based on their ability and parameters of the task. Using simulated and empirical samples, we show that WMV (1) offers a significant boost in performance across tasks and crowds of different size, (2) is not substantially affected by not knowing the true task parameters-a common scenario in the real-life setting, (3) significantly increases the chances of the crowd to outperform their smartest member, and (4) boosts the marginal contributions of additional agents. Also, using the example of general intelligence, we show that an agent’s ability can be estimated with appropriate accuracy using a relatively short screening test. We conclude with a discussion of the advantages of WMV and suggest that implementing ability- rather than performance-based metrics can offer great advantages in the context of crowdsourcing and multi-agent platforms. This talk is part of the Cambridge Psychometrics Centre Seminars series. This talk is included in these lists:
Note that ex-directory lists are not shown. |
Other listsBeyond Profit Think Tank BAS Aurora Innovation Centre IMS-MRL External Seminar seriesOther talksAnalytical Ultracentrifugation (AUC) Simulating wave propagation in elastic systems using the Finite-Difference-Time-Domain method Debtors’ schedules: a new source for understanding the economy in 18th-century England My Life in Science Seminar “Publishing in Science: an Inside Look" NatHistFest: the 99th Conversazione and exhibition on the wonders of the natural world. Cambridge-Lausanne Workshop 2018 - Day 1 |