Scoring open-ended questions

Add to your list(s) Download to your calendar using vCal

• Gunter Maris, CITO (Dutch National Institute for Educational Measurement)
• Sunday 13 September 2009, 11:45-12:25
• Lecture Theatre, CRI Cambridge, Li Ka Shing Centre, Robinson Way, Cambridge.

If you have a question about this talk, please contact Professor John Rust.

This talk has been canceled/deleted

POSTPONED DUE TO ADVERSE WEATHER CONDITIONS

Scoring open ended questions almost inevitably requires the use of one or more human raters to grade the responses. In the literature, several methods can be found for the measurement of ability from the responses to open ended questions. These methods either oversimplify (ignore the effect of human raters) or overspecify (model the behavior of individual raters). In this presentation, a method is presented that neither oversimplifies nor overspecifies. The method derives from three principles: Random assignment of raters to person-item combinations; If raters agree there is no reason to doubt their judgment; and If a first rater gives a positive (negative) rating, the probability that the second rating is positive (negative) as well is a monotone increasing (decreasing) function of ability. We extensively discuss and motivate these principles and show, in an informal manner, how a simple measurement model can be derived from them. The model is illustrated with real data from the examinations of Dutch as a foreign language.

This talk is part of the The Psychometrics Centre series.

This talk is included in these lists:

This talk is not included in any other list

Note that ex-directory lists are not shown.