Abstract

A common way to calculate confidence intervals in a testing situation is to assume that the standardised maximum likelihood estimator for the person parameter is normally distributed. Although maximum likelihood estimators are approximately normally distributed, this assumption is often questionable for short and medium test length. For example, when working with the Rasch model and a test length of n, there are only n + 1 different possible outcomes of the estimation and so the distribution of the standardized estimator will be quite discrete. Another disadvantage of the standard approach is that even when the normality assumption holds, it is not clear if the obtained confidence intervals meet any criterion of optimality, for example regarding some notion of “smallness”. A third undesirable property is that they will always depend on the specific choice of the estimator. Therefore, we will try to define confidence intervals in a more “exact” way by making use of the relationship between confidence intervals and hypothesis testing. We will outline three different approaches for confidence interval constructions and discuss how far these meet different optimality criteria.