The probability of exceedance as a nonparametric person-fit statistic for tests of moderate length

In this report we present a measure to identify unlikely patterns of correct/incorrect answers to test questions (commonly referred to as items). Such patterns may occur as a result of cheating behavior. The proposed measure is the probability of exceedance (PE). PE provides information about the probability of a correct/incorrect answer pattern, conditional on the test taker’s total score. Although this concept is not new, it is hardly if ever applied in practice. In this report we show how the PE of a response vector can be computed and how misfitting response patterns are detected. A simulation study is conducted to investigate the robustness of this procedure.

4