Detecting misfitting response patterns in educational testing : An empirical application

Abstract

Person-fit indices were applied to detect inconsistent patterns of correct/incorrect answers to test questions (items) on a high-stakes educational test. We were particularly interested in (a) the presence of aberrant item score patterns and their configurations of item scores (b) the relation of inconsistent answer patterns to given subgroups of respondents. We show that persons that were consistently flagged across subtests had relatively low scores, which may be the result of extensive guessing. We found no relation of inconsistent behavior between males/females and for most subgroups. However, we found significant differences in mean person-fit scores for one subgroup when compared to other subgroups. This particular subgroup has a large proportion of non-native English speakers. We argue that person-fit indices should be used to routinely monitor test behavior.