The Impact of Aberrant Response on Reliability and Validity

Abstract

ABSTRACT Aberrant response has an important impact on item parameter estimation, individuals’ evaluation, and other statistical analysis. There are various types of aberrant response behaviors in educational and psychological tests, like sleeping, guessing, and plodding. Random response is the most common one. The purpose of this research was to clarify the impact of random response on reliability and validity. Both simulation study and empirical study were conducted. In simulation study, one pilot experiment and two formal experiments were fulfilled to discuss the impact on construct validity. There were five factors that were taken into account. They were the rate of aberrant response individuals, the number of dimensions, the number of items per dimension, the correlations between dimensions, and the types of random response (global random response and local random response). Five fit indices based on confirmatory factor analysis model were calculated to evaluate construct validity. And response data were generated by multidimensional item response model. In empirical study, Nomophobia Scale and Freshman Adaptation Scale were used to investigate the impact of random response on criterion validity and test–retest reliability, respectively.