Mark J. Schmit

Frame-of-reference effects on personality scale scores and criterion-related validity.

Increased use of personality inventories in employee selection has led to concerns regarding factors that influence the validity of such measures. A series of studies was conducted to examine the influence of frame of reference on responses to a personality inventory. Study 1 involved both within-subject and between-groups designs to assess the effects of testing situation (general instructions vs. applicant instructions) and item type (work specific vs. noncontextual) on responses to the NEO Five-Factor Inventory (P. T. Costa & R. R. McCrae, 1989). Results indicated that a work-related testing context and workrelated items led to more positive responses. A second study found differences in the validity of a measure of conscientiousness, depending on the frame of reference of respondents. Specifically, context-specific items were found to have greater validity. Implications for personnel selection are discussed.

Test-taking dispositions: A missing link?

Test-taking dispositions and motivation have been proposed as moderators of employment test validity. In a simulated, multiorganization employment system, scores on the Test Attitude Survey (TAS) were found to moderate the validities of both an ability and a personality test used to predict a performance criterion. The criterion-related validity of the personality test was found to be higher for a subsample with less positive test-taking motivation than for a subsample with higher test-taking motivation. An opposite effect was found among the subgroups for the ability test. Range restriction and the predictability of individuals with high and low test-taking dispositions were examined as explanations for the findings

Measurement Equivalence Between Applicant and Incumbent Groups: An IRT Analysis of Personality Scales

Incumbents are often used in the development and validation of a wide variety of personnel selection instruments, including noncognitive instruments such as personality tests. However, the degree to which assumed motivational factors impact the measurement equivalence and validity of tests developed using incumbents has not been adequately addressed. This study addressed this issue by examining the measurement equivalence of 6 personality scales between a group applying for jobs as sales managers in a large retail organization (N = 999) and a group of sales managers currently employed in that organization (N = 796). A graded item response theory model (Samejima, 1969) was fit to the personality scales in each group. Results indicated that moderately large differences existed in personality scale scores (approximately 1/2 standard deviation units) but only one of the six scales contained any items that evidenced differential item functioning and no scales evidenced differential test functioning. In addition, person-level analyses showed no apparent differences across groups in aberrant responding. The results suggest that personality measures used for selection retain similar psychometric properties to those used in incumbent validation studies.

Effects of item context specificity on the measurement equivalence of a personality inventory.

Personality measures with items that ask respondents to characterize themselves across a range of situations are increasingly used for personnel selection purposes. Research conducted in a laboratory setting has found that personality items may have different psychometric characteristics depending on the degree to which that range is widened or narrowed (i.e., degree of contextualization). This study is an attempt to study the psychometric impact of contextualization in a large field sample (N = 1,078). Respondents were given either a contextualized (at work) or noncontextualized (in general) version of the six facets of the conscientiousness factor of the NEO PI-R. Analyses were conducted at the facet and item levels. Results were mixed but indicated that error variances tended to be slightly lower for the work-specific instrument in comparison to the noncontextualized instrument. Implications for personality inventory development, validation, and use are discussed.

Workplace Integrity: Differences in Perceptions of Behaviors and Situational Factors

Pre-employment integrity tests assume differences between honest and dishonest individuals in their perceptions of the degree of dishonesty of given behaviors. This exploratory study investigated whether individuals classified as dishonest by an overt integrity test had different perceptions of the honesty of behaviors than those classified as honest. Differences in perceptions of situational factors promoting or deterring honesty were also examined. Dishonest individuals had different perceptions of behaviors and situations than honest individuals, suggesting that the relative influence of situational interventions may vary with the general integrity of a workforce.