Christopher D. Nye

Vocational Interests and Performance: A Quantitative Summary of Over 60 Years of Research

Despite early claims that vocational interests could be used to distinguish successful workers and superior students from their peers, interest measures are generally ignored in the employee selection literature. Nevertheless, theoretical descriptions of vocational interests from vocational and educational psychology have proposed that interest constructs should be related to performance and persistence in work and academic settings. Moreover, on the basis of Holland’s (1959, 1997) theoretical predictions, congruence indices, which quantify the degree of similarity or person–environment fit between individuals and their occupations, should be more strongly related to performance than interest scores alone. Using a comprehensive review of the interest literature that spans more than 60 years of research, a meta-analysis was conducted to examine the veracity of these claims. A literature search identified 60 studies and approximately 568 correlations that addressed the relationship between interests and performance. Results showed that interests are indeed related to performance and persistence in work and academic contexts. In addition, the correlations between congruence indices and performance were stronger than for interest scores alone. Thus, consistent with interest theory, the fit between individuals and their environment was more predictive of performance than interest alone.

Assessing goodness of fit: Simple rules of thumb simply do not work

Confirmatory factor analytic (CFA) models are frequently used in many areas of organizational research. Due to their popularity, CFA models and issues about their fit have received a vast amount of attention during the past several decades. The purpose of this study was to examine several measures of fit and the appropriateness of previously developed ‘‘rules of thumb’’ for their interpretation. First, an empirical example is used to illustrate the effects of nonnormality on maximum likelihood (ML) estimation and to demonstrate the importance of diagonally weighted least squares (DWLS) estimation for organizational research. Then, the results of a simulation study are presented to show that appropriate cutoff values for DWLS estimation vary considerably across conditions. Finally, regression equations are described to aid researchers in selecting cutoff values for assessing the fit of DWLS solutions, given a desired level of Type I error. The results summarized here have important implications for the interpretation and use of CFA models.

Alleviating the Burden of Emotional Labor: The Role of Social Sharing

Difficult customer interactions cause service employees to experience negative emotions and to engage in emotional labor. The present laboratory study examined whether social sharing (i.e., talking about an emotionally arousing work event with one’s coworkers) can attenuate the residual anger lingering after a taxing service episode. Participants assumed the role of customer service representatives for a fictitious technical support hotline and encountered either neutral or difficult service interactions. After fielding three easy or three difficult calls, participants were given the opportunity to engage in social sharing by talking about (a) the facts that just transpired, (b) the feelings aroused by the encounters, or (c) the positive aspects of the experience, or they were asked to complete a filler task. Results from quantitative data revealed that participants who engaged in difficult (vs. neutral) customer interactions reported more surface acting and felt more anger. Engaging in social sharing was beneficial: All three types of social sharing were effective in reducing the anger aroused by handling demanding customers. Findings from qualitative analyses suggested that different mechanisms might have contributed to the effectiveness of the three types of social sharing. Future research directions and implications for practice are discussed.

Two-Step Testing in Employee Selection: Is Score Inflation a Problem?

Unproctored Internet testing in employee selection has become increasingly popular over the past few years. However, there is a concern that cheating during unproctored administrations may influence the test results in terms of score inflation. The current research attempts to determine the extent of cheating on an unproctored Internet test of perceptual speed by analyzing data from a parallel version of the test administered in a proctored setting. Among a sample of 856 job applicants from nine European countries, there was no evidence that cheating occurred. In fact, evidence of an effect operating in the opposite direction was obtained. These results indicate that the use of unproctored Internet testing is appropriate for this type of test.

Effect Size Indices for Analyses of Measurement Equivalence: Understanding the Practical Importance of Differences Between Groups

Because of the practical, theoretical, and legal implications of differential item functioning (DIF) for organizational assessments, studies of measurement equivalence are a necessary first step before scores can be compared across individuals from different groups. However, commonly recommended criteria for evaluating results from these analyses have several important limitations. The present study proposes an effect size index for confirmatory factor analytic (CFA) studies of measurement equivalence to address 1 of these limitations. The application of this index is illustrated with personality data from American English, Greek, and Chinese samples. Results showed a range of nonequivalence across these samples, and these differences were linked to the observed effects of DIF on the outcomes of the assessment (i.e., group-level mean differences and adverse impact).

From ABLE to TAPAS: A new generation of personality tests to support military selection and classification decisions

This article discusses 3 modern forced-choice personality tests developed for the U.S Armed Services to provide resistance to faking and other forms of response distortion: the Assessment of Individual Motivation, the Navy Computerized Adaptive Personality Scales, and the Tailored Adaptive Personality Assessment System. These tests represent the transition from Likert to forced-choice formats and from static to computerized adaptive item selection to meet the challenges of large-scale, high-stakes testing environments. For each test, we briefly describe the personality constructs that are assessed, the response format and scoring methods, and selected ongoing research and development efforts. We also highlight the potential of these tests for personnel selection, classification, and diagnostic screening purposes.

Proctored Versus Unproctored Internet Tests: Are Unproctored Noncognitive Tests as Predictive of Job Performance?

As the usage of unproctored Internet testing (UIT) increases in selection settings, concerns about the validity of such practices must be addressed. While recent examinations of the issue have focused primarily on the practicality, ethics, and potential legal ramifications of UIT, this paper provides an examination of the criterion-related validity of unproctored assessments. Using a database of validity evidence, we examine the predictive validity of several noncognitive assessments (i.e., personality and biodata) with respect to a number of subjective and objective job performance metrics. Results generally indicate that assessments administered in proctored and unproctored settings have similar validities. Limitations of this research and implications for practice are discussed.

Factor Structure of the Air Force Officer Qualifying Test Form S: Analysis and Comparison with Previous Forms

Due to its importance for assignment and classification in the U.S. Air Force, the Air Force Officer Qualifying Test (AFOQT) has received a substantial amount of research. Recently, the AFOQT was revised to reduce administrative burden and test-taker fatigue. However, the new version, the AFOQT Form S, was implemented without explicitly examining the latent structure of the exam. The current study examined the factor structure of Form S as well as its measurement equivalence across race- and sex-based groups. Results indicated that a bifactor model with a general intelligence factor and five content-specific factors fit the best. The measurement equivalence of the AFOQT across gender and racial/ethnic groups was also supported.

Too Good to Be True? Understanding Change in Organizational Outcomes

When evaluating organizational outcomes, strong experimental designs are often not practical. As a result, assessments of change may be confounded by a number of threats to their validity. Therefore, attempts have been made to detect or control for many of these artifacts. This article explores a method for detecting response artifacts, using measurement and relational equivalence techniques. The large drop in the reported levels of sexual harassment in the U.S. military between 1995 and 2002 provides the context for testing this method. The results suggest that at least part of the reported decrease in harassment was artifactual. The implications for evaluations of organizational change are discussed.

Personality Trait Differences Between Young and Middle-Aged Adults: Measurement Artifacts or Actual Trends?

A growing body of research demonstrates that older individuals tend to score differently on personality measures than younger adults. However, recent research using item response theory (IRT) has questioned these findings, suggesting that apparent age differences in personality traits merely reflect artifacts of the response process rather than true differences in the latent constructs. Conversely, other studies have found the opposite-age differences appear to be true differences rather than response artifacts. Given these contradictory findings, the goal of the present study was to examine the measurement equivalence of personality ratings drawn from large groups of young and middle-aged adults (a) to examine whether age differences in personality traits could be completely explained by measurement nonequivalence and (b) to illustrate the comparability of IRT and confirmatory factor analysis approaches to testing equivalence in this context. Self-ratings of personality traits were analyzed in two groups of Internet respondents aged 20 and 50 (n = 15,726 in each age group). Measurement nonequivalence across these groups was negligible. The effect sizes of the mean differences due to nonequivalence ranged from -.16 to .15. Results indicate that personality trait differences across age groups reflect actual differences rather than merely response artifacts.