Nigel Guenole

Preliminary Psychometric Properties of the Acceptance and Action Questionnaire-II: A Revised Measure of Psychological Inflexibility and Experiential Avoidance

The present research describes the development and psychometric evaluation of a second version of the Acceptance and Action Questionnaire (AAQ-II), which assesses the construct referred to as, variously, acceptance, experiential avoidance, and psychological inflexibility. Results from 2,816 participants across six samples indicate the satisfactory structure, reliability, and validity of this measure. For example, the mean alpha coefficient is .84 (.78-.88), and the 3- and 12-month test-retest reliability is .81 and .79, respectively. Results indicate that AAQ-II scores concurrently, longitudinally, and incrementally predict a range of outcomes, from mental health to work absence rates, that are consistent with its underlying theory. The AAQ-II also demonstrates appropriate discriminant validity. The AAQ-II appears to measure the same concept as the AAQ-I (r=.97) but with better psychometric consistency.

Formal Development Opportunities and Withdrawal Behaviors by Employees in Small and Medium-Sized Enterprises

This study explores direct and indirect relationships between involvement in formal training and development events, employee attitudes and withdrawal responses including turnover intentions and neglectful behavior for those employed in small and medium‐sized enterprises (SMEs). Questionnaire data were obtained from 185 staff employed in a diverse range of SMEs. Our results suggest that employees that participate in more training and development events are less likely to be considering leaving their employer and less likely to engage in neglectful behavior. However, the analysis revealed that the effects of participation in formal training and development are fully mediated by perceptions of organizational support and job satisfaction. In contrast to results from studies in large organizations, affective commitment was not found to be an influential determinant of employee exit intentions or neglect.

The NEO Personality Inventory-Revised: Factor Structure and Gender Invariance From Exploratory Structural Equation Modeling Analyses in a High-Stakes Setting

This study presents new analyses of NEO Personality Inventory–Revised (NEO-PI-R) responses collected from a large British sample in a high-stakes setting. The authors show the appropriateness of the five-factor model underpinning these responses in a variety of new ways. Using the recently developed exploratory structural equation modeling (ESEM) technique, the authors show that model fits improve markedly over conventional confirmatory factor analyses (CFA) of the same data set, but that (a) factor interpretations do not change under ESEM analyses, (b) ESEM factor scores, just like CFA factors scores, correlate at near unity with sums of observed scores, (c) NEO-PI-R facets under ESEM analyses are invariant across gender, and (d) ESEM highlights the inappropriateness of alpha and beta as a higher order representation of NEO-PI-R facets, whereas a CFA approach might lead researchers to believe in the appropriateness of these higher order factors. These results, coupled with the existing validity evidence for the NEO-PI-R, suggest that the five-factor structure is the most parsimonious structure for summarizing NEO-PI-R responses from high-stakes settings in the United Kingdom.

The consequences of ignoring measurement invariance for path coefficients in structural equation models

We report a Monte Carlo study examining the effects of two strategies for handling measurement non-invariance – modeling and ignoring non-invariant items – on structural regression coefficients between latent variables measured with item response theory models for categorical indicators. These strategies were examined across four levels and three types of non-invariance – non-invariant loadings, non-invariant thresholds, and combined non-invariance on loadings and thresholds – in simple, partial, mediated and moderated regression models where the non-invariant latent variable occupied predictor, mediator, and criterion positions in the structural regression models. When non-invariance is ignored in the latent predictor, the focal group regression parameters are biased in the opposite direction to the difference in loadings and thresholds relative to the referent group (i.e., lower loadings and thresholds for the focal group lead to overestimated regression parameters). With criterion non-invariance, the focal group regression parameters are biased in the same direction as the difference in loadings and thresholds relative to the referent group. While unacceptable levels of parameter bias were confined to the focal group, bias occurred at considerably lower levels of ignored non-invariance than was previously recognized in referent and focal groups.

Can Subject Matter Experts’ Ratings of Statement Extremity Be Used to Streamline the Development of Unidimensional Pairwise Preference Scales?

Interest in on-demand noncognitive assessment has flourished due to advances in computer technology and studies demonstrating noteworthy predictive validities for organizational outcomes. Computerized adaptive testing (CAT) based on the Zinnes-Griggs (ZG) ideal point item response theory (IRT) model may hold promise for organizational settings, because a large pool of items can be created from a modest number of stimuli, and the items have been shown to be resistant to some types of rater bias. However, sample sizes needed for marginal maximum likelihood (MML) estimation of statement parameters are quite large and could thus limit usefulness in practice. This article addresses that concern and its ramifications for CAT. Specifically, we conducted empirical and simulation studies to examine whether subject matter expert (SME) ratings of statement extremity (location) can be substituted for MML estimates to streamline test development and launch. Results showed that error in SME-based location estimates had little detrimental effect on score accuracy or validity, regardless of whether measures were constructed adaptively or nonadaptively. Implications for research involving small samples and CAT in field settings are discussed.

Placing Job Characteristics in Context: Cross-Temporal Meta-Analysis of Changes in Job Characteristics Since 1975

Despite frequent references to “the changing nature of work,” little empirical research has investigated proposed changes in work context perceptions. To address this gap, this study uses a cross-temporal meta-analysis to examine changes in five core job characteristics (e.g., task identity, task significance, skill variety, autonomy, and feedback from the job) as well as changes in the relationship between job characteristics and job satisfaction. An additional analysis of primary data is used to examine changes in two items related to interdependence. On average, workers perceived greater levels of skill variety and autonomy since 1975 and interdependence since 1985. In contrast, the results of a supplemental meta-analysis did not support significant changes in the association between the five core job characteristics and satisfaction over time. Thus, although there is some evidence for change in job characteristics, the findings do not support a change in the value placed on enriched work. Implications for researchers and organizations navigating the modern world of work are highlighted.

The Importance of Isomorphism for Conclusions about Homology: A Bayesian Multilevel Structural Equation Modeling Approach with Ordinal Indicators

We describe a Monte Carlo study examining the impact of assuming item isomorphism (i.e., equivalent construct meaning across levels of analysis) on conclusions about homology (i.e., equivalent structural relations across levels of analysis) under varying degrees of non-isomorphism in the context of ordinal indicator multilevel structural equation models (MSEMs). We focus on the condition where one or more loadings are higher on the between level than on the within level to show that while much past research on homology has ignored the issue of psychometric isomorphism, psychometric isomorphism is in fact critical to valid conclusions about homology. More specifically, when a measurement model with non-isomorphic items occupies an exogenous position in a multilevel structural model and the non-isomorphism of these items is not modeled, the within level exogenous latent variance is under-estimated leading to over-estimation of the within level structural coefficient, while the between level exogenous latent variance is overestimated leading to underestimation of the between structural coefficient. When a measurement model with non-isomorphic items occupies an endogenous position in a multilevel structural model and the non-isomorphism of these items is not modeled, the endogenous within level latent variance is under-estimated leading to under-estimation of the within level structural coefficient while the endogenous between level latent variance is over-estimated leading to over-estimation of the between level structural coefficient. The innovative aspect of this article is demonstrating that even minor violations of psychometric isomorphism render claims of homology untenable. We also show that posterior predictive p-values for ordinal indicator Bayesian MSEMs are insensitive to violations of isomorphism even when they lead to severely biased within and between level structural parameters. We highlight conditions where poor estimation of even correctly specified models rules out empirical examination of isomorphism and homology without taking precautions, for instance, larger Level-2 sample sizes, or using informative priors.

Can the Discretionary Nature of Certain Criteria Lead to Differential Prediction Across Cultural Groups

We examined the conjecture that relations between constructs across cultures may be susceptible to cultural moderation where the performance of the criterion construct is discretionary. This hypothesis was investigated using the relationship between personality and three performance constructs, with samples from the United States and New Zealand, two ideologically distinct cultures with respect to achievement orientation. All hypotheses were supported by results of hierarchical moderated regression analyses using bias free measures, suggesting that considering whether construct behaviors are discretionary is important when considering the merit of generalizing research findings across cultures.

Forced-Choice Assessment of Work-Related Maladaptive Personality Traits: Preliminary Evidence From an Application of Thurstonian Item Response Modeling:

This article describes an investigation of whether Thurstonian item response modeling is a viable method for assessment of maladaptive traits. Forced-choice responses from 420 working adults to a broad-range personality inventory assessing six maladaptive traits were considered. The Thurstonian item response model’s fit to the forced-choice data was adequate, while the fit of a counterpart item response model to responses to the same items but arranged in a single-stimulus design was poor. Monotrait heteromethod correlations indicated corresponding traits in the two formats overlapped substantially, although they did not measure equivalent constructs. A better goodness of fit and higher factor loadings for the Thurstonian item response model, coupled with a clearer conceptual alignment to the theoretical trait definitions, suggested that the single-stimulus item responses were influenced by biases that the independent clusters measurement model did not account for. Researchers may wish to consider forced-choice designs and appropriate item response modeling techniques such as Thurstonian item response modeling for personality questionnaire applications in industrial psychology, especially when assessing maladaptive traits. We recommend further investigation of this approach in actual selection situations and with different assessment instruments.

Are predictions based on situational judgement tests precise enough for feedback in leadership development

Situational judgement tests (SJTs) have much to recommend their use for personnel selection, but partly because of their low reliability, the role of SJTs in behavioural training is largely unexplored. However, research showing that SJTs cannot measure homogenous constructs very well is based exclusively on internal analyses, for example, alpha reliability and factor analysis. In this study, we investigated whether patterns of correlations with external criteria could be used to show that SJT dimension scores are homogenous enough for feedback purposes in leadership development. A multidimensional SJT was designed for 268 high-potential leaders on a development programme and used in conjunction with a multisource feedback instrument that measured the same competency framework. The SJT was criterion-keyed against the multisource feedback instrument using an N-fold cross validation strategy. Convergent and divergent correlations between the SJT scores and corresponding multisource dimension scores suggested that SJT scores can be constructed in a way that permits dimension-level feedback that would be useful in leadership development.