Chet Robie

The Relation between Job Level and Job Satisfaction

Two studies examined the relation between job level and job satisfaction. Study 1 was a metaanalysis of the relation between job level and job satisfaction using data drawn from 35 independent samples (N= 18,534). It was found that as job level increased, so did job satisfaction. Several possible moderators of this relation were identified. Study 2 was a primary study of the relation between job level and job satisfaction using 4 measures of job level and 5 facets of job satisfaction (N = 530 in 4 hospitals). There was a high degree of convergence among the measures of job level and a consistently positive relation between the job level and job facet satisfaction measures. These studies suggest that the relation between job level and job satisfaction is positive across most operationalizations of the 2 constructs and that other moderators should be investigated to account for the large residual variance in effect sizes identified.

A New Look at within-Source Interrater Reliability of 360-Degree Feedback Ratings

This study investigated within-source interrater reliability of supervisor, peer, and subordinate feedback ratings made for managerial development. Raters provided 360-degree feedback ratings on a sample of 153 managers. Using generalizability theory, results indicated that little within-source agreement exists; a large portion of the error variance is attributable to the combined rater main effect and Rater x Ratee effect; more raters are needed than currently used to reach acceptable levels of reliability; supervisors are the most reliable with trivial differences between peers and subordinates when the numbers of raters and items are held constant; and peers are the most reliable, followed by subordinates, followed by supervisors, under conditions commonly encountered in practice. Implications for the validity, design, and maintenance of 360-degree feedback systems are discussed along with directions for future research in this area.

Uncovering Faking Samples in Applicant, Incumbent, and Experimental Data Sets: An Application of Mixed-Model Item Response Theory

Most research on faking personality inventories has assumed that individuals are either faking or responding honestly; distinctions within these two groups are generally not made. A recently developed statistical technique, mixed-model item response theory, was used to identify subgroups within samples of individuals taking two different personality inventories under various conditions. For one personality test, the authors obtained a sample of applicants and incumbents. For the second test, a sample of honest respondents and two samples of respondents instructed to fake (coached and ad lib) were obtained. Across the applicant and incumbent data sets, the authors generally found that three classes were needed to model all response patterns. In the experimental faking study, an honest class and an extreme faking class were needed to model the data. Overall, these results demonstrate that previous assumptions about the nature of faking on personality inventories have been too restrictive.

Faking and the validity of conscientiousness: a Monte Carlo investigation.

The article reports the findings from a Monte Carlo investigation examining the impact of faking on the criterion-related validity of Conscientiousness for predicting supervisory ratings of job performance. Based on a review of faking literature, 6 parameters were manipulated in order to model 4,500 distinct faking conditions (5 [magnitude] x 5 [proportion] x 4 [variability] x 3 [faking-Conscientiousness relationship] x 3 [faking-performance relationship] x 5 [selection ratio]). Overall, the results indicated that validity change is significantly affected by all 6 faking parameters, with the relationship between faking and performance, the proportion of fakers in the sample, and the magnitude of faking having the strongest effect on validity change. Additionally, the association between several of the parameters and changes in criterion-related validity was conditional on the faking-performance relationship. The results are discussed in terms of their practical and theoretical implications for using personality testing for employee selection.

A process model of shiftwork and health

The authors developed and tested a process model of adaptation to shiftwork, which hypothesizes that various individual and situational variables influence the development of sleep and social and domestic disturbances. Both types of disturbances trigger various types of coping behavior, leading to several proximal outcomes. The end result is the development of chronic health problems in the form of digestive and cardiovascular symptoms. The model was tested with survey data collected from 2 samples of nurses (N = 1,532) in the United Kingdom and was cross-validated against a 3rd sample of industrial workers (N = 370). Results indicate support for the model across the 3 samples, although some sample-specific and subgroup effects were found. Results have direct implications for the development of shiftwork theory and interventions.

Measurement Equivalence Between Applicant and Incumbent Groups: An IRT Analysis of Personality Scales

Incumbents are often used in the development and validation of a wide variety of personnel selection instruments, including noncognitive instruments such as personality tests. However, the degree to which assumed motivational factors impact the measurement equivalence and validity of tests developed using incumbents has not been adequately addressed. This study addressed this issue by examining the measurement equivalence of 6 personality scales between a group applying for jobs as sales managers in a large retail organization (N = 999) and a group of sales managers currently employed in that organization (N = 796). A graded item response theory model (Samejima, 1969) was fit to the personality scales in each group. Results indicated that moderately large differences existed in personality scale scores (approximately 1/2 standard deviation units) but only one of the six scales contained any items that evidenced differential item functioning and no scales evidenced differential test functioning. In addition, person-level analyses showed no apparent differences across groups in aberrant responding. The results suggest that personality measures used for selection retain similar psychometric properties to those used in incumbent validation studies.

Effects of Nonlinearity and Heteroscedasticity on the Validity of Conscientiousness in Predicting Overall Job Performance

Most research on the relationship between personality and overall job performance assumes linearity and homoscedasticity. This study investigated the prevalence and nature of nonlinearities and heteroscedasticies in relationships between conscientiousness and supervisory ratings of overall job performance across five independent samples using both concurrent (k = 4) and predictive (k = 1) designs. Hierarchical polynomial regression analyses found evidence of robust linear effects but no evidence of statistically significant quadratic or cubic effects. A statistically and practically significant heteroscedastic effect was found in only one sample such that lower errors of prediction were evidenced in the ends in comparison to the middle of the bivariate distribution. Implications for the use of conscientiousness in personnel selection are suggested. Limitations of the current study and directions for future research are noted.

Structural Equivalence of a Measure of Cross-Cultural Adjustment

This study investigated the structural equivalence of a measure of cross-cultural adjustment in culturally dissimilar samples using confirmatory factor analysis. The two samples consisted of 155 international students studying at a midwestern American university and 153 American expatriate managers on assignment in either Taiwan or Belgium. Results indicated that the hypothesized factor structure was invariant and had a relatively good fit in both samples. The implication of these results is that adjustment may have similar dimensions across cultures. If substantiated through future studies, these results would enable researchers to have more confidence in cross-cultural comparisons of mean levels of adjustment or correlations of adjustment with external variables.

Effects of item context specificity on the measurement equivalence of a personality inventory.

Personality measures with items that ask respondents to characterize themselves across a range of situations are increasingly used for personnel selection purposes. Research conducted in a laboratory setting has found that personality items may have different psychometric characteristics depending on the degree to which that range is widened or narrowed (i.e., degree of contextualization). This study is an attempt to study the psychometric impact of contextualization in a large field sample (N = 1,078). Respondents were given either a contextualized (at work) or noncontextualized (in general) version of the six facets of the conscientiousness factor of the NEO PI-R. Analyses were conducted at the facet and item levels. Results were mixed but indicated that error variances tended to be slightly lower for the work-specific instrument in comparison to the noncontextualized instrument. Implications for personality inventory development, validation, and use are discussed.

Revising the JDI Work Satisfaction Subscale: Insights into Stress and Control.

The Job Descriptive Index (JDI) is a widely used facet measure of job satisfaction that has undergone several revisions since its first publication in 1969. A revision in 1985 added items that, in subsequent research, appeared to tap work stress rather than work satisfaction. To illuminate the contaminating effect of these items, the authors analyzed two samples (n = 1,623 and n = 314) that also contained test items hypothesized to tap job control. A multigroup confirmatory factor analysis supported a three-factor solution and provided evidence supporting the removal of the contaminating items from the JDI. The presence of factorially complex items, however, indicated that some content overlap remains in the measure. Hierarchical regression results supported predictions about relationships between satisfaction, stress, and control. Results of the study have implications for development of occupational satisfaction measures and further refinement of stress, control, and satisfaction constructs.