Charles A. Scherbaum

Estimating Statistical Power and Required Sample Sizes for Organizational Research Using Multilevel Modeling

The use of multilevel modeling to investigate organizational phenomena is rapidly increasing. Unfortunately, little advice is readily available for organizational researchers attempting to determine statistical power when using multilevel models or when determining sample sizes for each level that will maximize statistical power. This article presents an introduction to statistical power in multilevel models. The unique factors influencing power in multilevel models and calculations for estimating power for simple fixed effects, variance components, and cross-level interactions are presented. The results of simulation studies and the existing general rules of thumb are discussed, and the available power analysis software is reviewed.

Examining the Relationship Between Race-Based Differential Item Functioning and Item Difficulty:

Recent research examining racial differences on standardized cognitive tests has focused on the impact of test item difficulty. Studies using data from the SAT and GRE have reported a correlation between item difficulty and differential item functioning (DIF) such that minority test takers are less likely than majority test takers to respond correctly to easy test items. The statistical techniques used and the effect sizes reported in these studies have been heavily criticized. This study addresses these criticisms by examining the relationship between item difficulty and DIF by using alternative statistical techniques based on item response theory and a different standardized test. The results replicate previous research and provide support for the generalizability of the findings.

Testing a Computational Model of the Goal-Level Effect: An Example of a Neglected Methodology

To encourage the use of computational modeling in organizational behavior research, an example computational model is developed and rigorous tests of it presented. Specifically, a computational model based on control theory was created to test the theory’s explanation of the goal-level effect (e.g., higher goals lead to higher performance). Data from simulations of the model were compared with the behavior of 32 undergraduate students performing a scheduling task under various within-subject manipulations and across time. Correlational analyses indicated that the model accounted for most of the participants’data, with coefficients between the model and each participant’s behavior mostly in the high 90s.

Examining faking on personality inventories using unfolding item response theory models.

A concern about personality inventories in diagnostic and decision-making contexts is that individuals will fake. Although there is extensive research on faking, little research has focused on how perceptions of personality items change when individuals are faking or responding honestly. This research demonstrates how the delta parameter from the generalized graded unfolding item response theory model can be used to examine how individuals’ perceptions about personality items might change when responding honestly or when faking. The results indicate that perceptions changed from honest to faking conditions for several neuroticism items. The direction of the change varied, indicating that faking can operate to increase or decrease scores within a personality factor.

Examining the relationship between organizational change and financial loss

Purpose – Research on the effectiveness of organizational change initiatives tends to focus primarily on the positive benefits of organizational change including improved financial performance. Rarely are negative outcomes examined, such as financial losses resulting from change initiatives. However, negative outcomes are possible, common, and understudied. The purpose of this paper is to examine the relationship between organizational change and financial loss. Design/methodology/approach – The research used a database of insurance losses from a global reinsurance company over a 30-year period. Each loss event was examined to determine the cause of the loss, the amount of loss, and type of organizational change if any that preceded the loss. Findings – The results indicate that losses attributed to the organization and its employees are preceded by an organizational change initiative more often than not. In particular, the occurrence of losses attributable to the organization and its employees were prece...

Moderating Effect of Job Type on Job Characteristic—Worker Outcome Relationships

Justina M. Oliveira Southern New Hampshire University, Manchester, USA Charles Scherbaum Baruch College and the Graduate Center, City University of New York, New York, USA  There has been a great deal of research examining how characteristics of jobs impact employee work perceptions as well as worker outcomes and this question is typically viewed through the lens of the job characteristics model (JCM) (Oldham & Hackman, 1980). Despite the substantial literature on this model, questions are now being raised about the JCM’s generalizability across job types. This study tested job type as a moderator for job characteristic-worker outcome relationships through the use of a multilevel design and data from 11,718 employees in 24 countries. Findings indicate a moderating effect of job type on relationships between the job characteristics of autonomy, skill variety, and task significance with the worker outcomes of job satisfaction, organizational commitment, turnover intentions, and perceptions of stress. This has implications for work design across jobs and findings indicate these relationships are complicated in that they are dependent upon job type, the specific job characteristic, and the worker outcome in question. Thus, given the cost and time involved in work redesign, such redesign attempts should be tailored to the outcomes a given organization intends to improve instead of implementing higher levels of numerous job characteristics across all job types and situations.