Frank A. Bosco

Meta-Analytic Review of Employee Turnover as a Predictor of Firm Performance

Previous research has primarily revealed a negative relationship between collective employee turnover and organizational performance. However, this research also suggests underlying complexity in the relationship. To clarify the nature of this relationship, the authors conduct a meta-analytic review in which they test and provide support for a portion of Hausknecht and Trevor’s model of collective turnover. The authors’ meta-analysis includes 48 independent samples reporting 157 effect size estimates (N = 24,943), tests six hypothesized moderator variables, and provides path analyses to test alternative conceptualizations of the turnover–organizational performance relationship. Results indicate that the mean corrected correlation between turnover and organizational performance is −.03, but this relationship is moderated by several important variables. For example, the relationship is stronger in manufacturing and transportation industries (−.07), for managerial employees (−.08), in midsize organizations (−.07), in samples from labor market economies (−.05), and when organizational performance is operationalized in terms of customer service (−.10) or quality and safety (−.12) metrics. In addition, proximal performance outcomes mediate relationships with financial performance. The authors discuss implications of their results for theory and practice and provide directions for future research.

Correlational Effect Size Benchmarks

Effect size information is essential for the scientific enterprise and plays an increasingly central role in the scientific process. We extracted 147,328 correlations and developed a hierarchical taxonomy of variables reported in Journal of Applied Psychology and Personnel Psychology from 1980 to 2010 to produce empirical effect size benchmarks at the omnibus level, for 20 common research domains, and for an even finer grained level of generality. Results indicate that the usual interpretation and classification of effect sizes as small, medium, and large bear almost no resemblance to findings in the field, because distributions of effect sizes exhibit tertile partitions at values approximately one-half to one-third those intuited by Cohen (1988). Our results offer information that can be used for research planning and design purposes, such as producing better informed non-nil hypotheses and estimating statistical power and planning sample size accordingly. We also offer information useful for understanding the relative importance of the effect sizes found in a particular study in relationship to others and which research domains have advanced more or less, given that larger effect sizes indicate a better understanding of a phenomenon. Also, our study offers information about research domains for which the investigation of moderating effects may be more fruitful and provide information that is likely to facilitate the implementation of Bayesian analysis. Finally, our study offers information that practitioners can use to evaluate the relative effectiveness of various types of interventions.

First Decade of Organizational Research Methods: Trends in Design, Measurement, and Data Analysis Topics

The authors conducted a content analysis of the 193 articles published in the first 10 volumes (1998 to 2007) of Organizational Research Methods (ORM). The most popular quantitative topics are surveys, temporal issues, and electronic/Web research (research design); validity, reliability, and level of analysis of the dependent variable (measurement); and multiple regression/correlation, structural equation modeling, and multilevel research (data analysis). The most popular qualitative topics are interpretive, policy capturing, and action research (research design); surveys and reliability (measurement); and interpretive, policy capturing, and content analysis (data analysis). The authors found upward trends in the attention devoted to surveys and electronic/Web research, interpretive, and action research (research design); level of analysis of the dependent variable and validity (measurement); and multilevel research (data analysis). Implications for training doctoral students, retooling researchers, future research on methodology, the advancement of the organizational sciences, and the extent to which ORM is fulfilling its mission are discussed.

Meta-Analytic Choices and Judgment Calls: Implications for Theory Building and Testing, Obtained Effect Sizes, and Scholarly Impact

The authors content analyzed 196 meta-analyses including 5,581 effect-size estimates published in Academy of Management Journal, Journal of Applied Psychology, Journal of Management, Personnel Psychology, and Strategic Management Journal from January 1982 through August 2009 to assess the presumed effects of each of 21 methodological choices and judgment calls on substantive conclusions. Results indicate that, overall, the various meta-analytic methodological choices available and judgment calls involved in the conduct of a meta-analysis have little impact on the resulting magnitude of the meta-analytically derived effect sizes. Thus, the present study, based on actual meta-analyses, casts doubt on previous warnings, primarily based on selective case studies, that judgment calls have an important impact on substantive conclusions. The authors also tested the fit of a multivariate model that includes relationships among theory-building and theory-testing goals, obtained effect sizes, year of publication of the meta-analysis, and scholarly impact (i.e., citations per year). Results indicate that the more a meta-analysis attempts to test an existing theory, the larger the number of citations, whereas the more a meta-analysis attempts to build new theory, the lower the number of citations. Also, in support of scientific particularism, as opposed to scientific universalism, the magnitude of the derived effects is not related to the extent to which a meta-analysis is cited. Taken together, the results provide a comprehensive data-based understanding of how meta-analytic reviews are conducted and the implications of these practices for theory building and testing, obtained effect sizes, and scholarly impact.

Debunking Myths and Urban Legends About Meta-Analysis

Meta-analysis is the dominant approach to research synthesis in the organizational sciences. We discuss seven meta-analytic practices, misconceptions, claims, and assumptions that have reached the status of myths and urban legends (MULs). These seven MULs include issues related to data collection (e.g., consequences of choices made in the process of gathering primary-level studies to be included in a meta-analysis), data analysis (e.g., effects of meta-analytic choices and technical refinements on substantive conclusions and recommendations for practice), and the interpretation of results (e.g., meta-analytic inferences about causal relationships). We provide a critical analysis of each of these seven MULs, including a discussion of why each merits being classified as an MUL, their kernels of truth value, and what part of each MUL represents misunderstanding. As a consequence of discussing each of these seven MULs, we offer best-practice recommendations regarding how to conduct meta-analytic reviews.

Response to Comment on "Estimating the reproducibility of psychological science"

Gilbert et al. conclude that evidence from the Open Science Collaboration’s Reproducibility Project: Psychology indicates high reproducibility, given the study methodology. Their very optimistic assessment is limited by statistical misconceptions and by causal inferences from selectively interpreted, correlational data. Using the Reproducibility Project: Psychology data, both optimistic and pessimistic conclusions about reproducibility are possible, and neither are yet warranted.

The Happy Culture: A Theoretical, Meta-Analytic, and Empirical Review of the Relationship Between Culture and Wealth and Subjective Well-Being

Do cultural values enhance financial and subjective well-being (SWB)? Taking a multidisciplinary approach, we meta-analytically reviewed the field, found it thinly covered, and focused on individualism. In counter, we collected a broad array of individual-level data, specifically an Internet sample of 8,438 adult respondents. Individual SWB was most strongly associated with cultural values that foster relationships and social capital, which typically accounted for more unique variance in life satisfaction than an individual’s salary. At a national level, we used mean-based meta-analysis to construct a comprehensive cultural and SWB database. Results show some reversals from the individual level, particularly masculinity’s facet of achievement orientation. In all, the happy nation has low power distance and low uncertainty avoidance, but is high in femininity and individualism, and these effects are interrelated but still partially independent from political and economic institutions. In short, culture matters for individual and national well-being.

Executive attention as a predictor of employee performance : reconsidering the relationship between cognitive ability and adverse impact potential

The validity-adverse impact tradeoff associated with the relationships among cognitive ability, ethnicity, and employee performance represents one of the field’s most pressing concerns. In particular, the modal finding of a positive relationship between test validity and adverse impact potential ultimately asks practitioners to make decisions regarding acceptable reductions in test validity in order to meet diversity goals. While numerous and diverse, the attempts to remedy this concern scarcely involve the investigation of alternative frameworks and measures of cognitive ability, and this act may hold promise for addressing the validity-adverse impact tradeoff. One recent construct from cognitive psychology, executive attention, proposes a distinct view of cognitive ability that explains performance on cognitively simple and complex tasks including multi-tasking, an increasingly demanded organizational need. In a sample of 121 bank employees, we assessed the extent to which general cognitive ability and executive attention predict supervisory ratings of job performance and managerial simulation performance. Results indicated that, compared to general cognitive ability, executive attention was associated with (a) stronger relationships with performance criteria, (b) unique variance in performance criteria, (c) weaker relationships with employee ethnicity, and (d) lower levels of test bias. We discuss implications for theory and practice.

What’s Past (and Present) Is Prologue: Interactions Between Justice Levels and Trajectories Predicting Behavioral Reciprocity

Much of organizational justice research has tended to take a static approach, linking employees’ contemporaneous justice levels to outcomes of interest. In the present study, we tested a dynamic model emphasizing the interactive influences of both justice levels and trajectories for predicting behavioral social exchange outcomes. Specifically, our model posited both main effects and interactions between present justice levels and past justice changes over time in predicting helping behavior and voluntary turnover behavior. Data over four yearly measurement periods from 4,348 employees of a banking organization generally supported the notion that justice trajectories interact with absolute levels to predict both outcomes. Together, the findings highlight how employees invoke present fairness evaluations within the context of past fairness trends—rather than either in isolation—to inform decisions about behaviorally reciprocating at work.

Scientific findings as big data for research synthesis: The metaBUS project

We describe the metaBUS project, a large-scale research curation effort supported by the Digging into Data Challenge. This ongoing effort involves the extraction and cu-ration of a corpus of more than one million individual research findings from organizational research. The approach involves the development of a comprehensive hierarchical taxonomy containing thousands of variables studied in the scientific space. Technological enablements allow linkages between the taxonomy and research findings fostering the development of a research finding search engine at the level of individual primary studies that allows the conduct of instant meta-analyses on virtually any relation of interest in organizational research.