Charles A. Pierce

Effect size and power in assessing moderating effects of categorical variables using multiple regression: a 30-year review.

The authors conducted a 30-year review (1969-1998) of the size of moderating effects of categorical variables as assessed using multiple regression. The median observed effect size (f(2)) is only .002, but 72% of the moderator tests reviewed had power of .80 or greater to detect a targeted effect conventionally defined as small. Results suggest the need to minimize the influence of artifacts that produce a downward bias in the observed effect size and put into question the use of conventional definitions of moderating effect sizes. As long as an effect has a meaningful impact, the authors advise researchers to conduct a power analysis and plan future research designs on the basis of smaller and more realistic targeted effect sizes.

Cautionary Note on Reporting Eta-Squared Values from Multifactor ANOVA Designs.

The authors provide a cautionary note on reporting accurate eta-squared values from multifactor analysis of variance (ANOVA) designs. They reinforce the distinction between classical and partial eta-squared as measures of strength of association. They provide examples from articles published in premier psychology journals in which the authors erroneously reported partial eta-squared values as representing classical etasquared values. Finally, they discuss broader impacts of inaccurately reported etasquared values for theory development, meta-analytic reviews, and intervention programs.

Meta-Analytic Review of Employee Turnover as a Predictor of Firm Performance

Previous research has primarily revealed a negative relationship between collective employee turnover and organizational performance. However, this research also suggests underlying complexity in the relationship. To clarify the nature of this relationship, the authors conduct a meta-analytic review in which they test and provide support for a portion of Hausknecht and Trevor’s model of collective turnover. The authors’ meta-analysis includes 48 independent samples reporting 157 effect size estimates (N = 24,943), tests six hypothesized moderator variables, and provides path analyses to test alternative conceptualizations of the turnover–organizational performance relationship. Results indicate that the mean corrected correlation between turnover and organizational performance is −.03, but this relationship is moderated by several important variables. For example, the relationship is stronger in manufacturing and transportation industries (−.07), for managerial employees (−.08), in midsize organizations (−.07), in samples from labor market economies (−.05), and when organizational performance is operationalized in terms of customer service (−.10) or quality and safety (−.12) metrics. In addition, proximal performance outcomes mediate relationships with financial performance. The authors discuss implications of their results for theory and practice and provide directions for future research.

Correlational Effect Size Benchmarks

Effect size information is essential for the scientific enterprise and plays an increasingly central role in the scientific process. We extracted 147,328 correlations and developed a hierarchical taxonomy of variables reported in Journal of Applied Psychology and Personnel Psychology from 1980 to 2010 to produce empirical effect size benchmarks at the omnibus level, for 20 common research domains, and for an even finer grained level of generality. Results indicate that the usual interpretation and classification of effect sizes as small, medium, and large bear almost no resemblance to findings in the field, because distributions of effect sizes exhibit tertile partitions at values approximately one-half to one-third those intuited by Cohen (1988). Our results offer information that can be used for research planning and design purposes, such as producing better informed non-nil hypotheses and estimating statistical power and planning sample size accordingly. We also offer information useful for understanding the relative importance of the effect sizes found in a particular study in relationship to others and which research domains have advanced more or less, given that larger effect sizes indicate a better understanding of a phenomenon. Also, our study offers information about research domains for which the investigation of moderating effects may be more fruitful and provide information that is likely to facilitate the implementation of Bayesian analysis. Finally, our study offers information that practitioners can use to evaluate the relative effectiveness of various types of interventions.

First Decade of Organizational Research Methods: Trends in Design, Measurement, and Data Analysis Topics

The authors conducted a content analysis of the 193 articles published in the first 10 volumes (1998 to 2007) of Organizational Research Methods (ORM). The most popular quantitative topics are surveys, temporal issues, and electronic/Web research (research design); validity, reliability, and level of analysis of the dependent variable (measurement); and multiple regression/correlation, structural equation modeling, and multilevel research (data analysis). The most popular qualitative topics are interpretive, policy capturing, and action research (research design); surveys and reliability (measurement); and interpretive, policy capturing, and content analysis (data analysis). The authors found upward trends in the attention devoted to surveys and electronic/Web research, interpretive, and action research (research design); level of analysis of the dependent variable and validity (measurement); and multilevel research (data analysis). Implications for training doctoral students, retooling researchers, future research on methodology, the advancement of the organizational sciences, and the extent to which ORM is fulfilling its mission are discussed.

The Impact of Communal-Mastery Versus Self-Mastery on Emotional Outcomes During Stressful Conditions: A Prospective Study of Native American Women

Past research has examined the stress resiliency of individuals high in sense of personal-mastery. However, it has been theorized that within more collectivist cultures, a sense of shared efficacy, which we call communal-mastery, may be more central to peoples resiliency in the face of challenging life circumstances. We compared the impact of sense of self-mastery (i.e., “I am the key to my success”) to that of communal-mastery (i.e., “I am successful by virtue of my social attachments”) in a prospective study among a group of rural 103 Native American women residing on Indian Reservations in Montana. We found that women high in communal-mastery experienced less increase in depressive mood and anger, especially when faced with high stress circumstances, than women who were low in communal-mastery. In addition, the beneficial impact of communal-mastery was found to be more effective than self-mastery for these women.

Walking New Avenues in Management Research Methods and Theories: Bridging Micro and Macro Domains

One of the most critical challenges faced by management scholars is how to integrate micro and macro research methods and theories. This article introduces a special issue of the Journal of Management addressing this integration challenge. First, the authors describe the nature of the micro—macro divide and its challenge for the field of management. Second, the authors provide a summary of each of the four guest editorials and seven articles published in the special issue and how each piece, in its own unique way and adopting a different perspective, makes a novel contribution toward addressing this challenge. Finally, they offer suggestions for future research that they hope will stimulate greater integration of management research with the goal of bridging not only the micro—macro gap but also the science—practice gap.

Meta-Analytic Choices and Judgment Calls: Implications for Theory Building and Testing, Obtained Effect Sizes, and Scholarly Impact

The authors content analyzed 196 meta-analyses including 5,581 effect-size estimates published in Academy of Management Journal, Journal of Applied Psychology, Journal of Management, Personnel Psychology, and Strategic Management Journal from January 1982 through August 2009 to assess the presumed effects of each of 21 methodological choices and judgment calls on substantive conclusions. Results indicate that, overall, the various meta-analytic methodological choices available and judgment calls involved in the conduct of a meta-analysis have little impact on the resulting magnitude of the meta-analytically derived effect sizes. Thus, the present study, based on actual meta-analyses, casts doubt on previous warnings, primarily based on selective case studies, that judgment calls have an important impact on substantive conclusions. The authors also tested the fit of a multivariate model that includes relationships among theory-building and theory-testing goals, obtained effect sizes, year of publication of the meta-analysis, and scholarly impact (i.e., citations per year). Results indicate that the more a meta-analysis attempts to test an existing theory, the larger the number of citations, whereas the more a meta-analysis attempts to build new theory, the lower the number of citations. Also, in support of scientific particularism, as opposed to scientific universalism, the magnitude of the derived effects is not related to the extent to which a meta-analysis is cited. Taken together, the results provide a comprehensive data-based understanding of how meta-analytic reviews are conducted and the implications of these practices for theory building and testing, obtained effect sizes, and scholarly impact.

Debunking Myths and Urban Legends About Meta-Analysis

Meta-analysis is the dominant approach to research synthesis in the organizational sciences. We discuss seven meta-analytic practices, misconceptions, claims, and assumptions that have reached the status of myths and urban legends (MULs). These seven MULs include issues related to data collection (e.g., consequences of choices made in the process of gathering primary-level studies to be included in a meta-analysis), data analysis (e.g., effects of meta-analytic choices and technical refinements on substantive conclusions and recommendations for practice), and the interpretation of results (e.g., meta-analytic inferences about causal relationships). We provide a critical analysis of each of these seven MULs, including a discussion of why each merits being classified as an MUL, their kernels of truth value, and what part of each MUL represents misunderstanding. As a consequence of discussing each of these seven MULs, we offer best-practice recommendations regarding how to conduct meta-analytic reviews.

Attraction in organizations: A model of workplace romance

Antecedents and consequences of participating in a romantic relationship in the workplace are critically examined. These factors are derived from social-psychological theories of interpersonal attraction, romantic attraction, love, emotion, and social exchange, as well as from previous research addressing romance in the workplace. The antecedent factors explain the processes by which romantic relationships develop between two members of the same organization, and the consequent factors highlight both the beneficial and detrimental effects that such relationships have on the participants, coworkers, and the host organization. Based on these formation and impact factors, a conceptual model of workplace romance is developed along with a series of testable propositions in order to facilitate and direct future research on romantic organizational behavior.