Michael T. Braun

Advancing Multilevel Research Design Capturing the Dynamics of Emergence

Multilevel theory and research have advanced organizational science but are limited because the research focus is incomplete. Most quantitative research examines top-down, contextual, cross-level relationships. Emergent phenomena that manifest from the bottom up from the psychological characteristics, processes, and interactions among individuals—although examined qualitatively—have been largely neglected in quantitative research. Emergence is theoretically assumed, examined indirectly, and treated as an inference regarding the construct validity of higher level measures. As a result, quantitative researchers are investigating only one fundamental process of multilevel theory and organizational systems. This article advances more direct, dynamic, and temporally sensitive quantitative research methods designed to unpack emergence as a process. We argue that direct quantitative approaches, largely represented by computational modeling or agent-based simulation, have much to offer with respect to illuminating the mechanisms of emergence as a dynamic process. We illustrate how indirect and direct approaches can be complementary and, appropriately integrated, have the potential to substantially advance theory and research. We conclude with a set of recommendations for advancing multilevel research on emergent phenomena in teams and organizations.

Exploratory regression analysis: A tool for selecting models and determining predictor importance

Linear regression analysis is one of the most important tools in a researcher’s toolbox for creating and testing predictive models. Although linear regression analysis indicates how strongly a set of predictor variables, taken together, will predict a relevant criterion (i.e., the multiple R), the analysis cannot indicate which predictors are the most important. Although there is no definitive or unambiguous method for establishing predictor variable importance, there are several accepted methods. This article reviews those methods for establishing predictor importance and provides a program (in Excel) for implementing them (available for direct download at http://dl.dropbox.com/u/2480715/ERA.xlsm?dl=1). The program investigates all 2p – 1 submodels and produces several indices of predictor importance. This exploratory approach to linear regression, similar to other exploratory data analysis techniques, has the potential to yield both theoretical and practical benefits.

Social network approaches to leadership: : An integrative conceptual review.

Contemporary definitions of leadership advance a view of the phenomenon as relational, situated in specific social contexts, involving patterned emergent processes, and encompassing both formal and informal influence. Paralleling these views is a growing interest in leveraging social network approaches to study leadership. Social network approaches provide a set of theories and methods with which to articulate and investigate, with greater precision and rigor, the wide variety of relational perspectives implied by contemporary leadership theories. Our goal is to advance this domain through an integrative conceptual review. We begin by answering the question of why-Why adopt a network approach to study leadership? Then, we offer a framework for organizing prior research. Our review reveals 3 areas of research, which we term: (a) leadership in networks, (b) leadership as networks, and (c) leadership in and as networks. By clarifying the conceptual underpinnings, key findings, and themes within each area, this review serves as a foundation for future inquiry that capitalizes on, and programmatically builds upon, the insights of prior work. Our final contribution is to advance an agenda for future research that harnesses the confluent ideas at the intersection of leadership in and as networks. Leadership in and as networks represents a paradigm shift in leadership research-from an emphasis on the static traits and behaviors of formal leaders whose actions are contingent upon situational constraints, toward an emphasis on the complex and patterned relational processes that interact with the embedding social context to jointly constitute leadership emergence and effectiveness.

A Cautionary Note on Modeling Growth Trends in Longitudinal Data.

Random coefficient and latent growth curve modeling are currently the dominant approaches to the analysis of longitudinal data in psychology. The application of these models to longitudinal data assumes that the data-generating mechanism behind the psychological process under investigation contains only a deterministic trend. However, if a process, at least partially, contains a stochastic trend, then random coefficient regression results are likely to be spurious. This problem is demonstrated via a data example, previous research on simple regression models, and Monte Carlo simulations. A data analytic strategy is proposed to help researchers avoid making inaccurate inferences when observed trends may be due to stochastic processes.

Capturing the multilevel dynamics of emergence Computational modeling, simulation, and virtual experimentation

Emergent phenomena—those that manifest bottom-up from the psychological characteristics, perceptions, and interactions among individuals—are a fundamental dynamic process in multilevel theory, but have been treated in a very limited way in the research literature. In particular, treatments are largely assumed (rather than observed directly), retrospective, and static. This paper describes a research paradigm designed to examine directly the dynamics of micro-meso—individual, dyad, and team—emergent phenomena. We identify, describe, and illustrate the sequence of theoretical, measurement, computational, data analytic, and systematic research activities that are necessary to operationalize and utilize the paradigm. We illustrate the paradigm development process using our research, focused on learning and team knowledge emergence, and highlight key design principles that can be applied to examine other emergent phenomena in teams. We conclude with a discussion of contributions, strengths and limitations, and generalization of the approach to other emergent phenomena in teams.

Spurious Results in the Analysis of Longitudinal Data in Organizational Research

Organizational scientists increasingly focus on the dynamics of human behavior through longitudinal and event sampling methodologies. Random coefficient modeling such as hierarchical linear modeling and latent growth modeling is the dominant analytical method for longitudinal data. These models require that the covariance matrix of the errors is time invariant. Unfortunately, if unmeasured or unpredictable influences (i.e., unmeasured variables) consistently impact the dynamic process under investigation, the error term can become time-dependent. If random coefficient modeling is used to analyze data with time-dependent errors, then a serious inflation of Type I error rates, known as spurious regression, is observed. Monte Carlo simulation results from several common random coefficient models are presented to highlight the scope and severity of the problem, focusing on the potential mistaken inferences researchers can make. An analytic strategy is proposed to aid researchers in determining the underlying structure of the error covariance matrix, and alternative statistical techniques are given to analyze data that may contain a time-dependent error term.

Special Considerations for the Acquisition and Wrangling of Big Data

Organizational scientists must capitalize on the big data revolution to better understand the nomothetic, idiographic, multilevel, and/or dynamic processes that make up today’s workplace. Simultaneously, researchers must collect high-quality data and be careful, diligent, and deliberate during data wrangling and data analysis so that all results can be replicated and all inferences are appropriate. Unfortunately, big data create many uncommon challenges during data acquisition and data wrangling that must be considered and overcome to fulfill the promise and potential of big data. Specifically, during acquisition, organizational scientists must become familiar with concepts like web scraping and databases, determine how to divide big data files into manageable chunks for cleaning and analysis, all while ensuring not to violate data usage rules and regulations. Likewise, once acquired, to effectively wrangle data so that they are ready for analysis researchers must be able to handle multiple file formats and data encoding standards, utilize a variety of software to visualize and diagnose data structure, and be adept at using functions and algorithms to determine variable structure and evaluate records and variables for missing or erroneous information. The current article provides a concise definition of big data and addresses each of these novel challenges and concepts related to big data acquisition and wrangling, specifically focusing on providing guidance and recommendations. Finally, a detailed big data example, team development using play-by-play basketball data, is provided. Each step of the process of scraping the data from the web as well as wrangling the multilevel big data into tidy data form is discussed, accompanied by a supplemental R file that contains all of the code necessary for researchers to replicate the described procedure.

Spurious relationships in growth curve modeling: The effects of stochastic trends on regression-based models

Forward Eduardo Salas 1. Introduction Part 1: Statistical Analysis 2. Catastrophe Theory and Its Applications in Organizational Psychology Stephen J. Guastello 3. Longitudinal Growth Modeling Robert E. Ployhart and Youngsang Kim 4. Harnessing the Power of Social Network Analysis to Explain Organizational Phenomena Yuval Kalish 5. Latent Class Procedures: Recent Development and Applications Mo Wang and Le Zhou 6. Spurious Relationships in Growth Curve Modeling: The Effects of Stochastic Trends on Regression-based Models Michael Braun, Goran Kuljanin and Richard P. DeShon 7. Practical Applications of Data Mining for Organizational Research Jeffrey M. Stanton Part 2: Research Design and Measurement 8.Use of Conditional Reasoning to Measure the Power Motive Lawrence R. James, James M. LeBreton, Terence R. Mitchell, Daniel R. Smith, Justin A. DeSimone, Robert Cookson, and HyeJoo Lee 9. Qualitative Research Methods for Industrial and Organizational Psychology Robert P. Gephart, Jr. 10. Experience Sampling Methodology NikolaosDimotakis, Remus Ilies, and Timothy A. Judge 11. Synthetic Task Environments for Understanding Human Performance Eduardo Salas, Aaron S. Dietz, Mary Jane Sierra, & Kimberly Smith-Jentsch 12. Petri Nets: Modeling the Complexity of Modern Jobs Michael D. Coovert 13. A Brief Primer on Neuroimaging Methods Cory Adis and James C. Thompson 14. Knowledge and Skill Measurement: Insights from Outside of I/O Psychology Nikki Dudley-Meislahn, E. Daly Vaughn, Eric J. Sydell, &Marisa A. Seeds

The accuracy of dominance analysis as a metric to assess relative importance: The joint impact of sampling error variance and measurement unreliability.

Dominance analysis (DA) has been established as a useful tool for practitioners and researchers to identify the relative importance of predictors in a linear regression. This article examines the joint impact of two common and pervasive artifacts—sampling error variance and measurement unreliability—on the accuracy of DA. We present Monte Carlo simulations that detail the decrease in the accuracy of DA in the presence of these artifacts, highlighting the practical extent of the inferential mistakes that can be made. Then, we detail and provide a user-friendly program in R (R Core Team, 2017) for estimating the effects of sampling error variance and unreliability on DA. Finally, by way of a detailed example, we provide specific recommendations for how researchers and practitioners should more appropriately interpret and report results of DA.