Scott Tonidandel

Determining the Statistical Significance of Relative Weights

Relative weight analysis is a procedure for estimating the relative importance of correlated predictors in a regression equation. Because the sampling distribution of relative weights is unknown, researchers using relative weight analysis are unable to make judgments regarding the statistical significance of the relative weights. J. W. Johnson (2004) presented a bootstrapping methodology to compute standard errors for relative weights, but this procedure cannot be used to determine whether a relative weight is significantly different from zero. This article presents a bootstrapping procedure that allows one to determine the statistical significance of a relative weight. The authors conducted a Monte Carlo study to explore the Type I error, power, and bias associated with their proposed technique. They illustrate this approach here by applying the procedure to published data.

Multivariate relative importance: extending relative weight analysis to multivariate criterion spaces.

For years, organizational scholars have sought effective ways to evaluate the importance of predictors included in a regression analysis. Recent techniques, such as general dominance weights and relative weights, have shown great promise for guiding evaluations of predictor importance. Nevertheless, questions remain regarding how one should investigate relative importance in the presence of a multidimensional criterion variable. The purpose of this article is to extend understanding of relative importance statistics to multivariate designs. The authors review the concept of relative importance and discuss a new procedure for calculating estimates of importance in the presence of multiple correlated criteria. Finally, a published correlation matrix is reanalyzed and a Monte Carlo simulation conducted to compare the new procedure with another technique for estimating importance. Unlike canonical solutions, which are often uninterpretable, the proposed multivariate relative weights provide an intuitive index regarding the relationship between predictors and criteria. Implications for organizational researchers are discussed.

Assessing the Multigroup Ethnic Identity Measure for Measurement Equivalence Across Racial and Ethnic Groups

An increasing number of organizational researchers examine the effects of ethnic identity and other-group orientation. In doing so, many use Phinneys (1992) Multigroup Ethnic Identity Measure (MEIM), which purportedly allows simultaneous assessment of various groups. Although several studies demonstrate adequate validity and reliability for scores on the MEIM, the only two studies that have assessed its measurement equivalence across racial and ethnic groups (a) focus exclusively on the ethnic identity component, (b) use entirely adolescent samples, and (c) obtain somewhat mixed results. Because ethnic identity is still developing during adolescence, it cannot be assumed that equivalence or lack thereof among adolescents will generalize to adults. The present study examines the measurement equivalence of both components of the MEIM across racial and ethnic groups using a sample of 1,349 White, Hispanic, African American, and Asian American adults. The results suggest that Roberts et al.s revised version demonstrates evidence of measurement equivalence.

Determining the Relative Importance of Predictors in Logistic Regression: An Extension of Relative Weight Analysis:

Techniques such as dominance analysis and relative weight analysis have been proposed recently to evaluate more accurately predictor importance in ordinary least squares (OLS) regression. Similar questions of predictor importance also arise in instances where logistic regression is the primary mode of analysis. This article presents an extension of relative weight analysis that can be applied in logistic regression and thus aids in the determination of predictor importance. We briefly review relative importance techniques and then discuss a new procedure for calculating relative importance estimates in logistic regression. Finally, we present a substantive example applying this new approach to an example data set.

Residualized Relative Importance Analysis A Technique for the Comprehensive Decomposition of Variance in Higher Order Regression Models

The current article notes that the standard application of relative importance analyses is not appropriate when examining the relative importance of interactive or other higher order effects (e.g., quadratic, cubic). Although there is a growing demand for strategies that could be used to decompose the predicted variance in regression models containing such effects, there has been no formal, systematic discussion of whether it is appropriate to use relative importance statistics in such decompositions, and if it is appropriate, how to go about doing so. The purpose of this article is to address this gap in the literature by describing three different yet related strategies for decomposing variance in higher-order multiple regression models—hierarchical F tests (a between-sets test), constrained relative importance analysis (a within-sets test), and residualized relative importance analysis (a between- and within-sets test). Using a previously published data set, we illustrate the different types of inferences these three strategies permit researchers to draw. We conclude with recommendations for researchers seeking to decompose the predicted variance in regression models testing higher order effects.

Overworked in America

Purpose – Though a number of demographics (e.g. sex, age) have been associated with work overload, scholars have yet to consider the potential impact of immigrant status. This is important because immigrants constitute a significant proportion of the workforce, and evidence suggests many employers believe they are easier to exploit. This paper aims to examine work hours, interpersonal justice, and immigrant status as predictors of work overload.Design/methodology/approach – The hypotheses were tested using a large, national random telephone survey of employees in the United States (n=2,757).Findings – As expected, employees who worked more hours tended to perceive more work overload. Importantly, however, this effect interacted with interpersonal justice differently for immigrant and native‐born employees. Justice attenuated the effect of work hours for the former but seemed to exacerbate it somewhat for the latter. Of note, the interactive effect was more than five times larger for immigrants than for na...

Sample size and power calculations in repeated measurement analysis

Controlled clinical trials in neuropsychopharmacology, as in numerous other clinical research domains, tend to employ a conventional parallel-groups design with repeated measurements. The hypothesis of primary interest in the relatively short-term, double-blind trials, concerns the difference between patterns or magnitudes of change from baseline. A simple two-stage approach to the analysis of such data involves calculation of an index or coefficient of change in stage 1 and testing the significance of difference between group means on the derived measure of change in stage 2. This article has the aim of introducing formulas and a computer program for sample size and/or power calculations for such two-stage analyses involving each of three definitions of change, with or without baseline scores entered as a covariate, in the presence of homogeneous or heterogeneous (autoregressive) patterns of correlation among the repeated measurements. Empirical adjustments of sample size for the projected dropout rates are also provided in the computer program.

ISSUES IN USE OF SAS PROC.MIXED TO TEST THE SIGNIFICANCE OF TREATMENT EFFECTS IN CONTROLLED CLINICAL TRIALS

A project that originated with the aim of documenting the implications of dropouts for tests of significance based on general linear mixed model procedures resulted in recognition of problems in the use of SAS PROC.MIXED for this purpose. In responding to suggestions and criticisms, we have further analyzed simulated clinical trial data with realistic autoregressive structure, using alternative error model formulations, different approaches to the use of covariates to model dropout patterns, and different ways to include the critical time variable in the mixed model. Results emphasize the sensitivity of the PROC.MIXED tests of significance for GROUP and TIME × GROUP equal slopes hypothesis to less than optimal modeling of the error covariance structure. Even with the authoritatively recommended best available modeling of the error structure, model formulations that made use of the REPEATED statement did not maintain conservative test sizes when covariates were required to model dropout data patterns. Random coefficients models that employed the RANDOM statement did permit appropriate covariate controls, but the tests of significance for treatment effects were lacking in power. After examining a variety of alternative PROC.MIXED model formulations, it is concluded that none provided both Type I error protection and power comparable to that of simple two-stage analysis of covariance (ANCOVA) procedures for confirming the presence of true treatment effects in controlled clinical trials. Other issues examined in this article concern treating baseline scores as both covariate and initial repeated measurement to which a linear means model is fitted, failure to take advantage of the regression of repeated measurements on time in modeling time as an unordered categorical variable, and fitting linear regression models to nonlinear response patterns.

Determining the Number of Clusters by Sampling With Replacement.

A split-sample replication criterion originally proposed by J. E. Overall and K. N. Magee (1992) as a stopping rule for hierarchical cluster analysis is applied to multiple data sets generated by sampling with replacement from an original simulated primary data set. An investigation of the validity of this bootstrap procedure was undertaken using different combinations of the true number of latent populations, degrees of overlap, and sample sizes. The bootstrap procedure enhanced the accuracy of identifying the true number of latent populations under virtually all conditions. Increasing the size of the resampled data sets relative to the size of the primary data set further increased accuracy. A computer program to implement the bootstrap stopping rule is made available via a referenced Web site.

Big Data Methods: Leveraging Modern Data Analytic Techniques to Build Organizational Science

Advances in data science, such as data mining, data visualization, and machine learning, are extremely well-suited to address numerous questions in the organizational sciences given the explosion of available data. Despite these opportunities, few scholars in our field have discussed the specific ways in which the lens of our science should be brought to bear on the topic of big data and big datas reciprocal impact on our science. The purpose of this paper is to provide an overview of the big data phenomenon and its potential for impacting organizational science in both positive and negative ways. We identifying the biggest opportunities afforded by big data along with the biggest obstacles, and we discuss specifically how we think our methods will be most impacted by the data analytics movement. We also provide a list of resources to help interested readers incorporate big data methods into their existing research. Our hope is that we stimulate interest in big data, motivate future research using big data sources, and encourage the application of associated data science techniques more broadly in the organizational sciences.