S. Bartholomew Craig

Identifying Careless Responses in Survey Data.

When data are collected via anonymous Internet surveys, particularly under conditions of obligatory participation (such as with student samples), data quality can be a concern. However, little guidance exists in the published literature regarding techniques for detecting careless responses. Previously several potential approaches have been suggested for identifying careless respondents via indices computed from the data, yet almost no prior work has examined the relationships among these indicators or the types of data patterns identified by each. In 2 studies, we examined several methods for identifying careless responses, including (a) special items designed to detect careless response, (b) response consistency indices formed from responses to typical survey items, (c) multivariate outlier analysis, (d) response time, and (e) self-reported diligence. Results indicated that there are two distinct patterns of careless response (random and nonrandom) and that different indices are needed to identify these different response patterns. We also found that approximately 10%-12% of undergraduates completing a lengthy survey for course credit were identified as careless responders. In Study 2, we simulated data with known random response patterns to determine the efficacy of several indicators of careless response. We found that the nature of the data strongly influenced the efficacy of the indices to identify careless responses. Recommendations include using identified rather than anonymous responses, incorporating instructed response items before data collection, as well as computing consistency indices and multivariate outlier analysis to ensure high-quality data.

Leadership and the fate of organizations.

This article concerns the real-world importance of leadership for the success or failure of organizations and social institutions. The authors propose conceptualizing leadership and evaluating leaders in terms of the performance of the team or organization for which they are responsible. The authors next offer a taxonomy of the dependent variables used as criteria in leadership studies. A review of research using this taxonomy suggests that the vast empirical literature on leadership may tell us more about the success of individual managerial careers than the success of these people in leading groups, teams, and organizations. The authors then summarize the evidence showing that leaders do indeed affect the performance of organizations--for better or for worse--and conclude by describing the mechanisms through which they do so.

Do the Behaviors Related to Managerial Effectiveness Really Change With Organizational Level? An Empirical Test

Past research on how the requirements of managerial jobs differ across organizational levels has mostly been descriptive. No previously published study has directly tested whether the behaviors that predict effectiveness are different at different levels. The authors tested whether organizational level moderated the relationships between subordinate ratings on seven dimensions of managerial behavior and superior evaluations of overall effectiveness using a set of identical measures in a sample of 2,175 supervisors, middle managers, and executives representing 15 different industries and dozens of organizations in the United States. Multivariate analyses revealed significant differences in the pattern of behaviors associated with effectiveness across levels. Many differences were discontinuous (e.g., positive predictors of effectiveness at one level were negative predictors at another) and generally consistent with the dominant themes in the descriptive literature characterizing the unique requirements of ...

Differences in Managerial Jobs at the Bottom, Middle, and Top: A Review of Empirical Research

It has become popular in the practices of leadership development and talent management to segment managers at different organizational levels in order to focus on the unique requirements thought to characterize jobs at each level. This movement has been spurred by popular books that emphasize differences in the nature of managerial work at different hierarchical levels. Seemingly independent of popular work in this area has been scientific research to describe differences in managerial jobs across organizational levels. The present article summarizes the extensive research literature on level differences in managerial jobs in terms of three broad generalizations: The number of distinct management levels, five different ways to characterize work at each level, and how radical differences in work at each level pose adaptive challenges to managers who transition into positions of greater authority and responsibility. The article closes with commentary on the current state of theory and research and offers su...

Gender and Ethnicity Differences on the Abridged Big Five Circumplex (AB5C) of Personality Traits: A Differential Item Functioning Analysis.

The current study evaluates the measurement precision of the Abridged Big Five Circumplex (AB5C) of personality traits by identifying those items that demonstrate differential item functioning by gender and ethnicity. Differential item functioning is found in 33 of 45 (73%) of the AB5C scales, across gender and ethnic groups (Caucasian vs. African American). More differential item functioning is found by ethnic groups compared with gender groups. The amount of differential item functioning found in these personality measures suggests the need to consider differential item functioning in the construction of measures. The authors encourage those using the AB5C measure to discard or reword these problematic items.

Applying Item Response Theory to Multisource Performance Ratings: What are the Consequences of Violating the Independent Observations Assumption?

Multisource performance ratings involve multiple raters of a common target, thus violating the statistical assumption of independent observations. This study investigated the consequences of that violation on item response theory–based analyses of ratings in two domains of managerial performance. Using a differential item functioning framework, the authors found no significant differences in item parameters attributable to the violation. This finding suggests that item response theory can be applied to multisource ratings without discarding data in order to meet the independence assumption.

Assessing Leadership Performance in University-Based Cooperative Research Centers: Evidence from Four Studies

This chapter describes a program of research investigating leadership in the context of university-based cooperative research centers (URCs). S. Bartholomew Craig, Clara E. Hess, Jennifer Lindberg McGinnis, and Denis O. Gray report the results of four studies. Study 1 used content analysis of qualitative interview data to identify 15 categories of center director performance. Study 2 used the categories identified in Study 1 to create a standardized multisource (360°) leadership performance assessment instrument, which was subsequently refined to measure ten dimensions of directors’ leadership performance. Study 3 applied cluster analysis to scores from the new instrument to identify four distinct “types” of center directors, based on their patterns of leadership performance. Study 4 investigated the relations between center directors’ leadership performance and their personality scores as assessed with the Five Factor Model of personality. Results are discussed in terms of their implications for knowledge of leadership processes within URCs, selection and development of center directors, and for future research efforts. For complementary examinations, see the chapters by Rivers and Gray and by Davis and colleagues on center leaders’ industry member marketing practices and relations with university administrators, respectively.