Kevin D. Carlson

Understanding the Impact of Convergent Validity on Research Results

Using different measures of constructs in research to develop robust evidence of relationships and effects is seen as good methodological practice. This assumes these measures possess high convergent validity. However, proxies—alternative measures of the same construct—are rarely perfectly convergent. Although some convergence is preferred to none, this study demonstrates that even modest departures from perfect convergent validity can result in substantial differences in the magnitudes of findings, creating challenges for the accumulation and interpretation of research. Using data from published research, the authors find that substantial differences in findings between studies using desired and proxy variables occur even at levels of convergent validity as high as r = .85. Implications of using measures with less-than-ideal convergent validity for the interpretability of research results are examined. Convergent validities above r = .70 are recommended, whereas those below r = .50 should be avoided. Researchers are encouraged to develop and report convergent validity data.

Citing and Building on Meta-Analytic Findings: A Review and Recommendations

Meta-analysis is commonly used to quantitatively review research findings in the social sciences. This article looks at what happens next, after a meta-analysis is published. The authors examine how meta-analytic findings are cited in subsequent studies and whether the citing authors take full advantage of the information meta-analyses provide. A review of 1,489 citations to meta-analyses in 319 empirical studies published in three journals over two decades indicates that the frequency of citing meta-analyses is accelerating. An analysis of citing practices indicates that authors use data for a variety of purposes in subsequent research studies. However, the citing studies often underreported important aspects of meta-analytic data, and additional opportunities exist to build on the data provided by meta-analytic reviews.

Random and Systematic Error Effects of Insomnia on Survey Behavior

Insomnia is a prevalent experience among employees and survey respondents. Drawing from research on sleep and self-regulation, we examine both random (survey errors) and systematic (social desirability) effects of research participant insomnia on survey responses. With respect to random effects, we find that insomnia leads to increased survey errors, and that this effect is mediated by a lack of self-control and a lack of effort. However, insomnia also has a positive systematic effect, leading to lower levels of social desirability. This effect is also mediated by self-control depletion and a lack of effort. In supplemental analyses, we find that psychometric side effects of random and systematic error introduced by individuals high in insomnia negatively affect internal consistency estimates and measurement invariance on various organizational measures. Results were replicated across two studies, with alternative operationalizations of survey errors and social desirability and some alternative explanations examined. These findings suggest sleep may be a key methodological issue for conducting survey research. Recommendations from the sleep and self-regulation literature regarding potential strategies for counteracting the effect of insomnia on survey responses are discussed.

Evidence of differences in applicant pool quality

Despite general assumptions that recruitment is important to organizational success, little empiric evidence exists to confirm that different recruitment approaches lead to meaningful differences in attraction outcomes. This study begins to address this research need by examining the attraction outcomes of firms competing head‐to‐head for recruits for similar positions. Results of an analysis of 391 applicant pools representing 18 different job families suggest that applicant pool quality can vary substantially within and across job families. Utility estimates, based on the hiring of a single employee and using Grade Point Average (GPA) as a measure of applicant quality, produced differences within applicant pools for hiring a single individual valued as high as

Global Perceptions of the Fit between Person and Work Environment (P-E Fit): Development and Initial Validation of a New Measure:

15,000. The average difference between the highest and lowest quality applicant pools across 18 job families was

The Staffing Cycles Framework: Viewing Staffing as a System of Decision Events

6,394.45 (SD =

THE FORM OF FIT FUNCTIONS: AN ANALYSIS USING APPLICATION DECISIONS.

3,533.20).

The Illusion of Statistical Control Control Variable Practice in Management Research

Research investigating the role of fit between individuals and their work environments employs a growing number of measures assessing subsets of work environments. Global perceptions of fit should be useful in anchoring research studies and developing the literature. In this study, two measures of global fit perceptions are developed and initial validity data are offered. Using data from 363 upper-division college students in a simulated job search scenario, evidence of the unidimensionality, reliability, and predictive validity is provided. Suggestions are provided for use of these measures of global perceptions to advance research in Person-Environment ft.

Impact of experimental design on effect size : Findings from the research literature on training

We propose an alternative staffing framework that adopts a systems view of staffing and articulates the internal structures and influence mechanisms that determine “how” staffing outcomes occur. The staffing cycles framework (SCF) portrays the interrelationships among the actors, contexts, and activities of staffing as staffing cycles composed of decision events. A staffing cycle is a sequence of seven decision events controlled alternately—not mutually—by individuals and organizational decision makers. Staffing cycles are the smallest unit of analysis that still retains all staffing components and frames how and when staffing outcomes are affected. Decision events capture the primary decision activities of staffing systems. All decision events share a common structure that depicts how actors, contexts, and activities are combined to influence staffing outcomes.

RECRUITMENT EVALUATION: THE CASE FOR ASSESSING THE QUALITY OF APPLICANTS ATTRACTED

The article presents a study looking at how organizational or individual attributes are used to develop fit perceptions connected to application decisions. In the study the authors used polynomial ...