Adam S. Beatty

How Are We Doing After 30 Years? A Meta-Analytic Review of the Antecedents and Outcomes of Feedback-Seeking Behavior

This study provides meta-analytic estimates of the antecedents and consequences of feedback-seeking behavior (FSB). Clear support was found for the guiding cost/benefit framework in the feedback-seeking domain. Organizational tenure, job tenure, and age were negatively related to FSB. Learning and performance goal orientation, external feedback propensity, frequent positive feedback, high self-esteem, a transformational leadership style, and a high-quality relationship were positively associated with FSB. Challenging some of the dominant views in the feedback-seeking domain, the relationship between uncertainty and FSB was negative and the relationship between FSB and performance was small. Finally, inquiry and monitoring are not interchangeable feedback-seeking tactics. So FSB is best represented as an aggregate model instead of a latent model. In the discussion, gaps in the current FSB knowledge are identified and a research agenda for the future is put forward. Future research may benefit from (a) a systematic and integrative effort examining antecedents of both feedback-seeking strategies on the basis of a self-motives framework, (b) adopting a process perspective of feedback-seeking interactions, and (c) taking the iterative nature of feedback into account.

The Role of Socioeconomic Status in SAT-Grade Relationships and in College Admissions Decisions

This article examines the role of socioeconomic status (SES) in the relationships among college admissions-test scores, secondary school grades, and subsequent academic performance. Scores on the SAT (a test widely used in the admissions process in the United States), secondary school grades, college grades, and SES measures from 143,606 students at 110 colleges and universities were examined, and results of these analyses were compared with results obtained using a 41-school data set including scores from the prior version of the SAT and using University of California data from prior research on the role of SES. In all the data sets, the SAT showed incremental validity over secondary school grades in predicting subsequent academic performance, and this incremental relationship was not substantially affected by controlling for SES. The SES of enrolled students was very similar to that of specific schools’ applicant pools, which suggests that the barrier to college for low-SES students in the United States is a lower rate of entering the college admissions process, rather than exclusion on the part of colleges.

All validities are not created equal: Determinants of variation in SAT validity across schools

Previous research has demonstrated that cognitive test validities are generalizable and predictive of academic performance across situations. However, even after accounting for statistical artifacts (e.g., sampling error, range restriction, criterion reliability), substantial variability often remains around estimates of cognitive test–performance relationships suggesting the presence of additional moderators. In the present study, we examine the sources of institutional variation in Scholastic Aptitude Test (SAT) validity across a sample of 110 institutions. Institutional characteristics moderated the size of SAT validities, such that more selective schools and schools that emphasize traditional assessment techniques (i.e., school records, standardized tests) showed higher SAT validities while schools that were larger and where students demonstrated more financial need, schools that emphasized the usage of alternative assessment techniques (i.e., essays, letters of recommendations, extracurricular activities), and schools that enrolled higher percentages of historically disadvantaged minority students generally exhibited lower SAT validities. Future directions in the understanding of situational influences on SAT–grade point average validities are discussed.

Ability-Performance Relationships in Education and Employment Settings: Critical Tests of the More-Is-Better and the Good-Enough Hypotheses

The nature of the relationship between ability and performance is of critical importance for admission decisions in the context of higher education and for personnel selection. Although previous research has supported the more-is-better hypothesis by documenting linearity of ability-performance relationships, such research has not been sensitive enough to detect deviations at the top ends of the score distributions. An alternative position receiving considerable attention is the good-enough hypothesis, which suggests that although higher levels of ability may result in better performance up to a threshold, above this threshold greater ability does not translate to better performance. In this study, the nature of the relationship between cognitive ability and performance was examined throughout the score range in four large-scale data sets. Monotonicity was maintained in all instances. Contrary to the good-enough hypothesis, the ability-performance relationship was commonly stronger at the top end of the score distribution than at the bottom end.

Testing the generalizability of indirect range restriction corrections.

Range restriction is a common problem in personnel selection and other contexts in applied psychology. For many years researchers have used corrections that assume range restriction was direct, even when it was known that range restriction was indirect. Hunter, Schmidt, and Le (2006) proposed a new correction for cases of indirect range restriction that greatly increases its potential usefulness due to its reduced information requirements compared to alternatives. The current study examines the applicability of Hunter et al.s correction to settings where its assumed structural model is violated by including the measures that are to be involved in corrections in the original selection composite. We conclude that Hunter et al.s correction should generally be preferred when compared to its common alternative, Thorndikes Case II correction for direct range restriction. However, this is due to the likely violation of one of the other assumptions of the Hunter et al. correction in most applied settings. Correction mechanisms and practical implications are discussed.

Predictor content matters for knowledge testing: Evidence supporting content validation

ABSTRACT Organizations often rely on the match between job requirements and test content to justify test use. This practice has been questioned on the grounds that content validation has little relevance to criterion-related validation due to positive manifold among predictors. We analyze two large databases to assess the implications of test content for (a) test interchangeability and (b) criterion-related validity. Analyses of 15 knowledge tests administered (N = 80,394) as part of Project Talent demonstrate that test content is related to predictor interchangeability. Analyses of SAT and Advanced Placement test data compare correlations among predictors and criteria drawn from matched and unmatched content domains. We conclude that test-criterion content match is likely to result in stronger criterion-related validity.

Validity Stability Across Entering College Cohorts: Exploring the temporal generalizability of local validity estimates

Local validity studies rely on the assumption that validity estimates from one incumbent sample approximate validity for future applicant pools. We test this assumption using SAT scores and high school grades as predictors of first year college grade point average across multiple college applicant pools for over 100 schools. We present evidence for substantial absolute and rank order consistency in validity estimates. However, this consistency is far less than perfect, resulting in potentially meaningful utility differences over time. In addition, observed fluctuations are not fully explained by sampling error alone.