Avraham N. Kluger

The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory.

the total number of papers may exceed 10,000. Nevertheless, cost consideration forced us to consider mostly published papers and technical reports in English. 4 Formula 4 in Seifert (1991) is in error—a multiplier of n, of cell size, is missing in the numerator. 5 Unfortunately, the technique of meta-analysis cannot be applied, at present time, to such effects because the distribution of dis based on a sampling of people, whereas the statistics of techniques such as ARIMA are based on the distribution of a sampling of observations in the time domain regardless of the size of the people sample involved (i.e., there is no way to compare a sample of 100 points in time with a sample of 100 people). That is, a sample of 100 points in time has the same degrees of freedom if it were based on an observation of 1 person or of 1,000 people. 258 KLUGER AND DENISI From the papers we reviewed, only 131 (5%) met the criteria for inclusion. We were concerned that, given the small percentage of usable papers, our conclusions might not fairly represent the larger body of relevant literature. Therefore, we analyzed all the major reasons to reject a paper from the meta-analysis, even though the decision to exclude a paper came at the first identification of a missing inclusion criterion. This analysis showed the presence of review articles, interventions of natural feedback removal, and papers that merely discuss feedback, which in turn suggests that the included studies represent 1015% of the empirical FI literature. However, this analysis also showed that approximately 37% of the papers we considered manipulated feedback without a control group and that 16% reported confounded treatments, that is, roughly two thirds of the empirical FI literature cannot shed light on the question of FI effects on performance—a fact that requires attention from future FI researchers. Of the usable 131 papers (see references with asterisks), 607 effect sizes were extracted. These effects were based on 12,652 participants and 23,663 observations (reflecting multiple observations per participant). The average sample size per effect was 39 participants. The distribution of the effect sizes is presented in Figure 1. The weighted mean (weighted by sample size) of this distribution is 0.41, suggesting that, on average, FI has a moderate positive effect on performance. However, over 38% of the effects were negative (see Figure 1). The weighted variance of this distribution is 0.97, whereas the estimate of the sampling error variance is only 0.09. A potential problem in meta-analyses is a violation of the assumption of independence. Such a violation occurs either when multiple observations are taken from the same study (Rosenthal, 1984) or when several papers are authored by the same person (Wolf, 1986). In the present investigation, there were 91 effects derived from the laboratory experiments reported by Mikulincer (e.g., 1988a, 1988b). This raises the possibility that the average effect size is biased, because his studies manipulated extreme negative FIs and used similar tasks. In fact, the weighted average d in Mikulincers studies was —0.39; whereas in the remainder of the

Dispositional Effects on Job and Life Satisfaction: The Role of Core Evaluations

Past research has suggested that dispositional sources of job satisfaction can be traced to measures of affective temperament. The present research focused on another concept, core self-evaluations, which were hypothesized to comprise self-esteem, generalized self-efficacy, locus of control, and nonneuroticism. A model hypothesized that core self-evaluations would have direct effects on job and life satisfaction. It also was hypothesized that core self-evaluations would have indirect effects on job satisfaction. Data were collected from 3 independent samples in 2 countries, using dual source methodology. Results indicated that core self-evaluations had direct and indirect effects on job and life satisfaction. The statistical and logical relationship among core evaluations, affective disposition, and satisfaction was explored.

A meta-analysis of the association between DRD4 polymorphism and novelty seeking.

A meta-analytical review of 20 studies (n = 3907) of the association between DRD4 polymorphism and novelty seeking suggests the following conclusions: (a) on average, there is no association between DRD4 polymorphism and novelty seeking (average d= 0.06 with 95% CI of ± 0.09), where 13 reports suggest that the presence of longer alleles is associated with higher novelty seeking scores and seven reports suggest the opposite; (b) there is a true heterogeneity among the studies (ie, unknown moderators do exist) but the strength of the association between DRD4 polymorphism and novelty seeking in the presence of any (unknown) moderator is likely to be weak; (c) search for moderators has not yielded any reliable explanation for the variability among studies. We propose that to find such moderators, theory-driven research for potential interaction, coupled with larger sample sizes should be employed. The growing availability of powerful statistical techniques, high-throughput genotyping and large numbers of polymorphic markers such as single nucleotide polymorphisms makes such proposed studies increasingly feasible.

COMMUTE VARIABILITY AND STRAIN

The commute impedance model (e.g. Stokols, Novaco, Stokols and Campbell, 1978) is evaluated through a literature review and survey of commuters who drive to work (N=418). A new construct for the study of commuting - commute variability - is introduced and tested in the framework of the survey. Findings from the literature review indicated (a) partial support for the first proposition of the impedance model, namely, that impedance - measured as commute distance or time - causes strain (e.g. employee health deficits), but (b) no support for the second proposition of the model, namely that the impedance strain is moderated by control or the availability of choice. Similarly, the survey showed that (a) commute length (impedance) is consistently correlated with strain, (b) choice is weakly and negatively related to strain, but (c) there is no interactive relationship of choice and length with strain. Furthermore, the results suggest that (a) commute variability is the strongest correlate of commute strain, and that (b) once commute variability is partialled out, most of the partial correlations between impedance and strain are not significant. The paper concludes with an alternative commuting strain model which includes the new commute-variability construct.

The influence of selection test type on applicant reactions to employment testing

The impact of employment selection devices on the reactions of simulated applicants was investigated. In Study 1, 141 subjects were “failed” on either a biographical inventory, a cognitive ability test, a trainability test or a work sample test. Subjects rejected on the basis of biographical inventory scores perceived the test as less difficult, coped more adaptively, and had better moods than subjects tested with the other instruments. The biographical inventory was also seen as more fakable, but less amenable to improvement through feedback or training than the other devices. We hypothesized that differences among tests in cognitive demand is a primary cause of differences in applicant reaction. In Study 2 (N=151) the comparison of the effects of two biographical inventories to two cognitive ability tests supported our hypothesis, and replicated the findings of Study 1 across failure and success conditions.

Feedback, the various tasks of the doctor, and the feedforward alternative.

Medical Education 2010: 44: 1166–1174

Faking Biodata Tests Are Option-Keyed Instruments More Resistant?

Response biases in biodata scores derived with option-keying and item-keying procedures were investigated. Results indicated that (a) when subjects simulated responding as job applicants they distorted their responses in a socially desirable direction; (b) itemkeyed scores were susceptible to inflation due to socially desirable responding and specific job-title knowledge, but option-keyed scores were not; and (c) response biases were not reflected in response latencies. A supplementary analysis indicated that the two keying procedures may capture different aspects of criterion variance. Implications for reconciling conflicting reports about the susceptibility of biodata scores to response biases are discussed. Issues related to reliability and validity of the two keying procedures, and the generalizability of the results to personality tests, are also discussed.

Sabbatical Leave: Who Gains and How Much?

A rigorous quasi-experiment tested the ameliorative effects of a sabbatical leave, a special case of respite from routine work. We hypothesized that (a) respite increases resource level and well-being and (b) individual differences and respite features moderate respite effects. A sample of 129 faculty members on sabbatical and 129 matched controls completed measures of resource gain, resource loss, and well-being before, during, and after the sabbatical. Among the sabbatees, resource loss declined and resource gain and well-being rose during the sabbatical. The comparison group showed no change. Moderation analysis revealed that those who reported higher respite self-efficacy and greater control, were more detached, had a more positive sabbatical experience, and spent their sabbatical outside their home country enjoyed more enhanced well-being than others.

Research Notes: The Antecedents and Consequences of Union Commitment: A Meta-Analysis

To study union commitment antecedents and consequences, we performed a meta-analysis of 76 published and unpublished studies. Using the meta-analytically derived, corrected correlation coefficients, we examined four alternative structural equation models, each testing identical, key union commitment antecedents and consequences. Our results strongly suggest that union commitment has two potent antecedents (i.e., union attitudes and union instrumentality) and explains a moderate degree of the variance in union participation. The covariances show that these constructs are highly correlated but have clear divergent validity. Moreover, our findings suggest that the more dominant socially weaker groups in a unions overall membership, the greater the relative importance of union instrumentality as a predictor of union commitment. Nevertheless, in all of the subsamples examined, the findings indicate that union attitudes are a more powerful predictor of union commitment than union instrumentality.

Person- versus computer-mediated feedback

Abstract The effects of feedback provided by a person versus that provided by a computer on performance, motivation, and feedback seeking were studied. Employing a 2 × 3 experimental design, subjects were assigned to one of three feedback conditions: (a) no feedback, (b) feedback only upon request, and (c) automatic feedback with feedback provided either by a person or a computer. The results indicate that (a) subjects are more likely to seek feedback from a computer than from another person; (b) feedback from a person causes a decline in performance relative to a condition where a person is present but does not deliver feedback; (c) both human- and computer-mediated feedback reduce motivation in comparison to a control group that receives no feedback; and (d) personality — in this case, self-esteem and public and private self-consciousness — interacts with the receipt of person-mediated feedback to negatively affect performance.