Brian A. Nosek

Understanding and using the Implicit Association Test: I. An improved scoring algorithm.

In reporting Implicit Association Test (IAT) results, researchers have most often used scoring conventions described in the first publication of the IAT (A.G. Greenwald, D.E. McGhee, & J.L.K. Schwartz, 1998). Demonstration IATs available on the Internet have produced large data sets that were used in the current article to evaluate alternative scoring procedures. Candidate new algorithms were examined in terms of their (a) correlations with parallel self-report measures, (b) resistance to an artifact associated with speed of responding, (c) internal consistency, (d) sensitivity to known influences on IAT measures, and (e) resistance to known procedural influences. The best-performing measure incorporates data from the IATs practice trials, uses a metric that is calibrated by each respondents latency variability, and includes a latency penalty for errors. This new algorithm strongly outperforms the earlier (conventional) procedure.

Power failure: why small sample size undermines the reliability of neuroscience

A study with low statistical power has a reduced chance of detecting a true effect, but it is less well appreciated that low power also reduces the likelihood that a statistically significant result reflects a true effect. Here, we show that the average statistical power of studies in the neurosciences is very low. The consequences of this include overestimates of effect size and low reproducibility of results. There are also ethical dimensions to this problem, as unreliable research is inefficient and wasteful. Improving reproducibility in neuroscience is a key priority and requires attention to well-established but often ignored methodological principles.

Scientific Utopia: II - Restructuring Incentives and Practices to Promote Truth Over Publishability

An academic scientist’s professional success depends on publishing. Publishing norms emphasize novel, positive results. As such, disciplinary incentives encourage design, analysis, and reporting decisions that elicit positive results and ignore negative results. Prior reports demonstrate how these incentives inflate the rate of false effects in published science. When incentives favor novelty over replication, false results persist in the literature unchallenged, reducing efficiency in knowledge accumulation. Previous suggestions to address this problem are unlikely to be effective. For example, a journal of negative results publishes otherwise unpublishable reports. This enshrines the low status of the journal and its content. The persistence of false findings can be meliorated with strategies that make the fundamental but abstract accuracy motive—getting it right—competitive with the more tangible and concrete incentive—getting it published. This article develops strategies for improving scientific practices and knowledge accumulation that account for ordinary human motivations and biases.

Promoting an open research culture

Author guidelines for journals could help to promote transparency, openness, and reproducibility Transparency, openness, and reproducibility are readily recognized as vital features of science (1, 2). When asked, most scientists embrace these features as disciplinary norms and values (3). Therefore, one might expect that these valued features would be routine in daily practice. Yet, a growing body of evidence suggests that this is not the case (4–6).

Harvesting Implicit Group Attitudes and Beliefs From a Demonstration Web Site

Respondents at an Internet site completed over 600,000 tasks between October 1998 and April 2000 measuring attitudes toward and stereotypes of social groups. Their responses demonstrated, on average, implicit preference for White over Black and young over old and stereotypic associations linking male terms with science and career and female terms with liberal arts and family. The main purpose was to provide a demonstration site at which respondents could experience their implicit attitudes and stereotypes toward social groups. Nevertheless, the data collected are rich in information regarding the operation of attitudes and stereotypes, most notably the strength of implicit attitudes, the association and dissociation between implicit and explicit attitudes, and the effects of group membership on attitudes and stereotypes.

Mapping the Moral Domain

The moral domain is broader than the empathy and justice concerns assessed by existing measures of moral competence, and it is not just a subset of the values assessed by value inventories. To fill the need for reliable and theoretically grounded measurement of the full range of moral concerns, we developed the Moral Foundations Questionnaire on the basis of a theoretical model of 5 universally available (but variably developed) sets of moral intuitions: Harm/Care, Fairness/Reciprocity, Ingroup/Loyalty, Authority/Respect, and Purity/Sanctity. We present evidence for the internal and external validity of the scale and the model, and in doing so we present new findings about morality: (a) Comparative model fitting of confirmatory factor analyses provides empirical justification for a 5-factor structure of moral concerns; (b) convergent/discriminant validity evidence suggests that moral concerns predict personality features and social group attitudes not previously considered morally relevant; and (c) we establish pragmatic validity of the measure in providing new knowledge and research opportunities concerning demographic and cultural differences in moral intuitions. These analyses provide evidence for the usefulness of Moral Foundations Theory in simultaneously increasing the scope and sharpening the resolution of psychological views of morality.

Understanding and Using the Implicit Association Test: II. Method Variables and Construct Validity

The Implicit Association Test (IAT) assesses relative strengths of four associations involving two pairs of contrasted concepts (e.g., male-female and family-career). In four studies, analyses of data from 11 Web IATs, averaging 12,000 respondents per data set, supported the following conclusions: (a) sorting IAT trials into subsets does not yield conceptually distinct measures; (b) valid IAT measures can be produced using as few as two items to represent each concept; (c) there are conditions for which the administration order of IAT and self-report measures does not alter psychometric properties of either measure; and (d) a known extraneous effect of IAT task block order was sharply reduced by using extra practice trials. Together, these analyses provide additional construct validation for the IAT and suggest practical guidelines to users of the IAT.

Pervasiveness and correlates of implicit attitudes and stereotypes

http://implicit.harvard.edu/ was created to provide experience with the Implicit Association Test (IAT), a procedure designed to measure social knowledge that may operate outside awareness or control. Significant by-products of the websites existence are large datasets contributed to by the sites many visitors. This article summarises data from more than 2.5 million completed IATs and self-reports across 17 topics obtained between July 2000 and May 2006. In addition to reinforcing several published findings with a heterogeneous sample, the data help to establish that: (a) implicit preferences and stereotypes are pervasive across demographic groups and topics, (b) as with self-report, there is substantial inter-individual variability in implicit attitudes and stereotypes, (c) variations in gender, ethnicity, age, and political orientation predict variation in implicit and explicit measures, and (d) implicit and explicit attitudes and stereotypes are related, but distinct.

Math Male, Me Female, Therefore Math Me

College students, especially women, demonstrated negativity toward math and science relative to arts and language on implicit measures. Group membership (being female), group identity (self female), and gender stereotypes (math male) were related to attitudes and identification with mathematics. Stronger implicit math male stereotypes corresponded with more negative implicit and explicit math attitudes for women but more positive attitudes for men. Associating the self with female and math with male made it difficult for women, even women who had selected math-intensive majors, to associate math with the self. These results point to the opportunities and constraints on personal preferences that derive from membership in social groups.

Moderators of the relationship between implicit and explicit evaluation.

Automatic and controlled modes of evaluation sometimes provide conflicting reports of the quality of social objects. This article presents evidence for 4 moderators of the relationship between automatic (implicit) and controlled (explicit) evaluations. Implicit and explicit preferences were measured for a variety of object pairs using a large sample. The average correlation was r=.36, and 52 of the 57 object pairs showed a significant positive correlation. Results of multilevel modeling analyses suggested that (a) implicit and explicit preferences are related, (b) the relationship varies as a function of the objects assessed, and (c) at least 4 variables moderate the relationship: self-presentation, evaluative strength, dimensionality, and distinctiveness. The variables moderated implicit-explicit correspondence across individuals and accounted for much of the observed variation across content domains. The resulting model of the relationship between automatic and controlled evaluative processes is grounded in personal experience with the targets of evaluation.