Scott Clifford

Are samples drawn from Mechanical Turk valid for research on political ideology

Amazon’s Mechanical Turk (MTurk) is an increasingly popular tool for the recruitment of research subjects. While there has been much focus on the demographic differences between MTurk samples and the national public, we know little about whether liberals and conservatives recruited from MTurk share the same psychological dispositions as their counterparts in the mass public. In the absence of such evidence, some have argued that the selection process involved in joining MTurk invalidates the subject pool for studying questions central to political science. In this paper, we evaluate this claim by comparing a large MTurk sample to two benchmark national samples – one conducted online and one conducted face-to-face. We examine the personality and value-based motivations of political ideology across the three samples. All three samples produce substantively identical results with only minor variation in effect sizes. In short, liberals and conservatives in our MTurk sample closely mirror the psychological divisions of liberals and conservatives in the mass public, though MTurk liberals hold more characteristically liberal values and attitudes than liberals from representative samples. Overall, our results suggest that MTurk is a valid recruitment tool for psychological research on political ideology.

How Words Do the Work of Politics: Moral Foundations Theory and the Debate over Stem Cell Research

Moral considerations underlie partisan and ideological identification along with a variety of political attitudes, yet we know little about how elites strategically appeal to the public’s moral intuitions. Building on Moral Foundations Theory, we investigate the causes and consequences of elite moral rhetoric in the debate over stem cell research. Through content analysis of 12 years of coverage in the New York Times, we find that proponents and opponents of stem cell research engage in distinctive patterns of moral rhetoric and place different weight on the foundations. We also demonstrate that the prevalence of moral rhetoric increases during periods of legislative activity, and we find some evidence that moral rhetoric increases in response to the opposing side’s use of moral language. Merging our content analysis with seven national surveys, the analysis shows that moral rhetoric has had a substantial effect on public attitudes regarding the fundamental considerations underpinning the debate.

Moral foundations vignettes: a standardized stimulus database of scenarios based on moral foundations theory.

Research on the emotional, cognitive, and social determinants of moral judgment has surged in recent years. The development of moral foundations theory (MFT) has played an important role, demonstrating the breadth of morality. Moral psychology has responded by investigating how different domains of moral judgment are shaped by a variety of psychological factors. Yet, the discipline lacks a validated set of moral violations that span the moral domain, creating a barrier to investigating influences on judgment and how their neural bases might vary across the moral domain. In this paper, we aim to fill this gap by developing and validating a large set of moral foundations vignettes (MFVs). Each vignette depicts a behavior violating a particular moral foundation and not others. The vignettes are controlled on many dimensions including syntactic structure and complexity making them suitable for neuroimaging research. We demonstrate the validity of our vignettes by examining respondents’ classifications of moral violations, conducting exploratory and confirmatory factor analysis, and demonstrating the correspondence between the extracted factors and existing measures of the moral foundations. We expect that the MFVs will be beneficial for a wide variety of behavioral and neuroimaging investigations of moral cognition.

Validity and Mechanical Turk: An assessment of exclusion methods and interactive experiments

Social science researchers increasingly recruit participants through Amazons Mechanical Turk (MTurk) platform. Yet, the physical isolation of MTurk participants, and perceived lack of experimental control have led to persistent concerns about the quality of the data that can be obtained from MTurk samples. In this paper we focus on two of the most salient concernsthat MTurk participants may not buy into interactive experiments and that they may produce unreliable or invalid data. We review existing research on these topics and present new data to address these concerns. We find that insufficient attention is no more a problem among MTurk samples than among other commonly used convenience or high-quality commercial samples, and that MTurk participants buy into interactive experiments and trust researchers as much as participants in laboratory studies. Furthermore, we find that employing rigorous exclusion methods consistently boosts statistical power without introducing problematic side effects (e.g., substantially biasing the post-exclusion sample), and can thus provide a general solution for dealing with problematic respondents across samples. We conclude with a discussion of best practices and recommendations. Online participant recruitment has led to persistent concerns about data quality.Online participants are just as attentive as participants recruited offline.Online participants buy into experimental social interactions as much as in the lab.Rigorous exclusion methods can be used to improve data quality online and offline.

Linking Issue Stances and Trait Inferences: A Theory of Moral Exemplification

Considerable research has demonstrated the importance of perceptions of politicians’ character traits for vote choice. Yet, we know little about the antecedents of trait attributions. Drawing on Moral Foundations Theory, I argue that character traits correspond to particular moral foundations. I introduce a theory of moral exemplification, according to which individuals use their own moral motivations, and the character traits exemplifying these motivations, to interpret the behavior of politicians. The analysis of three separate studies reveals support for the theory. First, individuals’ moral foundations predict the accessibility of corresponding traits and thus their propensity to be used in evaluation. Second, across two experiments, politicians’ issue stances shape perceptions of their traits. As predicted, however, the type of trait inference made depends on the moral foundation associated with the individual’s issue stance. I conclude with a discussion of how moral exemplification theory provides insight into trait ownership theory and campaign strategy.

Moral Concerns and Policy Attitudes: Investigating the Influence of Elite Rhetoric

A growing body of research documents the crucial role played by moral concerns in the formation of attitudes and a wide range of political behaviors. Yet extant models of moral judgment portray a direct linkage between moral intuitions and policy attitudes, leaving little room for the influence of political context. In this article, we argue that political rhetoric plays an important role in facilitating the connection between moral intuitions and political attitudes. Using a unique combination of media content analysis of the stem cell debate and individual-level measures of the public’s moral foundations, we examine the role of rhetoric in linking a person’s moral foundations to his or her attitudes. Our results show that individuals who are the most likely to have been exposed to political rhetoric have the strongest connection between their moral foundations and their attitudes on the issue. We also find that rhetoric was persuasive on this moralized issue and present suggestive evidence that it was most persuasive among those who endorsed the relevant moral foundation.

Trust Me, I Believe in God Candidate Religiousness as a Signal of Trustworthiness

Religion plays a prominent role in American politics, and candidates often attempt to display their religiousness in a variety of ways. For example, in spite of the association between conservatism and religion, research shows that candidates of both parties routinely use religious language and seek to demonstrate personal religiousness. Existing research portrays religious rhetoric primarily as an ideological cue, failing to explain why Democrats would make religious appeals. Drawing on psychological theories of prejudice toward atheists, we argue that candidates emphasize their religiousness in order to enhance perceptions of their trustworthiness and morality. Using survey data, we show that voters are quite unlikely to support an atheist candidate, an effect that is strongly driven by the perception of atheists’ morality. Next, we show evidence that voters perceiving Hillary Clinton as religious also viewed her as more trustworthy, and were more likely to view her favorably. Finally, we show experimentally that religious candidates are perceived as more trustworthy, at least among a wide swath of the electorate. We conclude that displays of religiousness likely serve not only an ideological purpose but also the broader goal of increasing perceived candidate trustworthiness.