Paul E. Smaldino

The natural selection of bad science

Poor research design and data analysis encourage false-positive findings. Such poor methods persist despite perennial calls for improvement, suggesting that they result from something more than just misunderstanding. The persistence of poor methods results partly from incentives that favour them, leading to the natural selection of bad science. This dynamic requires no conscious strategizing—no deliberate cheating nor loafing—by scientists, only that publication is a principal factor for career advancement. Some normative methods of analysis have almost certainly been selected to further publication instead of discovery. In order to improve the culture of science, a shift must be made away from correcting misunderstandings and towards rewarding understanding. We support this argument with empirical evidence and computational modelling. We first present a 60-year meta-analysis of statistical power in the behavioural sciences and show that power has not improved despite repeated demonstrations of the necessity of increasing power. To demonstrate the logical consequences of structural incentives, we then present a dynamic model of scientific communities in which competing laboratories investigate novel or previously published hypotheses using culturally transmitted research methods. As in the real world, successful labs produce more ‘progeny,’ such that their methods are more often copied and their students are more likely to start labs of their own. Selection for high output leads to poorer methods and increasingly high false discovery rates. We additionally show that replication slows but does not stop the process of methodological deterioration. Improving the quality of research requires change at the institutional level.

The cultural evolution of emergent group-level traits

Many of the most important properties of human groups - including properties that may give one group an evolutionary advantage over another - are properly defined only at the level of group organization. Yet at present, most work on the evolution of culture has focused solely on the transmission of individual-level traits. I propose a conceptual extension of the theory of cultural evolution, particularly related to the evolutionary competition between cultural groups. The key concept in this extension is the emergent group-level trait. This type of trait is characterized by the structured organization of differentiated individuals and constitutes a unit of selection that is qualitatively different from selection on groups as defined by traditional multilevel selection (MLS) theory. As a corollary, I argue that the traditional focus on cooperation as the defining feature of human societies has missed an essential feature of cooperative groups. Traditional models of cooperation assume that interacting with one cooperator is equivalent to interacting with any other. However, human groups involve differential roles, meaning that receiving aid from one individual is often preferred to receiving aid from another. In this target article, I discuss the emergence and evolution of group-level traits and the implications for the theory of cultural evolution, including ramifications for the evolution of human cooperation, technology, and cultural institutions, and for the equivalency of multilevel selection and inclusive fitness approaches.

Increased Costs of Cooperation Help Cooperators in the Long Run

It has long been proposed that cooperation should increase in harsh environments, but this claim still lacks theoretical underpinnings. We modeled a scenario in which benefiting from altruistic behavior was essential to survival and reproduction. We used a spatial agent-based model to represent mutual cooperation enforced by environmental adversity. We studied two factors, the cost of unreciprocated cooperation and the environmental cost of living, which highlight a conflict between the short- and long-term rewards of cooperation. In the long run, cooperation is favored because only groups with a sufficient number of cooperators will survive. In the short run, however, harsh environmental costs increase the advantage of defectors in cooperator-defector interactions because the loss of resources leads to death. Our analysis sheds new light on the evolution of cooperation via interdependence and illustrates how selfish groups can incur short-term benefits at the cost of their eventual demise. We demonstrate how harsh environments select for cooperative phenotypes and suggest an explanation for the adoption of cooperative breeding strategies in human evolution. We also highlight the importance of variable population size and the role of socio-spatial organization in harsh environments.

A multilevel evolutionary framework for sustainability analysis

Sustainability theory can help achieve desirable social-ecological states by generalizing lessons across contexts and improving the design of sustainability interventions. To accomplish these goals, we argue that theory in sustainability science must (1) explain the emergence and persistence of social-ecological states, (2) account for endogenous cultural change, (3) incorporate cooperation dynamics, and (4) address the complexities of multilevel social-ecological interactions. We suggest that cultural evolutionary theory broadly, and cultural multilevel selection in particular, can improve on these fronts. We outline a multilevel evolutionary framework for describing social-ecological change and detail how multilevel cooperative dynamics can determine outcomes in environmental dilemmas. We show how this framework complements existing sustainability frameworks with a description of the emergence and persistence of sustainable institutions and behavior, a means to generalize causal patterns across social-ecological contexts, and a heuristic for designing and evaluating effective sustainability interventions. We support these assertions with case examples from developed and developing countries in which we track cooperative change at multiple levels of social organization as they impact social-ecological outcomes. Finally, we make suggestions for further theoretical development, empirical testing, and application.

An Institutional Mechanism for Assortment in an Ecology of Games

Recent research has revived Longs “ecology of games” model to analyze how social actors cooperate in the context of multiple political and social games. However, there is still a paucity of theoretical work that considers the mechanisms by which large-scale cooperation can be promoted in a dynamic institutional landscape, in which actors can join new games and leave old ones. This paper develops an agent-based model of an ecology of games where agents participate in multiple public goods games. In addition to contribution decisions, the agents can leave and join different games, and these processes are de-coupled. We show that the payoff for cooperation is greater than for defection when limits to the number of actors per game (“capacity constraints”) structure the population in ways that allow cooperators to cluster, independent of any complex individual-level mechanisms such as reputation or punishment. Our model suggests that capacity constraints are one effective mechanism for producing positive assortment and increasing cooperation in an ecology of games. The results suggest an important trade-off between the inclusiveness of policy processes and cooperation: Fully inclusive policy processes reduce the chances of cooperation.

The Origins of Options

Most research on decision making has focused on how human or animal decision makers choose between two or more options, posed in advance by the researchers. The mechanisms by which options are generated for most decisions, however, are not well understood. Models of sequential search have examined the trade-off between continued exploration and choosing one’s current best option, but still cannot explain the processes by which new options are generated. We argue that understanding the origins of options is a crucial but untapped area for decision making research. We explore a number of factors which influence the generation of options, which fall broadly into two categories: psycho-biological and socio-cultural. The former category includes factors such as perceptual biases and associative memory networks. The latter category relies on the incredible human capacity for culture and social learning, which doubtless shape not only our choices but the options available for choice. Our intention is to start a discussion that brings us closer toward understanding the origins of options.

Replication, Communication, and the Population Dynamics of Scientific Discovery.

Many published research results are false (Ioannidis, 2005), and controversy continues over the roles of replication and publication policy in improving the reliability of research. Addressing these problems is frustrated by the lack of a formal framework that jointly represents hypothesis formation, replication, publication bias, and variation in research quality. We develop a mathematical model of scientific discovery that combines all of these elements. This model provides both a dynamic model of research as well as a formal framework for reasoning about the normative structure of science. We show that replication may serve as a ratchet that gradually separates true hypotheses from false, but the same factors that make initial findings unreliable also make replications unreliable. The most important factors in improving the reliability of research are the rate of false positives and the base rate of true hypotheses, and we offer suggestions for addressing each. Our results also bring clarity to verbal debates about the communication of research. Surprisingly, publication bias is not always an obstacle, but instead may have positive impacts—suppression of negative novel findings is often beneficial. We also find that communication of negative replications may aid true discovery even when attempts to replicate have diminished power. The model speaks constructively to ongoing debates about the design and conduct of science, focusing analysis and discussion on precise, internally consistent models, as well as highlighting the importance of population dynamics.

Institutions and cooperation in an ecology of games

Social dilemmas have long been studied formally as cooperation games that pit individual gains against those of the group. In the real world, individuals face an ecology of games where they play many such games simultaneously, often with overlapping co-players. Here, we study an agent-based model of an ecology of public goods games and compare the effectiveness of two institutional mechanisms for promoting cooperation: a simple institution of limited group size (capacity constraints) and a reputational institution based on observed behavior. Reputation is shown to allow much higher relative payoffs for cooperators than do capacity constraints, but only if (1) the rate of reputational information flow is fast enough relative to the rate of social mobility, and (2) cooperators are relatively common in the population. When these conditions are not met, capacity constraints are more effective at protecting the interests of cooperators. Because of the simplicity of the limited-group-size rule, capacity constraints can also generate social organization, which promotes cooperation much more quickly than can reputation. Our results are discussed in terms of both normative prescriptions and evolutionary theory regarding institutions that regulate cooperation. More broadly, the ecology-of-games approach developed here provides an adaptable modeling framework for studying a wide variety of problems in the social sciences.

Social conformity despite individual preferences for distinctiveness

We demonstrate that individual behaviours directed at the attainment of distinctiveness can in fact produce complete social conformity. We thus offer an unexpected generative mechanism for this central social phenomenon. Specifically, we establish that agents who have fixed needs to be distinct and adapt their positions to achieve distinctiveness goals, can nevertheless self-organize to a limiting state of absolute conformity. This seemingly paradoxical result is deduced formally from a small number of natural assumptions and is then explored at length computationally. Interesting departures from this conformity equilibrium are also possible, including divergence in positions. The effect of extremist minorities on these dynamics is discussed. A simple extension is then introduced, which allows the model to generate and maintain social diversity, including multimodal distinctiveness distributions. The paper contributes formal definitions, analytical deductions and counterintuitive findings to the literature on individual distinctiveness and social conformity.

Agent-based modeling as a tool for studying social identity processes: The case of optimal distinctiveness theory

© Kramer, Roderick M.; Leonardelli, Geoffrey J.; Livingston, Robert W., May 23, 2011, Social Cognition, Social Identity, and Intergroup Relations : A Festschrift in Honor of Marilynn B. Psychology Press, Hoboken, ISBN: 9780203816790 AGENT-BASED MODELING AS A TOOL FOR STUDYING SOCIAL IDENTITY PROCESSES The Case of Optimal Distinctiveness Theory Cynthia L. Pickett, Paul E. Smaldino, Jeffrey W. Sherman, and Jeffrey Schank Researchers studying social identity and intergroup relations have tra- ditionally approached group behavior as an interaction between the individual, the group, and the social context in which the individual and group are embedded. This approach has been quite fruitful, as evi- denced by the proliferation of theories and studies over the last sev- eral decades that have identified the psychological and sociocontextual features that are likely to give rise to particular group behaviors (e.g., in-group bias, discrimination, intergroup hostility). However, these theories are based largely on how individuals are predicted to respond and behave under particular circumstances, often without explicit con- sideration of the interdependence among individuals or the group-level outcomes that may emerge as a result of the interactions among indi- vidual actors. This approach is similar to a traffic engineer attempt- ing to understand traffic patterns by examining the motivations and behaviors of individual drivers. Individual-level theories may tell the engineer that drivers attempt to maximize the speed of their car and avoid erratic fellow drivers. But understanding why traffic jams occur requires consideration of how the behavior of one driver affects the behavior of multiple other drivers and how these behaviors unfold over time. In this chapter, we echo the sentiment of other researchersR.W. Livingston, G.J. Leonardelli, R.M. Kramer, Rigor with Relevance: The Many Legacies of Marilynn Brewer. Part 1. Social Cognition. M. Karasawa, Categorization-Based Versus Person-Based Explanations of Behaviors: Implications from the Dual-Process Model. D.L. Hamilton, J.M. Chen, N. Way, Dynamic Aspects of Entitativity: From Group Perceptions to Social Interaction. L.R. Caporael, G.D. Reeder, New Evolutionary Perspectives on Theory of Mind. Part 2. Social Identity and Intergroup Relations. K. Schmid, M. Hewstone, Social Identity Complexity: Theoretical Implications for the Social Psychology of Intergroup Relations. G.J. Leonardelli, C.L. Pickett, J.E. Joseph, Y.D. Hess, Optimal Distinctiveness in Nested Categorization Contexts: Moving from Dueling Identities to a Dual Identity. C.L. Pickett, P. Smaldino, J. Sherman, J. Schank, Agent-Based Modeling as a Tool for Studying Social Identity Processes: The Case of Optimal Distinctiveness Theory. A.C. Rumble, Religion as Collective Identity. K. Gonsalkorale, W. von Hippel, Intergroup Relations in the 21st Century: Ingroup Positivity and Outgroup Negativity Among Members of an Internet Hate Group. M.M. McDonald, C.D. Navarrete, J. Sidanius, Developing a Theory of Gendered Prejudice: An Evolutionary and Social Dominance Perspective. W.D. Crano, V. Hemovich, Intergroup Relations and Majority or Minority Group Influence. M. Yuki, Intragroup Relationships and Intergroup Comparisons as Two Sources of Collectivism. Part 3. Applications and Implications. S. Schneider, W.M. George, S. Carroll, E.D. Middleton, How Leaders Transform Followers: Organizational Identity as a Mediator of Follower Attitudes in Two Samples. R.M. Kramer, Cooperation and the Commons: Lab and Field Explorations of a Persistent Dilemma. E. Castano, Moral Disengagement and Morality Shifting in the Context of Collective Violence. W.W. Maddux, A Movable Feast: How Transformational Cross-Cultural Experiences Facilitate Creativity. Y.-R. Chen, G. Zhao, J. Lee, Trust in the Manager as a Supervisor or a Group Leader? Toward a Relational Versus Collective Distinction in Procedural Justice. Part 4. Reflections and Conclusion. M.B. Brewer, In Retrospect.