Karthik Panchanathan

Indirect reciprocity can stabilize cooperation without the second-order free rider problem

Models of large-scale human cooperation take two forms. ‘Indirect reciprocity’ occurs when individuals help others in order to uphold a reputation and so be included in future cooperation. In ‘collective action’, individuals engage in costly behaviour that benefits the group as a whole. Although the evolution of indirect reciprocity is theoretically plausible, there is no consensus about how collective action evolves. Evidence suggests that punishing free riders can maintain cooperation, but why individuals should engage in costly punishment is unclear. Solutions to this ‘second-order free rider problem’ include meta-punishment, mutation, conformism, signalling and group-selection. The threat of exclusion from indirect reciprocity can sustain collective action in the laboratory. Here, we show that such exclusion is evolutionarily stable, providing an incentive to engage in costly cooperation, while avoiding the second-order free rider problem because punishers can withhold help from free riders without damaging their reputations. However, we also show that such a strategy cannot invade a population in which indirect reciprocity is not linked to collective action, thus leaving unexplained how collective action arises.

A tale of two defectors: the importance of standing for evolution of indirect reciprocity

Indirect reciprocity occurs when the cooperative behavior between two individuals is contingent on their previous behavior toward others. Previous theoretical analysis indicates that indirect reciprocity can evolve if individuals use an image-scoring strategy. In this paper, we show that, when errors are added, indirect reciprocity cannot be based on an image-scoring strategy. However, if individuals use a standing strategy, then cooperation through indirect reciprocity is evolutionarily stable. These two strategies differ with respect to the information to which they attend. While image-scoring strategies only need attend to the actions of others, standing strategies also require information about intent. We speculate that this difference may shed light on the evolvability of indirect reciprocity. Additionally, we show that systems of indirect reciprocity are highly sensitive to the availability of information. Finally, we present a model which shows that if indirect reciprocity were to evolve, selection should also favor trusting behavior in relations between strangers.

Balancing sampling and specialization: an adaptationist model of incremental development

Development is typically a constructive process, in which phenotypes incrementally adapt to local ecologies. Here, we present a novel model in which natural selection shapes developmental systems based on the evolutionary ecology, and these systems adaptively guide phenotypic development. We assume that phenotypic construction is incremental and trades off with sampling cues to the environmental state. We computed the optimal developmental programmes across a range of evolutionary ecological conditions. Using these programmes, we simulated distributions of mature phenotypes. Our results show that organisms sample the environment most extensively when cues are moderately, not highly, informative. When the developmental programme relies heavily on sampling, individuals transition from sampling to specialization at different times in ontogeny, depending on the consistency of their sampled cue set; this finding suggests that stochastic sampling may result in individual differences in plasticity itself. In addition, we find that different selection pressures may favour similar developmental mechanisms, and that organisms may incorrectly calibrate development despite stable ontogenetic environments. We hope our model will stimulate adaptationist research on the constructive processes guiding development.

Individual Differences in Developmental Plasticity May Result From Stochastic Sampling

The ability to adjust developmental trajectories based on experience is widespread in nature, including in humans. This plasticity is often adaptive, tailoring individuals to their local environment. However, it is less clear why some individuals are more sensitive to environmental influences than others. Explanations include differences in genes and differences in prior experiences. In this article, we present a novel hypothesis in the latter category. In some developmental domains, individuals must learn about the state of their environment before adapting accordingly. Because sampling environmental cues is a stochastic process, some individuals may receive a homogeneous sample, resulting in a confident estimate about the state of the world—these individuals specialize early. Other individuals may receive a heterogeneous, uninformative set of cues—those individuals will keep sampling. As a consequence, individual variation in plasticity may result from different degrees of confidence about the state of the environment. After developing the hypothesis, we conclude by discussing three empirical predictions.

The evolution of sensitive periods in a model of incremental development

Sensitive periods, in which experience shapes phenotypic development to a larger extent than other periods, are widespread in nature. Despite a recent focus on neural–physiological explanation, few formal models have examined the evolutionary selection pressures that result in developmental mechanisms that produce sensitive periods. Here, we present such a model. We model development as a specialization process during which individuals incrementally adapt to local environmental conditions, while receiving a constant stream of cost-free, imperfect cues to the environmental state. We compute optimal developmental programmes across a range of ecological conditions and use these programmes to simulate developmental trajectories and obtain distributions of mature phenotypes. We highlight four main results. First, matching the empirical record, sensitive periods often result from experience or from a combination of age and experience, but rarely from age alone. Second, individual differences in sensitive periods emerge as a result of stochasticity in cues: individuals who obtain more consistent cue sets lose their plasticity at faster rates. Third, in some cases, experience shapes phenotypes only at a later life stage (lagged effects). Fourth, individuals might perseverate along developmental trajectories despite accumulating evidence suggesting the alternate trajectory is more likely to match the ecology.

Two wrongs don't make a right: The initial viability of different assessment rules in the evolution of indirect reciprocity

Indirect reciprocity models are meant to correspond to simple moral systems, in which individuals assess the interactions of third parties in order to condition their cooperative behavior. Despite the staggering number of possible assessment rules in even the simplest of these models, previous research suggests that only a handful are evolutionarily stable against invasion by free riders. These successful assessment rules fall into two categories, one which positively judges miscreants when they refuse to help other miscreants, the other which does not. Previous research has not, however, demonstrated that all of these rules can invade an asocial population--a requirement for a complete theory of social evolution. Here, I present a general analytical model of indirect reciprocity and show that the class of assessment rules which positively judges a refusal to help scofflaws cannot invade a population of defectors, whereas the other class can. When rare, assessment rules which positively judge a refusal to help bad people produce a poor correlation between reputation and behavior. It is this correlation that generates the assortment crucial in sustaining cooperation through indirect reciprocity. Only assessment rules that require good deeds to achieve a good reputation guarantee a strong correlation between behavior and reputation.

The Evolution of Giving, Sharing, and Lotteries

A core feature of human societies is that people often transfer resources to others. Such transfers can be governed by several different mechanisms, such as gift giving, communal sharing, or lottery-type arrangements. We present a simple model of the circumstances under which each of these three forms of transfer would be expected to evolve through direct fitness benefits. We show that in general, individuals should favor transferring some of their resources to others when there is a fitness payoff to having social partners and/or where there are costs to keeping control of resources. Our model thus integrates models of cooperation through interdependence with tolerated theft models of sharing. We also show, by extending the HAWK-DOVE model of animal conflict, that communal sharing can be an adaptive strategy where returns to consumption are diminishing and lottery-type arrangements can be adaptive where returns to consumption are increasing. We relate these findings to the observed diversity in human resource-transfer processes and preferences and discuss limitations of the model.

Development: Evolutionary ecology's midwife

We agree with Henrich et al. that documenting cultural universality and variability provides an indispensable window into human nature. We want to stress the mediating role development plays between evolution and culture. Moving beyond the mere documentation of universality or variability, developmental approaches can provide mechanistic explanations, linking ecology to phenotype. Combining phylogeny and adaptationism, evolutionary approaches can explain the properties of developmental systems.

Enriching behavioral ecology with reinforcement learning methods

This article focuses on the division of labor between evolution and development in solving sequential, state-dependent decision problems. Currently, behavioral ecologists tend to use dynamic programming methods to study such problems. These methods are successful at predicting animal behavior in a variety of contexts. However, they depend on a distinct set of assumptions. Here, we argue that behavioral ecology will benefit from drawing more than it currently does on a complementary collection of tools, called reinforcement learning methods. These methods allow for the study of behavior in highly complex environments, which conventional dynamic programming methods do not feasibly address. In addition, reinforcement learning methods are well-suited to studying how biological mechanisms solve developmental and learning problems. For instance, we can use them to study simple rules that perform well in complex environments. Or to investigate under what conditions natural selection favors fixed, non-plastic traits (which do not vary across individuals), cue-driven-switch plasticity (innate instructions for adaptive behavioral development based on experience), or developmental selection (the incremental acquisition of adaptive behavior based on experience). If natural selection favors developmental selection, which includes learning from environmental feedback, we can also make predictions about the design of reward systems. Our paper is written in an accessible manner and for a broad audience, though we believe some novel insights can be drawn from our discussion. We hope our paper will help advance the emerging bridge connecting the fields of behavioral ecology and reinforcement learning.

Second-order free-riding problem solved? (reply): Human cooperation

We have shown that, if a system of indirect reciprocity is stable, exclusion from that system could deter collective-action cheats. Unlike direct punishment, indirect punishers benefit by avoiding donation, obviating the second-order free-rider problem. Fowler claims, however, that we assume away the second-order free-rider problem, and (by adding a new error term) argues that indirect-reciprocity defectors undermine cooperation.