Sendhil Mullainathan

Algorithmic Fairness and the Social Welfare Function

Social scientists have long been interested in discrimination and other inherent social inequities; and as such have developed models to evaluate policies through the dual lenses of efficiency and equity. More recently, computer scientists have illustrated show algorithms in many domains inherit and (sometimes inadvertently) bake in these same human biases and inequities. In this talk, I attempt to bring these two strands together: I embed concerns about algorithmic bias within a broader welfare economics framework. Instead of viewing the data as given, it begins with a model of the underlying social phenomena and their accompanying inequities. It then posits a social welfare function, where the social planner cares both about efficiency and equity. In particular, she places greater weight on equity than individual algorithm designers (firms or citizens) do. Intrinsic to this approach is that the social planners preferences imply desired properties of algorithm: the fairness of a given algorithm is not a primitive; instead, it is derived from the welfare of the outcomes it engenders. Several pieces of conventional wisdom do not hold true in this framework. For example, blinding the algorithm to variables such as race generally reduces welfare, even for the disadvantaged group. At the other extreme, I characterize situations where apparently fair algorithms can drastically increase inequities. Overall, I argue that it would be beneficial to model fairness and algorithmic bias more holistically, including both a generative model of the underlying social phenomena and a description of a global welfare function.

An opportunity for self-replication

To the Editor — We appreciate the Social Sciences Replication Project (SSRP) team’s work on these replications1. Naturally, we were disappointed to learn that our study 1 (ref. 2) did not replicate. Nevertheless, this is part of science and how it moves forward. Our paper was motivated by a question about why individuals in conditions of scarcity engage in certain financial behaviours, such as excessive borrowing. Previous explanations suggested these behaviours stemmed from the personality traits of the poor or structural barriers they face. We tested a different explanation — that resource scarcity itself can lead to these behaviours. We suggested that various forms of resource scarcity would have similar effects, and that a scarcity mindset would lead to attentional shifts that might drive behaviours such as over-borrowing. When the SSRP team contacted us, we welcomed the opportunity to have an independent replication of our study. We invited them to replicate all studies in the paper, but this was beyond the scope of their efforts. Their decision to replicate study 1 from each paper, they explained, was that “the first experiment typically provide[s] the first evidence of an hypothesized effect and the robustness of this effect is then typically demonstrated in the additional experiments.” In our paper, study 2 filled that role. While study 1 tested a peripheral hypothesis about cognitive fatigue, study 2 was the first to test the central hypothesis that scarcity leads to over-borrowing and all subsequent studies replicated that effect. The replication efforts made us want to revisit all of our results. We therefore conducted pre-registered replications of all five studies3. We invite other research teams to independently replicate these studies. We replicated three key results in this replication3. (1) Scarcity itself leads to overborrowing (the motivating hypothesis for the paper). (2) This is true for multiple kinds of resources. (3) Scarcity leads to greater focus. But we found weaker evidence for the hypothesis that scarcity-induced focus leads to neglect. Finally, we found no evidence that scarcity-induced focus leads to cognitive fatigue on subsequent tasks (study 1). Based on the SSRP’s findings1 and our own3, we believe that the original result2 was a false positive. Replication efforts that focus on a cross-section of studies provide useful overviews of the literature. However, they do not permit a deeper dive into individual research projects. Furthermore, the criteria used to select studies can overlook the most central hypotheses in a paper, as was the case with ours. Ultimately, to build a more reproducible social science, we need to understand which hypotheses and theories are robust. The SSRP highlights the need to replicate studies before publication. We replicated the most central findings, but were less vigilant about the introductory study. Today, we would have attempted to replicate that study as well — especially as it tests a hypothesis that was not central to our paper. Greater awareness and care of the kind raised by the SSRP should help more carefully balance central and peripheral claims in ways that increase publications’ reliability. ❐