Manipulation-Proof Machine Learning
MManipulation-Proof Machine Learning ∗ Daniel Bj¨orkegren † Brown University
Joshua E. Blumenstock ‡ U.C. Berkeley
Samsun Knight § Brown University
This version: April 9, 2020First version: November 30, 2018
Abstract
An increasing number of decisions are guided by machine learning algorithms. Inmany settings, from consumer credit to criminal justice, those decisions are madeby applying an estimator to data on an individual’s observed behavior. But whenconsequential decisions are encoded in rules, individuals may strategically alter theirbehavior to achieve desired outcomes. This paper develops a new class of estimatorthat is stable under manipulation, even when the decision rule is fully transparent. Weexplicitly model the costs of manipulating different behaviors, and identify decisionrules that are stable in equilibrium. Through a large field experiment in Kenya, weshow that decision rules estimated with our strategy-robust method outperform thosebased on standard supervised learning approaches.
Keywords : machine learning, manipulation, decisionmaking, targeting ∗ We are grateful for helpful conversations with Susan Athey, John Friedman, and Jesse Shapiro. Thisproject would not have been possible without the creative work of Chaning Jang, Simon Muthusi, NicholasOwsley, and the rest of the team at the Busara Center for Behavioral Economics. We thank numerousaudiences for helpful feedback. We are grateful for funding from the Brown University Seed Fund, the Bill andMelinda Gates Foundation, and the Digital Credit Observatory. Bj¨orkegren thanks the W. Glenn Campbelland Rita Ricardo-Campbell National Fellowship at Stanford University, and Microsoft Research for support.This study was pre-registered with the AEA RCT Registry (AEARCTR-0004649), and approved by the IRBsof UC Berkeley, Brown University, and the Kenya Medical Research Institute. † [email protected] ‡ [email protected] § samsun [email protected] a r X i v : . [ ec on . T H ] A p r Introduction
An increasing number of important decisions are being made by machine learning algorithms.Algorithms determine what information we see online (Perlich et al., 2014); who is hired, fired,and promoted (Brynjolfsson and Mitchell, 2017); who gets a loan (Bj¨orkegren and Grissen,2018), and whether to give bail and parole (Kleinberg et al., 2018). In the typical machinelearning deployment, an individual’s observed behavior is used as input to an estimator thatdetermines future decisions.These applications of machine intelligence raise two related problems. First, whenalgorithms are used to make consequential decisions, they create incentives for people toreverse engineer or ‘game.’ If agents understand how their behavior affects decisions, theymay alter their behavior to achieve the outcome they desire. Second, society increasinglydemands a ‘right to explanation’ about how algorithmic decisions are made (Goodman andFlaxman, 2016; Barocas et al., 2018). For instance, articles 13-15 of the European Union’sGeneral Data Protection Regulation mandate that “meaningful information about the logic”of automated systems be made available to data subjects (European Union, 2016). However,such transparency increases the scope for gaming: the more clearly that agents know howtheir behavior affects a decision, the easier it is to manipulate.These problems result from a simple core. The standard estimators that are used toconstruct decision rules assume that the relationship between the outcome of interest andhuman behaviors is stable. But this assumption tends to be violated as soon as a decisionrule is implemented: agents have incentives to change their behavior to achieve more favoredoutcomes. When decision rules are gamed, they can produce decisions that are arbitrarilypoor or unsafe. Lenders’ portfolios may be swamped with fraud, social media may be overrunby nefarious actors, self driving cars can be tricked into crashing (Eykholt et al., 2018). Thisproblem can undermine the use of machine learning in critical applications.There are two common approaches to deal with this problem. The first, familiar toeconomists, restricts models to predictors that are presumed to have a theoretical or structuralrelationship to the outcome of interest. This theory-driven approach amounts to havinga dogmatic prior that the cost of manipulation is either infinite (for included features) orzero (for excluded features). However, most behaviors are manipulable at some cost. Thesecond approach, which we refer to as the ‘industry approach’, keeps decision rules secret, and An extreme version of this restricts to predictors that causally affect the outcome of interest (Kleinbergand Raghavan, 2018; Milli et al., 2019). This may make manipulation desirable: for example, an exam mayinduce students to study and learn general knowledge. y i for each individual i .Each individual prefers a larger decision y i . We observe a training subset of cases thatpossess both features x i and optimal decisions y i . The policymaker seeks to estimate adecision rule ˆ y ( x i ) for cases in a testing subset where only features x i are observed. Standardmethods assume that x i ’s are fixed: training and test samples of ( x i , y i ) are drawn from samedistribution. Our method allows individuals to adjust behavior in response to the incentivesgenerated by the decision rule: x i (ˆ y ( · )) is a function of the decision rule. As a result, whileour training samples come from an unincentivized distribution ( x i (0) , y i ); test samples comefrom ( x i (ˆ y ( · )) , y i ). We assume individuals pay quadratic costs for manipulating behavior( x i ), and that these costs can be parametrized by a matrix C i . We describe several methodsto estimate this cost matrix, a new object needed to determine how behavior shifts whenincentivized.To sharpen intuition, we derive results for linear decision rules of the form ˆ y ( x ) = β x .The resulting estimator takes a simple nonlinear least squares form. Our method introducesa new notion of fit, which has analogues to other common linear regression approaches.3rdinary least squares (OLS) maximizes fit within sample; two stage least squares (2SLS)sacrifices fit within sample to estimate coefficients that have causal interpretations; penalizedleast squares (such as LASSO and ridge) sacrifice within-sample fit to better generalize toother samples drawn from the same population. Our method sacrifices fit within sample tomaximize equilibrium fit in the counterfactual where the decision rule is used to allocateresources, and agents manipulate against it. Our estimator is an example of a new class ofestimator that maximizes counterfactual fit –predictive fit in a counterfactual state of theworld.We use Monte Carlo simulations to compare this new strategy-robust approach to commonalternatives. OLS can perform extremely poorly when agents behave strategically. Theindustry approach, which periodically retrains the model, can also perform poorly andconverge slowly, or not at all. By contrast, our method adjusts the model to anticipatemanipulation. In simulations where agents respond to the decision rule, and manipulationcosts are known, our approach exceeds the performance of other estimators. Our approach canexceed the performance of others even if manipulation costs are misspecified for some cases.Under certain parameters, the presence of manipulation can improve predictive performance,if it signals unobservables associated with the outcome of interest (in the spirit of Spence,1973). In these cases, one may wish to use certain features that are manipulable by the typesthat you want to screen in, but not by those you want to screen out.In the second part of the paper, we implement and test our method in the contextof a field experiment in Kenya. This experiment allows us to compare the performanceof the strategy-robust estimator to standard machine learning algorithms in a real-worldenvironment. Specifically, we built a new smartphone app that passively collects data on howpeople use their phones, and disburses monetary rewards to users based on the data collected.The app is designed to mimic ‘digital credit’ products that are spreading dramaticallythrough the developing world (Francis et al., 2017). Digital credit products similarly collectuser data, and convert it into a credit score using machine learning, based on the insightthat historical patterns of mobile phone use can predict loan repayment (Bj¨orkegren, 2010;Bj¨orkegren and Grissen, 2019). However, as these systems have scaled, manipulation hasbecome commonplace as borrowers learn what behaviors will increase their credit limits(McCaffrey et al., 2013; Bloomberg, 2015). This field experiment produces several results. First, consistent with prior work, we showthat a person’s mobile phone usage behaviors ( x i (0)) can be used to predict characteristics of A recent survey in Kenya and Tanzania found that one of the top five reasons people report saving moneyin digital accounts is to increase the loan amount qualified for (FSD Kenya, 2018). Second,through the use of randomly-assigned experiments, we structurally estimate C i in our model,i.e., the relative costs of manipulating a variety of observed behaviors x i . Our experimentsoffer financial incentives to participants for altering behaviors that are observed through theapp, such as increasing the number of outgoing calls in a given week, or decreasing the numberof incoming text messages. The pattern of costs is intuitive: outgoing communications areless costly to manipulate than incoming communications; text messages, which are relativelycheap to send, are more manipulated than calls, which are relatively expensive. We also findthat complex behaviors (such as the standard deviation of talk time) are less manipulablethan simpler behaviors (such as the average duration of talk time).The next set of results demonstrate that strategy-robust decision rules, which accountfor the costs of manipulation, perform substantially better than standard machine learningalgorithms. We make this comparison by offering rewards to people who use their phoneslike a person of a particular type. For instance, some people receive a message that says,“Earn up to 1000 Ksh if the Sensing app guesses that you are a high income earner, based onhow you use your phone,” while others receive messages that offer rewards for acting likean “intelligent” person, and so forth. Across a variety of such decision rules, we show thatclassifications made with the strategy-robust algorithm are more accurate than classificationsfrom standard algorithms.Finally, we use our method to estimate the equilibrium cost of algorithmic transparency,i.e., the cost to the policymaker incurred for disclosing details of the decision rule. In theexperiment, we experimentally vary the amount of information subjects have about thedecision rule (e.g., the model used to predict the outcome), and show that the relativeperformance of the strategy-robust estimator increases with transparency. While predictiveperformance decreases by on average 23% under transparency for standard machine learningestimators, the strategy-robust estimator reduces this cost of transparency to approximately8%. Overall, this suggests that the equilibrium cost of moving from a regime where thedecision rules are secret, to one where they are disclosed, to be less than 8% in our setting.Our model allows policymakers to bound this equilibrium cost of transparency even withoutdisclosing decision rules to the world.Taken together, the paper develops and tests a new approach to supervised learning whenagents are strategic. This relates to papers from a variety of sub-literatures have confronted the Prior work has used mobile phone data to predict income and wealth (Blumenstock et al., 2015;Blumenstock, 2018), gender (Blumenstock et al., 2010; Frias-Martinez et al., 2010), and employment status(Sundsøy et al., 2016), and loan repayment (Bj¨orkegren and Grissen, 2018, 2019), .
The dilemma of manipulation is not new. Goodhart (1975), in what has since becomereferred to as ‘Goodhart’s Law’, noted that once a measure becomes a target, it ceases tobe a good measure. Lucas (1976) also famously observed that historical patterns can warpwhen economic policy changes. More broadly, our approach connects with literatures in botheconomics and computer science.Our problem can be viewed as a mechanism design problem. Canonical signaling models(Spence, 1973) rely on a single crossing condition to allow full revelation of individual types.In our setting, like the settings of Frankel and Kartik (2019) and Ball (2019), there are twoforms of heterogeneity: types θ i and the costs of manipulating behavior C i . Frankel andKartik (2019) show that unobserved heterogeneity in manipulation costs C i ‘muddles’ therelationship between behavior x i and types θ i , causing the single crossing condition to fail.That paper shows that muddling reduces the information available in a market. Ball (2019)extends that framework to multiple dimensions of behavior, and in a theoretical model similarto ours, characterizes and proves the existence of equilibrium. That paper also considers howthe problem is affected by the degree of commitment available to the policymaker. Relativeto this work, our paper builds a model that can be empirically estimated, which allows us toprobabilistically separate types and costs. Our paper is also related to the problem in public finance of setting taxes in environmentswhere agents adapt their behaviors. Our method weights predictors by the inverse of the In a related setting, Hussam et al. (2017) implement an incentive compatible mechanism that collectspeer reports to estimate an individual’s entrepreneurial ability. That method requires gathering peer reportsfrom a community during implementation; in contrast, our approach produces stand in replacements forstandard machine learning models, which can use arbitrary data on behavior. Also related, Holmstr¨om(1979) shows that a principal should use any information that has signal when contracting with an agent.Our method suggests how manipulable information be downweighted. Eliaz and Spiegler (2018) study arelated problem where a “statistician” is making decisions on behalf of an agent, with two-sided incompleteinformation: the agent knows his preferred behavior, but the statistician knows the decision rule. They focuson characterizing incentive-compatible estimators, and find that commonly-used regularized linear modelscreate incentive issues. Also related is the concept of ‘covariate shift’, whichconsiders scenarios where a test distribution differs from the training distribution. However,it is common to assume that the conditional distribution y | x is fixed, and the distributionof x ’s changes exogenously (Sayed-Mouchaweh and Lughofer, 2012). The manipulation weconsider induces the conditional distribution y | x to change endogenously when action is takenbased on the estimated relationship.Thus, papers from a variety of sub-literatures have confronted the notion that agentswill act strategically when their actions are used to determine allocations. Relative to priorwork, our paper makes two main contributions. First, we develop an equilibrium model ofmanipulation that can be estimated using data, which produces a machine learning estimatorthat functions well under manipulation even when the decision rule is fully transparent. Andsecond, to our knowledge for the first time in any literature, we design and implement afield experiment that stress-tests such an estimator in a real-world setting with incentivizedagents. Agents game decision rules in a wide variety of empirical settings. Manipulation has beendocumented in contexts ranging from New York high school exit exams (Dee et al., 2019)and health provider report cards (Dranove et al., 2003), to pollution monitoring in China For instance, Bruckner and Scheffer (2011) study adversarial prediction when the agent acts in responseto an observed predictive model, with an application to spam filtering. Dong et al. (2018) model an iteratedindustry approach where a policymaker observes how agents manipulate in response to previous rules, butdoes not know their utility functions or costs. y i ) based on easily observable characteristics or behaviors ( x i ) (Hanna and Olken,2018). The policymaker may infer a household’s type based on the levels of these variables, or,implicitly, on how they change in response to incentives. There is evidence that such decisionrules induce households to manipulate their observable features. For instance, Banerjee et al.(2018) find that adding a question about flat screen TV ownership to a census caused peopleto underreport ownership by 16% on a follow-up survey, in order to appear less wealthy. The method we develop is directly relevant to a variety of other settings where a policy-maker derives a decision from a prediction ( y i ) based on agent behaviors ( x i ). These includeother supervised settings where it is possible to obtain a ground truth value of y i for a trainingsample of individuals. For instance, in credit scoring applications, a decision about whetherit is prudent to provide a loan ( y i ) is made based on characteristics on the potential borrower(traditional credit scores are based on the borrower’s formal credit history, but increasinglythe characteristics x i include private data like mobile phone usage (Bj¨orkegren and Grissen,2019) and social network structure (Wei et al., 2015)). It also includes settings where nodefinite ground truth of y i exists. Search engines, social media, and spam filters attemptto determine the quality of a piece of content ( y i ) based on features that can be observed( x i : keywords, reputation of the sender, inbound links). Manipulating these features may becostly directly, or may undermine the author’s intent in distributing the content. Similarly,‘report cards’ for universities, hospitals, and doctors attempt to determine quality ( y i ) basedon indicators ( x i : alumni giving rates, endowment size, acceptance rates, graduation rates). Our method thus nests this latter case of self-targeting (Nichols and Zeckhauser, 1982; Alatas et al.,2016), which identifies beneficiaries based on willingness to engage with a costly “ordeal.” In other examples from the development literature, Camacho and Conover (2011) find that after aprogram eligibility decision rule was made transparent to local officials in Colombia, it was manipulated byan amount corresponding to 7% of the National Health and Social Security budget. They note, “there isanecdotal evidence of people moving or hiding their assets, or of borrowing and lending children.” Our model does not consider behaviors x i that have a causal relationship to y i , where manipulation canbe productive (Kleinberg and Raghavan, 2019). It thus would not cover report card variables that directlyinfluence quality, nor the case of a student who ‘games’ a test by studying ( x i ↑ ), and as a result improves This section introduces the model underlying our estimator, and demonstrates the intuitionwith simulations.
A policymaker observes a training subset of cases that possess both features x i and optimaldecisions y i . The policymaker also obtains information on the costs of manipulating features,which will be detailed later. The policymaker would like to estimate the parameters of adecision rule ˆ y ( x i ) for cases in a testing subset where only features x i are observed, and maybe manipulated.A policymaker has a preferred action y i for each individual i , denominated in units ofindividuals’ utility. The action y i can be projected onto i (cid:48) s bliss behavior x i by the equation y i = b + b (cid:48) x i + e i , with e i ⊥ x i representing idiosyncratic preference.However, the policymaker observes an individual’s actual behavior x i , which may differfrom their bliss level x i . It selects a deterministic decision rule of the form :ˆ y ( x i ) = β + β (cid:48) x i Individuals can manipulate their behavior x i away from their bliss level x i at some cost. i earns utility from the decision minus these costs: u i = ˆ y ( x i ) − c ( x i , x i )For simplicity, we consider the case where the utility from the decision exactly coincideswith the policymaker’s prediction. their knowledge ( y i ↑ ). The approach could be extended to cover such cases. Although randomizing a decision rule may make it harder to manipulate, it undermines a major goal oftransparency: that people know how they are evaluated. That is, we consider the case where the utility of the decision u (ˆ y ) = ˆ y , which holds in our experiment.Under more general functions u ( · ), our model would represent a linear approximation. One could easilygeneralize our framework to allow for more general functional forms. i are heterogeneous in two respects, bliss behaviors x i and gaming ability γ i (as in Frankel and Kartik (2019)).Manipulation costs are quadratic: c ( x i , x i ) = 12 ( x i − x i ) (cid:48) C i ( x i − x i )for matrix C i : C i = 1 γ i α · · · α K ... . . . ... α K · · · α KK Different behaviors may be differentially hard to manipulate, by themselves (the diagonal α kk ) or in conjunction with other behaviors (the off diagonals α kj ). And different people mayfind it easier or harder to manipulate ( γ i ): for example, people with more technical savvy orlower opportunity cost of time may find it easier to game decision rules.When i knows the decision rule ˆ y ( x i ) and receives benefits according to it, he will optimallymanipulate behavior to level: x ∗ i ( β ) = x i + C − i β When behavior is not incentivized ( β = ), optimal behavior equals the bliss level( x ∗ i ( ) = x i ). However, as β moves away from zero, behavior moves in the same direction,downweighted by the cost of manipulation (as highlighted in blue). Decision rules.
The policymaker faces expected squared loss: L (ˆ y ( · )) = E i (cid:2) [ y i − ˆ y ( x i (ˆ y ( · )))] + M ( · ) (cid:3) The first term represents fit of the model in the counterfactual where the model isimplemented and agents manipulate behavior. If the policymaker additionally cares aboutthe costs that individuals incur manipulating, this manipulation cost results in additionalterm M ( · ).Our strategy-robust decision rule is given by: β stable = arg min β (cid:32) N (cid:88) i (cid:2) y i − β − β (cid:48) (x i + C − i β ) (cid:3) + . . . (cid:33) (1)which deviates from ordinary least squares due to the term C − i β which captures manipu-10ation in response to β . Additional terms ‘ . . . ’ can include any weight M ( · ) the policymakerplaces on manipulation costs incurred by agents, and any regularization terms R λ decision ( · ). Discussion
If the policymaker only cares about targeting performance ( M ( · ) ≡
0) and there are noadditional regularization terms ( R ( · ) ≡ E (cid:2) x i · (cid:0) y i − β − β (cid:48) (x i + C − i β ) (cid:1)(cid:3) = − E (cid:2) C − i β · (cid:0) y i − β − β (cid:48) (x i + C − i β ) (cid:1)(cid:3) This suggests that the estimator imposes that equilibrium errors in the counterfactual are less than orthogonal to individual types x i : they equal the negative of an adjustment factor2 C − i β that accounts for the fact that β induces a marginal incentive to respond. When C i ≡ ∞ , the resulting estimator corresponds to OLS.When the policymaker cares about not only the resulting allocation, but also the manipu-lation costs that individuals incur, this is accompanied by the term M ( · ), which can take adifferent form depending on policymaker preferences. An entity that is narrowly concernedwith its own objective (e.g., profits in the case of a firm) may thus select different decisionrules from those that maximize social welfare (for example, a firm may be satisfied with anequilibrium where all individuals expend welfare gaming a test, where a social planner maynot). To reduce overfitting in small samples, one may also include common forms of regu-larization; for example, R LASSOλ decision ( β ) = λ decision (cid:80) k> | β k | or R ridgeλ decision ( β ) = λ decision (cid:80) k> β k .Hyperparameter λ decision can be set with cross validation in the baseline sample. Under theseregularization terms, when M ( · ) ≡ C i ≡ ∞ the resulting estimator corresponds toLASSO, or ridge, respectively. We demonstrate the method with Monte Carlo simulations.We derive desired payments y , from individual types x and payment rule b , with deviations e . We then assess decision rules ˆ y ( x ) based on observed behaviors x generated with differentestimators. Our strategy-robust estimators anticipate that behaviors x may change when For example, the policymaker may place weight w on the sum of manipulation costs: M ( · ) = w (cid:80) i c (x i + C − i β , x i ). The Supplemental Appendix derives a microfounded term for the case of proxy means testing. C . This section assumesthat manipulation costs are known. Comparative Statics
We consider a case where x is more predictive than x in baseline behavior, but would beeasily manipulated if used in a decision rule ( b > b but α (cid:28) α ).Figure 1 compares our method to OLS and LASSO, which mistakenly place most weighton x . OLS maximizes predicted performance within the unincentivized sample ( x i ( ) , y i ); asshown in Figure 1a, it performs poorly as manipulation becomes easier. Figure 1b shows thatfor a given cost of manipulation, LASSO shrinks these coefficients. However, when LASSOselects variables, it does exactly the wrong thing: it kicks x out of the regression first. Incontrast, our method considers how predictive features will be in equilibrium when the decisionrule is implemented: ( x i ( β ) , y i ). As shown in Figure 1c, when manipulation costs are high,our method approaches OLS, but as manipulation becomes easier, our method substantiallypenalizes x . Our method can also be combined with LASSO or ridge penalization to finetune out of sample fit. If each feature is equally costly to manipulate, our method shrinks them together,similar to ridge regression, as shown in Figure A2. If all individuals have the same gamingability ( γ i ≡ γ ), then manipulation shifts behavior uniformly and does not affect predictiveperformance. However, even though predictive performance is high, individuals’ can spendsubstantial utility on manipulation. Figure A1 develops this intuition further, by showinghow the strategy-robust method penalizes indicators that are easy to shift: Figure A1a showsthe effect of scaling the cost of one behavior ( x ). As the cost of manipulating that particularbehavior ( α ) decreases, it is penalized, and weight is shifted to other behaviors. The methodalso penalizes indicators that make it easier to shift other predictive indicators (in a mannersimilar to Ramsey (1927) taxation). Figure A1b shows that the effect of cost interactions:when manipulating x makes it easier to manipulate x ( α sufficiently negative), our methodfurther reduces weight on x . Performance
Table 1 shows the results of an example Monte Carlo simulation, chosen to demonstrate howstandard approaches can fail. In this simulation, type x has a large weight in the desired See Appendix Figure A2 for a comparison to ridge regression, as well as a demonstration of combiningour method with ridge penalization. β OLS ( / γ ) (b) β LASSO ( λ ; γ = 1) (c) β stable ( / γ ) B2B1 C oe ff i c i en t B1B2 B1B2 Manip Cost Scale S q . E rr o r LASSO Lambda
Manip Cost Scale
Manipulation Cost = 1
Note:
The first behavior is more predictive ( b > b ), but is easily manipulable ( α (cid:28) α ). (a) OLS performance deteriorates substantially when behavior can be manipulated. ( b ) LASSOpenalization favors x , which will be manipulated as soon as the decision rule is implemented. (c) Our method anticipates that x will be manipulated if it is incentivized. It shifts weight to x asbehavior becomes manipulable.x i iid ∼ N (0 , b = [1 . , C het = γγ i (cid:20) (cid:21) , γ heti iid ∼ U nif orm [0 , e i iid ∼ N (0 , . b = 3) relative to the other two dimensions ( b = b = 0 . x is much easier to manipulate ( α = 1 vs. α = 2 and α = 4).In this environment, OLS considers the static relationship in the unmanipulated data.This rule would perform well if behavior were held fixed (no manipulation column); however,once consumers adjust to the rule, it makes terrible decisions (manipulation column).The industry approach would retrain (refresh) this model after this manipulation. Ifwe observe how consumers adjust their behavior and reestimate OLS, we obtain β OLS (1) ,which places negative weight on the manipulated x . However, its also makes terribledecisions when consumers respond to it. We can try to do better by repeatedly allowingindividuals to best respond, and then reestimating the decision rule. But even with perfectinformation and no changes in the environment, this process can make poor decisions enroute to convergence, or may not converge at all. If we estimate β OLS ( r ) using data from allprior periods (1 , . . . , r − β OLS ( r ) using only data from theprior period ( r − x (see Table A1). Thus standard approaches canperform poorly even in ideal cases. If there were noise or frictions in learning, the risksof this approach are greater: the rule may appear to be performing well, and suddenlybe devastatingly undermined (for example, Gonzalez-Lira and Mobarak (2019) find thatincreased enforcement of a ban on selling an endangered fish can lead vendors to learn aboutthe decision rule, and more effectively undermine it).In contrast, our strategy-robust estimator ( β stable ) anticipates that including a behavior inthe decision rule will shift that behavior. It penalizes the easily manipulable behavior x , andshifts weight to behaviors that are harder to manipulate ( x and x ). It sacrifices performancein the environment in which it is trained (in sample, no manipulation) for performance inthe counterfactual where there is manipulation. When individuals manipulate as describedin the model, our estimator exceeds the performance of other estimators.Our method can reduce risk even if manipulation costs are misestimated. We consider acase with two measurement mistakes: (a) all off diagonal elements are set to zero, and (b) theestimated costs of manipulation are two times too large. Performance deteriorates relative to14he case where we know the true cost matrix, but our method still outperforms OLS in thepresence of manipulation. One can use our method as a first step towards equilibrium, andthen follow it with the industry approach; as shown in the bottom rows, doing so skips theterrible decisions made in the first two iterations of the industry approach. Manipulation can improve performance
Manipulation can improve performance, if ease of manipulation ( γ i ) is correlated with theoutcome ( y i ). In that case, manipulation itself represents a signal of the underlying type, asin Spence (1973), and applications of self-targeting (Nichols and Zeckhauser, 1982; Alatas etal., 2016). An example is shown in Table A2: manipulation improves the performance evenof na¨ıve estimators, as shown in the first two rows. Our method can additionally exploit costheterogeneity, and thus further improves performance as shown in the third row. Our model can be fully estimated with experimental data. To estimate manipulation costs,we hire study participants to undermine component parts of the model, and gauge howsensitive these manipulations are to incentives.We observe multiple time periods. Each period, an individual may desire to deviate frombliss behavior due to manipulation, or shocks that are common ( µ t ) or individual specific( (cid:15) it ): u it = ˆ y ( x i ) − c ( x i , x i ) + ( µ t + (cid:15) it ) · ( x i − x i )where both components are mean zero: E µ t = and E (cid:15) it = . Then, in week t we willobserve behavior: x ∗ it ( β ) = x i + µ t + (cid:15) it + C − i β (2)We parameterize the inverse of the cost matrix as follows: C − i = γ i · C − with elements of inverse costs defined for convenience as:15able 1: Manipulation Can Harm Prediction (Monte Carlo) Decision Rule Performance (squared loss) β β β β No manip. Manipulation
Panel A:
Data generating process b DGP
Panel B:
Standard Approaches β OLS ‘Industry’ Approach (estimated cumulatively) β OLS (1) after β OLS -0.798 0.061 2.090 -1.675 3.275 625.762 β OLS (2) after β OLS -2.174 0.174 0.436 0.143 12.861 8.369 β OLS (3) after β OLS -1.376 0.165 0.573 0.483 9.343 4.415... β OLS (100) after β OLS -1.619 0.316 0.753 -0.059 8.442 2.105... β OLS (1000) after β OLS -1.854 0.489 0.582 -0.124 9.211 1.959
Panel C:
Strategy Robust Method β stable -1.813 0.503 0.536 -0.096 9.155 1.939 If costs are misestimated: β stable ˆ C i =2 diag ( C i ) -1.566 0.658 0.719 -0.352 6.893 10.826 Followed by Industry Approach (estimated cumulatively): β OLS (1) after β stable ˆ C i =2 diag ( C i ) -2.045 0.800 0.042 0.418 10.891 4.447 β OLS (2) after β stable ˆ C i =2 diag ( C i ) -2.022 0.558 0.327 0.137 10.685 2.453 Notes : Monte Carlo simulation results. Panel A shows the coefficients that relate the outcome ( y ) tobehaviors ( x ) under the data generating process (DGP). Panel B shows coefficients from OLS; Panel C showscoefficients estimated with the strategy robust method. Performance is assessed on the same sample ofindividuals, under behavior without manipulation: x i ( ), or with: x i ( β ). Parameters: C = . . . . . . . . . , x iid ∼ N , .
11 2 10 . , γ i = (cid:40) i ≤ .
210 x i > . e i iid ∼ N (0 , . − =: c · · · c K ... . . . ... c K · · · c KK Gaming ability includes two types of heterogeneity: γ i = e − ω z i + v i It is allowed to vary with characteristics z i that are observable in the training sample(but need not be observed in an implementation sample; for example, we survey participantson tech savviness). It also includes unobserved heterogeneity v i ∼ V with Ev i = 0, whichwill enter the model as random effects.We estimate strategy-robust decision rules in two steps. We first estimate primitives: types x, cost parameters ω and C − , and the distribution ofunobserved gaming ability V . Types
We infer types x by observing baseline behavior prior to the implementation of a decisionrule. When β = , behavior will not be manipulated. We can estimate types and time periodfixed effects with moment conditions derived from the equation: x ∗ it ( ) = x i + µ t + (cid:15) it (3)including only time periods where β = . Costs
Our main specification recovers manipulation costs experimentally. Each week we randomlyassign individuals to a decision rule β it . The decision rule may be a control, in which case β it ≡
0. Or, it may be a treatment group that incentivizes one behavior k ∈ ...K , bydisclosing a rule that pays incentives for k : β itk > β itj = 0 for j (cid:54) = k . These treatments make it possible to recover the inverse cost matrix (diagonal and17ff-diagonal elements), as well as heterogeneous gaming ability (observed ω and unobserved V ). Moment Conditions
We recover all parameters jointly with the following moment conditions.Incentives are orthogonal to idiosyncratic behavior shocks ( E [ β itk (cid:15) itj ] = 0). For each pairof behaviors jk (including j = k ) this yields sample moment condition:0 = 1 N N (cid:88) i =1 β itk (cid:2) x ijt − x ij − µ jt − β itj (cid:0) e − ω z i · c kj (cid:1)(cid:3) We also have E [ (cid:15) itj ] = 0: for each time period t and behavior k , we obtain: µ kt = 1 N N (cid:88) i =1 (cid:2) x ikt − x ik − β it (cid:0) e − ω z · C − (cid:1)(cid:3) For each individual i and behavior k , we obtain:x ik = 1 T T (cid:88) t =1 (cid:2) x ikt − µ kt − β it (cid:0) e − ω z · C − (cid:1)(cid:3) given T observations.Unobserved heterogeneity is mean zero ( E [ v i ] = 0), yielding:0 = 1 T (cid:88) i,k,t where k incentivized (cid:20) x ikt − x ik − µ kt C − β it − e − ω z i (cid:21) Each heterogeneity characteristic z ∈ z is orthogonal to unobserved heterogeneity( E [ z i v i ] = 0), yielding:0 = 1 T z (cid:88) i z i (cid:88) k,t where k incentivized (cid:20) x ikt − x ik − µ kt C − β it − e − ω z i (cid:21) These moment conditions jointly identify x, C − , and ω . Joint Estimation
We jointly solve for the parameters to minimize the squared distance from zero: L (x , C − , ω ) + R λ costs costs ( C − , ω )18here L ( · ) represents the associated general method of moments (GMM) loss function. Penalization and Cross Validation
We make include two adjustments to reduceoverfitting of the cost matrix to our limited dataset. First, we impose the constraint thatincentivizing a behavior increases it: c jj >
0. Second, we regularize the cost estimates: R λ costs costs ( · ) = (cid:34) λ costsdiagonal (cid:88) k c kk + λ costsoffdiagonal (cid:88) j (cid:54) = k c jk (cid:35) (cid:34)(cid:88) i e − ω z (cid:35) where we allow the possibility of using separate hyperparameters λ costs = { λ costsdiagonal , λ costsoffdiagonal } for diagonal and off diagonal costs. These penalize the cost of manipulation towards infinity(ease of manipulation towards zero), which will tend to penalize our method’s estimatestowards standard methods (OLS/LASSO/etc).We jointly solve for parameters x, C − , and ω , and hyperparameters λ costs to minimizeout of sample prediction error, using cross validation. Then, we impose the optimal λ costs and jointly estimate x, C − , and ω on the full sample. Unobserved Gaming Ability
After estimating these parameters, we back out the distribution of unobserved gaming ability V in two steps. First we compute whether each individual manipulates more or less thanpredicted during incentivized weeks:˜ v i = 1 K i (cid:88) k T i (cid:88) t where k incentivized (cid:20) x ikt − x ik − µ kt C − β it − e − ω z i (cid:21) Second, to reduce the impact of noise and outliers, we shrink and winsorize these backedout shocks. We form the empirical distribution V = { max( φ · ˜ v i , v) } i , where v is the lowestvalue of ˜ v that leads to a nonnegative implied gaming ability. We set the shrinkage factor φ to 0.005 so that less than 5% of distribution is winsorized. This yields a distribution ofcosts C i . Given these primitives, a strategy robust decision rule is given by: That is, v = min i ( ˜ v i | ˜ v i ≥ min j ( e − ω z j )). After shrinkage, 4.1% of observations are winsorized. stable = arg min β E (cid:34) N (cid:88) i (cid:2) y i − β − β (cid:48) (x i + C − i β ) (cid:3) + R λ decision decision ( β , y , C ) (cid:35) taken over expectation over C i , and given decision rule regularization term R λ decision ( · ).Hyperparameter λ decision is set through cross validation in the unmanipulated sample (wherewe can observe ground truth): λ decision = c.v. arg min λ cv (cid:34) min β naive (cid:34) N (cid:88) i (cid:2) y i − β naive − β naive x i (cid:3) + R λ cv decision ( β naive , y , C ) (cid:35)(cid:35) We designed a field experiment to test the performance of our strategy-robust estimator in areal-world setting. Design started in 2017. Working with the Busara Center for BehavioralEconomics in Nairobi, we developed and deployed a new smartphone-based application (‘app’)to 1,557 research subjects. The app was designed to mimic the key features of the ‘digitalcredit’ apps that are quickly transforming consumer credit in developing countries (Francis etal., 2017). In Kenya, at the time of our study, CGAP (2018) estimates that 27% of all adultshad an outstanding ‘digital credit’ loan. These phone-based apps construct an alternativecredit score ( ˆ y i ) based on how each applicant uses their phone ( x i ; Bj¨orkegren (2010);Bj¨orkegren and Grissen (2019)). The app we built similarly collects data on how each subjectuses their phone, and uses that data to make cash transfer decisions. This section describesthe app and experimental design (Section 4.1); estimates costs of manipulation and derivesstrategy-robust decision rules using our method; and compares the performance of these newestimators to traditional learning algorithms (Section 4.3). Our design was pre-specified in apre-analysis plan registered in the AEA RCT registry under AEARCTR-0004649. Our experiment is intended to create an environment with incentives similar to those of a‘digital credit’ lending app. These apps run in the background on a smartphone, and collectrich data on phone use (including data on communications, mobility, social media behavior,and much more). Digital credit apps use this information to allocate loans to people whoappear creditworthy (i.e., for whom ˆ y i exceeds some threshold). Since financial regulationsprevented us from actually underwriting loans to research subjects, we instead focused on20nalogous problems where a decisionmaker wishes to allocate resources to individuals withspecific characteristics—for instance, by paying individuals who have a certain income level,or other characteristic (e.g., intelligence, level of activity, education). This allows us tofocus on the mechanics of manipulation in a prediction task, which is the same regardless ofwhich outcome is predicted.
Smartphone app
The ‘Smart Sensing’ app we built has has two key features. First, it runs in the backgroundon the smartphone to capture anonymized metadata on how individuals use their phones,such as when calls or texts are placed, which apps are installed and used, geolocation,battery usage, wifi connections, and when the screen was on. In total, we extract over¯
K > ,
000 behavioral features — Appendix Figure A3 shows the correlation between 80different behavioral indicators (“features”) collected through the app. Second, the appprovides a platform to deliver weekly “challenges” to research subjects (see Figure 2). Thesechallenges appear on the subject’s phone, and offer financial incentives based on their behavior.The challenges can be very simple (‘You will receive 12 Ksh. for every incoming call youreceive this week’) or more complex (‘Earn up to 1000 Ksh. if the Sensing app guesses youare a high-income earner’). Users are paid a base amount of 100 Ksh. for uploading data,plus any challenge winnings, directly via M-PESA at the conclusion of each week.
Study population and recruitment
The subject population consists of Kenyans aged 18 years or older who own a smartphoneand are able to travel to the Busara center in Nairobi. Participants were recruited through inperson solicitations in public spaces in neighborhoods around Nairobi. From this master listof potential participants, every third individual was saved for a ‘top up’ sample; we drewinvited individuals from this list to participate later in the experiment, to form a fresh testsample. The remaining sample was invited at the beginning. All individuals were sequentiallyinvited for an enrollment session at the Busara center. (The center had a capacity to enroll200 people per week.) During enrollment, participants complete a survey on a tablet ondemographics and technology usage. These responses will form the ground truth about users While these target predictions may bear little resemblance to credit-worthiness, there are many settingswhere characteristics like these are being inferred by digital traces (for example, welfare programs that targetunmarried women, or digital advertisers who target college students). The app is designed to capture this data with minimal impact on battery life and performance. Data isuploaded to secure Busara servers at a set frequency, or can be uploaded manually. (a) Installation Screen (b) Challenge with Hint (c) Earnings Calculator that we seek to infer based on phone usage behavior.Prospective participants were given the opportunity to install the Sensing App on theirphones for about 16 weeks. Participants were told the dimensions of behavior that wouldbe captured and used anonymously, and assured that no content of calls or text messageswould be recorded. Participants were given the opportunity to ask questions. Participantsshowed understanding of the privacy tradeoffs involved, and voiced trust in Busara based onits positive reputation in this community. Participants who opted in to the study were offeredhelp installing the Sensing App, which provided the main interaction of the study. Duringinstallation, participants had the opportunity to view the Android permissions required and todecide whether to accept. Our sample includes only participants who opted in. Participantscould elect to receive challenges in English, Swahili, or both. 82.6% elected English, 15.9%elected Swahili, and 1.4% elected both.
Weekly rhythm
The study follows a weekly rhythm. Each Wednesday at noon, each user receives a genericnotification, ‘Opt in to see this week’s challenge!’, via Android notifications and a textmessage. When a user opens the app, it will ask them to opt in to a challenge for thatweek. Only after a user opts in are the details of their challenge for that week revealed (see22igure 2). Challenges are valid until 6pm Tuesday. At the conclusion of the challenge,users have 16 hours to ensure that their data is uploaded (until 10am Wednesday). Busarathen computes and sends any payments to users via M-PESA by noon Wednesday, and usersreceive the next challenge.Each week, participants could attrit in two ways: by not uploading their data, or bynot opting in to the challenge. Participants who failed to upload or opt in were sent textmessage reminders, or called by Busara staff, following an attrition protocol detailed inAppendix A1.2. We include in our analysis only participant-weeks where the participantopted in, and uploaded during the end-of-week upload window.
Predicting user characteristics
We begin the experiment with baseline weeks that have no incentives (no active challenges).These baseline weeks allow us to estimate each individual’s type in absence of manipulation,x. We estimate each dimension of type using Equation 3, with week fixed effects to absorbidiosyncratic weekly shocks.Consistent with prior work (Blumenstock et al., 2015; Bj¨orkegren and Grissen, 2019),we find that characteristics of users can be predicted from phone behaviors. Results forseveral outcomes, based on OLS, are shown in Table 2. For characteristics such as monthlyincome, intelligence (Ravens Matrices), and overall phone activity, R values range from 0.02to 0.15. To make these rules easier for participants to interpret, we will focus on threevariable decision rules selected via LASSO; the last row of Table 2 shows that these obtainsimilar R when cross validated. Evidence that app-based challenges induce manipulation
We will eventually use variation in behavior induced by our randomized experiment toestimate the cost of manipulating different behaviors, C ( z i ). This exogenous variation comes To minimize the possibility of differential attrition, the pre-opt-in notification was the same for all usersregardless of their assigned challenge. As some participants may upload data sparsely throughout the week, only those who upload withinthe 21-hour window at the end of the challenge-week (between 1pm Tuesday and 10am Wednesday) will becounted as having fully uploaded all of their weekly data. In these ‘control’ weeks, the subject receives a challenge of the form, ‘Dear user, you do not have todo anything for this week’s challenge. You will receive an extra Ksh 50 for accepting this challenge.’ Ourmethod could also be used without these control weeks, as long as there is variation in incentives betweenweeks; one would then need to net out the manipulation in estimation.
Monthly Income Intelligence Activity PCA
OLS (Ravens)Average Duration of Workday Calls -6 .
877 (0 . . .
6) -0 . . .
746 (0 . . . . . .
747 (0 . .
006 (0 . . . .
477 (0 . .
016 (0 . .
003 (0 . .
962 (0 . .
001 (0 . .
005 (0 . .
904 (0 . . . . . .
739 (0 . . . . . .
950 (0 . .
003 (0 . . . .
130 (0 . .
002 (0 . . . .
666 (0 . . . .
001 (0 . .
556 (0 . .
001 (0 .
14) 0 .
004 (0 . .
762 (0 .
6) 0 .
002 (0 . .
001 (0 . .
547 (0 . .
071 (0 . .
956 (0 . . . . . . . Notes : Each column indicates a different prediction target. P-values in parentheses. N represents individuals.10-fold cross-validated R2 is reported for a LASSO regression where the regularization parameter is set inorder to achieve a 3-covariate model. M for each additional x j you do’, where amount M and behavior j are assigned randomly. For example, one challenge was, ‘You will receive 3Ksh. for each text you send this week, up to Ksh. 250.’ In the long run, individuals mayidentify new, easier ways to manipulate these indicators. To mimic this, we held focus groupsto identify the most effective ways to manipulate different features, and during onboarding,exposed each participant to a discussion of how one could change different types of behavior(this is similar to hiring ‘white hat’ hackers to uncover security weaknesses).People response to these challenges, as anticipated by our theory (Equation 2). Forintuition, Table 3 shows how behavior changed in response to simple challenges. Each columnshows a regression of an outcome on different incentives (randomly assigned). Individualsmanipulate the particular behaviors that were incentivized, as shown by the diagonal, whichis positive and significant for these outcomes. Incentivizing one behavior also affects others,as shown in the off diagonal elements. For example, incentivizing missed incoming callsalso increases the number of texts sent (presumably requests to contacts to be called). Ourmethod can theoretically exploit these cross elasticities.Since we have a limited sample on which to estimate costs, our challenges focus onincentivizing a subset of K focal behaviors (from the full set of ¯ K ). Specifically, we selectbehaviors x C that are useful in predicting the set of user characteristics that form the basisfor our ‘complex’ challenges. To identify this subset, we run LASSO regressions for each y to induce variable selection, and include the selected variables { x k | β naivek (cid:54) = 0 } . For each ofthese variables, we pair an additional behavior that measures a similar concept but which weanticipate may be differently easy to manipulate (for example, if a na¨ıve regression selectsoutgoing calls, we will also include the variable incoming calls). Note that by including onlya subset of variables, our procedure implicitly assumes that omitted variables are costlessto manipulate (and therefore should not be included in any decision rule); we will thusunderestimate the performance that could be attained with our method if costs were fullyestimated. In Section 5, we evaluate other potential methods to lower the expense of We determined “similar” behaviors as those that met at least one of the following conditions: (1)correlated with the primary behavior with a coefficient of at least 0.75; (2) was a ‘close cousin’ of the primarybehavior, in that it was a different transformation of a similar underlying behavior (e.g., for ‘weekly numberof late-night calls’, ‘maximum number of late-night calls in a single day’ would be considered a close cousin);(3) a cross validated LASSO regression that excluded the principal behavior from the feature set then newlypicked out this variable in its optimal set. From this list of similar behaviors, we picked alternates based onour intuition of which behaviors would substitute the best, and which would be the easiest to explain in achallenge. Note that this procedure will perform poorly if baseline predictiveness and manipulation cost are highly
BEHAVIOR OBSERVED (outgoing) (incoming) (M-F, 9am-5pm)change in actions per ¢ of incentiveBEHAVIOR INCENTIVIZED -0.052 -0.836 -0.305 -0.022(0.0)*** (0.929) (0.337) (0.161) (0.953) -0.206 0.324 -2.022 -0.056 1.234 0.015 (0.481) (0.916) (0.113) (0.94) (0.0)*** Week and Individual Fixed Effects X X X X XN (person-weeks) 7976 7976 7976 7976 7976R2 0.705 0.637 0.552 0.604 0.491
Notes:
P-values in parentheses. Bold indicates diagonal: effect on behavior j when behavior j is incentivized.N represents person-weeks when no “incentive challenge” was assigned to the given participant. Individualand weekly fixed effects included, excluding the first week and first individual hash. Each column representsa separate regression, over the full set of covariates assigned; only the first five coefficients reported here. * p < < < Estimation
Finally, we use the data from all weeks of the experiment to jointly estimate types andmanipulation costs (using GMM with the moment conditions outlined in Section 3.1). Weallow manipulation cost to differ by behavior, by whether a person reports having high techskills, and by an unobserved random effect by person. Table 4 summarizes these estimatedcosts. With our sample size, we find that off diagonal elements are noisily estimated, so wepenalize them to zero ( λ costsoffdiagonal → ∞ ); this results in a diagonal cost matrix C .Several intuitive patterns can be discerned from the estimated manipulation costs in thetop panel of Table 4 (here we present only behaviors selected by models; see SupplementalAppendix for all estimated costs). Outgoing communications are less costly to manipulatethan incoming communications. Text messages, which are relatively cheap to send, are moremanipulated than calls, which are relatively expensive. We also find that complex behaviors(such as the standard deviation of talk time; estimated but not shown on this summarydiagram) are less manipulable than simpler behaviors (such as the average duration of talktime).Costs are also heterogeneous across people, as shown in the bottom panel of Table 4.On average it is 10%pt easier for individuals who report advanced or higher tech skills tomanipulate their mobile phone behaviors. Overall, including unobserved heterogeneity ingaming ability, the 90th percentile finds it 2.5 times easier to game than the 10th percentile. The final and most important stage of the experiment compares decisions made by standardmachine learning algorithms to the decisions made by our new strategy-robust estimator thataccounts for the cost of manipulating behavior. The robust decision rules can be directlyestimated with Equation 1, which relies on the estimates of x and C i that come from previousstages of the experiment.In this final stage, subjects receive complex challenges that reward them for their ultimate negatively correlated: in that case we may omit a behavior k which is less predictive at baseline but is morepredictive in the counterfactual because it is difficult to manipulate. We have allowed for a single dimension of observed heterogeneity in costs z ; with the rest absorbed intounobserved heterogeneity V . Thus Spence signaling will only be captured in that dimension z . With a largersample one could estimate a more nuanced functional form for the observable portion, which would bettercapture the correlations between gaming ability γ i and bliss behavior x i . Heterogeneity by Behavior ( C diagonal; subset of behaviors selected by models) lll llllll ll l l lllllllll lllllllllll text message you sendtext you receivetext in the evening (6pm−10pm)call you makesecond of your shortest weekend callperson you textcall you receivecall with someone not in your contactsmaximum daily texts receivedcall you make that's missedperson who texts yousecond of your average evening (6pm−10pm)second of your average call duration 0.03 0.04 0.06 0.59 0.64 1.01 1.33 2.24 2.30 4.64 6.01 19.01741.46 ( 0.0000)( 0.0000)( 0.0000)( 0.0001)( 0.0001)( 0.0002)( 0.0006)( 0.0007)( 0.0016)( 0.0428)( 0.0074)( 5.0605)(16612.8388) . .
001 0 .
01 0 . C: a jj (¢ action ) Heterogeneity by Person ( γ i ) γ i = e − ωz i + v i Low tech skills 1.00 den s i t y High tech skills 1.10
In top panel: Red: used in a LASSO model; blue: used in SR model. Line segment represents standarderror. Parameters estimated using GMM. In cost matrix, off diagonal elements α jk ; j (cid:54) = k regularized to zero( λ costsoffdiagonal → ∞ ), diagonal elements regularized with λ costsdiagonal = 1 .
0, set via cross validation. Standarderrors estimated from PD approximation of inverse Hessian. Shown here with v i winsorized at top andbottom of range; in implementation, only bottom is winsorized, to maintain assumption of non-negative γ i .Only behaviors selected by models shown in Panel I; for all behaviors see Supplemental Appendix. y .’ We consider a focalchallenge of the form, ‘Earn up to 1000 Ksh. if the Sensing app guesses you are a high-incomeearner.’ These challenges are designed to mimic real world applications of machine learning,where depending on how they are classified, users may receive a loan (digital credit), grant(targeted aid), or other benefits. Estimating Decision Rules
In order to keep decision rules simple and interpretable for our participants, we considerdecision rules of up to three features. We regularize na¨ıve decision rules to three features,selecting λ decision = max( λ cv , λ var ), where λ var is the smallest hyperparameter that resultsin a 3 variable LASSO model. We use the same hyperparameter to penalize our strategyrobust decision rule, and allow it to select only among three variable models. Treatments
Participants are randomly assigned into different targets (ˆ y ), decision rules (standard: β LASSO ,or robust β stable ), and whether the decision rule is kept opaque or revealed transparently tothe user. Under the opaque treatment, users are told only the outcome and the reward. Underthe transparent treatment, users see the coefficients of the decision rule, which reveals howmuch they are rewarded for changing which behaviors. We included an interactive interfacethat can be used to compute the payments that would result from different behavior (seeFigure 2c). Because the transparent treatment reveals information about potential decisionrules, after a person has seen a transparent challenge for ˆ y , we do not assign them to anopaque challenge for the same outcome.Table 5 summarizes the effect of decision rule incentives on behavior. High income peoplemake more outgoing calls, and send fewer texts but receive more. If we pay people to ‘act likea high-income earner,’ without revealing the decision rule, the response is noisy and oftenin the wrong direction (participants place fewer calls and send more texts). Participantswho are transparently presented with the decision rule change their behavior, closer to thedirection incentivized by the algorithm, though the response is still noisy. For a given λ decision that selects three variables in a LASSO model, the strategy robust model will tendto select more than three variables, because it induces some penalization on its own. Instead of restricting tothree variable models, one could alternately increase λ decision . Weekly Challenge:
Use your phone like a high-income earner!
Panel I: Incentives Generated by Algorithm ( ¢ /action) β LASSO
Panel II: x it Assigned to challenge, -6.5573 14.3701 12.0135 1.1672 -6.8104 algorithm opaque (9.949) (16.405) (20.583) (3.473) (7.002)
Assigned to challenge, algorithm transparent (9.083) (14.976) (18.79) (3.17) (6.392)
N (Person-weeks) 1664 1664 1664 1664 1664
Notes:
The first panel reports the decision rule associated with the challenge. The second reports the resultsof a regression of behavior on challenge assignment. Regressions estimated based on dummy indicatorsfor complex challenge assignment for participants assigned “income” challenge, over person-weeks whenthe income challenge was assigned or when no challenge was assigned (“control” weeks). Simple challengeassignment person-weeks, used in estimating costs, are not included. Standard errors in parentheses.
Performance of decision rules
We compare performance of na¨ıve vs. robust decision rules in Table 6. The first two columns(under ‘Income’) show results for the challenge that incentivized participants to use theirphones like a high-income earner; the last two columns show the performance averaged acrossseveral different challenges. The decision rules and associated manipulation costs are shownin the top panel (“Decision Rule”); the relative performance of the different estimators isshown below (under “Prediction Error”). We note several results.First, in the top panel, we observe important differences in the decision rules estimatedby β LASSO vs. β act . LASSO places weight on the behaviors that were most correlated atbaseline: outgoing calls, outgoing texts, and incoming texts. However, the estimated costsof manipulating some of these behavior – and in particular the costs of manipulating textmessaging behavior – are low, and therefore likely to be manipulated when incentivized.Thus, our strategy robust decision rule both selects less manipulable behaviors (evening textsrather than incoming texts), and shrinks manipulable behaviors (especially outgoing texts).We evaluate prediction error using root mean squared error (RMSE), in units of dollars,in the middle panel. The magnitude of error is similar to the average payout, around $ Income
Costs
All Outcomes (Pooled)
Income, Intelligence, Activity PCA
Decision Rule β LASSO β stable α jj ¢ /action ¢ /action2 Prediction Error
RMSE ( $ ) RMSE ( $ ) Baseline Data: Control 3.55 3.55 3.70 3.75Baseline Data: Predicted Transparent 4.66 3.83 4.34 3.85Implemented: Opaque 3.24 3.23 4.00 3.80Implemented: Transparent 3.87 3.66 4.93 4.31Predicted Cost of Transparency ≤ ≤ ≤ ≤ $ ) 3.30 3.24 3.23 2.98N (Control Person-Weeks) 3781 3781 3781 3781N (Treatment Person-Weeks, Opaque) 85 85 230 230N (Treatment Person-Weeks, Trans.) 91 74 252 216 Notes:
The first panel reports the decision rule associated with the challenge, and the costs associated withthese behaviors. The second reports the performance of the different models over the groups they wereassigned to; on the left, the naive LASSO regression, and on the right, this paper’s strategy-robust (SR)model. Performance figures estimated using a regression of model indicators on week-model RMSE, weightedby number of person-weeks. ‘Transparent Predicted’ RMSE denotes the RMSE that our theoretical modelexpected, given costs of manipulation and behavioral incentives. ‘Predicted Cost of Transparency’ denotesthe difference between predicted transparent RMSE under the SR model and baseline RMSE under the naiveLASSO. ‘Equilibrium Cost of Transparency’ denotes the difference between implemented transparent SRmodel RMSE and opaque naive model RMSE. Pooled performance is estimated using this same regressionapproach, after combining all model-weeks over the three outcomes investigated: a PCA of phone activity,intelligence, and monthly income. Full regression results and standard errors reported in appendix. $ $ $ $ $ $ $ $ Cost of transparency
Our framework provides a way to bound a key cost of imposing algorithmic transparency(Akyol et al., 2016). Many tech firms argue that imposing transparency would reduce thequality of machine decisions, because rules may perform better if they can rely on opacityto prevent manipulation. Our method allows us to bound this performance cost. We cancompare the performance arising from the optimal opaque rule (under the assumption thatopacity will prevent it from being manipulated) to the optimal equilibrium transparentrule (factoring in equilibrium manipulation). Because the opaque rule also faces the threatof manipulation, this difference is the upper bound of the performance cost of imposingtransparency, arising from increased manipulation.The most straightforward way to measure this cost of transparency would require disclosingthe decision rule to a subset of users, and assessing any drop in performance after a processof equilibration. But for the most consequential decisions, once the decision rule is revealedto some, it can leak out to the entire market. Such disclosure irreversibly tips the market totransparency, and thus is a nonstarter for policy discussions.Crucially, under the assumptions of our model, this quantity can be estimated withoutrevealing the decision rule: it only requires the estimation of types and costs (the first part32f our experiment). Our method makes it possible for regulators or firms to assess the costthat transparency would impose—prior to making their model transparent. Our model basedestimates suggest that transparency introduces a performance cost of ≤ $0 .
28 (8% of baselineerror) for our income targeting rule, or ≤ $0 .
15 (4%) for all outcomes pooled together. Thesenumbers are shown in the final rows of the middle panel of Table 6.When we actually implement transparency in our experiment, we find that the performancecost is similar to these model based estimates: ≤ $0 .
41 (13%) for income, or ≤ $0 .
31 (8%)for all outcomes pooled together. (To mitigate the problem of leakage, we only assess opaqueperformance prior to each individual observing a transparent challenge for that outcome.Because our decision rules were not going to be used later in production, we were unconcernedabout them leaking out after the experiment.)
Our method requires estimating C and γ i , which are new objects. The experimental approachwe use is likely not feasible in many settings. We offer suggestions on alternative approachesto measure these costs. Expert elicitations.
We evaluate how well experts can predict the costs of manipulatingdifferent behaviors, using a method similar to DellaVigna and Pope (2016). We sent a surveyto 177 experts with different backgrounds (PhDs from different fields, research assistants,Busara staff who had not worked on the experiment, and Mechanical Turk workers in theUS) to predict how Kenyans would manipulate different phone behaviors when incentivized.Results are shown in Figure 3. In panel A, we compare the predicted change in behaviorfrom a given incentive to the actual experimental estimate (∆ jj := x j ( β j ) − ¯ x j ). In Panel B,we compare the implied structural cost estimates (for predicted costs ˆ α jj = ¯ γ · β j ∆ jj ); althoughexperts predict that costs are too low, the correlation is 0.75. This suggests that it may bepossible to use expert elicitations to estimate manipulation costs. Partially estimated.
The costs of behavior k may be related to that of behavior k (cid:48) . Becauseof this, we may be able to predict unknown cost α kj based on correlations between types xand known costs, for some prediction function: ˆ α kj = f ( C, x). Our method of estimating costs does requires revealing the existence of features to users, but does notrequire specifying whether those features are included in the model, or with what weights (one could estimatecosts for a large set of features, hiding the features critical to the model). (a)
Reduced Form Shift in Behavior (b)
Structural Cost Estimates ll l llll l l llll lll
Experimental D jj E x pe r t P r ed i c t ed D jj l lll l l lll llll ll l l Experimental a jj ($ action ) E x pe r t I m p li ed a jj ( $a c t i on ) For structural costs we set ¯ γ = 1 and ˆ α jj = ¯ γ · β j max(0 . , ∆ jj ) . To sharpen intuition, this paper focuses on linear decision rules. While many modernmachine learned decision rules are nonlinear, agents’ beliefs about those rules may be wellapproximated by linear functions. In such a context, our derivations could be viewed aslinear approximations to both these beliefs, and the actual functions. Additionally, it may bethat some benefits of extreme nonlinearities that can surface in modern machine learningare lessened when manipulation is taken into account: contract theory suggests that lineardecision rules are more robust (Holmstrom and Milgrom, 1987; Carroll, 2015). Our approach could also be extended to work in nonlinear settings. In nonlinear environ-ments there may also be multiple equilibria. In such a setting, if iterative learning converges,it may converge to a local optimum, whereas an approach like ours could be used to select aglobal equilibrium. This paper considers the possibility that the implementation of machine decisions changesthe world they describe. We focus on the case where individuals manipulate their behavior inorder to game decision rules. Our chief contribution is to derive decision rules that anticipatethis manipulation, by embedding a behavioral model of how individuals will respond. This With the exception that linear models can be subject to the influence of outliers; one may thus want totamp down inputs as they approach the boundaries of the distribution of training data. Thanks to Glen Weyl for this point. any proposed decision rule of a givenform would be manipulated. This allows us to compute decision rules that are optimal inequilibrium.We demonstrate our method in a field experiment in Kenya, by deploying a tailor-madesmartphone app that mimics the ‘digital credit’ loan products that are now commonplacein sub-Saharan Africa. We find that even some of the world’s poorest users of technology– who are relatively recent adopters of smartphones and to whom whom the concept of an‘algorithm’ is quite foreign (Musya and Kamau, 2018) – are savvy enough to change theirbehavior to game machine decisions. In this setting, we show that our strategy robustestimator outperforms standard estimators on average by 13% when individuals are giveninformation about the scoring rule. This framework also allows us to quantify the “cost oftransparency”, i.e., the loss in predictive performance associated with moving from “securitythrough obscurity” (with a naive decision rule) to a regime of full algorithmic transparency(with our strategy-robust rule). We estimate this loss to be roughly 8% in equilibrium –substantially less than the 23% loss associated with making the naive rule transparent.Our discussion focuses on the simple case of linear models with a small number ofpredictor variables, where subjects have either no information or full transparency of thescoring rule. We envision useful extensions to more complex models and more nuancedbeliefs. More generally, our approach of embedding a model of behavior within a machinelearning estimator may be relevant to a wide range of contexts where machine learningsystems face a changing human environment. In this sense, it offers a machine learninginterpretation of Lucas (1976), where algorithmic decisions change the context of the systemsthey model. For example, financial forecasts may affect the underlying financial processesthey attempt to describe, personalized news recommendations may change the informationseeking behaviors of consumers, and predictions about the intensity of a disease may affectindividuals’ protective behaviors and thus its realized intensity.35 eferences
Akyol, Emrah, Cedric Langbort, and Tamer Basar , “Price of Transparency in Strate-gic Machine Learning,” arXiv:1610.08210 [cs] , October 2016. arXiv: 1610.08210.
Alatas, Vivi, Abhijit Banerjee, Rema Hanna, Benjamin A. Olken, Ririn Purna-masari, and Matthew Wai-Poi , “Self-Targeting: Evidence from a Field Experiment inIndonesia,”
Journal of Political Economy , March 2016, (2), 371–427.
Ball, Ian , “Scoring Strategic Agents,” arXiv:1909.01888 [econ] , November 2019. arXiv:1909.01888.
Banerjee, Abhijit, Rema Hanna, Benjamin A Olken, and Sudarno Sumarto , “The(lack of) Distortionary Effects of Proxy-Means Tests: Results from a Nationwide Experimentin Indonesia,” Working Paper 25362, National Bureau of Economic Research December2018.
Barocas, Solon, Moritz Hardt, and Arvind Narayanan , Fairness and Machine Learn-ing , fairmlbook.org, 2018.
Bj¨orkegren, Daniel , “’Big data’ for development,” 2010. and Darrell Grissen , “The Potential of Digital Credit to Bank the Poor,”
AmericanEconomic Association Papers and Proceedings , 2018. and , “Behavior Revealed in Mobile Phone Usage Predicts Credit Repayment,”
TheWorld Bank Economic Review , 2019.
Bloomberg , “Phone Stats Unlock a Million Loans a Month for Africa Lender,”
Bloomberg.com , September 2015.
Blumenstock, Joshua E. , “Estimating Economic Characteristics with Phone Data,”
AEAPapers and Proceedings , 2018, , 72–76.
Blumenstock, Joshua Evan, Dan Gillick, and Nathan Eagle , “Who’s Calling? De-mographics of Mobile Phone Use in Rwanda,” in “2010 AAAI Spring Symposium Series”March 2010. , Gabriel Cadamuro, and Robert On , “Predicting poverty and wealth from mobilephone metadata,”
Science , November 2015, (6264), 1073–1076.
Borrell Associates , “Trends in Digital Marketing Services,” 2016.
Bruckner, Michael and Tobias Scheffer , “Stackelberg Games for Adversarial PredictionProblems,” in “Proceedings of the 17th ACM SIGKDD International Conference onKnowledge Discovery and Data Mining” KDD ’11 ACM New York, NY, USA 2011,pp. 547–555.
Brynjolfsson, Erik and Tom Mitchell , “What can machine learning do? Workforceimplications,”
Science , 2017, (6370), 1530–1534.36 amacho, Adriana and Emily Conover , “Manipulation of Social Program Eligibility,”
American Economic Journal: Economic Policy , May 2011, (2), 41–65. Carroll, Gabriel , “Robustness and Linear Contracts,”
American Economic Review , February2015, (2), 536–563.
CGAP , “Kenya’s Digital Credit Revolution Five Years On,”
CGAP , March 2018.
Dee, Thomas S., Will Dobbie, Brian A. Jacob, and Jonah Rockoff , “The Causesand Consequences of Test Score Manipulation: Evidence from the New York RegentsExaminations,”
American Economic Journal: Applied Economics , July 2019, (3),382–423. DellaVigna, Stefano and Devin Pope , “Predicting Experimental Results: Who KnowsWhat?,” Working Paper 22566, National Bureau of Economic Research August 2016.
Dong, Jinshuo, Aaron Roth, Zachary Schutzman, Bo Waggoner, and Zhi-wei Steven Wu , “Strategic Classification from Revealed Preferences,” in “Proceedings ofthe 2018 ACM Conference on Economics and Computation” EC ’18 ACM New York, NY,USA 2018, pp. 55–70.
Dranove, David, Daniel Kessler, Mark McClellan, and Mark Satterthwaite , “IsMore Information Better? The Effects of “Report Cards” on Health Care Providers,”
Journal of Political Economy , June 2003, (3), 555–588.
Eliaz, Kfir and Ran Spiegler , “Incentive-Compatible Estimators,” 2018.
European Union , “EU General Data Protection Regulation (GDPR),” 2016.
Eykholt, Kevin, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati,Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song , “RobustPhysical-World Attacks on Deep Learning Models,” arXiv:1707.08945 [cs] , April 2018.arXiv: 1707.08945.
Francis, Eilin, Joshua Blumenstock, and Jonathan Robinson , “Digital Credit: ASnapshot of the Current Landscape and Open Research Questions,”
CEGA White Paper ,2017.
Frankel, Alex and Navin Kartik , “Muddled Information,”
Journal of Political Economy ,August 2019, (4), 1739–1776.
Frias-Martinez, Vanessa, Enrique Frias-Martinez, and Nuria Oliver , “A Gender-Centric Analysis of Calling Behavior in a Developing Economy Using Call Detail Records,”in “2010 AAAI Spring Symposium Series” March 2010.
FSD Kenya , “Tech-enabled lending in Africa,” 2018.
Gonzalez-Lira, Andres and Ahmed Mobarak , “Slippery Fish: Enforcing Regulationunder Subversive Adaptation,” IZA Discussion Paper 12179, Institute of Labor Economics(IZA) February 2019. 37 oodhart, Charles , Monetary Relationships: A View from Threadneedle Street , Universityof Warwick, 1975. Google-Books-ID: GKwJMwEACAAJ.
Goodman, Bryce and Seth Flaxman , “European Union regulations on algorithmicdecision-making and a ”right to explanation”,” arXiv:1606.08813 [cs, stat] , June 2016.arXiv: 1606.08813.
Greenstone, Michael, Guojun He, Ruixue Jia, and Tong Liu , “Can Technology Solvethe Principal-Agent Problem? Evidence from Pollution Monitoring in China,” 2019.
Hanna, Rema and Benjamin A. Olken , “Universal Basic Incomes versus Targeted Trans-fers: Anti-Poverty Programs in Developing Countries,”
Journal of Economic Perspectives ,November 2018, (4), 201–226. Hardt, Moritz, Nimrod Megiddo, Christos Papadimitriou, and Mary Wootters ,“Strategic Classification,” in “Proceedings of the 2016 ACM Conference on Innovations inTheoretical Computer Science” ITCS ’16 ACM New York, NY, USA 2016, pp. 111–122.
Holmstr¨om, Bengt , “Moral Hazard and Observability,”
The Bell Journal of Economics ,1979, (1), 74–91. Holmstrom, Bengt and Paul Milgrom , “Aggregation and Linearity in the Provision ofIntertemporal Incentives,”
Econometrica , 1987, (2), 303–328. Hu, Lily, Nicole Immorlica, and Jennifer Wortman Vaughan , “The Disparate Effectsof Strategic Manipulation,”
Proceedings of the Conference on Fairness, Accountability, andTransparency - FAT* ’19 , 2019, pp. 259–268. arXiv: 1808.08646.
Hussam, Reshmaan, Natalia Rigol, and Benjamin N. Roth , “Targeting High AbilityEntrepreneurs Using Community Information: Mechanism Design in the Field,” November2017.
Kleinberg, Jon and Manish Raghavan , “How Do Classifiers Induce Agents To InvestEffort Strategically?,” arXiv:1807.05307 [cs, stat] , July 2018. arXiv: 1807.05307. and , “How Do Classifiers Induce Agents to Invest Effort Strategically?,” in “Proceedingsof the 2019 ACM Conference on Economics and Computation” EC ’19 ACM New York,NY, USA 2019, pp. 825–844. event-place: Phoenix, AZ, USA. , Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, and Sendhil Mul-lainathan , “Human Decisions and Machine Predictions,”
The Quarterly Journal ofEconomics , February 2018, (1), 237–293.
Lucas, Robert E. , “Econometric policy evaluation: A critique,”
Carnegie-Rochester Con-ference Series on Public Policy , January 1976, (Supplement C), 19–46. McCaffrey, Mike, Olivia Obiero, and George Mugweru , “M-Shwari: Market Reac-tions and Potential Improvements,” Technical Report 139 2013.38 illi, Smitha, John Miller, Anca D. Dragan, and Moritz Hardt , “The Social Costof Strategic Classification,” in “Proceedings of the Conference on Fairness, Accountability,and Transparency” FAT* ’19 ACM New York, NY, USA 2019, pp. 230–239. event-place:Atlanta, GA, USA.
Mirrlees, J. A. , “An Exploration in the Theory of Optimum Income Taxation,”
The Reviewof Economic Studies , 1971, (2), 175–208. Musya, Mercy and Grace Kamau , “How do you say “algorithm” in Kiswahili?,” Decem-ber 2018. Library Catalog: medium.com.
National Institute of Standards and Technology , “Guide to General Server Security,”
NIST Special Publication , July 2008, (800-123).
Nichols, Albert L. and Richard J. Zeckhauser , “Targeting Transfers through Restric-tions on Recipients,”
The American Economic Review , 1982, (2), 372–377. Niehaus, Paul, Antonia Atanassova, Marianne Bertrand, and Sendhil Mul-lainathan , “Targeting with Agents,”
American Economic Journal: Economic Policy ,2013, (1), 206–238. Perlich, Claudia, Brian Dalessandro, Troy Raeder, Ori Stitelman, and FosterProvost , “Machine learning for targeted display advertising: Transfer learning in action,”
Machine learning , 2014, (1), 103–127. Ramsey, F. P. , “A Contribution to the Theory of Taxation,”
The Economic Journal , 1927, (145), 47–61. Sayed-Mouchaweh, Moamar and Edwin Lughofer , Learning in Non-Stationary En-vironments: Methods and Applications , Springer Science & Business Media, April 2012.Google-Books-ID: qFWM2nva7xQC.
Spence, Michael , “Job Market Signaling,”
The Quarterly Journal of Economics , 1973, (3), 355–374. Sundsøy, P˚al, Johannes Bjelland, Bjørn-Atle Reme, Eaman Jahani, Erik Wetter,and Linus Bengtsson , “Estimating individual employment status using mobile phonenetwork data,” arXiv:1612.03870 [cs] , December 2016. arXiv: 1612.03870.
Wei, Yanhao, Pinar Yildirim, Christophe Van den Bulte, and Chrysanthos Del-larocas , “Credit Scoring with Social Network Data,”
Marketing Science , October 2015, (2), 234–258. Publisher: INFORMS. 39 ppendicesA1 Experimental Design A1.1 Pre Analysis Plan
This study was pre-registered with the AEA RCT Registry (AEARCTR-0004649) prior tothe experiment (September 3, 2019). Our implementation deviated in several respects from the pre analysis plan: at the startof phase 2 the cloud server account ran out of storage space, and the Busara center was hitby a power outage due to construction on a nearby road. These two events disrupted serversfor several hours during the upload window, and caused some participants’ phones to becomeoverloaded with records. It took several weeks to recover the affected participants. Becauseof the disruption, we extended phase 2 and delayed the expert cost surveys.
A1.2 Attrition Management
Attrition in the context of this study had two dimensions: first, there were participants whodo not regularly upload data through their app, and second, there were participants who didnot participate in the assigned weekly challenges. (As some participants may have uploadeddata sparsely throughout the week, only those who uploaded within the 21-hour window atthe end of the challenge-week [between 1pm Tuesday and 10am Wednesday] were counted ashaving fully uploaded all of their weekly data.)In order to minimize both such types of attrition, participants were sent regular remindersvia text to encourage engagement. Every participant in the study was sent a text everyTuesday at 1pm to remind them to upload their data through the Smart Sensing app.Additionally, on Wednesday, Thursday and Friday, participants who still had not uploadeddata or activated their challenge respectively were contacted by phone and surveyed by theBusara team. Specifically, the protocol was as follows: Prior to the collection of the main outcomes in phase 2, we amended the registration, adding one sentencethat specifies that the focal performance measure will be mean squared error (which corresponds with theobjective minimized by the method; January 15, 2020). We later noticed that the registration still containedtext in another section that appeared to specify that the focal measures would be R or AUC; prior to thecompletion of phase 2 and prior to analysis of the main outcomes, we amended the registration to delete thatsentence (February 4, 2020). On Wednesday, participants who had not uploaded any data during the five day periodending on Wednesday at 12pm were contacted and surveyed, as were those who uploadedsome data in this period but not during the ‘end-of-week upload window’ (between6pm Tuesday and 10am Wednesday) • On Thursday, participants whose phones showed that they did not receive a challengeby Thursday 12pm were contacted and surveyed, as were participants whose phonesshow that they did receive a challenge but who had not opted in to accept the challenge. • On Friday, participants whose phones showed that they still had not received andopted-in to a challenge were contacted and surveyed, as were participants whose phonesshowed that they did receive a challenge but who had not opted in to accept thechallenge.For all of the above categories, any participant who did not answer a call on the first attemptwould be re-contacted once more by the surveyor after the rest of the calls were complete.Finally, to mitigate the effects of attrition during the analysis stage, any participant-weekswherein the participant did not opt in and/or did not upload during the end-of-week uploadwindow were dropped from the sample prior to all analysis. During baseline weeks, a singlepassive challenge was assigned to all participants, offering a flat bonus to upload data withinthe upload window; in this way, we ensured that our analysis control groups would also berestricted to those who opt in to this passive challenge, and were thus a valid comparisongroup to the restricted panel during the challenge weeks.
A1.3 Communicating Decision Rules
In focus groups we found that individuals had difficulty understanding decimals or complicatedmathematical operations (e.g., standard deviation). We stuck to simple behaviors andformatted decision rules as follows, to make it easier for participants to understand how theirmarginal behavior affects their payment: • Each coefficient is rounded to the nearest integer. If the nearest integer is zero, thedenominator was inflated by factors of 10 until it became nonzero. (If the unit wasseconds or minutes, the denominator was instead inflated by factors of 60.) • The order of indicators was randomized between three orderings (ABC, CAB, BCA forindicators A, B, and C). 41
The constant term was reported last, unless the first coefficient was negative, in whichcase the constant was reported first. 42
Figure A1: Comparative Statics
Note:
The first behavior is more predictive in the baseline behavior ( b > b ), but iseasily manipulable ( α (cid:28) α ). Below panels show weights on coefficients asmanipulation costs are scaled for: (a) x (b) Interaction x , x β stable ( α ) β stable ( α ) α = 0 α = 32 B2B1
Alpha22 C oe ff i c i en t B1B2
Alpha12 C oe ff i c i en t As x becomes cheaper to manipulate ( α decreases), β stable places less weight on it, andadjusts the weight placed on x . If manipulating one variable makes it easier tomanipulate the other ( α sufficiently negative), β stable reduces weight on both.x i iid ∼ N (0 , b = [1 . , C = γγ i (cid:34) α α α (cid:35) , γ i iid ∼ U nif orm [0 , (cid:15) i iid ∼ N (0 , . i g u r e A : A dd i t i o n a l C o m p a r a t i v e S t a t i c s C a nd γ i h e t e r og e n e o u s T h e fi r s t b e h a v i o r i s m o r e p r e d i c t i v e i n t h e b a s e li n e b e h a v i o r( b > b ) , bu t i s e a s il y m a n i pu l a b l e ( α (cid:28) α ) . C h o m og e n o u s : F e a t u r e s e q u a ll y c o s t l y t o m a n i pu l a t e γ i h o m og e n e o u s : S a m e ga m i n ga b ili t y β R i d g e ( λ ; γ fi x e d ) β s t a b l e ( λ | γ = ) ( d ) β s t a b l e ( / γ ) ( e ) β s t a b l e ( / γ ) B B . . . . Coefficient B B . . . . . Coefficient B B . . . . Coefficient B B . . . Coefficient R i dge La m bda Sq. Error R i dge La m bda Sq. Error Sq. Error Sq. Error M an i p C o s t S c a l e Loss incl Manip M an i p C o s t S c a l e Loss incl Manip M a n i pu l a t i o n C o s t = L i k e L A SS O , r i d g e p l a ce s m o r e w e i g h t o n x . O u r m e t h o d c a nb e c o m b i n e d w i t h o t h e r f o r m s o f p e n a li z a t i o n ( s u c h a s r i d g e s h o w nh e r e ) , t o m o r e fin e l y m a n ag e o u t o f s a m p l e fi t . W h e n f e a t u r e s a r ee q u a ll y c o s t l y t o m a n i pu l a t e , o u r m e t h o dp e n a li ze s i n a s i m il a r m a nn e r t o r i d g e . W h e n ga m i n ga b ili t y i s h o m og e n o u s , e v e r y o n e s h i f t s b e h a v i o r e q u a ll y . P r e d i c t i v e p e r f o r m a n ce r e m a i n s h i g h , bu t u t ili t y i s w a s t e d o n m a n i pu l a t i o n . x iii d ∼ N ( , ) , b = [ . , ], C h e t = γγ i (cid:20)
40 032 (cid:21) , C h o m = γγ i (cid:20)
80 08 (cid:21) , γ h e t i ii d ∼ U n i f o r m [ , ], γ h o m i = , e iii d ∼ N ( , . ) . S q u a r e d e rr o r m e a s u r e d o n a n o u t o f s a m p l e d r a w f r o m t h e s a m e p o pu l a t i o n ,i n ce n t i v i ze d t o t h a t d ec i s i o n r u l e . Feature variables F ea t u r e v a r i ab l e s Correlations between feature variables:Incentivized features and other LASSO-selected features
Each row and column represent a feature of behavior. Features are clustered into similar groups. The diagonalindicates that the correlation of a feature with itself is +1. Table A1: Manipulation Can Harm Prediction (Monte Carlo): “Industry Approach”
Decision Rule Performance (squared loss) β β β β No manip. Manipulation
Panel A:
Data generating process b DGP
Panel B:
Standard Approaches β OLS ‘Industry’ Approach (estimated with just data from that period) β OLS (1) after β OLS -0.798 0.061 2.090 -1.675 3.275 625.762 β OLS (2) after β OLS β OLS (3) after β OLS -0.755 0.120 2.077 -1.671 3.071 619.059... β OLS (1000) after β OLS -0.393 3.741 -1.341 1.566 1.375 11611.884 β OLS (1001) after β OLS -0.404 0.704 1.861 -1.526 1.674 565.383
Notes : Monte Carlo simulation results. Panel A shows the coefficients that relate the outcome ( y ) tobehaviors ( x ) under the data generating process (DGP). Panel B shows coefficients from OLS, underbehavior without manipulation: x i ( ), or with manipulation: x i ( β ). Parameters: C = . . . . . . . . . , x iid ∼ N , .
11 2 10 . , γ i = (cid:40) i ≤ .
210 x i > . e i iid ∼ N (0 , . Decision Rule Performance (squared loss) β β β No manipulation Manipulation
Panel A:
Data generating process b DGP
Panel B:
Standard Approach β OLS
Panel C:
Strategy Robust Method β stable Notes : Monte Carlo simulation results. Panel A shows the coefficients that relate theoutcome ( y ) to behaviors ( x ) under the data generating process (DGP). Panel B showsestimated coefficients from OLS; Panel C shows coefficients estimated with the strategyrobust method. Performance is assessed on the same sample of individuals, under behaviorwithout manipulation: x i ( ), or with: x i ( β ). Parameters: C = . . , x i iid ∼ N , . . , γ i = 0 . u i − (cid:15) i + Bu i ∼ N (0 , B set so min γ i = 0 . (cid:15) i iid ∼ N (0 , a b l e A : P e r f o r m a n ce o f D ec i s i o n R u l e s C o s t s A ll o u t c o m e s ( p oo l e d ) I n c o m e R a v e n s ( i n t e lli g e n ce ) A c t i v i t y P C A a b o v e m e d i a n α jj β L A SS O β s t a b l e β L A SS O β s t a b l e β L A SS O β s t a b l e ¢ /a c t i o n ¢ /a c t i o n ¢ /a c t i o n ¢ /a c t i o n D e c i s i o n R u l e c a ll c o un t o u t . -- . . . . t e x t c o un t i n c o m i n g0 . -- . . . . . t e x t c o un t o u t . --- . - . t e x t c o un t e v e n i n g0 . --- . . . c a ll s n o n c o n t a c t s . --- . - . c a ll c o un t o u t go i n g m i ss e d . --- . m a x d a il y t e x t s i n c o m i n g2 . -- . P r e d i c t i o n E rr o r B a s e li n e D a t a : R M S E ( $ ) R M S E ( $ ) R M S E ( $ ) R M S E ( $ ) C o n t r o l . . . . . . . . ( . )( . )( . )( . )( . )( . )( . )( . ) P r e d i c t e d T r a n s p a r e n t . . . . . . . . ( . )( . )( . )( . )( . )( . )( . )( . ) I m p l e m e n t e d : O p a q u e . . . . . . . . ( . )( . )( . )( . )( . )( . )( . )( . ) T r a n s p a r e n t . . . . . . . . ( . )( . )( . )( . )( . )( . )( . )( . ) C o s t o f T r a n s p a r e n c y . ( . ) . ( . ) . ( . ) . ( . ) . ( . ) - . ( . ) . ( . ) . ( . ) E q m .: P r e d i c t e d . ( . ) . ( . ) - . ( . ) . ( . ) E q m .: I m p l e m e n t e d . ( . ) . ( . ) - . ( . ) . ( . ) A v e r ag e P a y o u t( $ ) . . . . . . . . N N p e r s o n - w ee k s N o t e s : T h e fi r s t p a n e l r e p o r t s t h e d ec i s i o n r u l e a ss o c i a t e d w i t h t h ec h a ll e n g e , a nd t h ec o s t s a ss o c i a t e d w i t h t h e s e b e h a v i o r s . T h e b e l o w p a n e l s r e p o r tt h e p e r f o r m a n ce o f n a i v e L A SS O a nd o u r s t r a t e g y - r o bu s t m o d e l, b y o u t c o m e a ndp oo l e d a c r o ss o u t c o m e s , r e s p ec t i v e l y . P e r f o r m a n ce m e t r i c s e s t i m a t e du s i n ga r e g r e ss i o n o f m o d e li nd i c a t o r s o n w ee k - m o d e l R M S E , w e i g h t e db y nu m b e r o f p e r s o n - w ee k s . O p a q u e ( T r a i n i n g W ee k s ) r e p r e s e n t s t h e a v e r ag e p e r f o r m a n ce o f m o d e l s i n c o n t r o l p e r s o n - w ee k s , w h e nn o b e h a v i o r w a s i n ce n t i v i ze d . T r a n s p a r e n t( M o d e l ) r e p r e s e n t s t h e a v e r ag ee x p ec t e dp e r f o r m a n ce o f m o d e l s g i v e n t h e t h e o r e t i c a l m o d e l, b e h a v i o r i n ce n t i v e s a nd e s t i m a t e d c o s t s . I m p l e m e n t e d O p a q u e r e p r e s e n t s t h e a v e r ag e p e r f o r m a n ce o f m o d e l s w h e n a ss i g n e d w i t h o u tt r a n s p a r e n c y h i n t s . I m p l e m e n t e d T r a n s p a r e n t r e p r e s e n t s t h e a v e r ag e p e r f o r m a n ce o f m o d e l s w h e n a ss i g n e d w i t h t r a n s p a r e n c y h i n t s . C o s t o f t r a n s p a r e n c y r e p r e s e n t s t h e d i ff e r e n ce b e t w ee n t r a n s p a r e n t a nd o p a q u e R M S E f o r n a i v e L A SS O a nd s t r a t e g y - r o bu s t( S R ) m o d e l s , r e s p ec t i v e l y .’ E q m .: P r e d i c t e d ’ d e n o t e s t h e d i ff e r e n ce b e t w ee np r e d i c t e d t r a n s p a r e n t R M S E und e r t h e S R m o d e l a ndb a s e li n e R M S E und e r t h e n a i v e L A SS O .’ E q u ili b r i u m C o s t o f T r a n s p a r e n c y ’ d e n o t e s t h e d i ff e r e n ce b e t w ee n i m p l e m e n t e d t r a n s p a r e n t S R m o d e l R M S E a nd o p a q u e n a i v e m o d e l R M S E . A v e r ag e p a y o u t s r e p r e s e n t s t h e a v e r ag e p a y o u t f r o m t h e a ss i g n e d c h a ll e n g e s a ss o c i a t e d w i t h t h e m o d e l. N r e p r e s e n t s t h e nu m b e r o f m o d e l - w ee k s t h a tt h e r e g r e ss i o n i s e s t i m a t e d o v e r , N p e r s o n - w ee k s r e p r e s e n t s t h e nu m b e r o f p e r s o n - w ee k s t h a tt h e s e m o d e l - w ee k s i n c l ud e , a s w e ll a s t h e s u m o f t h e w e i g h t s u s e d i n r e g r e ss i o n . C o s t s r e p r e s e n t s t r u c t u r a l c o s t s e s t i m a t e du s i n ga b o v e p r o ce du r e . S t a nd a r d e rr o r s i np a r e n t h e s e s , c l u s t e r e d a t w ee k - o u t c o m e l e v e l.l.