FFriend-Based Ranking in Practice
Francis Bloch
Paris School of Economics
Matthew Olckers
Monash UniversityJanuary 11, 2021
Abstract
A planner aims to target individuals who exceed a threshold in acharacteristic, such as wealth or ability. The individuals can rank theirfriends according to the characteristic. We study a strategy-proof mech-anism for the planner to use the rankings for targeting. We discuss howthe mechanism works in practice, when the rankings may contain er-rors. a r X i v : . [ ec on . GN ] J a n BLOCH & OLCKERS
1. Introduction
Through social interactions we learn a great deal about our friends, includ-ing characteristics such as wealth and ability. Although we may not be ableto put a precise number to these characteristics, we can rank our friends withease—from poorest to richest, from least to most business savvy, or from shyto outgoing.These rankings can be extremely valuable for planners, such as govern-ment agencies and banks. For example, a bank may want to know whichentrepreneur is most profitable, or a government agency may want to knowwhich household is in the most need.In Bloch and Olckers (forthcoming) we study how a planner can designa mechanism to extract a ranking of a unidimensional characteristic froma network of individuals. We assume that each individual only observes anordinal ranking of the characteristics of their friends—their neighbors in thenetwork. We show that a planner can design a mechanism to extract all avail-able information if and only if every pair of friends has a friend in common.In many social networks this property is satisfied—most friends have manyfriends in common.In Bloch and Olckers (forthcoming) we made the assumption that indi-viduals have perfect ordinal information about their friends. Individuals donot mistakenly rank friends in the incorrect order. This assumption is un-likely to be satisfied in practice. The observed rankings may be perturbed,especially when friends are very similar. The mechanism we derived reliedon the assumption of perfect information. If the planner observed conflict-ing reports, she could conclude that at least one of the individuals was lying.In this article, we extend the model of Bloch and Olckers (forthcoming)by allowing individuals to make errors when observing the ranking of theirfriends. We introduce a mechanism to target a group of agents rather than
RIEND-BASED RANKING IN PRACTICE github.com/matthewolckers/fbr-in-practice ) so that practitioners canapply our methods to their ranking data.
2. Model
We consider a community N of individuals i = 1 , , ..n with characteristic θ i ∈ R . The individuals belong to a social network g . The planner designsa mechanism to select a targeted group of individuals, denoted by the set T ⊂ N according to their characteristic θ i .The characteristics θ i are not directly observed by the planner who relieson the reports r i sent by individuals. Individuals have local, ordinal informa-tion: their types are ranks of their friends. As opposed to Bloch and Olckers(forthcoming), we do not assume that the rankings are perfect. Instead, indi-viduals make mistakes and may observe an incorrect ranking of their friends.We do not make any specific assumption on the information structure gen-erating an individual’s ranking as a function of the true characteristics of hisfriends.The objective of the planner is to maximize the number of individuals in T for which θ i > θ and minimize the number of agents in T for which θ i < θ ,where θ is a fixed threshold. Each individual obtains a payoff of if he is BLOCH & OLCKERS chosen in the targeted set T and otherwise. Building on Mattei et al. (2020), we construct the following three-step mech-anism T assigning a subset T of N to every vector of individual reports r =( r , ..., r n ) . Step 1 - Collect rankings
Each individual reports a ranking of his friends in the social network.
Step 2 - Estimate scores
The planner uses an algorithm to map the reports r to a score s i ∈ R foreach individual i . To prevent manipulation, i ’s report is excluded from thecalculation of his score, i.e. s i ( r ) = s i ( r (cid:48) ) for any r , r (cid:48) such that r − i = r (cid:48)− i . Step 3 - Choose the targeted group T If s i > s , a predetermined cutoff, individual i is included in T . If s i ≤ s , i is excluded from T .
3. Properties of the Mechanism
We first recall that a mechanism is strategy-proof if no individual has an in-centive to misreport and group-strategy-proof if no subset of individuals hasan incentive to misreport.
Definition 1.
A mechanism is strategy-proof if for every i , every type r i of i ,every report r − i of the other individuals, Pr( i ∈ T ( r i , r − i ) ≥ Pr( i ∈ T ( r (cid:48) i , r − i ) for any other report r (cid:48) i of individual i . Proposition 1.
The three-step mechanism T is strategy-proof. RIEND-BASED RANKING IN PRACTICE i , s i , is independent of i ’s report, and individual i is only included in the tar-get set T if and only if s i > s , i cannot change her probability of inclusion bychanging her report. Hence, the mechanism is strategy-proof.We say that the scoring function s ( r ) is monotonic if s i ( r ) > s i ( r (cid:48) ) when-ever r ranks player i higher than r (cid:48) . When the scoring rule is monotonic, thethree-step mechanism reacts to changes in the reports of the individuals. Weshow however that it is not immune to deviations by groups of players.
Definition 2.
A mechanism is manipulable by a coalition S at the report r ifthere exists a vector of reports ( r (cid:48) j ) j ∈ S such that, for every individual i ∈ S , P r ( i ∈ T (( r (cid:48) j ) j ∈ S , ( r k ) k ∈ S ) ≥ P r ( i ∈ T ( r )) , with strict inequality for some individual. A mechanism T is group-strategyproof if it is not group manipulable by a coalition S at any type profile r . Proposition 2.
Suppose that n ≥ . If the scoring rule is monotonic, thereexists a threshold level s such that the three-step mechanism is not group-strategy-proof. The proof of Proposition 2 only requires a counter-example. Considertwo connected individuals i and j . Let k be an individual connected to j .Suppose that j observes θ k > θ i , and consider a profile r such that s i ( r ) >s j ( r ) . Because the scoring rule is monotonic, if j reports θ i > θ k , the scoreof individual i is higher under the new vector of reports r (cid:48) , s i ( r (cid:48) ) > s i ( r ) . Nowchoose s ∈ ( s i ( r ) , s i ( r (cid:48) )) . The three-step mechanism is group manipulableby the coalition ( { i, j } ) at the profile r . We say that r ranks player i higher than r (cid:48) if, whenever i beats j under r (cid:48) , i beats j under r and there is an instance where i beats j under r but not under r (cid:48) . If no such profile exists, we revert the roles of i and j in the example. BLOCH & OLCKERS
Finally, we note that the threshold for inclusion s must be an absolute,rather than a relative value. Consider a variation of the mechanism, wherethe size of the targeted group is fixed, and agents are included according totheir relative score. Formally, let the fraction of targeted individuals be equalto α = | T | n and let individuals be assigned to T if and only if |{ j, s j > s i }| ≥ n (1 − α ) . Consider a community of three individuals, i, j, k connected in a triangle,and let α = . Consider a scoring rule which is monotonic and anonymous.A scoring rule s is anonymous if, for any permutation of the individuals π ,for any player i , s π ( i ) ( π ( r )) = s i ( r ) .Let the report r be such that individual i ranks j above k , individual j ranks i above k and individual k ranks j above i . By anonymity, and becauseall vertices are similar in a triangle, the scoring rule only depends on thenumber of times that an individual beats another individual. Hence individ-ual j is chosen with probability after the report r . If however i changes herereport and ranks k above j , all the individuals have an equal chance to beincluded in the target set.
4. Aggregating Reports using HodgeRank
The planner must choose an algorithm to map the reported rankings intothe scores, s i . We propose using HodgeRank, an approach that can handlecycles and incomplete ranking data (Jiang et al., 2011). Also, the HodgeRankmethod does not rely on any assumption on the distribution of errors, andhence can be applied when the planner does not have any information onthe community. In this section, we explain the HodgeRank algorithm.We start with reported ranks of the form r kij ∈ {− , } where r kij = 1 if k RIEND-BASED RANKING IN PRACTICE i above j and r kij = − if k ranks i below j . We collect the reportedranks into a weighted and directed ranking graph Y with edge set E . Eachnode corresponds to an individual. The weighted edges Y ij aggregate thereported ranks about each pair of individuals. If there are multiple reportedranks on a specific pair, we take the mean of the reported ranks.HodgeRank chooses a score s i for each individual i to minimize the squareddifference between the scores and the aggregated rankings. HodgeRank solvesthe problem: min (cid:88) { i,j }∈ E (( s i − s j ) − Y ij ) Jiang et al. (2011) showed that the residual of this least squares problem cor-responds to the cycles in the ranking graph. The cycle ratio provides a mea-sure of goodness-of-fit for the estimated scores.Cycle ratio = (cid:80) { i,j }∈ E ((ˆ s i − ˆ s j ) − Y ij ) (cid:80) { i,j }∈ E ( Y ij ) We now provide several examples to illustrate how HodgeRank estimatesscores. Let us start with a simple case with four individuals. i j k l
From the graph, it is clear the rank has no cycles and the scores s i = − . , s j = − . , s k = 0 . , s l = 1 . solve the least squares problem. The cycle ratio iszero as the scores explain all of the variation in the ranking graph.Let’s add a cycle of length three to the graph. The scores are unique up to an additive constant. We set the scores to sum to zero.
BLOCH & OLCKERS i j kl
The scores become s i = − . , s j = 0 . , s k = 0 . , s l = 0 . . The cycle ratiois 0.75. For each pair on the cycle, the estimated scores predict no differenceon the ranking graph, but these pairs have a difference of 1 unit.Now let’s add a cycle of length four to the first graph. ij kl The scores become s i = 0 , s j = 0 , s k = 0 , s l = 0 . The cycle ratio is equal toone as the cycle explains all of the variation in the ranks.The cutoff score s requires a minimal level of consensus in the rankinggraph for individuals to be included in the targeted set T . If s = 1 , individual l would be targeted in the first example whereas the targeted set would beempty for the examples with cycles.Finally, we consider an example where the cycle ratio may be positive dueto the magnitude of the edge weights of the ranking graph. In the examplebelow, the scores are s i = − , s j = 0 , s k = and the cycle ratio is . i jk Note that the HodgeRank algorithm assumes the pairwise comparisons arecardinal. The path ( i, j, k ) , composed of ( i, j ) = 1 and ( j, k ) = 1 , implies that RIEND-BASED RANKING IN PRACTICE ( i, k ) = 2 . However, in this example ( i, k ) = 1 . Wedo not find the same magnitude by taking two different paths between i and k . Thus, the discrepancy is reflected in the scores and the cycle ratio.Recall that the edges Y ij are formed by taking an average over all individ-uals who rank the pair ( i, j ) . Therefore, Y ij can range from − to . Given thisrestriction on Y ij , we are likely to find discrepancies by following differentpaths along the ranking graph. Other methods of aggregating the reportedranks r kij into the weighted edges Y ij may lead to more accurate scores. Weleave this problem to future research.
5. Comparison to Traditional Community-Based Targeting
One application of friend-based ranking is poverty targeting. In a tradi-tional community-based targeting approach, a community meets togetherand agrees on a ranking of households from poorest to richest. Friend-basedranking does not require a centralized meeting. Rankings can be collectedfrom each individual separately.Using data from 423 Indonesian hamlets collected by Alatas et al. (2012,2016), we can compare the community-based targeting and friend-basedranking approaches. The data contains four essential ingredients: ground-truth data on wealth as measured by a detailed consumption survey, socialnetwork data, the outcome of a community-based targeting exercise, andindividual rankings of wealth reported by nine households.Although this data contains all the ingredients required to compare friend-based ranking and community-based targeting, the setting is not ideal sincethere are only nine households who reported rankings. We anticipate thatfriend-based ranking will be of greater benefit in much larger networks.The individual rankings required each of the nine households to rank theother eight from poorest to richest. The respondent was allowed to indi-0
BLOCH & OLCKERS
Figure 1: Comparing Targeting Methods
Friend−based rankingCommunity−based targeting T a r ge t ed Friend−based rankingCommunity−based targeting0 500 1000 1500 2000
Consumption N o t t a r ge t ed Note: Sample size of 3 522 individuals. The targeted set contains 1 084individuals for both methods. For community-based targeting, we usethe targeting quota specified for each hamlet. For friend-based ranking,we set the cutoff score s to include a similar number of individuals in thetargeted group. This cutoff is fixed and does not vary between hamlets. cate that he or she did not know the ranking of some individuals. In a briefempirical exercise, we compare the distribution of consumption for the tar-geted and excluded individuals for each of the two methods: friend-basedranking and community-based targeting.Figure 1 shows the distribution of consumption for households targetedand not targeted by each of the two methods. The distributions are similar—friend-based ranking achieves similar targeting performance to community-based targeting. RIEND-BASED RANKING IN PRACTICE
Cycle Ratio
Note: Sample size of 423 hamlets. The cycle ratio measures the propor-tion of variance in the reported ranks that can be attributed to cycles inthe ranking graph.
For the friend-based ranking method, we can also report the cycle ratiofor each of the 423 hamlets. Figure 2 shows the distribution of the cycle ratio.Most hamlets have a cycle ratio of around 0.15. Some of these cycles mayderive from the weights on the edges of the ranking graph.Although most hamlets have low cycle ratios, 10 outliers have cycle ratiosof greater than 0.3. Interestingly, these 10 outliers also have low social net-work density—an average density of 0.25 in the outliers in comparisons to0.7 in the remaining networks.
References
Alatas, Vivi, Abhijit Banerjee, Arun G Chandrasekhar, Rema Hanna, and Ben-jamin A Olken, “Network Structure and the Aggregation of Information:Theory and Evidence from Indonesia,”
American Economic Review , 2016, (7), 1663–1704., , Rema Hanna, Benjamin A Olken, and Julia Tobias, “Targeting the Poor:2
BLOCH & OLCKERS
Evidence from a Field Experiment in Indonesia,”
American Economic Re-view , 2012, (4), 1206–40.Bloch, Francis and Matthew Olckers, “Friend-Based Ranking,”
AmericanEconomic Journal: Microeconomics , forthcoming.Jiang, Xiaoye, Lek-Heng Lim, Yuan Yao, and Yinyu Ye, “Statistical Rank-ing and Combinatorial Hodge Theory,”
Mathematical Programming , 2011, (1), 203–244.Mattei, Nicholas, Paolo Turrini, and Stanislav Zhydkov, “PeerNomina-tion: Relaxing Exactness for Increased Accuracy in Peer Selection,” arXivpreprint arXiv:2004.14939arXivpreprint arXiv:2004.14939