Random Walks with Erasure: Diversifying Personalized Recommendations on Social and Information Networks
RRandom Walks with Erasure: Diversifying PersonalizedRecommendations on Social and Information Networks
Bibek Paudel
Stanford UniversityStanford, [email protected]
Abraham Bernstein
University of ZürichZürich, [email protected]
ABSTRACT
Most existing personalization systems promote items that matcha user’s previous choices or those that are popular among similarusers. This results in recommendations that are highly similar to theones users are already exposed to, resulting in their isolation insidefamiliar but insulated information silos. In this context, we developa novel recommendation framework with a goal of improving in-formation diversity using a modified random walk exploration ofthe user-item graph. We focus on the problem of political contentrecommendation, while addressing a general problem applicable topersonalization tasks in other social and information networks.For recommending political content on social networks, we firstpropose a new model to estimate the ideological positions for bothusers and the content they share, which is able to recover ide-ological positions with high accuracy. Based on these estimatedpositions, we generate diversified personalized recommendationsusing our new random-walk based recommendation algorithm.With experimental evaluations on large datasets of Twitter dis-cussions, we show that our method based on random walks witherasure is able to generate more ideologically diverse recommenda-tions. Our approach does not depend on the availability of labelsregarding the bias of users or content producers. With experimentson open benchmark datasets from other social and informationnetworks, we also demonstrate the effectiveness of our method inrecommending diverse long-tail items.
CCS CONCEPTS • Computing methodologies → Machine learning ; •
Informa-tion systems → Recommender systems ; Social networks . KEYWORDS diverse recommendations, social networks, random walks.
ACM Reference Format:
Bibek Paudel and Abraham Bernstein. 2021. Random Walks with Erasure:Diversifying Personalized Recommendations on Social and InformationNetworks. In
Proceedings of the Web Conference 2021 (WWW ’21), April19–23, 2021, Ljubljana, Slovenia.
ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3442381.3449970
This paper is published under the Creative Commons Attribution 4.0 International(CC-BY 4.0) license. Authors reserve their rights to disseminate the work on theirpersonal and corporate Web sites with the appropriate attribution.
WWW ’21, April 19–23, 2021, Ljubljana, Slovenia © 2021 IW3C2 (International World Wide Web Conference Committee), publishedunder Creative Commons CC-BY 4.0 License.ACM ISBN 978-1-4503-8312-7/21/04.https://doi.org/10.1145/3442381.3449970
Users often rely on recommender systems and Online Social Net-works (OSNs) to select news and other information they consume [2,33]. However, these algorithmic systems have also been criticizedfor increasing polarization and influencing political processes andevents around the world. This makes it imperative to investigatebetter strategies for recommending personalized content to users.High-quality as well as balanced news consumption is vital for afunctioning democracy [15, 31, 34]. To address these needs, thispaper introduces recommender algorithms that are designed with thegoal of increasing the reader’s exposure to diverse information. To promote the diversity of views , we propose a random-walkbased algorithm that can generate diverse as well as accurate rec-ommendations. We introduce a modified random-walk explorationof the user-item feedback graph in which random-walk traversalsto certain nodes are systematically erased in order to lower theirimportance with respect to the starting node of the random walk.We call this new approach random walk with erasure . Our approachbased on this modified random walk exploration provides a generalmathematical and algorithmic framework for diversifying recom-mendations that can be used in various domains. Our evaluation,however, mainly focuses on the problem of political content recom-mendations on social media platforms (e.g., Twitter).Towards this goal, we also propose a method that can identifythe political leanings of both users and the news items they shareon OSNs. We collected datasets of tweets on political discussionsfrom various countries, related to events such as general electionsor referendums. We exploit the sharing behaviour of users on socialmedia related to particular political events in order to estimatetheir ideological positions on a one-dimensional scale. Based onsuch information, our recommendation approach can suggest newsitems to users that purposefully exposes them to different viewpointsand increases the diversity of their information “diet.” A common way to diversify content is by including viewpointsfrom different outlets, assuming that ideological positions of politi-cal elites and news outlets are fixed over long durations. In highlycontested political events, however, this approach is likely to sufferfrom a major problem: a set of viewpoints from politicians or newssources belonging to different ideologies can still be homogeneous.The left-right classification of users or politicians based on theirlong-term behavior like speeches, voting habits or social mediafollower patterns is also likely to fall short.Table 1 shows two examples of how content from the same outletwere shared by groups of people with opposing political viewpointsabout the 2016 Brexit referendum in the UK. Although The Daily Note that users can still choose which stories they want to read. As such, we donot impose a news “diet” on them but provide a balanced suggestions. a r X i v : . [ c s . S I] F e b WW ’21, April 19–23, 2021, Ljubljana, Slovenia B. Paudel and A. Bernstein
BBC Brexit: Gibraltar in talks with Scotland to stay in EU (link)Second EU referendum petition investigated for fraud (link)The Nigel Farage: £350 million pledge to fund the NHS was ’a mistake’ (link)Telegraph Britain remains a great country with a great future (link)
Table 1:
News stories from the same outlet (BBC in the top row and TheDaily Telegraph in the bottom row) that were shared by people with opposingviewpoints about the 2016 EU referendum (Brexit) held in the UK.
Telegraph is known as a conservative British newspaper that sup-ported the Leave campaign, its report about the backtracking of acampaign promise by Leave campaigners (first example in the bot-tom row, Table 1) was popular among the supporters of the Remaincampaign, while other pieces were popular among the supportersof the Leave campaign (second example in the bottom row, Table 1).The two opposing groups also shared articles from the BBC (toprow in Table 1) differently: the first article was shared more bythe Remain supporters and the second article was shared more byLeave supporters.As a first step towards tackling this problem, we propose a novelapproach to recommendation diversification that incorporates ideo-logical positions about particular political events learned from socialmedia signals . To the best of our knowledge this is the first workto deal with this problem. Our proposed solution has two compo-nents: (i) learning ideological positions of users, political elites, andweb content as well as (ii) using the ideological positions to diver-sify recommendation based on random walks with erasure and adiversification strategy that exploits weak ties in social networks.In this work, we use the one-dimensional ideological positions(left-right) for users and political content. The key difference fromother approaches is that we identify such positions for politicalelites, users, and individual content (rather than content outlets) de-pending on the sharing patterns on social networks during specificpolitical events. Additionally, we also propose a novel and effectiverecommendation strategy based on ideological positions.With experimental evaluations on social network datasets, weshow that our method is able to generate more politically diversifiedrecommendations to the users without overly sacrificing accuracy.To show that random walk with erasure (RWE) is a general methodthat is also effective in other social and information networks, wealso evaluate it on benchmark datasets from other domains likemovie and restaurant recommendations. We find that RWE canrecommend both highly accurate and diverse items to the users. In summary, our contributions in this paper are the fol-lowing: (i) we describe a new method to estimate ideological po-sitions of not only users and elites, but also web content shared onsocial networks such as Twitter, (ii) we introduce random walk witherasure (RWE), a novel modified random walk based explorationof bi-partite feedback graphs that is useful for diversifying recom-mendations, (iii) on datasets of social media discussions we showexperimentally that our recommendation method based on RWE isable to diversify political content recommendations —in many caseswithout a loss of accuracy, while state-of-the-art recommender sys-tems generate recommendations that are less ideologically diverse, https://yougov.co.uk/news/2017/03/07/how-left-or-right-wing-are-uks-newspapers/ and (iv) on open benchmark datasets from other domains, we showexperimentally that our algorithm can provide a general frameworkfor diversified recommendations .The rest of the paper is structured as follows. We review relatedwork and describe the challenges of political content recommenda-tion in Section 2. We introduce notations and background conceptsin Section 3. Then we introduce our new method random walkswith erasure in Section 4, and two diversification strategies inSection 5. In Section 6, we describe our methods for the identifica-tion of political leanings jointly using elite- and content-sharingsignals. Finally, we describe our experimental setup and discuss theresults of our evaluation in Section 7, and conclude in Section 8. In this section, we review relevant literature from various fields.
Recommendation Diversity.
Several previous works [1, 3, 10,19, 30, 35, 38, 40–42, 50] have investigated the issue of recommen-dation diversity. They diversify recommendations by exploitingtopics and tags, post-processing of recommendations, promotinglong-tail items, and so on. As a result, each of the proposed ap-proaches provides a different kind of diversity to the users. Sincethe question of what constitutes a diverse recommendation doesnot have a clear answer, we think it is desirable to have multipleapproaches and strategies for recommendation diversification.
Ideology Detection from Social Media.
Methods that use so-cial media behavior to estimate political leanings of users [9, 12, 13]can be compared to the multidimensional scaling method famouslyknown as DW-NOMINATE [44], that measures ideology of parlia-mentarians by analyzing legislative voting behavior. Some recentworks approach the problem of recommendation diversity usingideological positions [5, 18, 29, 32, 36], but they either rely on outletspecific positions, or do not provide a complete recommendationframework. We not only address the problem with outlet-specificpositions, but also provide an end-to-end recommendation frame-work, with extensive evaluations with state-of-the-art methods.
Political Content Diversity.
In context of political content,there are additional challenges regarding the question of recommen-dation diversity. Exposure to diverse viewpoints, and cross-cuttingdiscussions between users of different viewpoints may help widentheir perspective and can be desirable for a healthy democracy.However, it is not enough to just diversify information without re-gard for several factors that influence opinion formation. Researchhas shown that exposure to diverse political viewpoints can alsolead to further polarization [7], especially in case of individualswho hold a strong viewpoint on a particular side of the debate [49].These behaviors have been explained by selective exposure theoryand confirmation bias in social sciences. Other works have foundthat reinforcement of strong beliefs is weaker in individuals withmoderate viewpoints, or in case of two-sided neutral debates andcross-cutting exposure [21, 27, 43], as explained by social scienceconcepts like moderation theory and cross-cutting discussions.
Weak Ties.
There is also evidence from social network theorythat weak ties are important for exposure to diverse information [20].A recent study also shows that weak-tie discussion frequency ispositively correlated to online political participation [48]. andom Walks with Erasure WWW ’21, April 19–23, 2021, Ljubljana, Slovenia
Political Polarization on Social Networks.
In the context ofpolitical news, the role of OSNs and recommender systems in po-litical polarization has recently become subjects of public concern.This has led to an increased attention on studying the systemicbias of filtering algorithms and developing ways to correct them.A study by Facebook[8] demonstrates how algorithmic filteringaffects users’ exposure to news in OSNs. To get around the nega-tive effects of information filtering, some news organizations havestarted to offer curated lists containing diverse viewpoints in orderto balance their readers’ exposure. Browser extensions have alsobeen developed to inject news links to “burst the bubble.”The challenge of diversifying recommendations can be seen aspart of the research on AI and machine-learning biases [4, 22,39, 52]. Variations of random-walks on networks have been usedbefore to diversify rankings or improving predictions [6, 40, 51],but a general recommendation framework has been lacking.
Scholars have argued that exposure to diverse viewpoints helps broaden their acceptance, if not agreement, with such view-points. They are deemed essential for promoting political toler-ance and deliberative democracy. Insulated discussions among like-minded participants, in contrast, can breed excessive confidence,extremism, contempt for others, and also violence [47]. The ex-change of ideas among people with politically dissimilar groupsis known as cross-cutting discussions . Research in this field hasshown that awareness of rationales for oppositional viewpointsincreases with the exposure to disagreement, and that affectivepolarization is negatively related to involvement in cross-cuttingdiscussions [37, 43]. This implies that greater network diversityreduces polarization by facilitating cross-cutting discussions.
We begin by introducing the concepts and notations relevant forlater sections of the paper.
The user-item feedback dataset is a 𝑚 × 𝑛 matrix A . The entries of thefeedback matrix contain the feedback from user 𝑢 𝑖 ( 𝑖 ∈ . . . 𝑚 ) onitem 𝑖 𝑗 ( 𝑗 ∈ . . . 𝑛 ) . In our case, we use implicit feedback, meaningthat these entries are either 1 for present or 0 for missing feedback.In this work, we also use the bi-partite graph representation G of the user-item feedback dataset. We model G as unweighted andundirected graph (all edges have the same weight), but we couldalso generalize the definitions to a weighted version. The adjacencymatrix of the bi-partite user-item graph has the dimension ( 𝑚 + 𝑛 ) × ( 𝑚 + 𝑛 ) and is constructed as shown in (1): A G = (cid:2) AA 𝑇 (cid:3) (1)The transition-probability matrix P for A G is obtained by row-normalizing its entries: P = D − A G , where D 𝑖𝑖 = ∑︁ A G 𝑖 . (2) D is the degree matrix which has the degree of the nodes of thegraph in its diagonal elements. The transition-probability matrix P has some interesting properties. Its entries 𝑃 𝑖 𝑗 encode the prob-ability of a random-walk starting at node 𝑖 arriving at node 𝑗 inone step. Every odd-power of P (e.g., P ) represents the transitionprobabilities for random walks starting at one of the user verticesand arriving at one of the item vertices. Consider a row-vector 𝑣 𝑠 ∈ { , } ( 𝑚 + 𝑛 ) all of whose entries are zero, except at index 𝑠 .Then, the vector-matrix multiplication 𝑣 𝑠 P gives the transitionprobabilities for three-step random walks starting at node 𝑠 .Instead of starting with an initial state probability of 1, if therandom walkers start with a mass of 𝑒 𝑠 , the index 𝑠 of 𝑣 𝑠 contains 𝑒 𝑠 instead of 1, and the transition probabilities are obtained by vector-matrix multiplication as before: 𝑣 𝑠 P k , where 𝑘 is the number ofsteps in the random walk. A seminal work about the estimation of ideological positions isby Poole and Rosenthal [44], who used roll-call data from the UnitedStates Congress to recover the political positions of its members,called their ideal points. These approaches place the politicians ona latent dimension, which is usually a point in the one-dimensionalleft-right scale. Using this ideological dimension, a politician can besaid to be left- or right-wing depending on whether her estimatedposition is towards the left or right of the center.In this work, we estimate ideological positions for not only polit-ical elites, but also common users and the political content (URLs)they share on social media. We denote the ideal points of user 𝑢 ,elite 𝑒 , and content 𝑖 by 𝜃 𝑢 , 𝜙 𝑒 , and 𝜓 𝑖 . By learning these positionsjointly, we place them on a common ideological scale where com-parisons can be made about their relative positions. For example,user 𝑢 𝑝 and URL 𝑖 𝑞 can be said to share similar political stance iftheir ideological positions are nearby, i.e., | 𝜃 𝑝 − 𝜓 𝑞 | is small. In this section we define our new random-walk method, calledrandom walk with erasure (RWE). In RWE, we introduce variationon the usual random walk in the forms of erasures.At certain steps in the random walk, erasures cause a fractionof the mass reaching the destination vertices to be erased and sentback to the origin vertex. In other words, for a vertex that receivesa mass of 𝑝 from a random walk, a portion 𝑝 × 𝑞 is erased, where0 ≤ 𝑞 < erasure probability 𝑞 defines the probability with which the walk reaching a destinationvertex is erased and sent back to the origin vertex. At the nextiteration, instead of the usual mass of 1, the walker starting at theorigin vertex starts her walk with the mass accumulated from theerasures in the previous iteration. It is important to note that ateach iteration the initial mass in the starting vertex gets smaller andis always less than 1. In this way, RWE induces different randomwalk transition probabilities than the usual random walk.The intuition behind RWE is to allow different probability dis-tributions than those induced by the degree distribution of the WW ’21, April 19–23, 2021, Ljubljana, Slovenia B. Paudel and A. Bernstein graph. This provides the flexibility to favor certain nodes duringthe random-walk exploration, based on their attributes, or simi-larity with the origin vertex. We exploit this property of RWE todiversify recommendations by proposing two different strategiesdescribed below. First we start with a formal definition of RWE.
RWE proceeds like a regular random walk except for two importantdifferences. The first involves a erasure-matrix Q which encodesthe node-specific erasure probabilities. The second is the erasureprocess itself. We now describe them in detail.The amount by which the walks arriving a vertex are erased isnot the same for all vertices—they differ for each pair of randomwalk origin and current vertex. These quantities are encoded in Q ∈[ , ) ( 𝑚 + 𝑛 )×( 𝑚 + 𝑛 ) . The entries Q 𝑖 𝑗 indicate the erasure probabilitiesfrom destination vertex 𝑗 to the origin vertex 𝑖 .Note the distinction between PageRank and RWE : while therestart probability and the erasure probability in the two methodseem similar, the crucial difference is that the erasure probability isdifferent for each pair of vertices and the number of walks that geterased varies in each iteration, until convergence.At each iteration of the random-walk, RWE proceeds as follows:(a) start regular random walks of odd number of steps 𝑘 from originvertex 𝑠 , and as specified by the transition probabilities P in (2), (b)at the destination vertex 𝑗 , with probability Q 𝑖 𝑗 , erase the walk;with probability 1 − Q 𝑖 𝑗 , do not erase the walk, and (c) at the seconditeration, start new random walks from the origin vertex with thefollowing probability ( P 𝑘 ◦ Q ) ◦ I .,𝑠 (3)Here is a 𝑚 + 𝑛 dimensional vector with all ones, ◦ is theHadamard product, and P 𝑘 ◦ Q encodes erasures from the previousiteration. Multiplication with sums up all the erasures arriving ateach origin-vertex. Considering 𝑠 as the origin user-vertex, I .,𝑠 isthe 𝑠 𝑡ℎ column of ( 𝑚 + 𝑛 ) × ( 𝑚 + 𝑛 ) dimensional identity matrixand the final Hadamard product gives the initial state probabilitymodified due to erasures.This process is continued for sufficient number of iterations andfinally the number of walks at the destination vertices that were noterased is used to estimate the probability of a 𝑘 -step RWE startingat 𝑖 and reaching 𝑗 without being erased. We use this probability toscore item-nodes for recommendation tasks. The Erasure matrix Q can be defined by the service providers ac-cording to their strategy for diversifying recommendations. In otherwords, the strategy determined by Q can be defined to favor diverseitems that would be less traversed by regular 𝑘 -hop random walks.Note that the items in the final recommendation list are those inthe local neighborhood (when 𝑘 is not large) of the user vertices;only the probability of traversal to those vertices are changed dueto Q . RWE diversifies the recommendations by promotingdiverse items connected by weak links , and is less likely torecommend items that are too dissimilar or unfamiliar to the users.In this section, we present two examples of diversification strate-gies: one for long-tail and one for political content diversity. − − 𝑢 𝑢 𝑢 𝑢 Figure 1:
An example showing four users 𝑢 . . .𝑢 with their ideologicalpositions. 𝐷 ) To use RWE for promoting long-tail diversity, as in 𝑅𝑃 𝛽 [40], onecan can define erasure matrix Q 𝐷 as given in (4), where D definedin (2) is the diagonal matrix containing vertex degrees, and 𝛽 is aparameter that can be used to tune the erasure probabilities. Thisstrategy depends only on the degree of item vertices and has theeffect of preferring low-degree (long-tail) items. Q 𝐷𝑖,𝑗 = − D 𝛽𝑗,𝑗 (4) 𝐵 ) Before describing our strategies for diversifying political contentrecommendation, we illustrate example ideological positions of fourusers in Figure 1. Those towards the left in the ideological scale(e.g., 𝑢 ) are called left-leaning and those towards the right (e.g., 𝑢 , 𝑢 , 𝑢 ) are called right-leaning. These are relative comparisonswith respect to other users, and we can also compare the distancesand similarities between users based on their ideological positions: 𝑢 is more right leaning than 𝑢 but more left leaning than 𝑢 , thedistance between 𝑢 and 𝑢 is higher than between 𝑢 and 𝑢 , and 𝑢 is more similar to both 𝑢 and 𝑢 than 𝑢 is.In this work, we also identify ideological positions for politi-cal content (e.g., videos, news, social media posts, etc.) and elites.The ideological positions for user 𝑢 , elite 𝑒 , and content 𝑖 are 𝜃 𝑢 , 𝜙 𝑒 , 𝑎𝑛𝑑 𝜓 𝑖 , respectively. Based on these positions, we define similarity between a content-user or a elite-user pair as: oneminus the normalized absolute difference in their ideologiacal posi-tions: 𝑠𝑖𝑚 ( 𝑖, 𝑢 ) = − (| 𝜓 𝑖 − 𝜃 𝑢 |)/( 𝑚𝑎𝑥 𝑝 − 𝑚𝑖𝑛 𝑝 ) for content 𝑖 anduser 𝑢 , and 𝑠𝑖𝑚 ( 𝑒, 𝑢 ) = − (| 𝜙 𝑒 − 𝜃 𝑢 |)/( 𝑚𝑎𝑥 𝑝 − 𝑚𝑖𝑛 𝑝 ) for elite 𝑒 and user 𝑢 , where [ 𝑚𝑎𝑥 𝑝 , 𝑚𝑖𝑛 𝑝 ] is the range of (all) ideologicalpositions. It is symmetric and bounded between [ , ] It is possible to transform the ideological positions (e.g., into 𝑧 -scores), or to define similarity as a non-linear transformation ofdistance (e.g., using logarithmic transforms). We leave the study onthe effect of such transformations for future work.As discussed in Section 2, the definition of diversity in the contextof political content is far from clear. A diversification strategy thatoffers viewpoints radically different to a user’s position is not likelyto be appreciated by that user. Viewpoints that are different but nottoo far from the user’s own ideological position—i.e., connected viaweak links—can be expected to appeal more to the user than thosethat are at a greater distance. Such different viewpoints are likelyto be reachable through others who are close to the user but in theopposite side of the political spectrum. We call them bridge users ,and in Figure 1, 𝑢 could be a bridge user between those on herright ( 𝑢 𝑎𝑛𝑑 𝑢 ) and those on her left ( 𝑢 ). Similar notions applyin case of elites and content. Bridges are weak ties whose ideologicalpositions are on the opposite side of the user’s own position . andom Walks with Erasure WWW ’21, April 19–23, 2021, Ljubljana, Slovenia Based on these motivations, we present the RWE strategy forbridging diverse political viewpoints and define the correspondingerasure matrix Q 𝐵 for user-content pairs as: Q 𝐵𝑢,𝑖 = (cid:40) 𝑠𝑖𝑚 ( 𝑢, 𝑖 ) , if i is a bridge with respect to 𝑢𝜖, otherwise , (5)where the values in Q are less than 1, and the parameter 𝜖 < 𝜖 causes random walksreaching non-bridge elites or content to be erased at a higher rate.The erasure matrix Q 𝐵𝑢,𝑒 for user-elite pairs is defined similarly.
In the previous sections, we discussed how we can use the ideolog-ical positions to find candidates for diversifying recommendations.Now we provide the details about our method to identify ideologicalpositions for users, elites, and content. For this purpose, we considertwo user-item feedback graphs: the elite-endorsement graph andthe content-share graph. We construct these datasets from socialmedia discussions around particular political events like elections,protests, or referendums. We use datasets of Twitter discussions,but the approach can be applied to similar datasets from any othersocial network. In the elite-endorsement graph, there are 𝑚 users U and 𝑛 𝑒 elites E . Elites are those individuals who are endorsedoften, so it is possible for the same real-life person to be presentboth as a user and an elite in our dataset. The same users U and 𝑛 𝑖 content-identifiers I constitute the content-share graph.We treat retweets and content-sharing as acts of endorsementsof elites and content by users with similar ideological positions. Weconsider any URL present in the tweets as a web-content and theseURLs could refer to news, videos, pictures, or other social mediaposts. Similarly, we consider a retweet as an elite-endorsement.From these feedback graphs, we can construct two matrices similarto the feedback matrices defined in Section 3.1: R of dimension 𝑚 × 𝑛 𝑒 for the elite-endorsement graph and S of dimension 𝑚 × 𝑛 𝑖 for the content-share graph. The entries R 𝑢,𝑒 are 1 if user 𝑢 hasretweeted the elite 𝑒 and likewise entries in S 𝑢,𝑖 are 1 if user 𝑢 hasshared the content 𝑖 . The remaining entries are zero. We assume a one-dimensional ideological space and want to recoverthe ideal points 𝜃 ∈ R for users and 𝜙 ∈ R for elites in this space. If auser and an elite share similar ideological positions, we assume thedistance between them in this space to be low, and model these asquadratic utility functions similar to [9, 11]. With this assumption, R 𝑢,𝑒 = 𝑢 and 𝑒 is small in thisspace. We model this in probabilistic terms, and state the probabilityof the user endorsing an elite using the logistic function: 𝑝 ( R 𝑢,𝑒 = | 𝜃 𝑢 , 𝜙 𝑒 , 𝛼 𝑢 , 𝛽 𝑒 ) = 𝑒𝑥𝑝 (−∥ 𝜃 𝑢 − 𝜙 𝑒 ∥ + 𝛼 𝑢 + 𝛽 𝑒 ) (6)where the terms 𝛼 𝑢 and 𝛽 𝑒 are bias terms associated with 𝑢 and 𝑒 ,and account for the differences among users and elites respectively.Using Bayesian inference, Bernoulli probability mass function,and under the assumption that all observed endorsements R 𝑢,𝑒 areindependent, we get the following: 𝑝 ( 𝜃, 𝜙, 𝛼, 𝛽 | R ) ∝ (cid:214) ( 𝑢,𝑒 ) ∈U×E 𝑝 ( R 𝑢,𝑒 = | 𝜃 𝑢 , 𝜙 𝑒 , 𝛼 𝑢 , 𝛽 𝑒 ) 𝑎 𝑢,𝑒 (7) ( − 𝑝 ( R 𝑢,𝑒 = | 𝜃 𝑢 , 𝜙 𝑒 , 𝛼 𝑢 , 𝛽 𝑒 )) − 𝑎 𝑢,𝑒 The parameter 𝑎 𝑢,𝑒 is used to assign confidence to the observedendorsement of 𝑒 by 𝑢 , and it could be a function of the number oftimes 𝑢 has endorsed 𝑒 .To simplify the notation, we write Π 𝑢,𝑒 = −∥ 𝜃 𝑢 − 𝜙 𝑒 ∥ + 𝛼 𝑢 + 𝛽 𝑒 .After placing standard normal priors on 𝜃 and 𝜙 , and taking the logof posterior, we get the following log-likelihood function with L-2regularization terms:argmin 𝜃,𝜙,𝛼,𝛽 𝑝 ( 𝜃, 𝜙, 𝛼, 𝛽 | R ) ∝ ∑︁ ( 𝑢,𝑒 ) ∈U×E 𝑎 𝑢,𝑒 ( Π 𝑢,𝑒 ) − 𝑙𝑜𝑔 ( + 𝑒𝑥𝑝 ( Π 𝑢,𝑒 )) (8) − 𝜆 ∥ 𝜃 ∥ − 𝜆 ∥ 𝜙 ∥ To also identify the ideological positions of the URLs shared byusers, we use the content-share graph and elite-endorsement graphtogether in a joint probabilistic framework. We assume that web-content shared by users have ideological positions 𝜓 ∈ R in thesame shared latent space described in Section 6.1. In case of content, S 𝑖,𝑘 = 𝑢 𝑖 and 𝑖 𝑘 is small in this space. We also model the probability ofa user sharing a web-content using logistic function: 𝑝 ( S 𝑢,𝑖 = | 𝜃 𝑢 ,𝜓 𝑖 , 𝛼 𝑢 , 𝛾 𝑖 ) = 𝑒𝑥𝑝 (−∥ 𝜃 𝑢 − 𝜓 𝑖 ∥ + 𝛼 𝑢 + 𝛾 𝑖 ) (9)where 𝜆 𝑖 is the bias term associated with 𝑖 .As before, using Bayesian inference and the assumption that allobserved S 𝑢,𝑖 ’s are independent, we arrive the following expression: 𝑝 ( 𝜃,𝜓, 𝛼, 𝛾 | S ) ∝ (cid:214) ( 𝑢,𝑖 ) ∈U×I 𝑝 ( S 𝑢,𝑖 = | 𝜃 𝑢 ,𝜓 𝑖 , 𝛼 𝑢 , 𝛾 𝑖 ) 𝑏 𝑢,𝑖 (10) ( − 𝑝 ( S 𝑢,𝑖 = | 𝜃 𝑢 ,𝜓 𝑖 , 𝛼 𝑢 , 𝛾 𝑖 )) − 𝑏 𝑢,𝑖 The parameter 𝑏 𝑢,𝑖 is used to assign confidence to the observedendorsement of 𝑖 by 𝑢 , for example the number of endorsements. Optimization.
Instead of learning the ideologies in (6) and (9)separately, we formulate a joint optimization to learn all theideological positions together. The reason for doing so is to alignthe positions learned by (6) and (9). When there is not enoughobserved data in R or S , one model is also expected to compensatefor the lack of data in the other when learning 𝜃 ’s, 𝜙 ’s and 𝜓 ’sjointly. In other words, we use these two models to regularize eachother such that they share the same latent dimension and learnrelative distances between them in that shared space. Graphicalmodels for both approaches are shown in Figure 2. More details, including source code are available online https://github.com/bibekp/random_walks_erasure
WW ’21, April 19–23, 2021, Ljubljana, Slovenia B. Paudel and A. Bernstein
After placing standard normal priors on 𝜓 as before, and addingcontribution from (10) as an additional regularizer on (8), we getthe following log-likelihood function for joint optimization, where 𝜇 trades-off the contribution from elite-endorsement graph:argmin 𝜃,𝜙,𝜓,𝛼,𝛽,𝛾 𝑝 ( 𝜃, 𝜙, 𝛼, 𝛽 | R , S ) ∝ 𝜇 ∑︁ ( 𝑢,𝑒 ) ∈U×E 𝑎 𝑢,𝑒 ( Π 𝑢,𝑒 ) − 𝑙𝑜𝑔 ( + 𝑒𝑥𝑝 ( Π 𝑢,𝑒 ))− ∑︁ ( 𝑢,𝑖 ) ∈U×I 𝑏 𝑢,𝑒 ( Π 𝑢,𝑖 ) − 𝑙𝑜𝑔 ( + 𝑒𝑥𝑝 ( Π 𝑢,𝑖 ))− 𝜆 ∥ 𝜃 ∥ − 𝜆 ∥ 𝜙 ∥ − 𝜆 ∥ 𝜓 ∥ (11)The local maxima of (8) and (11) can be found via a gradient-based optimization in which all but one parameter are fixed at eachstep and they are updated alternatively. Figure 2:
Graphical models for the ideology detection in social networksusing endorsement of elites 𝐸 𝑗 by users 𝑈 𝑖 (left), and endorsements of elitesand content 𝐼 𝑘 by users (right), where the 𝜎 𝑥 are the priors for the elites, users,and items. Now we describe the setup and the findings of our experiments.
To evaluate the performance of our method and baselines on politi-cal content diversification, we crawled tweets during three politicalevents and created these datasets: (a)
UK2016 from the 2016 EUreferendum in the UK, (b)
US2016 from the 2016 US presidentialelections, and (c)
DE2017 from the 2017 German federal elections.We used Twitter’s search API to crawl the tweets containing anyone of the common terms related to the online discussion about theevents. We included the terms from [24, 25, 28] and those appearingin Twitter’s trending topic lists. These search terms are shown inTables 2, 3, and 4.For UK2016 and US2016 datasets, we gathered roughly equalnumber of tweets for each major campaign position (Remain andLeave) or presidential candidate (Donald Trump and Hillary Clin-ton). For DE2017 dataset, we included search terms representingeach major political party and some general terms related to theelection. For UK2016, we collected tweets from the day of the ref-erendum to until about ten days later. For US2016, we collectedtweets from a week before till the day of the election. For DE2017,
Search TermsPro-Remain
Table 2:
Search terms used to create the UK2016 dataset.
Search TermsPro-Trump trump, donald trump, @realDonaldTrumpPro-Clinton clinton, hillary clinton, @HillaryClinton
Table 3:
Search terms used to create the US2016 dataset.
Search Terms
Table 4:
Search terms used to create the DE2017 dataset.Name
US-RT
DE-RT
Content-endorsementUK-URL
US-URL
DE-URL
Recommender System BenchmarkML-1M
Yelp
Table 5:
Statistics of elite-endorsement and content-endorsement datasetscreated from the three tweet-datasets, along with the benchmark recom-mender datasets. we collected tweets from a day before the election to about ten dayslater. To filter suspicious users and content, we removed tweetsthat were not retweeted more than 50 times, and also the tweets byusers who had few followers or who did not tweet often. Note thatthis step may not filter out bots and automated accounts.We created two user-item feedback graphs for each dataset: (a)elite-endorsement, and (b) content-endorsement. We treat eachTwitter user who is retweeted more than five times as an eliteand each URL that is included in more than five tweets as a web-content. For URLs, we apply some preprocessing steps like un-shortening and resolving re-directions. The rows of both matricesdenote users and columns denote elites (in R 𝑢,𝑒 ) and web-content(in S 𝑢,𝑖 ), respectively. Additionally, we evaluate RWE- 𝐷 on twobenchmark datasets from recommender systems: Movielens-1Mand Yelp-Restaurants. Table 5 shows the statistics of these datasets. We evaluate our models introduced in Section 6 in identifyingpolitical positions of political elites in the three datasets describedin Section 7.1. The ideological positions detected by the joint model andom Walks with Erasure WWW ’21, April 19–23, 2021, Ljubljana, Slovenia in Section 6.2 are given in Figure 3, 4, and 5 for US2016, UK2016,and DE2017 respectively.
Figure 3:
Estimated ideological positions of political elites in 2016 US Pres-idential elections (US2016 dataset).
Figure 4:
Estimated ideological positions of political elites in 2016 EU Ref-erendum in the UK (UK2016 dataset).
Figure 5:
Estimated ideological positions of political elites in 2017 GermanFederal elections (DE2017 dataset).
Dataset Elite endorsement Joint learning from elite-only and content-endorsement
US2016
Pearson correlation coefficients for estimated ideological positionscompared with baselines results [9]. Better result in each row is boldfaced.UK2016 shows a weak correlation; traditional left-right positions in the UKpolitics were not strongly reflected in the Brexit campaign.
The average and standard deviations from three separate runsare plotted as ideological positions on the x-axis. For Figure 3 and 4,the elites on the y-axis are the Twitter users that are common toboth our dataset and [9]. For Figure 5, the common Twitter users(with [9]) mostly included media personalities, so we manuallychose the users close to each major party with the highest numberof Twitter followers.
Result I: Our method accurately identifies ideological posi-tions from social network signals.
We compare the performanceof two algorithms in Section 6: positions learned from elite en-dorsement only and positions learned jointly by using by elite-endorsement and content-endorsement. The comparisons with [9]are given in Table 6. We see that for all datasets, positions learnedjointly by using both social network signals perform better thanthose learned by using the endorsement signal only. The correla-tion is especially strong for US2016 dataset but is weak for UK2016
WW ’21, April 19–23, 2021, Ljubljana, Slovenia B. Paudel and A. Bernstein dataset. The reason is that during the EU referendum of 2016, tra-ditional left-right distances in the UK politics were not stronglyreflected in the campaign endorsements by the political elites.In all datasets we see a general separation of political elites intotheir left and right positions. For example, Democratic primary can-didates Hillary Clinton and Bernie Sanders are located on the left,and Republican primary candidates Ted Cruz and Donald Trumpare located on the right in Figure 3. An interesting observation canbe made in Figure 4 where most Labour party leaders are located inthe left and most Conservative party leaders are located in the right.However Conservative party leaders Nick Hurd and Ed Vaizy cam-paigned for the Remain campaign and our method has located themon the left. The Scottish National Party (SNP) also advocated forthe Remain position. Labour MP Gisela Stuart campaigned for theLeave campaign, and has been located rightmost among all Labourpoliticians by out method. A list of MPs and their endorsementsfor the Leave and Remain campaigns can be found online. In caseof Figure 5, we can see that the traditional left-right alignment ofGerman parties like Green, Linke, ALDE, CDU, FDP, and ALD isreflected on the detected ideological positions.
To choose baselines for comparing RWE, we referred to a recentwork that analyzed several popular recommender algorithms [16,17]. Since RWE is based on graph exploration, we searched for base-lines representing graph-based algorithms, as well as other methodsthat have shown to produce highly accurate and diverse recommen-dations. For political content diversification, we include the state-of-the art graph based method RP 𝛽 [40], which also deals with long-tailitem diversity. Additionally, we include P [14], which achieves highaccuracy but does not deal with diversity. Note that RP 𝛽 outper-forms the popular matrix factorization baseline BPRMF [45]. Forthis reason, we include [26], which is a strong matrix-factorizationbaseline (MF). Finally, we also compare with item-based collabora-tive filtering (CF) [46]. For long-tail diversity, we compare RWE- 𝐷 against CF, P and RP 𝛽 since previous work has shown that theyall outperform latent-factor models [40].Among the several measures for evaluating the accuracy of arecommender system, we use the common ones: AUC, Mean Rank(MR), Hit-rate (HR), and Precision(P) at top-10. Higher values ofthese measures indicate better accuracy.For measuring long-tail diversity, we borrow the measures Gini-Diversity (GiniD@20), Personalization (Pers@20), Surprisal (Surp@20),and Average Item Degree (AvgDeg@20) from the literature [1, 40].Higher values of GiniD, Pers and Surp, as well as lower value ofAvgDeg indicate better diversity.For ideological diversity, we measure the average range of ideo-logical positions in the top-k recommendations as 𝑅𝑒𝑐𝑅𝑎𝑛𝑔𝑒 @ 𝑘 = | 𝑈 | (cid:205) 𝑢 ∈ 𝑈 𝑚𝑎𝑥 ( 𝑝𝑜𝑠 ( 𝑅 𝑢 [ 𝑘 ])) − 𝑚𝑖𝑛 ( 𝑝𝑜𝑠 ( 𝑅 𝑢 [ 𝑘 ])) , where 𝑅 𝑢 is the ranked list of recommendations for user 𝑢 and 𝑝𝑜𝑠 ( 𝑥 ) givesthe ideological position of 𝑥 .To evaluate the recommender algorithms, we divide the datasetsinto train-test splits in the following way: for each user with morethan three interactions, we randomly select 30% of the items into test-set. The remaining interactions are selected into train-set. Werepeat this process three times, creating three independent test-trainsplits. We run each algorithm three times (once on each test-trainsplit) and report the average results.For RWE-based algorithms, we use a parameter 𝜈 to change thevalues of erasure matrix as Q ◦ 𝜈 (this is an element-wise operatorand preserves the properties of the erasure matrix), and use 𝜖 = . 𝜈 , 𝛽 ,and number of nearest neighbors 𝑘 in case of RWE , RP 𝛽 , and CFrespectively. In case of MF, we do a grid search on the number ofcomponents 𝑘 and regularization constant. We choose the parame-ters that result in the best AUC and report the corresponding resultsfor all eight datasets in Table 8, where measures related to accuracyand long-tail diversity are in columns 2-5 and 6-9 respectively. Result II: RWE generates accurate and diverse long-tail rec-ommendations.
We can see from Table 8 that on all datasets RWEis able to achieve best accuracy result (measured by AUC, Hit Rate,Precision, and Mean Rank). On ML-1M and Yelp datasets, whereRWE- 𝐷 ’s erasure matrix is designed to promote long-tail diversity,it achieves similar or second-best results to RP 𝛽 (measured by Gini,Average Degree, Personalization, and Surprisal). It shows that inaddition to political content, RWE is also suitable for general recom-mendation tasks (e.g., movies, or restaurants). In terms of long-taildiversity, on the Twitter-based datasets RWE does not perform sowell because its erasure matrices are designed to promote diversityof political positions. While CF generates better long-tail diversity,it does so at a high cost to accuracy. Result III: RWE generates ideologically diverse recommen-dations.
In case of Twitter-based datasets for political content, welack measures that comprehensively capture the recommendationdiversity. In this section, we use four methods: (i) average range ofideological positions in the top-10 recommendations
𝑅𝑒𝑐𝑅𝑎𝑛𝑔𝑒 @10(ii) visual comparison of ideological distribution of items in top-krecommendations, (iii) Kolmogorov-Smirnoff statistic to quantifythe difference in distributions of political ideology in top-k recom-mendations, and (iv) new measures to numerically and visuallyinspect the ideological diversity for users across the spectrum. Thefirst three results are presented below and the fourth can be foundin Appendix A.1. In all cases we compare RWE- 𝐵 against the algo-rithms which are most competitive in terms of accuracy (P , andRP 𝛽 ). Unless otherwise specified, the parameters corresponding tothe best accuracy measure in Table 8 are used.In Table 7, we list the 𝑅𝑒𝑐𝑅𝑎𝑛𝑔𝑒 @10 values for the algorithmswhich are most competitive in terms of accuracy. We can observethat RWE- 𝐵 outperforms other methods in all but one dataset.To illustrate the diversification of recommendations by RWE- 𝐵 ,we compare the recommendations of the most accurate baselineswith our approach on the US-RT dataset (results for other datasetsare similar and omitted for space reasons). In this example, the taskis to recommend elites to users (elites are items in the traditionalrecommender system terminology). First, we classify users withideological positions less than − . Left-leaning , with positions andom Walks with Erasure WWW ’21, April 19–23, 2021, Ljubljana, Slovenia
Dataset RWE- 𝐵 RP 𝛽 P3US-RT 1.71*
US-URL 2.15***
UK-RT 2.83***
UK-URL 2.02***
DE-RT 3.14***
DE-URL
Table 7:
Average spread of ideological positions in the top-10 recommenda-tions (
𝑅𝑒𝑐𝑅𝑎𝑛𝑔𝑒 @10 ) by different algorithms in three independent runs oneach dataset. Best results are boldfaced and statistically significant differencewith the second best results are indicated with asterisks (one tailed Welch’st-test, * p-value < 0.05, *** p-value < 0.001). RWE produces more iedologicallydiverse recommendations on all but the smallest dataset. greater than 0 . Right-leaning , and the rest as
Center-leaning .Then, we compare the distribution of ideological positions of itemsin the top-10 recommendations for all three types of users. Theresults are shown in Figure 6.All the baselines shown in the figure achieve good accuracy, butthey do so by recommending different kinds of political content.On the x-axis of these plots are the average ideological positionsin the top-10 recommendations (elites), and colors of the densityplots indicate the political ideology of the users for whom the rec-ommendations are generated. The first three plots are for baselinealgorithms: collaborative filtering (CF), three-hop random walk( 𝑃 ), and 𝑅𝑃 𝛽 , which is a diverse recommender system based on 𝑃 .The bottom plot is obtained from the recommendations of RWE- 𝐵 , which uses bridging diversification strategy. We can see thatthe baselines’ recommendations are more polarized, i.e., there arevery few recommended items from the middle of the political spec-trum, and most recommendations are either strongly left-leaning,or strongly right-leaning. Although biases in recommender systemshave been studied before, they have not been examined before in thecontext of political content.
In contrast, our algorithm is able to recommend more items fromthe middle of the spectrum. Similarly, the content is less polarized.As already discussed, even though it is not clear what constitutesa good diversification strategy, RWE- 𝐵 is flexible enough to allowdifferent diversification strategies to be plugged in to the system.Our strategy in Figure 6 shows that it is able to recommend moreitems that are both (i) dissimilar to the users’ ideological positionsand (ii) are not too far from the center of the spectrum.We also used Kolmogorov-Smirnov statistic with the nullhypothesis that the distribution of political ideology in the top-10 recommendations generated by RWE- 𝐵 is similar to those ofbaseline algorithms. With very high probability (p value « 0.0001),we are able to reject the null hypothesis (against all baselines).Additional results demonstrating the higher ideological diversityof RWE- 𝐵 ’s recommendations are presented in Appendix A.1. In this paper, we described the problem of diversifying person-alized recommendations and its several challenges, especially inthe context of political content. We proposed a novel approachto diversify recommendations in social and information networks (a)
Item-based Collaborative Filtering (b)
Three-hop random walk ( 𝑃 ) (c) Long-tail diversifying three-hop graph random walk ( 𝑅𝑃 𝛽 ) (d) Random walk with erasure (Bridging strategy) (RWE- 𝐵 ) Figure 6:
Distribution of ideological positions in top-10 recommendationsfor left-, right-, and center-leaning users by the most competitive algorithmsin the US-RT dataset. The colors indicate the ideological position of the usersfor whom the recommendations are generated and values in x-axis representideological positions of recommended items. RWE is able to generate rec-ommendations that are more balanced and less polarized (more items fromthe middle), whereas other algorithms recommend more items from the ex-tremes and less from the middle of the spectrum for all types of users. TheKolmogorov-Smirnov statistic was used to determine whether the recommen-dations from RWE were statistically different from those of other algorithms.
WW ’21, April 19–23, 2021, Ljubljana, Slovenia B. Paudel and A. Bernstein
Model AUC HR@ P@ MR Gini@ AvgDeg@ Pers@ Surp@10 10 20 20 20 20
UK-RT
RWE- 𝐵 ( 𝜈 = 𝛽 ( 𝛽 = . ) 0.89 0.13 0.06 257.49 0.49 119.85 0.96 8.24CF ( 𝑘 = ) 0.74 0.04 0.02 634.27 MF ( 𝑘 = ) 0.85 0.10 0.05 377.0 0.16 392.36 0.53 4.92P US-RT
RWE- 𝐵 ( 𝜈 = . ) 𝛽 ( 𝛽 = . ) 0.91 0.27 0.12 81.81 0.25 332.26 0.80 5.43CF ( 𝑘 = ) 0.86 0.02 0.01 138.29 MF ( 𝑘 = ) 0.87 0.26 0.12 120.38 0.16 405.30 0.73 4.62P DE-RT
RWE- 𝐵 ( 𝜈 = . ) 𝛽 ( 𝛽 = . ) 0.89 0.28 0.13 92.60 0.49 73.70 0.94 6.37CF ( 𝑘 = ) 0.86 0.08 0.04 119.42 MF ( 𝑘 = ) 0.82 0.19 0.09 149.14 0.21 222.20 0.42 3.67P UK-URL
RWE- 𝐵 ( 𝜈 = . ) 𝛽 ( 𝛽 = . ) 0.84 0.23 0.09 197.00 𝑘 = ) MF ( 𝑘 = ) 0.75 0.16 0.07 307.57 0.15 254.35 0.48 4.38P US-URL
RWE- 𝐵 ( 𝜈 = . ) 𝛽 ( 𝛽 = . ) 0.90 0.29 0.12 72.16 0.39 167.02 0.87 5.66CF ( 𝑘 = ) 0.86 0.08 0.03 98.88 MF ( 𝑘 = ) 0.84 0.25 0.12 113.62 0.19 336.31 0.46 3.64P DE-URL
RWE- 𝐵 ( 𝜈 = . ) 𝛽 ( 𝛽 = . ) 0.82 0.31 0.14 75.03 0.61 31.35 0.92 4.87CF ( 𝑘 = ) 0.81 0.21 0.10 80.43 MF ( 𝑘 = ) 0.73 0.14 0.06 112.18 0.26 77.53 0.58 3.10P ML-1M
RWE- 𝐷 ( 𝜈 = . ) 𝛽 ( 𝛽 = . ) 𝑘 = ) 0.91 0.06 0.03 331.90 P Yelp
RWE- 𝐷 ( 𝜈 = . ) 𝛽 ( 𝛽 = . ) 𝑘 = ) 0.92 0.02 0.01 892.16 P Table 8:
Comparison of RWE-based algorithms with other baselines on po-litical content diversification (first six datasets) and long-tail diversity (lasttwo datasets). The dashed lines separate our methods with the baselines andthe chosen parameter for each method is given in parentheses. The first fourcolumns represent measures related to accuracy, the next four columns rep-resent measures related to long-tail diversity. Best results are boldfaced. and showed that it is able to generate both long-tail and ideolog-ically diverse recommendations. Our recommendation algorithmis based on a novel random-walk based algorithm, called RandomWalk with Erasure (RWE). RWE iteratively samples the nodes in auser-item graph by preferring certain nodes over others, as speci-fied by an erasure matrix. For ideological diversity, our approachconsists of two parts: (a) detection of ideological positions of notjust users and elites but also web-content by exploiting social me-dia signals about important political debates, (b) diversification ofrecommendations using the detected ideological positions. To thebest of our knowledge, this is the first work to present a frameworkfor political content diversification and a joint learning of ideolo-gies positions. To evaluate the performance of our algorithms, wecompared the ideological positions of political elites during recentimportant events in three Western countries: Brexit referendum,US presidential elections, and German federal elections and showedthat our joint learning framework can accurately detect ideologicalpositions from social network signals. We also compared the recom-mendation performance of RWE with several baselines and showed that it is able to generate accurate and diverse recommendations.Our work has the following assumptions and limitations. First, weassume one-dimensional ideological positions, which is a simpli-fication of real-world political debates. Second, a proper measurefor assessing political content diversification is still lacking andmay be the subject of debate [23]. Third, we need to test RWE inan real-world, interactive scenario. Last, in the absence of bridgeusers, finding content that are both diverse and agreeable could bechallenging. In practice, this could be addressed by introducing athreshold in our method to specify the maximum extent of diversifi-cation, or the minimum number of bridge-users needed to promoteitems. We plan to explore these limitations and ways to diversifyrecommendations based on additional dimensions in future work.Recent events have shown that the power of OSNs, online newsoutlets, and automated recommendations should not be underesti-mated. With our work, we hope to contribute to the endeavour ofmaking machine learning algorithms and recommender systemssupport healthy debates to enable a better society.
ACKNOWLEDGMENTS
We would like to thank the Hasler Foundation for their generoussupport of some of the work presented here.
REFERENCES [1] Gediminas Adomavicius and YoungOk Kwon. 2012. Improving aggregate rec-ommendation diversity using ranking-based techniques.
IEEE Transactions onKnowledge and Data Engineering
24, 5 (2012), 896–911.[2] Muhammad Aljukhadar, Sylvain Senecal, and Charles-Etienne Daoust. 2012.Using recommendation agents to cope with information overload.
InternationalJournal of Electronic Commerce
17, 2 (2012), 41–70.[3] Arda Antikacioglu and R Ravi. 2017. Post Processing Recommender Systems forDiversity. In
Proceedings of the 23rd ACM SIGKDD International Conference onKnowledge Discovery and Data Mining . ACM, 707–716.[4] Mario Arduini, Lorenzo Noci, Federico Pirovano, Ce Zhang, Yash Raj Shrestha,and Bibek Paudel. 2020. Adversarial Learning for Debiasing Knowledge GraphEmbeddings.
MLG at ACM SIGKDD Conference on Knowledge Discovery and DataMining (2020).[5] Cigdem Aslay, Antonis Matakos, Esther Galbrun, and Aristides Gionis. 2018. Max-imizing the diversity of exposure in a social network. In . IEEE, 863–868.[6] Lars Backstrom and Jure Leskovec. 2011. Supervised random walks: predictingand recommending links in social networks. In
Proceedings of the fourth ACMinternational conference on Web search and data mining . 635–644.[7] Christopher A Bail, Lisa P Argyle, Taylor W Brown, John P Bumpus, HaohanChen, MB Fallin Hunzaker, Jaemin Lee, Marcus Mann, Friedolin Merhout, andAlexander Volfovsky. 2018. Exposure to opposing views on social media canincrease political polarization.
Proceedings of the National Academy of Sciences
Science
Political Analysis (2013).[10] Pablo Castells, Neil J Hurley, and Saul Vargas. 2015. Novelty and diversity inrecommender systems. In
Recommender Systems Handbook . Springer, 881–918.[11] Joshua Clinton, Simon Jackman, and Douglas Rivers. 2004. The statistical analysisof roll call data.
American Political Science Review
98, 2 (2004), 355–370.[12] Elanor Colleoni, Alessandro Rozza, and Adam Arvidsson. 2014. Echo chamber orpublic sphere? Predicting political orientation and measuring political homophilyin Twitter using big data.
Journal of Communication
64, 2 (2014), 317–332.[13] Michael D Conover, Bruno Gonçalves, Jacob Ratkiewicz, Alessandro Flammini,and Filippo Menczer. 2011. Predicting the political alignment of twitter users.In
Privacy, Security, Risk and Trust (PASSAT) and 2011 IEEE Third InernationalConference on Social Computing (SocialCom) . IEEE, 192–199.[14] Colin Cooper, Sang Hyuk Lee, Tomasz Radzik, and Yiannis Siantos. 2014. Randomwalks in recommender systems: exact computation and simulations. In
Proceed-ings of the 23rd International Conference on World Wide Web . ACM, 811–816.[15] Sheila Coronel. 2003. The role of the media in deepening democracy.
NGO MediaOutreach: Using the (2003). andom Walks with Erasure WWW ’21, April 19–23, 2021, Ljubljana, Slovenia [16] Maurizio Ferrari Dacrema, Simone Boglio, Paolo Cremonesi, and Dietmar Jannach.2021. A troubling analysis of reproducibility and progress in recommendersystems research.
ACM Transactions on Information Systems (TOIS)
39, 2 (2021).[17] Maurizio Ferrari Dacrema, Paolo Cremonesi, and Dietmar Jannach. 2019. Are wereally making much progress? A worrying analysis of recent neural recommen-dation approaches. In
Proceedings of the 13th ACM Conference on RecommenderSystems . 101–109.[18] Kiran Garimella, Gianmarco De Francisci Morales, Aristides Gionis, and MichaelMathioudakis. 2017. Reducing controversy by connecting opposing views. In
Proceedings of the Tenth ACM International Conference on Web Search and DataMining . 81–90.[19] Mouzhi Ge, Fatih Gedikli, and Dietmar Jannach. 2011. Placing high-diversityitems in top-n recommendation lists. In
Workshop chairs . 65.[20] Mark S Granovetter. 1977. The strength of weak ties. In
Social networks . Elsevier.[21] Kyle A Heatherly, Yanqin Lu, and Jae Kook Lee. 2017. Filtering out the otherside? Cross-cutting and like-minded discussions on social networking sites.
NewMedia & Society
19, 8 (2017), 1271–1289.[22] Hoda Heidari, Michele Loi, Krishna P Gummadi, and Andreas Krause. 2019. Amoral framework for understanding fair ML through economic models of equalityof opportunity. In
Proceedings of the Conference on Fairness, Accountability, andTransparency . 181–190.[23] Natali Helberger. 2019. On the Democratic Role of News Recommenders.
DigitalJournalism
7, 8 (2019), 993–1012. https://doi.org/10.1080/21670811.2019.1623700[24] Philip N Howard and Bence Kollanyi. 2016. Bots,
Computational Propaganda Project:Working Paper Series (2016).[26] Christopher C Johnson. 2014. Logistic matrix factorization for implicit feed-back data.
In NIPS 2014 Workshop on Distributed Machine Learning and MatrixComputations
27 (2014).[27] Rune Karlsen, Kari Steen-Johnsen, Dag Wollebæk, and Bernard Enjolras. 2017.Echo chamber and trench warfare dynamics in online debates.
European journalof communication
32, 3 (2017), 257–273.[28] Bence Kollanyi and Philip N Howard. 2017. Junk News and Bots during theGerman Parliamentary Election: What are German Voters Sharing over Twitter?[29] Preethi Lahoti, Kiran Garimella, and Aristides Gionis. 2018. Joint non-negativematrix factorization for learning ideological leaning on Twitter. In
Proceedings ofthe Eleventh ACM International Conference on Web Search and Data Mining .[30] Neal Lathia, Stephen Hailes, Licia Capra, and Xavier Amatriain. 2010. Temporaldiversity in recommender systems. In
Proceedings of the 33rd international ACMSIGIR conference on Research and development in information retrieval . ACM.[31] Jane Mansbridge, James Bohman, Simone Chambers, Thomas Christiano, ArchonFung, John Parkinson, Dennis F Thompson, and Mark E Warren. 2012. A systemicapproach to deliberative democracy.
Deliberative systems: Deliberative democracyat the large scale (2012), 1–26.[32] Antonis Matakos, Evimaria Terzi, and Panayiotis Tsaparas. 2017. Measuring andmoderating opinion polarization in social networks.
Data Mining and KnowledgeDiscovery
31, 5 (2017), 1480–1505.[33] Amy Mitchell, Jeffrey Gottfried, and Katerina Eva Matsa. 2015. Millennials andpolitical news.
Pew Research Center
Comparing mass media in established democracies: patterns ofmedia performance . Springer.[35] Sean A Munson, Daniel Xiaodan Zhou, and Paul Resnick. 2009. Sidelines: AnAlgorithm for Increasing Diversity in News and Opinion Aggregators.. In
ICWSM .[36] Cameron Musco, Christopher Musco, and Charalampos E Tsourakakis. 2018.Minimizing polarization and disagreement in social networks. In
Proceedings ofthe 2018 World Wide Web Conference . 369–378.[37] Diana C Mutz. 2002. Cross-cutting social networks: Testing democratic theoryin practice.
American Political Science Review
96, 1 (2002), 111–126.[38] Tien T Nguyen, Pik-Mai Hui, F Maxwell Harper, Loren Terveen, and Joseph AKonstan. 2014. Exploring the filter bubble: the effect of using recommendersystems on content diversity. In
Proceedings of the 23rd international conferenceon World wide web . ACM, 677–686.[39] Gourab K Patro, Arpita Biswas, Niloy Ganguly, Krishna P Gummadi, and AbhijnanChakraborty. 2020. Fairrec: Two-sided fairness for personalized recommendationsin two-sided platforms. In
Proceedings of The Web Conference 2020 . 1194–1204.[40] Bibek Paudel, Fabian Christoffel, Chris Newell, and Abraham Bernstein. 2017.Updatable, accurate, diverse, and scalable recommendations for interactive appli-cations.
ACM Transactions on Interactive Intelligent Systems (TiiS)
7, 1 (2017).[41] Bibek Paudel, Thilo Haas, and Abraham Bernstein. 2017. Fewer flops at the top:Accuracy, diversity, and regularization in two-class collaborative filtering. In
Proceedings of the Eleventh ACM Conference on Recommender Systems . 215–223.[42] Bibek Paudel, Sandro Luck, and Abraham Bernstein. 2018. Loss aversion in recom-mender systems: Utilizing negative user preference to improve recommendationquality.
CARS-BDA at the 12th ACM International Conference on Web Search andData Mining (2018). [43] Erik Peterson and Ali Kagalwala. 2021. When Unfamiliarity Breeds Contempt:How Partisan Selective Exposure Sustains Oppositional Media Hostility.
AmericanPolitical Science Review (2021), 1–14. https://doi.org/10.1017/S0003055420001124[44] Keith T Poole and Howard Rosenthal. 1985. A spatial model for legislative rollcall analysis.
American Journal of Political Science (1985), 357–384.[45] Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme.2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv preprintarXiv:1205.2618 (2012).[46] Badrul Munir Sarwar, George Karypis, Joseph A Konstan, John Riedl, et al. 2001.Item-based collaborative filtering recommendation algorithms.
WWW
Echo chambers: Bush v. Gore, impeachment, and beyond .Princeton University Press Princeton, NJ.[48] Sebastián Valenzuela, Yonghwan Kim, and Homero Gil de Zúñiga. 2011. Socialnetworks that matter: Exploring the role of political discussion for online politicalparticipation.
International journal of public opinion research
24, 2 (2011), 163–184.[49] Magdalena Wojcieszak. 2009.
When deliberation divides: How people with strongviews respond to political disagreement . University of Pennsylvania.[50] Yuan Cao Zhang, Diarmuid Ó Séaghdha, Daniele Quercia, and Tamas Jambor. 2012.Auralist: introducing serendipity into music recommendation. In
Proceedings ofthe fifth ACM international conference on Web search and data mining . ACM.[51] Xiaojin Zhu, Andrew B Goldberg, Jurgen Van Gael, and David Andrzejewski.2007. Improving diversity in ranking using absorbing random walks. In
NAACL .[52] James Zou and Londa Schiebinger. 2018. AI can be sexist and racist—it’s time tomake it fair.
Nature (2018).
A APPENDIXA.1 Additional Results
In addition to the results presented in Section 7.5 of the main paper,here we introduce some new measures in order to help understandthe difference in the ideological diversity of recommended items forusers across the spectrum. The top half of Table 9 contains measuresfor average ideological position of recommendations (Rec-pos) andtraining items (Train-pos), difference between Rec-pos and user’sideological position (User-shift) and between Rec-pos and Train-pos (Train-shift), and the range of recommended items’ positions(Rec-range). In the remainder of this section, we use a cutoff of 𝑡𝑜𝑝𝑘 =
10 for evaluating recommendation diversity.We show these measures for 𝑃 , 𝑅𝑃 𝛽 , and RWE- 𝐵 on the US-URLdataset in Figure 7. The relations of 𝜃 𝑢 and Train-pos with Rec-poscan be seen in the first two rows, and their relations with User-shiftand Train-shift can be seen in the third and fourth rows. From thefirst two rows, we see that RWE- 𝐵 recommends more items from theupper and lower quadrants to users in the left and right respectively,and items from the center to users throughout the spectrum. Fromthe third and fourth rows, we see that RWE- 𝐵 has bigger positiveshift for users in the left and bigger negative shift for users inthe right. In comparison, other algorithms have considerably lessshift for most users and items are concentrated in the user’s ownquadrant, showing that their recommendations are not diverse. Thefifth row shows the relationship between 𝜃 𝑢 and Rec-range, whereRWE- 𝐵 has fewer recommendations with a narrow range (bottomof the plot) for users throughout the spectrum.To summarize the above measures for all users in a dataset, weintroduce five measures based on the weightings of the measuresdescribed above by user’s positions or training items’ average po-sition. First, let us revisit the two (proposed) desirable propertiesfor diverse recommendations: (i) recommendations should leantowards the center relative to a user’s ideological position and (ii)recommendations should span a wider range of ideological posi-tions for users in the extremes of the spectrum. The first conditionimplies that for users on the left-side (or right-side) of the spectrum,a bigger positive difference (or negative difference) between the WW ’21, April 19–23, 2021, Ljubljana, Slovenia B. Paudel and A. Bernstein (a) 𝑃 (b) 𝑅𝑃 𝛽 (c) RWE- 𝐵 ( 𝜈 = . ) Figure 7:
Distribution of different ideological diversity measures for allusers and top-10 recommendations in the US-URL dataset. Compared to themost competitive algorithms, RWE- 𝐵 is able to (i) recommend items whoseideological positions is different from the user and her observed preferences,(ii) shift the recommendations towards more moderate positions from userand her observed preferences, (iii) recommend items that span a wider rangeof ideological positions for all users. Name Measure Description 𝐽 𝑢 = 𝑝𝑜𝑠 ( 𝑅 𝑢 [ 𝑘 ] Positions of top-k recs 𝐿 𝑢 = 𝑝𝑜𝑠 ( 𝑇 𝑢 [ : ]) Positions of trainsRec-pos 𝑝𝑜𝑠 ( 𝑅 𝑢 ) = 𝑘 (cid:205) 𝐽 𝑢 Avg position of top-k recsTrain-pos 𝑝𝑜𝑠 ( 𝑇 𝑢 ) = | 𝑇𝑢 | 𝐿 𝑢 Avg position of traininsUser-shift 𝑠ℎ𝑖𝑓 𝑡 ( 𝑅 𝑢 ,𝑢 ) = 𝑝𝑜𝑠 ( 𝑅 𝑢 ) − 𝑝𝑜𝑠 ( 𝑢 ) Shift of recs from userTrain-shift 𝑠ℎ𝑖𝑓 𝑡 ( 𝑅 𝑢 ,𝑇 𝑢 ) = 𝑝𝑜𝑠 ( 𝑅 𝑢 ) − 𝑝𝑜𝑠 ( 𝑇 𝑢 ) Shift of recs from trainsRec-range 𝑟𝑎𝑛𝑔𝑒 ( 𝑅 𝑢 ) = 𝑚𝑎𝑥 ( 𝐽 𝑢 ) − 𝑚𝑖𝑛 ( 𝐽 𝑢 ) Ideological range of recsUW-Recs 𝜃 𝑢 × 𝑝𝑜𝑠 ( 𝑅 𝑢 ) Rec pos weighted by 𝜃 𝑢 UW-Shift 𝜃 𝑢 × 𝑠ℎ𝑖𝑓 𝑡 ( 𝑅 𝑢 ,𝑢 ) User shift weighted by 𝜃 𝑢 TW-Recs 𝑝𝑜𝑠 ( 𝑇 𝑢 ) × 𝑝𝑜𝑠 ( 𝑅 𝑢 ) Rec pos weighted by TrainposTW-Shift 𝑝𝑜𝑠 ( 𝑇 𝑢 ) × 𝑠ℎ𝑖𝑓 𝑡 ( 𝑅 𝑢 ,𝑇 𝑢 ) Train shift weighted byTrain posUW-Range 𝑎𝑏𝑠 ( 𝜃 𝑢 ) × 𝑟𝑎𝑛𝑔𝑒 ( 𝑅 𝑢 ) Rec range weighted by 𝜃 𝑢 Table 9:
Measures for ideological diversity in recommendations (recs = rec-ommended items, trains = training items, pos = ideological position, 𝜃 𝑢 = user 𝑢 ’s ideological position, 𝑅 𝑢 are items recommended for user 𝑢 , 𝑇 𝑢 are train-ing items for user 𝑢 , 𝑎𝑏𝑠 (·) is absolute value of 𝑥 , k is the threshold for top-kmeasures). Measure RP 𝛽 P RWE- 𝐵 RWE- 𝐵 RWE- 𝐵 ( 𝜈 = .
0) ( 𝜈 = .
0) ( 𝜈 = . UW-Recs
TW-Recs
UW-Shift -0.84 -0.92 -1.04* -1.53** -1.31**
TW-Shift -0.11 -0.24 -0.33 -0.59* -0.49*
UW-Range
AUC
HR@10
Table 10: