Use of a controlled experiment and computational models to measure the impact of sequential peer exposures on decision making
Soumajyoti Sarkar, Ashkan Aleali, Paulo Shakarian, Mika Armenta, Danielle Sanchez, Kiran Lakkaraju
UUse of a controlled experiment and computational models tomeasure the impact of sequential peer exposures on decisionmaking
Soumajyoti Sarkar , Paulo Shakarian , Danielle Sanchez , Mika Armenta , KiranLakkaraju School of Computing, Informatics, and Decision Systems Engineering, Arizona StateUniversity, Tempe, AZ, USA Computer Science and Applications, Sandia National Laboratories, Albuquerque,NM, USA* [email protected]
Abstract
It is widely believed that one’s peers influence product adoption behaviors.This relationship has been linked to the number of signals a decision-makerreceives in a social network. But it is unclear if these same principles hold whenthe “pattern” by which it receives these signals vary and when peer influence isdirected towards choices which are not optimal. To investigate that, wemanipulate social signal exposure in an online controlled experiment using agame with human participants. Each participant in the game makes a decisionamong choices with differing utilities. We observe the following: (1) even in thepresence of monetary risks and previously acquired knowledge of the choices,decision-makers tend to deviate from the obvious optimal decision when theirpeers make similar decision which we call the influence decision , (2) when thequantity of social signals vary over time, the forwarding probability of theinfluence decision and therefore being responsive to social influence does notnecessarily correlate proportionally to the absolute quantity of signals. To betterunderstand how these rules of peer influence could be used in modelingapplications of real world diffusion and in networked environments, we use ourbehavioral findings to simulate spreading dynamics in real world case studies.We specifically try to see how cumulative influence plays out in the presence ofuser uncertainty and measure its outcome on rumor diffusion, which we model asan example of sub-optimal choice diffusion. Together, our simulation resultsindicate that sequential peer effects from the influence decision overcomesindividual uncertainty to guide faster rumor diffusion over time. However, whenthe rate of diffusion is slow in the beginning, user uncertainty can have asubstantial role compared to peer influence in deciding the adoption trajectory ofa piece of questionable information.
The connections and interactions of individuals that comprise social networks aregenerally believed to impact decision-making in many domains including productselection and decision making in uncertain environments [1, 2]. While there is generaltheoretical consensus that social influence, the phenomenon by which an individual’sopinions, behaviors, and decisions are influenced by other people [3], facilitatesproduct selection, the empirical literature is actually quite torn. According toindividual utility models, people adopt technologies when its benefits exceed itsJune 8, 2020 1/41 a r X i v : . [ c s . S I] J un osts [4]. Because comparing every option can be cognitively costly and timeconsuming, individuals employ cognitive strategies and shortcuts that reduce thenumber of alternatives until one superior option is left [5]. Social signals factor intothe strategies because they provide a cost-efficient means of acquiring information.Some find that social information predicts selection decisions [6] and others havereported interactions between a decision-maker’s experience and/or knowledge of theproduct and their general susceptibility to social influence [7]. It is critical thatresearch agendas pursue understanding of these nuances because as technologiesbecome more sophisticated, widely used, and powerful, so does their potential to beused for harm. The recent slew of worldwide cyber-attacks is a potent example of howthe selection of cyber-defense venders will affect billions of people [8]; if cyber-defense‘shoppers’ are subject to ‘sub-optimal’ social influence, how will their decisions beaffected? The choice to follow the herd may not be the best strategy when the herdchooses ‘wrong’. Similarly. the widespread rise in misinformation has detrimentalimpacts in society and where social influence has known to be adversely impacting thedecision making process leading to undesirable contagion [9, 10]. In both thesesituations, the choice to follow the herd may not be the obvious optimal choice andsocial influence can play a detrimental role. In this paper, we investigate the role of patterns of influence (PoI) or the manner in which an individual is repeatedly subjectto the same piece of information over time by being embodied in a connectedenvironment with other individuals, on behavior diffusion. We extend our recent workon understanding the role of PoI towards individual decision making using a controlledexperimental setup [11, 12].We develop two sets of studies in this paper. First, we start off by developing anonline controlled framework to question the longstanding notions of the magnitude ofpeer signals as a reliable predictive factor of behavior diffusion. To this end, we developan experimental framework to characterize the exposure effect under multiple signalsbut when the pattern of influence or PoI could be controlled (we show an example ofwhat a “pattern” is in Fig 1) - the experimental framework allows us to measure socialinfluence while avoiding confounding effects. It allows us to analyze how the signalproportion when paired with its temporal treatment impacts the selection choices ofusers in environments where the influence decision is not the best choice. The firststudy was designed to avoid network effects as confounders in our understanding ofpeer influence effect on behavior diffusion. Following this, in a separate second study,we attempt at testing some of the rules obtained from our behavioral findings in thecontrolled experiment on sub-optimal choice selections in networked environments inthe real world. We specifically try to measure the role of peer signals in networkedenvironments and in the presence of user uncertainty on diffusion of information withquestionable veracity. The objective of this second work is to understand the extent towhich our behavioral findings from the experimental data can be applied toobservational data - to this end we simulate spreading on real networks.For our first work, we use Amazon Mechanical Turk (AMT) to run an online,controlled decision-making game and recruit participants for the game conducted overseveral time steps. At each time step, participants selected one technology among 6choices with differing utilities (only one among them was the optimal technology) - wemodulate the latter part of the game by subjecting participants to peer signalsdirected towards a technology which was not optimal. We focus on understanding theeffects of PoI on the choices made by individuals in such a setting. While the firststudy relies on experimental data, in our second work we adopt a data-drivenapproach to simulate diffusion using multi-agent models but where the agents aremapped to real world users. The diffusion model incorporates the rules of peerinfluence observed from the experimental setup.June 8, 2020 2/41ig 1: This illustration demonstrates a pattern of influence . At the first time step, oneamong 6 neighbors of a user shares a message, in the following time step, another usershares the same message and in the final step, two additional users. The user is thusexposed to a “pattern” of social signals we represent as V = { , , } for that message.Our work has been influenced from existing studies on peer exposures and behaviordiffusion [13, 14], however to the best of our knowledge, the extant work does notexamine the effect of signal exposure when the signals promote sub-optimal choices. Inan observational study on the impact of repeated exposures on informationspreading [15], the authors show that an overwhelming majority of message samplesare more probable to be forwarded under repeated exposures, compared to thoseunder only a single exposure. However what is often understudied or left out in theseresearch studies is the sequential exposure mechanism or what we refer to as PoI in thepaper. Additionally, observing and mapping these PoI in the real world to analyze thecause and effect synopsis is not straightforward because of the opacity problem [16].Similarly, the experiment conducted in [17] reported that individual adoption ismuch more likely when participants receive social reinforcement from multipleneighbors in the social network relative to a single exposure, however it did notdifferentiate the effects of network structure from the sequential exposure mechanismor what we refer to as a pattern of influence. One of the main results from [17] showthat the influence monotonically increases, although the increased likelihood ofinfluence from k signals compared to k − k =2. Our results, assummarized below, have the following observations: • We cluster participants in the online experiment into 5 groups - 2 control groupsand 3 treatment groups differing by the PoI or the pattern of informationcascade induced among the neighbors. Upon analyzing the selections made byparticipants aggregated over the lifecycle of the game, we observe the following -compared to participants who were treated to a single controlled sub-optimalpeer signal (reflecting the influence decision) over all time steps, the probabilityof selecting that influence decision was significantly higher for participants whowere treated early to a large quantity of such controlled sub-optimal peer signals. • Through multiple analyses on the effect of sequential peer exposures, we observethat the number of exposures alone does not explain successful social influencecontrasting conclusions from several previous studies. Surprisingly, a delayedstimulus in the form of sudden increase in peer signals is a more effectiveinfluence strategy for later stages than peer signals administered through auniform build up when comparing the same time stage.June 8, 2020 3/41
Finally, as a step towards understanding how the rules from the behavioralfindings play out in real world diffusion, we develop data-driven agent basedmodels that simulate rumor diffusion. We use data to learn agent specificparameters and evaluate the spreading dynamics of a group of questionableinformation in Twitter networks. We find that while social influence based onsequential exposures can result in faster diffusion compared to the factor ofsimply the number of peer exposures, individual behavior uncertainty can alsoplay an important role that can impact the influence factor itself.The rest of the paper is organized as follows: we first discuss the related literatureunderpinning this study in Section 2 and the hypotheses that we will investigate inSection 3. We present the experimental setup and methods designed for measuringsocial influence in Section 4. We analyze the controlled experiment results inSection 5. Finally, we develop an agent based model drawing upon the conclusions ofthe controlled experimental results and evaluate its results with real world data inSection 6.
Informational social influence, the tendency to accept information from others asevidence about reality, tends to affect financial decision-making and product selectionmore than other forms of social influence [18–20]. We are more often swayed byothers’ decisions and behavior when we lack knowledge about the object of ourdecision, such as when we must choose a product that we do not know much about, isnot well-described, or that we have little experience with [21]. This is because theinformation we seek can be more cheaply acquired by observing others than by seekingit ourselves. Conventional studies suggest that as the consensus of entities in a socialnetwork increases – more signalers make the same signal – we assume that theinformation peers are conveying is valid and we are more likely to adopt the signaledbehavior or decision [19, 22]. The literature documents several influences on theadoption of behaviors including network structure – who is connected to who and theproperties associated with these connections [17] – an individual’s information parsingprocesses, their perceptions of product utility, and the number of signals they receive.
Number of signals
The relationship between the number of signals an individual receives in its network,social influence, and the likelihood that said individual will adopt the behaviorindicated by the signals is closely related to the linear threshold model in which anactor adopts a behavior after the signal count reaches an optimal threshold [23].What, though, is the impact of repeated signals on the decision-making process, andmore specifically how many signals are required to reliably influence an individual’sdecision-making? There are mixed findings regarding the benefit of multiple exposureson the diffusion of information necessary to reach this threshold [24]. People mayprefer multiple confirmations from their peers to reassure themselves before making adecision [15, 25]. Experimental human studies using games from behavioral economicslike the Prisoner’s Dilemma tend to find that the impact of zero to three signalersincreases behavioral and decision-adoption in a linear fashion, and that a keythreshold for maximum social influence exists between four and five signalers. Thismeans that two signalers exert more influence than one, three exert more than two,and four to five exert more influence than three. There is not much difference betweenthe impact of five and six or more. However, debate still exists – some have found theJune 8, 2020 4/41hreshold to be two signalers [26] while others report it at three [17, 27]. Still, othershave found the reverse trend. In one study, repeated exposure to online signals in asocial networking site slowed the subsequent spread of information. This might affectdecision-making and behavioral adoption by inhibiting social influence [24, 28].
Network Structure
A network structure’s describes characteristics of the network as a whole. This caninclude properties like clustering/decentralization (to what degree do actors form ties),modularity (how densely connected nodes are within clusters) or motif structures [29],homophily/heterophily (similarity or dissimilarity of actors predicts tie-forming), andcentralization (how connected are actors). For instance, the density of adecision-maker’s social network can influence the choices they make because signalers’intentions are more ambiguous in densely connected networks [30]. The dynamism, orhow easily entities can move in and out of networks, also influences decision-making.Social decision-making modelers have found that the more dynamic and mobile anetwork, the greater concurrence of their decisions in behavioral economicsgames [31, 32]. Because the present studies are primarily interested in the effects of a)the number of signals and b) the pattern by which they are received, we attempt tocontrol for the effects of network structure by holding the structure constantthroughout all experimental conditions in both studies. Please see Methods sectionsfor further details.
Information Parsing
The manner in which individuals search for information affects their decision-making.Individuals employ search strategies to reduce the number of choices [5]. This includesrevising their initial opinions by processing and averaging the different influencesacting on them [33] and social information provides one mechanism through which thisis achieved. These manners of decision making have also been linked to the concept ofdual process theory - the notion that two different systems of thought co-exist; aquick, automatic, associative, and affective-based form of reasoning and a slow,thoughtful, deliberative process [34]. Fast thinking involves conditions of “cognitiveease” and so social influence factors into this process of slowing down the decisionmaking system by presenting alternating evidences for reconsideration. When socialsignals point towards a specific outcome or opinion, individuals will often adopt theopinions and behaviors of signalers [22, 35], however while adopting behaviors based offsocial influence can be cost effective, it does not always lead to the most effective orefficient decision. Individuals must trade-off between trusting their own knowledge andtrusting other’s opinions [13]. Biased social signals can influence individuals to choosewrong or less effective answers in a variety of domains, particularly if multiple peersback the behavior or decision [15, 22, 35].
Product Utility
When making product decisions, outcomes related to the quality and need for aproduct change its the utility or perceived value and therefore the risk of the selection.For instance, when decision-makers were asked to purchase songs in an online marketwhere they could see the decisions of other purchasers, the perceived quality of thesongs predicted their choices even when they witnessed peers purchasing songs of poorquality. Songs of medium quality were most subject to effects of social influence [36].The need for a quality product also influences selection decisions. Kraut andcolleagues found that when deciding between different video-teleconferencingJune 8, 2020 5/41echnologies, people with the most communication intensive work – those who reliedon video-teleconferencing the most – placed greater value on the product’s ability thanthose with less need [4]. The perceived value of a product predicts how much anindividual will search for information to inform their selection [21], and consumers anddecision-makers are more likely to engage in information search – including socialinformation – when purchasing products is risky – such as when the costs of making asub-optimal choice are high – and when they lack prior knowledge about thetechnology [21]. When we feel that we know enough about a product, we believe thatwe already have enough information stored in memory to make the best decision, andtherefore additional social information is unnecessary.
We perform an online controlled experiment that manipulates the number of socialsignals and the signal pattern over time. We hypothesize that successful socialinfluence requires more than just receiving signals or exposures to information frompeers, as both the utility of the technology and informational influences are at play. Inour experiment, any decision made produces varying degrees of monetary gain basedon utility. So, we speculate that successful social influence should be reflective of themechanism through which information diffuses that ultimately instigates individualsto change their beliefs and therefore their adoption behavior. We present the followingtwo hypotheses that we test in this paper with regards to the objective mentioned.HYPOTHESIS H1.
Individuals will be more likely to choose a sub-optimalcyber-defense provider when they observe peers choosing the sub-optimal provider.
Through our first hypothesis, we try to examine the aggregated results of thecascading exposure arising from varying temporal patterns of influence on the outcomeof interest - whether users follow the decision made by their peers. In this hypothesis,time takes a backseat and we try to measure the extent to which early, uniform ordelayed exposure to peer influence can successfully achieve our desired outcome insituations of sequential decision making when aggregated over all time steps of theexperiment. Subsequently, the hypothesis attempts to examine findings in [17] whichshow that behaviors spread to a larger portion of the population in a clusterednetwork, indicating that additional social signals have significant effect on influence.However, the results on behavior diffusion reported by this paper are heavilyassociated with the clustered network organization that dictates the exposure to socialinfluence and as such the sequential nature of exposures and its effect was largelyignored in the study. Following this, in another study on empirical data fromTwitter [37], authors show that the number of active neighbors is a positive indicatorof influence, which is a similar finding reported by [13, 17]. In both these studies, theauthors did not segregate varying temporal influence patterns that might force usersto revise their beliefs over time in different manners. We build on these experiments totest the aggregated effect of the peer signals and the extent to which the manner ofsignal dissemination among peers can act as an impetus towards coercing users tochange their decisions, especially when users weigh their own private informationagainst external influence.HYPOTHESIS H2.
The cascading pattern of peer signals or the temporal patternsof influence will impact the adoption behavior of individuals more than just thequantity of signals.
June 8, 2020 6/41able 1: Table of Symbols
Symbol Description u a user or an agent C u influence decision for uT treat treatment phase of controlled experiment, time steps13 to 18 in our setting N u number of times user u has adopted the influencedecision C u in T treat A u ( t ) number of peers of user u who have adopted C u attime td i technology choice/decision i , i ∈ [1 ,
6] for the con-trolled experiment D u ( t ) decision or technology adopted by user u at time step tn number of individuals in a group in the controlledexperiment G = ( V, E ) a network G consisting of nodes V and edges E , V ( G ) denotes vertices relevant to network G . p u ( t ) probability of user u adopting a sub-optimal decisionat time tµ u ( t ) utility obtained by user u upon choosing the optimaldecision at time tζ u ( t ) utility obtained by user u upon choosing the influ-ence decision C u at time tz u ( t ) binary variable that assumes value 1 if user u chooses C u at time t , else zero η u , β u Agent specific parameters in the ABM model q , V q information cascade and the users/nodes participat-ing in qDF q [ t ] Diffusion node set for cascade q obtained at time step t from our ABM simulationIn this hypothesis, our goal is to understand whether the same quantity of peersignals have different outcomes when individuals are subjected to them throughdifferent cascading patterns. We again note that the authors in [17] conclude that thelikelihood of a user adopting a behavior at k signals compared to k − k =3. The authors attribute this to the clustered network organizationthat allowed for users to receive multiple signals before they chose to adopt a behavior.What we instead posit is that “time” has an important role to play in the revisedbeliefs and opinions of individuals - an early exposure to peer influence can can resultin a substantially dynamics of adoption than situations where influence is delayed. Wedeliberately downplay the role of networks to be able to control and study the natureof cascading and we consider that all peers of an individual are homogeneous withrespect to the influence they can exert on it. This allows us to control the pattern ofinfluence i.e. the number of signals sent over each time step to an individual.As we point out later, a successful social influence constitutes situations whereindividuals not only deviate from the optimal decision but they also select the optionthat majority of their peers choose.June 8, 2020 7/41ig 2: Example screen in the cyber-defense provider selection task. Participants in theUniform Messages (UM) condition of the study have access to a screen that looked likethis. The F eedback section displays the number of attacks the participant preventedafter each time step. The
Decisions section displays the 6 provider choices that ithas. Finally, the
M essages section is displayed after time step 12, from where on theparticipants can view what their peers selected in the previous time step.
The study with human participants was approved by Human Studies Board at SandiaNational Laboratories. The consent was obtained in written from the board. Subjectshad informed consent through written means. The “documentation of informedconsent” was waived since this was an online experiment that had no more thanminimal risk of harm to subjects. To test our hypotheses, we ran an online, controlleddecision-making game in which participants took on the role of a security officer at abank. Participants were told that they and several of their peers at different bankswere being asked to invest in a cyber-defense technology provider once a month for 18time steps or time steps. We separated participants into 5 groups based on pattern ofsocial signal exposure which will be described in details in the
Design subsectionfollowing this. At each time step, participants were able to choose from 6 differenttechnology providers - among which only one was optimal, preventing 7 attacks. Theremaining 5 providers prevented 5 attacks each (from that perspective, all sub-optimaltechnologies had the same utility). For every attack they prevented in any time step,participants received $ $ Participants were randomly assigned to five groups with each group having uniquemembers not involved in decision making as part of other groups. The entire gamewas partitioned into two phases. For the first phase comprising 12 time steps, no otherinformation but a short excerpt about six potential providers was given. After theparticipants made their selection for a given time step, they saw the number of attackstheir provider had prevented in that step. As mentioned earlier, in the absence of theJune 8, 2020 8/41ig 3: The signal vs time step plots for the 4 patterns - note that for the NM group(not shown here), no peer signal in the form of pre-selected sub-optimal technologieswere sent to the participants at any time step.knowledge of the utilities of the technology providers (or the number of attacks itprevents), the first 12 time steps allow for individual decision making and exploration.In the second phase of the experiment which started at time step 13, we introducedinterventions by allowing participants access to extra information from 6 otherindividuals which are supposedly their peers (and which are bots controlled by us).After participants make their choice at a time step, they can view the selections madeby their peers in the previous time step. Each participant in the all groups bar one aresubjected to 6 peers - we call the decisions of these peers that the participant views inthe second phase as peer signals in this work. Fig 2 shows the screen of a participantfrom time steps 13 to 18 when they were able to view the decision of their peers. Theplatform was hosted by the Controlled Large Online Social Experimentation (CLOSE)platform and developed at Sandia National Laboratories [38]. While the participant isable to view the technology selections of all its peers at each time step, we controlsocial influence by administering a randomly selected sub-optimal technology C u (among the 5 providers) through the peers of the participant u - so using ournomenclature, C u is the influence decision for u . For each u , this technology C u wasselected as the choice that would be disproportionately signaled by its peers over time(this pattern of influence or PoI would be manipulated by us). The motivation behindthis deliberate selection of sub-optimal C u (controlled by us) as the influence decision is to investigate whether participants would be tempted to select this technology C u inthe presence of its peer feedback. Note that we attempt to avoid network effects byrandomizing this technology or the influence decision C u specific to the user u - thisallows us to avoid any deliberate collisions among peer choices of different users thatcould be representative of network effects in the real world.We denote by A u ( t ) the number of peers of the participant u , who we administerthe sub-optimal technology/influence decision C u at time step t (so A u ( t ) ≤ A dropping the subscripts to generalize for allusers or P oI as we denote it, for the 5 groups (the first 2 being the control groups forcomparison with the next 3 treatment groups) (Fig 3 shows the signal patterns for thegroups):1.
No Message (NM) : Participants receive no message from the peers, so the lastsix time step are exactly same for the participants as the first 12.June 8, 2020 9/41.
Uniform Message (UM) : Here we send C using one peer of a participant at eachtime step. So A = { , , , , , } denotes uniform influence.3. Linear Cascade (LC) : Here we incrementally activate one peer with thetechnology C at each time step. So A = { , , , , , } denotes uniformlyincreasingly influence as shown in Fig 4.4. Delayed Cascade (DC) : Here we send only one signal for the first 3 time stepsand send 4, 5 and 6 signals at the last 3 time steps in order. So A = { , , , , , } . The objective is to see whether the sudden change in the numberof signals acts as a catalyst for successful influence at the later stages of theexperiment.5. Early Cascade (EC) :In this setup, we send higher magnitude of signals from thebeginning setting A = { , , , , , } . This pattern allows us to ask if an earlytrigger is able to sustain the levels of influence, or whether participants willreturn to the optimal choice at the later stages.Accordingly, A u ( t ) would be different for users in each group, for e.g. for a u in LCgroup, A u ( t = 1) = 1, A u ( t = 2) = 2 while for EC group, A u ( t = 1)=4, A u ( t = 2)=5and so on. An example of the linear cascade setting is shown in Fig 4, where aparticipant receives social signals from its six neighbors - our influence decision C uniformly cascades through the peers of the participant.At time step 13 (start of the second phase), a signaler u selects a sub-optimalprovider C u , and over the next five time steps, the remaining peers adopt the same influence decision one after another. Note that although we program only selectedpeers (bots) of a subject to administer C u over time, the subjects are able to view alltheir peers’ decisions in their dashboard for all the last 6 time steps - the rest of thenon-controlled peers at a time step show random technologies to the participant. Also,note that in all conditions, users can switch back to any choice in the next time stepafter having selected an option in the current time step. We consider the NM and UMgroups as our baseline groups and LC, EC, DC groups as our treatments groups ofinterest.For both the hypotheses H H
2, the outcomes of interest are the decisionsmade by participants in the last six time steps, in the presence of social signals frompeers. We explore whether decision-makers will be more likely to choose cyber-defenseproviders which are not the optimal choice when they have knowledge about theutilities and when they observe peers opting for choices which are not optimal. Wenote that people get feedback about their choice on the very next screen—and sochoosing a technology during an intermediate time step is more of a data-gatheringexploration rather than their final choice. In order to allow for this initial bandwidthfor exploration, we keep the first 12 time steps ( time steps) same for all subjectsdevoid of any interference This helps in overcoming bias related to an individual’s ownknowledge about the utilities in the second phase of the experiment when they aretreated to social signals.
We recruited a total of 357 participants for this study to play the same cyber-defenseprovider game. Based on the responses provided by the participants regarding theirdemographics in the form of a survey response , we have 151 males and 190 males inthe study. Most of our participants are in the age group of 26-35 years and full details Some participants declined to provide a response regarding their demographics
June 8, 2020 10/41ig 4: Illustration of the linear cascade diffusion. The technology C u chosen by us asthe sub-optimal technology ( influence decision for user u (in dots) cascades through thepeers of u over the 6 time steps. Colored nodes denote the activated peers with respectto C u (manually preprogrammed by us) at each time step. Note that although at timesteps starting at 13 and ending at 18, there are subjects (uncolored) among peers whohave not adopted C u , their selections (which may not be C u ) are visible to u . However,which users among the peers have been preprogrammed manually is by default unknownto the target subject u .regarding their age groups have been provided in Tables 1, 2 and 3 in Appendix A inS1 Appendix. Additionally, majority of the participants in our experiment had aBachelor’s degree and details have been provided in the Appendix. Participants werepaid $ $ $ N M
55 104.77 (10.99)
U M
71 106.44 (9.46) LC
79 103.87 (9.91) DC
81 104.87 (8.39) EC
71 103.50 (8.38)Table 2: Average number of attacks prevented by subjects in each group. The lowerattacks suggest participants deviated more from the optimal decision responding tosocial influence.
Table 2 shows the distribution of attacks prevented by subjects in each group. Weobserve that, on average subjects in the EC and LC groups prevent less attackscompared to others. However based on 2 sampled t-test, we did not find anystatistically significant differences between the groups based on the means of thedistributions. Based on a survey analysis, we found that none of the traits likecomputer anxiety, computer confidence, computer liking, intuition or neuroticism werecorrelated to the number of attacks prevented in all groups. The details of the surveyanalysis is presented in Appendix A in S1 Appendix.June 8, 2020 11/41ig 5: P ( d i ) - proportion of users in each group who were administered technology ordecision d i as the influence decision . Note that only the decisions that are not optimalare sent as prospective influence decisions/social signals in the second phase of theexperiment. As a first step towards investigating hypothesis H
1, we analyze the kinds of socialsignals or the cyber security technologies (which were not the optimal technology)chosen by the peers of each participant and whether they are uniform across all thegroups. To simplify nomenclature hereon, we denote the 6 available technology choicesas decision d i , i ∈ [1 , d as the influence decision - it is the optimal choice preventing 7attacks, while the rest of the technologies prevented 5 attacks and are being termed assub-optimal choices. As mentioned in Section 4.1, the influence decision C u ∈ [ d , d ]is randomized for each u , so we first observe the distribution of the sub-optimaldecisions as the choices for C u . For each group, we define P ( d i ) = |{ u | C u = d i }|| u | as theproportion of users in the group who were administered technology or decision d i , i ∈ [2 ,
6] as the influence decision in the second phase. From Fig 5, we observe that therandom selection of C u introduces some disproportionate values of P ( d i ) among thegroups. For UM group, around 25% of users were administered d as the influencedecision C u (this proportion P ( d ) for UM is the highest among all other decisions)while 29% of users in the EC group were sent d ( P ( d ) being the highest for EC) astheir infuence decisions C u and 28% of users in the DC were sent d ( P ( d ) being thehighest for DC). However, we see that for LC group, P ( d i ) was similar for all decisions d i that could be selected as C u .Having observed that there was not one pre-programmed peer choice C u as thestrategical obvious sub-optimal choice across all groups, we proceed with investigating H
1. In order to detect any implicit occurrence of a selection bias over the participants,we analyze whether there is any significant difference in the groups with respect tochoices made in the first 12 time steps. To accomplish that, we plot the probabilitythat an individual makes each decision when aggregated over the first 12 time steps.We find that there is clearly no evidence of differences in the mean statistics of thedistributions of all choices between the treatments groups (LC, DC, EC) and thecontrol groups (UM, NM) (Refer to Fig 1 in Appendix B in S1 Appendix). Before weconduct pairwise t -tests to check for differences between the control and treatmentgroups for the decision distributions made by the participants in the second phase, weconducted three one-way ANOVA tests considering the sample means of the twocontrol groups and each treatment group, one at a time. We conducted these 3 testsfor each decision from decision 1 to decision 6. The null hypothesis for each testconstituted the situation where the means of the number of times a decision waschosen by the participants belonging to the 2 control groups and one of the treatmentJune 8, 2020 12/41 a)(b) Fig 6: Probability of decisions made in time steps 13 to 18. (a) Probability of makingthe optimal decision. (b) Probability of making the sub-optimal (other 5) decision. Theerror bars denote standard error over the distributions.groups, are the same for the technology or decision in consideration. We find thefollowing significant results: for the LC group, we find that for d , the null hypothesisis rejected ( F (2 , . p = . d as well ( F (2 , . p = .
04) and for the EC group, we finddifferences approaching statistical significance for d ( F (2 , . p = . p -valuesfor each treatment group with respect to the control groups) in Tables 4, 5 and 6 inAppendix C in S1 Appendix suggest no significant difference in the distributionsamong the groups. This rules out any bias among the participants themselves in theabsence of externalities. However, in the second phase of the experiment ( time steps13 to 18 aggregated), we find differences in the selection patterns among thedecision-makers in their respective groups. We find the following observations fromFigs 6(a) and (b) for our treatment groups (ssee Tables in Appendix C in S1 Appendix): 1. LC : With respect to the NM group, there are no statistically significantdifferences in the decisions taken by the participants in LC - we carried out asimilar statistical test comparing the group pairwise means as done for the firstJune 8, 2020 13/412 steps. On the other hand, we find that there is a statistically significantdifference for the LC group participants ( M = 3 , SD = 2 .
4) in the means of thedistributions compared to the UM group participants ( M = 3 . , SD = 2 .
53) forthe optimal decision or d ( t (149) = 1 . p = .
04) at α = 0 .
05. The differenceshows that a significantly less number of participants are tempted to choose theoptimal technology provider in the presence of linear cascading signal patternthan when a single signal is sent across all time steps.2. DC : When considering the number of times participants choose d , we find thatthe participants in the DC group ( M = 0 . , SD = 1 . M = 0 . , SD = 0 .
61) and this difference is statistically significant( t (155) = − . p = . EC : When we considering the choice of d , the technology that was administeredto majority of the EC group participants, the users in this group( M = . , S = 1 .
08) differ from the NM group ( M = 0 . , SD = 0 .
66) in itsselection and the difference is significant ( t (128) = − . p = . d (the optimal) at each time step similar to what has been shown inFig 7. However, we do not find any clear distinctions among the groups in terms of thefraction of subjects who move away from optimal decision when aggregated over allthe 6 time steps in the second phase - just the fact that all users eventually move awayfrom the optimal decision when exposed to social signals does not contribute much indistinguishing the PoI. This first analysis of H1 does not shed any light on the temporal variations in thedecision making process exhibited by users in different groups - it shows somestatistically significant differences in the choices made for specific decisions (ortechnology providers) over the entire second phase. While it did show that not allcascade patterns successfully influenced users towards deviating from “decision1” orthe optimal provider, it brings up the question that is posed for hypothesis H
2: whatconstitutes successful influence and if so, does the manner in which the signals are sentdetermine successful influence?To this end, we analyze how the proportion of users who switch to the influencedecision ignoring the optimal choice, evolves over the times steps. We note that foreach user u , the influence decision C u is randomly selected before the second phasestarts. Fig 7 shows the fraction of users in each group adopting the influence decisionJune 8, 2020 14/41ig 7: Fraction of users in each group adopting the influence decision chosen by theirpeers.from time steps 13 to 18, when the experimental participants observe their peers’decisions. Following from the observations regarding H
1, we find that the probabilityof successful influence for participants in EC has the strongest effects ondecision-making in the early stages of the second phase. Participants in EC are mostlikely to deviate from selecting the optimal provider as shown in Fig 6 - for the first 3time steps in the second phase, participants in EC group exhibit the maximumadoption compared to other groups denoted by higher fraction of adoptedparticipants. Additionally, for the EC group. time step 15 had the maximum retentionwhere 35% of users adopted their peer behavior, and when participants were exposedto all their 6 peers selecting the same technology C u - this may be due to new usersadopting the influence decision or due to cumulative build-up from previous timesteps who do not switch back. We will explore this in the following sections.The participants in the DC group exhibit successful response towards the suddenincrease in signals at time step 16 whic is shown by a 65% increase in user adoption ofthe influence decision compared to the previous step. These results become close tothe 25% of users making influence decision selection at time step 16 and which is alsothe maximum among all groups surpassing the early cascade adoption ratio. However,there is no substantial increase in the adoption fraction for the users in the linearcascade group - we do note that the adoption peaks at time step 15 for the linearcascade users before it drops again. These observations from Fig 6 and Fig 7 suggestthat while an early burst of social signals successfully persuades users in EC to adoptthe influence decision, causing an aggregated overall maximum selection of theinfluence decision, a sudden impulse in the quantity of social influence also successfullysteers users towards the influence decision in the later stages. In this section, we investigate whether the quantity of signals alone stand out as thesole factor of influence. Before going into the analysis, we define a few notations: wedenote a subject in this study as u where u can belong to any group. We denote thedecision taken by a subject u at time t , t ∈ [1 ,
18] by D u ( t ). We define T treat as thesequence of time points during which the subjects receive signals from their peers i.e. T treat = [13 , t , t ∈ T treat when anindividual u first switches to C u as t fu . For each signal quantity s , we measure theproportion of individuals (in each group) who made their first switch to theirinfluence decision only after they were exposed to s signals. Formally, for any t ∈ T treat , R ( s ) = |{ u | D u ( t )= C u (cid:86) A u ( t )= s (cid:86) t = t fu }||{ u }| ( R (1) in LC group denotes theproportion of individuals in LC group who made their first switch to their peerJune 8, 2020 15/41 a) (b) Fig 8: Plots of adoption under the influence of s exposures. (a) For each signal quantity s , proportion of individuals who made their first switch to their influence decision afterbeing exposed to s signals, (b) Proportion of individuals who adopt their influencedecision after being exposed to signals from s influence signals.influence decision after being exposed to just 1 signal. Similarly, R (6) in EC groupdenotes the proportion of individuals in EC group who made their first switch to theirpeer influence decision only after being exposed to 6 signals, so this can happen in anyof the last 4 time steps in EC). The denominator in the formula here denotes thenumber of individuals in the group.We bin the values from R ( s ) based on s and take the mean for each group, sincefor some groups, there can be multiple time steps with the same number of exposuresor signals. From Fig 8(a), we observe that for EC group, ∼
30% of users under theinfluence of 4 signalers, for DC, ∼
18% of users at 4 signalers and for LC, ∼
15% ofusers at 3 signalers (all these being the maximum ratio) made their first switch to theinfluence decisions in the second phase of the experiment. However, on closeobservation, we find that the number of exposures alone does not explain the adoptionbehavior. When we compare the EC participants with those in DC group, we find thatwith 4 exposures in T treat (at time step 13 for EC and at time step 16 for DC), theproportion of adopters in EC making their first influence decision switch (30% ofusers) is higher than the proportion in DC (18% of users), although the same 4quantity of signals are delivered at different time steps for the 2 groups. However,while there is a constant decrease in the number of adopters making first switches inthe EC group going from 4 to 6 exposures denoted by the corresponding mean R ( s ),we see that the same quantity for the DC group does not decrease the same way for 4to 5 to 6 exposures. This suggests that the sudden stimulus from the delayed exposuresomehow succeeds in influencing more users to make their first switch to the influencedecision compared to the EC group (note that all the users under the quantity R ( s )for each s are unique since we measure their first switch). On the other hand, for thelinear cascade group we do not find one quantity that is most effective in the influence- in the LC setup, there is no one exposure that impacts the adoption behavior themost in terms of successful influence. In addition, we measure the cumulative adoptionratio for each group defined as: for any t ∈ T treat , Z ( s ) = |{ u | D u ( t )= C u (cid:86) A u ( t )= s }||{ u }| .In simple terms, it measures the number of individuals in a group who adopt theinfluence decision under s exposures irrespective of whether it is the first switch. Thisis demonstrated in Fig 8(b). When we combine the results obtained in Fig 8(a) withFig 8(b), we find an interesting observation for the LC group. The linear cascadepattern is able to retain most of the users even after first switch at time step 13 as theJune 8, 2020 16/41 inear Cascade Delayed Cascade Early Cascade Time Signals Inter. Time Signals Inter. Time Signals Inter.
M0-FE
M1-RE
M2-RE
M3-RE -0.02(.27) 0.29(.23) -1.87(.96) -0.37(.029) 0.44(.074) -0.23(.025) -0.23(0.12) 0.29(.25) -0.28(.42)
Table 3: Results of Fixed Effects (FE) and Random Effects (RE) modeling. “Inter.”denotes intercept in the regression models. Values in brackets denote standard errors.cumulative ratio increases up to 3 exposures (which occurs at time step 15). Thissuggests that the LC pattern is effective in terms of retention in the early stages of thecascade.
To understand the patterns of change over time by incorporating the heterogeneityamong individuals in each group and among the groups, we resort to the widely usedstatistical tool of growth modeling through random coefficient models [39]. Briefly, thetechnique of growth modeling allows us to test the longitudinal effects of the peersignals on the users and test any source of heterogeneity in decision making amongindividuals that lead to the observed outcomes. One of the advantages of growthmodeling comes from treating time as a predictor of influence outcome in the absenceand presence of the peer signals. The error-covariance matrix from these modelsinform us about the variations among the individuals in the presence and absence ofpeer signals over the phases of the experiment.We utilize 4 regression models where the outcome of interest denotes whether anindividual selected the influence decision C u ( t ) at time t in the second phase of theexperiment. Let the linear predictor, η be the combination of the fixed and randomeffects excluding the residuals. Considering a linear mixed effects model LMEM y = X β + U χ + (cid:15) , where y ∈ R k , (and k = n ∗ [ t ]) denotes the outcome response ofindividuals over all time points t ∈ [1 , [ t ]] - k thus denotes the number of instances inthe model. In this work, we have [ t ]= [ T treat ]=6. X ∈ R k × f denotes the design matrixof endogenous variables, f being the number of predictors, U ∈ R k × w denotes thematrix with random effects (the random complement to the fixed X ), w being thenumber of predictors with random effects , (cid:15) denotes the residuals, and β and χ arethe parameters of interest corresponding to the fixed effects and the random effects inthe model. Since the outcome in our work is binary, we use Generalized Linear MixedEffects models (GLMEM) [40] in this work. In a GLMEM model, we use the sameequation as LMEM but with a linear predictor η such that η = X β + U χ + (cid:15) where g ( E [ y ]) = η , g ( . ) being the link function. To accommodate for the binary outcomes,we use the logit link function. To quantify the effects using growth modeling, for eachindividual, we use the following regression models for modeling the growth functions:June 8, 2020 17/41 − FE : η i = [ β + β ( T ime i )] + (cid:15) i (1) M1 − RE : η ij = [ β + β ( T ime ij ] + χ j + (cid:15) ij (2) M2 − RE : η ij = [ β + β ( T ime ij ] + [ χ j + χ j ( T ime ij )] + (cid:15) ij (3) M3 − RE : η ij = [ β + β ( T ime i ) + β ( Signals i )] + [ χ j + χ j ( T ime i )] + (cid:15) ij (4)where the indices i denote the observation instance number (or the row number ina table of data) and j denotes the individual in the group of all users. So, M0-FE represents the fixed effects model where the only independent variable is the time.Intuitively, this model just tests how individual responses to peer signals evolve overtime, when time itself is the only factor in consideration. From the results in Table 3,we find that among the 3 treatment groups, the probability of outcome is mostpositively correlated with time for the Delayed Cascade group. This simple regressionmodel ignores the fact that the observations are nested within individuals andaccordingly, the next step in growth modeling is to add a component of randomintercept to the model - this is denoted by
M1-RE . Note that in this model, therandom intercept χ j is specific to each individual j . On comparing the parameterestimates, we find that the time coefficient β remains similar for both models evenafter incorporating between-person differences for all the groups. This shows that timeis significantly correlated to the outcome in the absence of the knowledge of peersignal treatment.Next, we add the “Time” variable to the random components, so that time canrandomly vary among the users. This is denoted by the model M2-RE given byEquation 3. The correlation between the slope and intercept for this model for the LC,DC and EC groups are respectively 0.7, 0.7 and 0.4 respectively. The positivecorrelation indicates that individuals whose have a high propensity to move towardsthe influence decision in the beginning tend to have strong slopes - this weaklysuggests that individuals with some degree of uncertainty towards optimal decision atthe beginning tend to be more susceptible to influence over time. Following this, wenow explore the idea that “Signals” have a role to play on the outcome of theindividuals. We add to M2-RE, the fixed effects from Signals shown in
M3-FE . Fromthe results in Table 3, we find that for all the 3 groups, the number of signals ispositively correlated with the outcome while the time factor is now being negativelycorrelated for the groups. Among all the groups we find that for the DC group, theSignals factor has the highest coefficient suggesting the strong correlation between thetreatment with signals and the outcome. These results suggest the positive correlationof the treatment of signals on the outcome when time has an important role to play -the interaction between time and signals are important here given that the correlationof time changes once the effect of signals is considered.
We end this study with a retrospective analysis to understand the dynamics ofadoption under a slightly relaxed setting. In real-world networks, not all individualswould be susceptible to social change emanating from their neighborhood, some peoplehave stronger beliefs than others [41]. In an attempt to quantify the effect of theinfluence on subjects in a more constrained setting, we consider only those users u who have been influenced to adopt the influence decision C u at least once betweentime steps 13 to 18 in the second phase. We measure at every time step, the ratio ofindividuals who adopted the influence decision at that time step to the number ofindividuals in the group they belong to, who switched to their influence decision atleast once within their lifecycle ( T treat ). Note that this is different from previousJune 8, 2020 18/41ig 9: Success Ratio.measures in 2 ways: first we retrospectively filter out users who never adopted theirinfluence decision (in the real world these are users who would not be susceptible toinfluence or are immune as such). Second, we analyze this ratio at the end of theirexploration phase, in time step 18, when everybody have supposedly settled down. Wedefine a symbol N ( u ) as the number of time steps for which a user u adopts C u in T treat (this is measured retrospectively aggregating all time steps beforehand).Formally it is defined as: Success ratio ( t ) = |{ u | D u ( t )= C u }||{ u | N ( u ) ≥ }| . The denominator denotesthe number of individuals who have adopted the influence decision at least once fromtime steps 13 to 18. The comparison shown in Fig 9 among the four groups (the Nomessage group does not have any influence decision) demonstrates that while the ECgroup adopts the influence decision more quickly than other groups, the stimulus insignals quantity at time step 16 in DC group affected the participants. This isconfirmed when the effects of DC strongly outstrip those observed from EC in the lasttime step where both groups receive 6 signals. At the end of the game, at time step18, we find that the highest number of such susceptible individuals come from the DCgroup - these results reinforce some of the conclusions we had from Section 5.3 andFig 7 regarding the late retention capabilities of the DC group strategy - however, thismeasure makes the differences clearer when we consider individuals who are moreprone to social influence. In this second study, we extend our work on understanding the behavioral aspect ofresponses to social influence in sub-optimal choice diffusion settings to real worldscenarios where the distinction between the utilities among choices may not beoutright evident. Additionally, as mentioned in the introduction, it is generally noteasy to observe these patterns of influence at scale and also in networked settings inthe real world which makes it difficult to study the effects it might have on behaviordiffusion. The opaque nature of the effect of exposures in the real world responsiblefor influence makes it more difficult to analyze the characteristics of these PoI - thefact that the data about who-exposed-whom in real world information cascades israrely available makes studying peer influence more challenging [16]. To this end, wetry to bridge the gap between experimental hypothesis and real world scenarios bytrying to model the behavior of agents or users when subject to peer influences and bycapturing the sequential nature and bursts in influences towards diffusion. We takethe case of rumor diffusion when the piece of information that propagates as a rumorturns out to be false. In such cases, the action of resharing by individuals issub-optimal from the perspective of information sharing. In such situations, socialinfluence plays an important role in persuading individuals with benign intent towardsJune 8, 2020 19/41 a) Single agent behavior model (b) Multi-agent behavior simulation
Fig 10: Agent based models for measuring social influence with multiple choices. (a)Single agent behavior model, (b) Multi-agent behavior simulationresharing when in fact these individuals might otherwise be reluctant to participate.We observe the trade-off between individual decisions and the influence decisionthrough the cascading effect from peers in such environments where resharing amessage would be the wrong choice.While the growth models in Section 5.5 provided us with evidence about theimportance of signals across time for the 3 treatment groups, the analysis fromSection 5.4 showed that the compounding effect of influence can lead to differentprobability outcomes. Specifically, we find that the number of signals as the proxy forinfluence can have different effects when it comes to users adopting the influencedecision - we observe the values of R ( s ) being different among the LC and the ECgroups for the same s signals despite them being administered at the same steps of thegame. This non-markovian nature of influence calls for developing models that notonly take the effect of the magnitude of peer signals into account but its effect relativeto what the user has been exposed to so far across the time steps thus far. However,observing such influence patterns can be non-trivial when it comes to mapping andfiltering these chains of patterns in real world cascades. The challenge is exacerbatedwhen we try to weigh the users’ private information against the factor of influence - wedo not observe these private cognitive factors in real world data which makes it moredifficult to understand the real world implications of our experimental conclusions.To this end, we define our models of influence where in we take into account theimpact of sequential exposures and perform simulations of the spread of adoptionusing an influence based multi-agent model on real world data. This would also helpus measure the extent to which the observations from our controlled setup can bereplicated in real world cases through simulations. Our agent based model (ABM)differs from the traditional models of diffusion in that (1) agents as influencers arehomogeneous unlike in traditional models, where each influencer has its owncontribution towards the group influence function and therefore in our case, pairwiseinfluences between users and their peers are similar for all peers and (2) behaviordiffusion in the simulation is directed towards sub-optimal decision making where thechoice of following peers or the influence decision may not be the optimal decision.June 8, 2020 20/41 .1 Simulation setup for ABM There are two ways in which we can model agent behavior to simulate behaviordiffusion - the single agent behavioral model which predicts individual behavior whenindividuals are observed in isolation and the multi agent behavioral model thatextends the single agent behavioral model to a population level by executing thesimulation in a multi-agent environment. In such scenarios, since the agents influenceeach other through their own actions over a period of time, the behavior of individualscan no longer be measured without taking the environmental factors into account.Fig 10(a) shows the agent based model for a single agent where an agent makes adecision based on its utilities at every time step. Once the utilities have beendetermined, the agent picks a decision corresponding to the maximum utility. Oncethe agent has acted, the environment is updated and external factors that might alsoimpact the agent’s decision in the next step is accounted for. Prior to the next timestep, the utilities of the choices are updated based on the previous decision. Howeverin this lifecycle of the decision making process, the agent is observed in isolation.In our work, we use this single-agent behavior model as a generative element tosimulate agent behavior but we observe the behavior at a population level especiallywhen the agents influence each other by making actions and by virtue of being in anetworked environment. In order to simulate and evaluate the effect of social influencethrough multiple exposures in the real world, we choose a specific real world casestudy to implement the environment. As shown in Fig 10(b) which represents themulti-agent behavioral model lifecycle, agent behavior is simulated using the singleagent behavior model in Fig 10(a) with specific input parameters for the agent whichwould be learnt from the empirical data. We run the ABM and evaluate the resultsfrom the ABM using different training and test splits but from the same distributionof the empirical data. The agent specific parameters are learnt prior to the start of thesimulation iterations since we map the agents to real world users. Although we do notuse feedback from the evaluation obtained from one run of the simulation from thereal world to optimize the simulation environment for future runs, this step can beadditionally performed to obtain monotonically increasing evaluation performance. Inour environment, each agent has a choice to reshare a piece of information in thesituation where resharing is not the optimal choice. We now describe the agents andtheir choices in details in the following subsections.
In our multi-agent simulation environment, the agents and the users in the socialnetwork of the real world are one-one mappings and so by default the agents areembodied in a networked environment where they can now share and be exposed toothers’ messages. We model a social network as a directed graph N = ( V, E ), where anode v ∈ V represents and individual and the edge ( u, v ) ∈ E exists if u follows v , inour case v influences u ’s decision. So in the context of this social network, our agentsare the nodes in these networks. Each agent can play both the role of a neighborinfluencing a user or a susceptible user who is being influenced. For each agent u , asmentioned before, we consider the set of agent’s peers to be homogeneous with respectto the influence they exert on u . In the context of the ABM, we separately model theprobability that each agent is being influenced based on factors that we are going todiscuss in the next section.June 8, 2020 21/41 .3 Modeling agent behavior towards sub-optimal decisionmaking Instead of considering multiple choices for the agent decision stages, we consider abinary choice model where at each time step, each agent has to make a selectionamong two choices which are reciprocal to each other, and the selection is based off onthe choice that comes with the higher probability of activation. In the real world casestudy used for the simulation, the action of resharing a piece of information is asub-optimal choice and so the peers of an agent who reshared the same piece ofinformation, are going to exert influence towards sub-optimal decision making.Also, note that this is a simpler version of the online controlled experimental setupwhere the user had 5 choices and only one of them was optimal and our setup can beextended to include multiple choices for a relevant real world case study. Followingthis, the agent can only be in two states based on the choice it makes, that is it haseither reshared a piece of information identifying the cascade or it has not.Additionally, as in most real world adoption scenarios with a binary choice model, theuser cannot transition back to the state it arrived from. Before delving into thetechnical details of the components for the agent utilities, agent states and theexposure effect based ABM, we describe in details how we quantify the individualdecision factor and the peer influence through exposures.
The agent in our model starts with being agnostic about whether the choice ofresharing a message or the activation choice is the optimal choice, however it is in theinterest of a rational actor to make a choice that is optimal in the real world, yetconform to the general choices made by the population. So in the absence of anyexternal factors, our model posits that the probability that agent u makes asub-optimal decision is given by: p u ( t ) = 11 + exp ( − ( ζ u ( t ) − µ u ( t ))) (5)where µ u ( t ) denotes the negative utility from the agent u ’s own decision to selectthe optimal choice at time t using its acquired knowledge of the real world event (thisquantity represents the utilities from the knowledge acquired by users in the firstphase of the controlled experiment) i.e. it represents the cost from disagreeing withthe dominant influence decision which is not optimal, ζ u ( t ) denotes the utility thatcomes from selecting the peer choice C u . An important point to note here is thatwhile µ u ( t ) denotes the loss from selecting the optimal decision, the utility ζ u ( t )denotes the utility from adopting the influence decision and we incorporate that in thefunction definition described later. So p u ( t ) here denotes the probability of selectingthe sub-optimal decision which is key to the way we handle the simulation later.We note that unlike other propagation models [42–44], we do not consider externaleffects for the propagation like infectiousness parameter of the cascade, time of the dayas those factors can be added to our ABM model as well the baselines we use tomeasure the effect of exposures as a proxy for influence. The goal of this ABM modelbased simulation of spread is two fold: (1) we measure the extent to which thespreading on real world networks based on the exposure effect from our model differsfrom that of the Bass model and which was the basis of the controlled experimentin [17] which considers only the number of active individuals (using the networkstructure to diffuse signals) at a time as a measure of influence, and (2) how close theresults obtained from the spreading simulation are to real world diffusion. We repeatthat any other external factors that influence adoption can be added to our model toJune 8, 2020 22/41mprove the fit to real world data - however we test exposure rates for social influenceas opposed to just active neighbors, exclusively through this ABM. We next go on todescribe several models that define these two probability measures in Equation 5. Most of the diffusion models that measure the impact of behavior are somewhatmechanical and not strategic, meaning that the probability that an agent adopts aspecific behavior is proportional to the infection rate of her/his peers. However, thecontrolled experiments show that social influence based exclusively on the quantity ofpeer signals at any time does not always determine the outcome and that individualchoices of rational agents can determine the diffusion process quite significantly.Additionally, each agent wants to maximize her/his utility through intentionallyselecting behaviors. Intuitively, considering the stochastic and non-stationary natureof human decision, it is essential to accommodate uncertainty when users infer utilityfrom interactions. We achieve this individual decision component through thefollowing latent variable. Let the utilities associated with the choice of making theoptimal decision be given by a latent unobserved variable x u ( t ) that determines theindividual utility that drives user u ’s decision making at time t . Since in most realworld studies, there is no straightforward way to determine the individual intentionsbehind resharing a message, we use this variable to capture it. In the context ofsub-optimal decision making, this utility from an agent’s standpoint towards optimalchoices now counts as a loss towards the net utility for making sub-optimal decisions.We do note that as mentioned in several existing studies [45], there can be severalother factors like the infectiousness of the current event, user’s other intrinsic factorsthat contribute to the utility - we repeat that all these factors can be incorporated tomake the model more realistic. The utility that a user gets from making the optimaldecision is then µ ( t ) = x u ( t ) + (cid:15) (6)Here (cid:15) is an iid random variable drawn from some generating distribution thataccounts for the uncertainty in the behavior of individuals beyond their own utility fora decision. As will be mentioned later, we start the simulation after already observingthe initial set of reshares for a cascade. Following this in this work, we consider thatthe utility a user gains from selecting the optimal choice as per its own knowledge isconstant over time after the initial stage. Social influence phenomenon arising out of individual interactions is measured throughpairwise influences that result in a complex contagion. The basic assumption is thatthe probability of an agent being activated is dependent on the heterogeneous pairwiseprobabilities between the agent and its peers. In the simplest case for a specificindividual, when we just measure the number of peers who have already adopted aparticular message as a measure of social influence [46], the probability p (cid:48) u ( t ) that u isactivated at time t is given by: p (cid:48) u ( t ) = 11 + exp ( − [ η u A u ( t ) + β u ]) (7)where η u , β u are coefficients to be estimated. Equivalently, ζ u ( t ) = ln (cid:16) p (cid:48) u ( t )1 − p (cid:48) u ( t ) (cid:17) = η u A u ( t ) + β u (8)June 8, 2020 23/41here coefficient η u measures the social influence or social correlation effect for u .Intuitively, the right hand side of Equation 8 denotes the utility of the agent u obtained from adopting the influence decision in situations where resharing is not theoptimal choice.One of the key observations from the controlled experiments shown in Fig 8(a) and(b) is that the slow compounding effect on behavior outcome from linear influencecascades may not be the best in terms of the desired outcome at all time steps. Weobserve that the sudden spike in signals at time step 4 for the delayed cascadeparticipants (when both LC and DC participants had 4 peer exposures) allows formore users in DC to respond to social influence compared to the LC group. Weintroduce a scaling factor for the quantity of peer signals that capture this spikingeffect. To this end, instead of using the number of exposures directly as the peereffect, as an alternative we substitute A u ( t ) with the following A u ( t ) = A u ( t ) .e σ ( A u ( t ) − A u ( t − − (9)The intuition behind the augmented exposures is that the sudden spike makes anamplifying effect on the social influence measure and so should be accounted for. Here σ is the parameter that controls the amplifying exponential curve. Note that for thelinear cascade pattern, where there is a single increase in the peer exposure at t withrespect to t − σ is heldconstant for all agents. Using the above two components, we arrive at two models of activation based onEquation 5 and we use these 2 models to run our simulation procedure:1.
Base model (BM) : We use Equations 6 and 7 to arrive at the followingprobability of activation p u ( t ) = 11 + exp (cid:16) − (cid:104) ( η u A u ( t ) + β u ) − ( x u ( t ) + (cid:15) ) (cid:105)(cid:17) (10)2. Augmented Exposure model (AEM) : We use Equations 6 and 9 to arrive atthe following probability of activation with the augmented peer exposures p u ( t ) = 11 + exp (cid:16) − (cid:104) ( η u ( A u ( t ) .e σ ( A u ( t ) − A u ( t − − ) + β u ) − ( x u ( t ) + (cid:15) ) (cid:105)(cid:17) (11)The above two probabilities represent the situation when the agent decides toreshare the message after weighing the utilities from the two components. Since weadopt a binary choice model, we do not explicitly model the utilities of the otherchoice which is to not reshare the message. The probability of an agent not sharingthe message is then just 1 − p u ( t ). With the models of activation stated as above, we now describe how we set theparameters of the model in the simulation procedure. We specifically work withinformation cascades representative of real world diffusion [47] as will be describedlater while discussing the dataset. Since we consider rumor diffusion as the real worldstudy and we calibrate the parameters of the models to this dataset by splitting it intoJune 8, 2020 24/41raining and evaluation sets as is prevalent in machine learning setups. Specifically, westart the behavior diffusion simulation of the agents after observing part of thediffusion cascades till T thresh . That is, we first observe the cascades from theirbeginning to a specific time span T thresh for learning and leave the rest of each cascadeafter T thresh to be used for evaluation of the ABM. This helps us in 2 ways: first, itallows us to perform simulation with the assumption that the agents had the time toform some opinion of their own using the exploration strategy as setup in thecontrolled experiments. Second, it allows us to learn the parameters specific to eachagent in a data-driven way prior to start of simulation and allows us to performevaluation of the ABM based diffusion process after T thresh .In our work, we treat the latent utility factor x u ( t ) as a parameter of interest andwe consider that the agent’s individual decision utility is fixed. So for our work, weconsider x u ( t ) = x u for all time steps t ∈ [1 , T ]. Specifically, the parameters of interestspecific to agents are θ u = { x u , η u , β u } and the parameter σ which we set to aconstant during our evaluation. Since it is not easy to map the controlledexperimental environment to situations in observational studies, we do not consideragent histories and so instead of learning individual agent parameters { x u , η u , β u } , foreach u , we instead divide all the agents into L latent groups. This also captures thenotion that agents in a connected network belong to a latent block structure orspecifically stochastic block models [48]. In a stochastic block model, each agent isassigned to a block and the pattern of influence between different agents depends onlyon their block assignment. Following this, the overall probability that u belongs toclass l ∈ [1 , L ] is given by p l = P [ l u = l ] with (cid:80) Ll =1 p l = 1. So all the individual agentspecific parameters { x u , η u , β u } are now replaced by { x l , η l , β l } . Let θ l = { x l , η l , β l } bethe parameters specific to the latent class l . Denoting z u ( t )=1 if the agent resharesthe message (sub-optimal decision) and 0 otherwise, Z u = { z u ( t ) } , ∀ t ∈ [1 , T ], thelikelihood contribution of u belonging to latent class l is then given by: f ( Z u ; θ l ) = (cid:89) t p u ( t ) z u ( t ) (1 − p u ( t )) (1 − z u ( t )) (12)Denoting the set of parameters Θ = [ θ , . . . , θ L ], P = [ p , . . . , p L ] and A u = { A u ( t ) } , ∀ t ∈ [1 , T ], the likelihood of the model is then given by: L (Θ , P | D, A ) = (cid:89) u ( (cid:88) l p l f ( Z u ; θ l )) (13)So our log-likelihood is: l (Θ , P ) = (cid:88) u log (cid:0) (cid:88) l p l f ( z u ( t ); θ l ) (cid:1) (14)We attempt to compute the posterior distribution of the parameters given theobservations: P (Θ l , p l | A u , Z u ) = p l f ( z u ( t ); θ l ) (cid:80) l p l f ( z u ( t ); θ l ) (15)Denoting W u = { A u , Z u } for all agents u , we first attempt to compute the posteriordistribution of p l,u = P ( l u = l | W u ), given the observations. And formally it is given by: P ( l u = l | W u ) = k u,l = P ( W u | l u = l ) P ( l u = l ) P ( W u ) = π l f ( Z u ; θ l ) (cid:80) l π l f ( Z u ; θ l ) (16)June 8, 2020 25/41he lower bound log-likelihood following Equation 14 takes the form ll = (cid:88) u log E l ∼ k u,l (cid:104) π l f ( Z u ; θ l k u,l (cid:105) ≥ (cid:88) u (cid:88) l k u,l log π l f ( Z u ; θ l ) k lu (17)Taking the derivative of ll with respect to x u and keeping other parameters fixed,we get ∇ x l ll = ∇ x l (cid:88) u (cid:88) l k u,l log π l f ( Z u ; θ l ) k lu = ∇ x l (cid:88) u (cid:88) l k u,l log π l (cid:104) (cid:81) t p u ( t ) z u ( t ) (1 − p u ( t )) (1 − z u ( t )) (cid:105) k lu = ∇ x l (cid:88) u (cid:88) l (cid:104) k u,l log π l k u,l + k u,l log (cid:104) (cid:89) t p u ( t ) z u ( t ) (1 − p u ( t )) (1 − z u ( t )) (cid:105)(cid:105) = ∇ x l (cid:88) u (cid:88) l (cid:104) k u,l log π l k u,l + k u,l (cid:88) t (cid:104) z u ( t ) log p u ( t ) + (1 − z u ( t )) log(1 − p u ( t )) (cid:105)(cid:105) = (cid:88) u (cid:88) l (cid:104) k u,l (cid:88) t (cid:104) z u ( t ) ∇ x l log p u ( t ) + (1 − z u ( t )) ∇ x l log(1 − p u ( t )) (cid:105)(cid:105) = (cid:88) u (cid:88) l (cid:104) k u,l (cid:88) t (cid:104) z u ( t ) p u ( t ) ∇ x l p u ( t ) − (cid:16) − z u ( t )1 − p u ( t ) (cid:17) ∇ x l p u ( t ) (cid:105)(cid:105) = (cid:88) u (cid:88) l (cid:104) k u,l (cid:88) t (cid:104) z u ( t ) p u ( t ) − (cid:16) − z u ( t )1 − p u ( t ) (cid:17)(cid:105) ∇ x l p u ( t ) (cid:105) (18)The derivative of p u ( t ) considering the base model BM with respect to x u keepingother parameters fixed is ∇ x l p u ( t ) = ∇ x l
11 + exp (cid:16) − (cid:104) ( η u A l ( t ) + β l ) − ( x l ( t ) + (cid:15) ) (cid:105)(cid:17) = exp (cid:16) − (cid:104) ( η l A l ( t ) + β l ) − ( x l ( t ) + (cid:15) ) (cid:105)(cid:17)(cid:16) exp (cid:16) − (cid:104) ( η l A l ( t ) + β l ) − ( x l ( t ) + (cid:15) ) (cid:105)(cid:17)(cid:17) ∇ x l ( − x l ( t ) − (cid:15) )= − exp (cid:16) − (cid:104) ( η l A l ( t ) + β l ) − ( x l ( t ) + (cid:15) ) (cid:105)(cid:17)(cid:16) exp (cid:16) − (cid:104) ( η l A l ( t ) + β l ) − ( x l ( t ) + (cid:15) ) (cid:105)(cid:17)(cid:17) (19)Similarly, the gradients with respect to other parameters η l , β l and p l can also becalculated and the parameter updates in the M step can be performed via gradientdescent using the following procedure. This is a standard finite mixture model wherethe parameters are estimated by the Expectation Maximization (EM) framework [49].The brief steps to obtain the parameter estimates are as follows: We use the Twitter dataset released publicly by authors in [50] which analyzed howpeople orient to and spread rumors in social media. As discussed in that study,June 8, 2020 26/41 harlie Hebdo Putin Missing Ferguson UnrestNetwork Nodes
Network Edges
74 11 53
Avg. in-degree • Charlie Hebdo Shooting : Two brothers forced their way into the offices ofthe French satirical weekly newspaper Charlie Hebdo in Paris, France killing 11people and wounding 11 more, on January 2015. • Ferguson unrest : The citizens of Ferguson in Michigan, USA, protested afterthe fatal shooting of an 18-year-old African American, Michael Brown, by awhite police officer on August 9, 2014. • Putin missing : Numerous rumors emerged in March 2015 when the Russianpresident Vladimir Putin did not appear in public for 10 days. He spoke on the11 th day, denying all rumors that he had been ill or was dead.Since the publicly released dataset only contained a sample of the threads ascompared to the those used in [50], we picked these 3 events which had relativelylarger proportion of rumor threads among all the events. The authors in the studycurated the annotations for the threads as to whether they were rumors or not withthe help of several journalists. We consider all the threads for the events which weretagged as rumors such that the stories related to these threads were later verified asfalse. Fact checking for these threads were performed by a group of annotators postthese events and while the entire event might have been later described as true, therewere threads related to those events that spread misinformation. The statistics of thedata pertaining to these 3 events is provided in Table 4. For each of the events, thedataset provides us with the following segregated information modules:1. Initialize the parameters Θ , P and evaluate the log likelihood of the model usingEquation 14.2. E-Step : Evaluate the posterior probabilities using the current values of Θ , P ,with Equation 16.3. M-Step : Update the parameters Θ , P with the current values of the posteriorusing the gradients obtained through maximization of the log likelihood withrespect to parameters.June 8, 2020 27/41 lgorithm 1: Simulating the diffusion of cascade q based on social influence andindividual decisions. Input: T thresh , Activated Set A (till time T thresh ), Time limit T sim , Θ , P , σ ,users V q , G , Model Type M T
Output:
Diffusion Node Set DF q [ t ], ∀ t ∈ [1 , T sim ]activated ← A ; DF q [0] ← {} ;; for t=1 to T sim do curr activated ← {} ; DF q [ t ] ← DF q [ t −
1] ; for each agent u ∈ V q \ activated do l v ← arg max l p l f ( z u ( T thresh ); θ l ) (cid:80) l p l f ( z u ( T thresh ); θ l ) ; /* Calculate individual factor µ u ( t ) */ (cid:15) ∼ N (0 ,
1) ;compute µ u ( t ) with Equation 6 using (cid:15) and x l v ;; /* Calculate utility from influence ζ u ( t ) */ if MT == BM then compute ζ u ( t ) with Equation 8 ; else compute A u ( t ) using G with Equation 9 ;compute ζ u ( t ) with Equation 8;; /* Calculate probability of activation p u ( t ) */ p u ( t ) ← exp ( − [ ζ u ( t ) − µ u ( t )] ;; if p u ( t ) > . then DF q [ t ] ← DF q [ t ] ∪ { v } ;curr activated ← curr activated ∪ { v } ;activated ← activated ∪ curr activated;return DF q
4. Evaluate the log-likelihood with the new parameter estimates. If thelog-likelihood has changed by less than some small (cid:15) , stop, else reiterate Steps 2and 3.1.
Who-follows-who network : This social network is sampled to cater to specificusers who participated through either replying to tweets or retweeting thatspecific event - this allows us to focus our simulation for each event using thisnetwork N instead of using one large social network common to all events. Ithelps in part by enabling us in the evaluation part where we compare our set ofactivated individuals at each time step to the actual users who were activated.2. Retweet cascades : In our work, we consider retweet cascades and do not includeusers who simply replied to a particular tweet since it is challenging to deducewhether a user agreed to the agenda of the cascade while replying - the notionthat induces a cascade of like minded individuals. So we restrict users in thecascade to those who only retweeted the source tweets.June 8, 2020 28/41ig 11: Degree distribution of the followee networks.So, we operationalize our simulation of behavior diffusion described in details inthe next section, for each event separately. We use the follower networks for eachevent to simulate the diffusion and use the retweet cascades to learn the agentparameters specific to each event.
We now describe the algorithm for operationalizing the simulation of behaviordiffusion based on the influence setup described in Section 6.3. As mentioned before,agents refer to users in the social network and a one-one mapping to the actualnetwork of the events from the dataset. In our work, we use the follower networksrelevant to each event, which are directed in its edges and we refer to such networks as G . The algorithm for the agent based model for social influence based diffusionprocess is described in Algorithm 1.The diffusion simulation unfolds in discrete time steps and at each step, multipleagents can change their states (initial state being the state where the user/node hasnot reshared the message) - however, once they transition from non-shared to sharedstate, they cannot switch back. We observe the users who participated in the first T thresh steps of the cascades and use that to learn the parameters, specifically thelatent class l specific parameters θ l , p l for all classes l ∈ [1 , L ]. We then use theactivated nodes (who have already reshared the rumor prior to T thresh ) along withthese learnt parameters as input to the simulation algorithm. From thereon, we runthe algorithm for a span of T sim discrete time steps. For each t ∈ [1 , T sim ], thealgorithm outputs the number of agents who reshared the rumor message at time t orwere activated at time t . In each step t , we loop through all the agents in the networkthat are yet to reshare the message. Since we projected each agent into a mixture oflatent classes L , prior to the simulation step, we need to categorize each agent into oneof the latent classes in order to compute p u ( t ) with the respective parameters of thelatent class they belong to. We observe D u ( t ), A u ( t ) of each user u till T thresh in thereal world data and then the latent class can be decided by the maximum of posteriorarg max l k u,l (Equation 16) from the data. Then the probability of activation iscomputed based on Equations 10 and 11 depending on whether the base model or theaugmented exposures model is used (the algorithm mentions the general form of theequation based on Equation 5). If this calculated probability is higher than 0.5, theagent is activated and the simulation continues for other agents for that time step.June 8, 2020 29/41 .7 Evaluation of ABM One of the specific goals we had while setting up the ABM was to compare the 2models we proposed - the base model which relies on the magnitude of peer signals asthe factor for social influence and the augmented model which allows foraccommodating the sequential changes in peer signals. To this end, we compare thediffusion trajectory of the cascades from simulations based on each model and the realworld data. We note that all the parameters of the models represented inEquations 10 and 11 are learnt from the data, except for the noise random variable η that adds uncertainty to the individual utilities. We sample (cid:15) ∼ N (0 ,
1) and followingthis, we execute 100 runs of the simulation algorithm for each cascade (thread) foreach event in the data to account for this uncertainty. For learning the parameters asmentioned before, we observe the cascades for each event till time T thresh . Since thetime span of resharing actions for each cascade is different and in the absence of anynormalization procedure that could be applied to decide on a single T thresh for allcascades, we instead observe the first 40% of each cascade (in terms of total number ofreshares in the cascade) in the chronological order of reshares. This allows us to keepthe T thresh dynamic for each cascade while allowing the rest of the cascade to be usedfor ABM evaluation.To evaluate the simulation results, for each cascade q , we consider the set of users V q ∈ V ( G ) for each network G relevant to the event, such that all users in V q reshared q in the time span of the cascade. We run each simulation round for T sim =20 steps.For each time step t in our simulation, we compute the following metric for eachcascade: | V q ∩ DF q [ t ] || V q | , the number of actual activated users which are also part of theactivated users from the simulation algorithm at time t . We call this measure the TrueDiffusion Rate of our simulation algorithm. The metric does not measure predictionresults here since there is no way in which we can precisely map a time step in oursimulation to a numeric time interval in the real world dataset. The metric allows usto measure recall over the users who reshared the rumors while allowing us to simulatethe trajectory of the diffusion over time. Figs 12 show the results of the ABMsimulation for the 2 simulations. For each event, we run the 2 models as described inSection 6.3. So we have a total of 6 models and we learn 6 sets of parameters and run100 simulations for each model that learns the parameters of the ABM.Figs 12(a), (b) and (c) show the plots corresponding to the true diffusion rate overtime and it compares the two models: the BM model where the peer influence at time t is characterized by the number of neighbors of a user who have reshared the rumormessage till t and the AEM model where the peer influence at time t is characterizedby the augmented exposures till t given in Equation 9. For all the results, we plot themean of the True Diffusion Rate at each time step taking all cascades for therespective events into account. We discuss the results from the two events CharlieHebdo and Putin missing and then Ferguson arrest events below:
Charlie Hebdo and Putin missing:
For the cascades related to both these events,the AEM model which takes into account our notion of augmented exposures, exhibitsfaster diffusion rate compared to the BM model. The results are not surprising in thisscenario since the additive factor of influence coming from the spike in neighborinformation in the AEM model augments the utility from social influence resulting infaster diffusion. Not only this, but we also observe from that for the cascades in PutinMissing events, the diffusion rate for the AEM model is an order of magnitude higherin the initial stages till 7 time steps and the AEM model reaches the saturation pointof the curve faster than the BM model.On closer analysis, we find that the network structure in this case has an importantrole to play. The degree distribution for the network used for Putin missing eventJune 8, 2020 30/41 a) Charlie Hebdo (b) Putin missing(c) Ferguson unrest
Fig 12: ABM results for the simulation using Algorithm 1 on 3 Twitter who-follows-whom network for the events (a) Charlie Hebdo, (b) Putin missing and (c) Fergusonunrestdisplayed in Fig 11 shows that there are only a few nodes with high in-degree or thenumber of potential peer signals. So the spikes in the adoption curve for the Putinmissing event happens when these few nodes with high in-degree (or higher potentialof exposure to peer siganls) are now exposed to the message from multiple peers andthey reshare the message - this happens earlier for the AEM model than the BMmodel. Consequently, this result shows that faster diffusion in real world networks canoften be attributed to the presence of a few nodes who are more susceptible toexposure, resulting in faster adoption at a population level. It shows how peer signalsin the initial stages can drive diffusion faster for the rest of the trajectory.We also observe that in both these events, the dynamics of adoption do not varymuch in that at no point does the adoption rate induced by the BM model surpass theAEM model - this also suggests that when population-level adoption is faster shownby the fact that in both cases, almost 70% of the actual users who reshared themessage were activated in our AEM model by time step 7, user uncertainty does notaccount for much and the peer signals drive the dynamics.
Ferguson arrest : For the cascades belonging to Ferguson arrest event, we find thatin contrast to the results discussed above, the AEM model results in slower diffusionthan the base model after time step 4 as shown in Fig 12(c). On closer analysis, thereasons behind this can be attributed to two key observations: (1) slow initial diffusionprior to start of simulation for cascades belonging to Ferguson arrest event : it took anaverage of T thresh =25 and 31 hours for the cascades to reach 40% of their finalaffected population for the Charlie Hebdo and the Putin missing events respectively,while it took roughly T thresh =134 hours to reach the same 40% of the final size for thecascades in the Ferguson unrest events. This also led to the models setting high valuesJune 8, 2020 31/41f µ u ( t ) from the learning procedure prior to start of simulation i.e. the useruncertainties were learned to be higher at the start of the simulation resulting inslower diffusion through the simulation given that it took 10 time steps to reach 70%of the population in contrast to 7 time steps for the other two events. (2) the slowerdiffusion led to many nodes not experiencing any increase in peer signals, resulting inlower probability of activation over time, since note that the factor e A u ( t ) − A u ( t − − could be less than 1 when A u ( t ) = A u ( t −
1) i.e. there are no increase in peerexposures which happens to be the case in this situation.Consequently, the social influence factor drops in AEM model after time step 4compared to the BM model which explains the plot Fig 12(c). This suggests that innetworked situations unlike our experimental environment, slower initial diffusion canresult in initial higher uncertainty which can eventually result in the decay of theinfluence factor later on. In such cases delayed stimulus through interventions wouldbe the only way for peer influence to play a bigger role which of course would beundesirable in such sub-optimal choice diffusion scenarios. As mentioned before, one ofthe limitations of this model lies in the setup that a user cannot transition back to astate it has explored - this limits us in measuring the retention effect concurrent towhat we tested for the Early Cascade phenomenon. However we believe that ourbinary choice model can be extended to multiple choice data given relevant casestudies - this would then allow for understanding the retention effect in real world andwhether early exposures are not an effective tool for retention over the long run thatwe concluded from our controlled experiment setup.
The primary goal of our studies was to examine how individuals’ decisions areinfluenced by the decisions of others, particularly when they are exposed to differentcascading peer influence patterns. Through a behavioral experiment and a data-drivenagent-based model, we explore how social influence can play a role in sub-optimalbehavior diffusion. Specifically, we investigated how temporal patterns of influence, byand large, affect decision-makers when the decisions have utilities. We conducted twosets of studies: in study 1, we developed a controlled experimental setup that dividedparticipants into 5 groups based on the manner in which they were treated to peersignals or the PoI over time. Our first hypothesis, that studied this effect of PoI onbehavior outcome, confirmed that an early exposure to signals commands the mostsuccess in terms of desired outcome when aggregated over all time steps of the game.However, it did not shed much light on the temporal variations between-groups indecision making. Based on a second hypothesis that attempted to study this aspect,we analyzed the influence of the quantity of peer signals across the time steps. We findthat the effect of the same quantity of signals can have a substantially different effectbased on the PoI - while early exposure to a large peer influence can decay very fastthus failing the retention effect one would hope would come from early exposures, adelayed stimulus in peer signals proves to be a successful strategy in resurgence of thedesired outcome of influence.The first study was conducted to focus on the nature of peer influence onsub-optimal choice diffusion. However, in real world applications, information diffusesthrough networked environments. the Additionally, the controlled settings implicitlydo not allow us to measure the role of individual uncertainty or private informationwhich is an additional cognitive factor that remain difficult to be measured. Tocomplement the online controlled study, we conducted a simulation on real worldTwitter networks to measure the impact of influence decisions towards rumordiffusion. We find that over a long a period of time, the influence effect from majorityJune 8, 2020 32/41f an individual’s peers are sufficiently large to persuade users to follow theirneighbors despite having to reshare a message of questionable veracity. While that isintuitive, we find that surprisingly when information diffusion is slow at the beginningof a cascade, individual uncertainty can play a substantial role as time progresses andcan itself impact this influence effect thus impacting the trajectory of adoption. Ourconclusions deviate from the existing notions of networks being the main constituentscontrolling both the probability of successful social influence and the resharing models.Our conclusions point to adversarial situations where the network organization can bemanipulated to now be used for devising successful social influence mechanisms. Thesestrategies or patterns of influence can become the confounding factors behind theoutcomes and therefore these deserve to be studied in more detail.Such conclusions can have diverse implications in the real world, where strategiesto encourage harmful decisions could be weaponized in adversarial situations. Socialinfluence is key to technology adoption, and research on the role of persuasion insecurity technology adoption indicates that various social influence factors impact auser’s decision when making decisions to purchase or use a given technology . However,these studies have primarily investigated the role of benign social influence and nothow it can be harnessed to harm users, e.g. by cyber-adversaries. Specifically, socialinfluence has primarily been studied in the context of it having a net positive impacton society, especially when considering the utility of the decisions made throughinfluence. Given the slew of recent events in which cyber warriors exploit social mediawith malicious intent, researchers and policy-makers are reconsidering the role ofsocial influence as a tool for change. Together, these simulations and the behavioralexperiment illustrate the power of multidisciplinary, complimentary work in thecomputational social sciences. Use of modeling and the principles of experimentationallow us to more holistically study the effect of social influence on decision-making.The current research thus sheds on the principled manner in which influencepatterns can be harnessed to achieve a desired outcome that could be harmful inmyriad ways. While it is evident from our growth models, that the role of themagnitude of signals on the decision outcome is significant across time, we also findthat the absolute quantity of peer signals can command different probabilities ofsuccess when disseminated using different mechanisms. Following this, we see ourresearch being extended to multiple directions: a straightforward extension would beto test these patterns at scale when the number of peers that a user is subject to islarge and so the duration of the game is also proportionally extended. A seconddirection can be extended to situations where the users also stand to lose money forcertain decisions - in the real world these could be factors like costing users’ theircredibility and reputation for the wrong choice and it remains to be seen whetherusers would still be tempted to explore or would they exploit the best option moreoften. Similarly, the agent based model can be extended to multi-armed banditsituations where specific algorithms could be devised based on regret achieved from aconvex combination of utilities derived from social influence and its own experience.
Acknowledgments . Some of the authors are supported through the Army ResearchOffice (ARO) grant W911NF-15-1-0282 and W911NF-19-1-0066. Sandia NationalLaboratories is a multimission laboratory managed and operated by NationalTechnology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary ofHoneywell International Inc., for the U.S. Department of Energy’s National NuclearSecurity Administration under contract DE-NA0003525. This paper describesobjective technical results and analysis. Any subjective views or opinions that mightbe expressed in the paper do not necessarily represent the views of the U.S.Department of Energy or the United States Government. We would like to thankJune 8, 2020 33/41lory Emmanuel-Avi˜ n a and Victoria Newton for their insights and assistance withpreliminary experiment design. References
Supporting Information
S1 Appendix
Appendix
Gender Age in years (bins)18-25 26-35 35-45 46-55 56-65 years 65 TotalFemale
11 67 38 22 8 5 151
Male
18 99 34 24 14 1 190
No Answer
Total
29 173 76 49 22 6
Gender EducationSome HighSchool(9-11 grades) High SchoolGraduate Some College(no degree) Associate’sDegreeFemale
Male
No Answer
Total
Bachelor’s Degree Some Graduate (no degree)Female
64 2 20 5
Male
83 6 27 4
No Answer
Total
153 9 49 9
There was no main effect of the signal pattern on the number of attacks prevented ( 2)or on the final amount of money participants earned [ANOVA: F(4, 341) = 1.099, p =0.357, η = 0.013]. Unexpectedly, not a single survey measure correlated significantlywith the number of attacks prevented after correcting for multiple comparisons, butJune 8, 2020 37/41 ender IncomeLess than $ $ $ $ $ $ $
19 51 41 17
Male
15 59 52 32
No Answer
Total
35 115 97 51 $ $ $ $ $
14 4 5 5
Male
No Answer
Total
24 12 18 9there were some interesting correlations between survey measures that I think areworth exploring more.For instance: • It’s interesting and somewhat expected that people who have a more competitiveapproach to interpersonal conflict might be less likely to want to be similar toothers. • IC Competing ∼ Social Influence similarity, r = -0.23, p < • Neuroticism was negatively correlated with being a rational decision-maker:neurotScore ∼ Rational r = -0.24, p < • All measures of the computer subscale were negatively correlated with desire tobe similar to others (Social Influence similarity): Computer.Anxiety ∼ Social Influence similarity r= -0.35; Computer.Confidence ∼ Social Influence similarity r=-0.4 , Computer.Liking ∼ Social Influence similarity r=-0.43; Computer.Usefulness.CASU ∼ Social Influence similarity r=-0.5, p’s < • People who were higher in self-control also prefer risk when it comes to finance.STP Self.control ∼ Risk Pref Finance, r=0.56, p < • Yet people who are intuitive decision-makers tend to prefer risk when it comes tofinance intuition ∼ Risk Pref Finance r = 0.28, p < • People who have a more dependent approach to interpersonal conflict are moresusceptible to normative social influence. • Dependent ∼ Social Influence Norm r = 0.361747405 p < • People who considered themselves more rational consider themselves morerisk-averse when it comes to finance, Rational ∼ Risk Pref Finance r = -0.23, p < • People who have a more open personality type also favor a collaborative stylewhen approaching interpersonal conflicts. openScore ∼ IC Collaborating, r =0.41, p < .3 Distributions of decisions by individuals: time steps 1 to12 (No Social Signals) (a)(b) Fig 13: Probability of decisions made in time steps 1 to 12. (a) Probability of makingthe optimal decision. (b) Probability of making the sub-optimal (other 5) decision. Theerror bars denote standard error over the distributions.In order to detect any implicit occurrence of a selection bias over the participants,we analyze whether there is any significant difference in the groups with respect tochoices made in the first 12 time steps. To accomplish that, we plot the probabilitythat an individual makes each decision when aggregated over the first 12 time steps.We find that there is clearly no evidence of differences in the mean statistics of theprobability distributions between the treatments groups(LC, DC, EC) and the controlgroups (UM, NM) shown in Fig 6. The results (p-values for each group) in Tables 5, 6and 7 in the Appendix suggest no significant difference in the distributions among thegroups. This rules out any bias among the participants themselves in the absence ofexternalities.
The tables below - one for LC, EC and DC each, show how the probability of selectingthe 6 decisions compares between each treatment group among LC, EC and DC andthe control groups UM, NM. The figures in the table show the p -values of the t -testthat compares the difference in the means of the distributions over the time steps. Wedisplay the results for time steps 1-12 and time steps 13-18 separately. The figures inJune 8, 2020 39/41rackets show the difference in the mean probabilities of the distributions between thetreatment group and the control group (control group among UM, NM correspondingto the row in consideration). p-values Decisions Time Steps (-0.13) 0.32(0.01) 0.16(0.03) (0.03) 0.75 (-0.008) 0.41(0.02)Table 5:
Linear Cascade group - The tables shows the p-values of the hypothesis testthat measures the difference in the means of the probability distributions (probabilityof choosing/adopting the influence decision) of the users in linear cascade group andthe control groups. Bold values denote significant differences considering α =0.05. p-values Decisions time steps (0.04) .91 (-0.003) .68 (-0.01)Table 6: Delayed Cascade - The tables shows the p-values of the hypothesis testthat measures the difference in the means of the probability distributions (probabilityof choosing/adopting the influence decision) of the users in delayed cascade group andthe control groups. Bold values denote significant differences considering α =0.05.June 8, 2020 40/41 -values Decisions time steps (0.07) .99(0.0001)UniformMes-sage (-0.07) .03 (0.25) .31(0.02) 0.12(0.03) .24(0.03) .69(0.01)Table 7: Early Cascade - The tables shows the p-values of the hypothesis test thatmeasures the difference in the means of the probability distributions (probability ofchoosing/adopting the influence decision) of the users in early cascade group and thecontrol groups. Bold values denote significant differences considering αα