Information-sharing and aggregation models for interacting minds
Piotr Migdał, Michał Denkiewicz, Joanna Rcaczaszek-Leonardi, Dariusz Plewczynski
IInformation-sharing and aggregation models for interacting minds
Piotr Migdał a,b , Joanna Rączaszek-Leonardi c , Michał Denkiewicz d , Dariusz Plewczynski e a Institute of Theoretical Physics, University of Warsaw, Warsaw, Poland b ICFO–Institut de Ciències Fotòniques, 08860 Castelldefels (Barcelona), Spain c Institute of Psychology, Polish Academy of Sciences, Warsaw, Poland d Department of Psychology, University of Warsaw, Warsaw, Poland e Interdisciplinary Centre for Mathematical and Computational Modelling, University of Warsaw,Pawińskiego 5a, 02-106 Warsaw, Poland
Abstract
We study mathematical models of the collaborative solving of a two-choice discriminationtask. We estimate the difference between the shared performance for a group of n observersover a single person performance. Our paper is a theoretical extension of the recent work ofBahrami et al. (2010) from a dyad (a pair) to a group of n interacting minds. We analyzeseveral models of communication, decision-making and hierarchical information-aggregation.The maximal slope of psychometric function (closely related to the percentage of rightanswers vs. easiness of the task) is a convenient parameter characterizing performance.For every model we investigated, the group performance turns out to be a product of twonumbers: a scaling factor depending of the group size and an average performance. Thescaling factor is a power function of the group size (with the exponent ranging from to ), whereas the average is arithmetic mean, quadratic mean, or maximum of the individualslopes. Moreover, voting can be almost as efficient as more elaborate communication models,given the participants have similar individual performances. Keywords: group decision making, two-alternative forced choice, decision aggregation,group information processing, discriminative judgments, accuracy, discriminationdifficulty, bias, information sharing, group size, two-choice decision, distributive cognitivesystems, communication models, cognitive process modeling
1. Introduction
Anyone who has ever taken part in group decision making or problem solving has mostlikely asked themselves at one point or another whether the process actually made any sense.Would it not be better if the most competent person in the group simply made the decision?In other words, it is an open question whether a group can ever outperform its most capable
Email addresses: [email protected] (Piotr Migdał), [email protected] (Dariusz Plewczynski)
URL: http://migdal.wikidot.com/en (Piotr Migdał), http://cognitivesystems.pl (DariuszPlewczynski)
Preprint submitted to Journal of Mathematical Psychology, 10.1016/j.jmp.2013.01.002 August 26, 2018 a r X i v : . [ phy s i c s . s o c - ph ] M a r ember. There have been many studies that have reported group decisions to be lessaccurate (Corfman and Kahn, 1995). Some studies, however, have concluded that groups— even when they merely use simple majority voting — can make better decisions thantheir individual members (Grofman, 1978; Kerr and Tindale, 2004; Hastie and Kameda,2005). We ask a more general question: how does the group performance depend uponthe individual performances of its participants and the ways in which those participantscommunicate?This question is given new light by recent trends in cognitive psychology, which after ahalf a century of fascination with isolated cognition in the individual, has finally admitted theindividual interaction with the social environment. It is increasingly understood that jointactions and joint cognition are not limited to situations of committee/voter decisions, butinstead, they pervade everyday life and require the constant coordination and integrationof cognitive and physical abilities. This new approach, typically called distributed cogni-tion (Hutchins and Lintern, 1995), or extended cognition within the social domain (Clark,2006), brings the focus of research to the mechanisms of cognitive and physical coordina-tion (Kirsh, 2006) that affect this integration. It also brings attention to the comparisonof the performance of the group to the performance of the individual. For some tasks thatrequire different types of knowledge and abilities from group participants, groups are likelyto outperform individuals (Hill, 1982). For other tasks, such as simple discrimination tasksor estimations, a question arises if a group is indeed better than the best of its members. Ifthere are such situations, it is important to know when they arise.Group decision making obviously involves members interacting with each other. Castinga vote requires a minimum amount of communication for the individual (only to informother group members about his or her choice). However, other group decisions allow forextensive communication and negotiations of the decision. Our questions are: 1) whichforms of communication are most likely to facilitate an improved outcome, and 2) what isactually being communicated in successful groups? Recent experiments by Bahrami, Olsen,Latham, Roepstorff, Rees, and Frith (2010) have shown that cooperation can be beneficial,even in simple task, and that this benefit is best explained by the participants communicatingtheir relative confidences. In their study, dyads (pairs) performed a perceptual two-choicediscrimination task. On every trial participants had to decide which of two consecutivestimuli (sets of Gabor patches) contained a patch with higher contrast. First, decisionswere collected from both persons; then, if the decisions were different, the participants wereallowed to communicate to reach a joint decision.The decision data obtained from each person was used to fit a psychometric function,i.e., the probability of that person giving a specific answer, as a function of the differenceof the contrast between Gabor patches. These functions describe the person’s skill in thetask. Similarly, a function describing the skill of the group as a whole can be estimatedfrom the group decisions. As was described by Bahrami: "In experiments (...) psychometricfunctions were constructed for each observer and for the dyad by plotting the proportion oftrials in which the oddball was seen in the second interval against the contrast difference atthe oddball location" (Bahrami et al. (2010), Supplementary Materials, p. 3).Various assumptions about the nature of within-group interactions during the joint2ecision-making process can be made. From these assumptions, we can derive theoreti-cal relationships between the parameters of members’ functions and the parameters of thegroup function. These are the models of decision making. The correctness of each jointdecision model can then be tested against empirical data.Bahrami et al. (2010) described and evaluated four such models. One was own, inwhich group members communicate their confidence in their individual choices. Anothermodel stemmed from signal detection theory (Sorkin et al., 2001). If members know eachother’s relative discriminatory ability (i.e., their psychometric functions), the group canmake a statistically optimal choice. Thus, under certain conditions, we have an upperbound on group performance. The third model suggested that the dyad is only as goodas its best member. Finally, the last model tested was a control model involving randomresponse selection. The study concluded that, when similarly skilled persons meet, they canboth benefit from cooperation. A model in which participants communicate their relativeconfidences best explains this benefit.We extend the models from Bahrami et al. (2010) to groups of n participants and comparetheir predictions. Furthermore, we add a model in which a participant either knows thecorrect answer, or guesses. Importantly, in the case of larger groups, it may be the case thatonly small subgroups of participants can communicate simultaneously. Thus, we addressthis issue by considering hierarchical schemes of decision aggregation, in which decisionsare first made by subgroups, and then some of these subgroups interact to reach a shareddecision.The paper is organized as follows. In Section 2, we present the Bahrami et al. (2010)approach to integrating individual discrimination functions in pairs of participants. We useit to assess performance in groups. In Section 3, we proceed to formulating a series of modelsof communication, which express the performance of a group of n persons as a function oftheir individual performances. In Section 4, we investigate how each model works, assumingseveral schemes of decision aggregation. Section 5 compares the introduced models andprovides insight into further experimental and theoretical work. Section 6 concludes thepaper.
2. Model of discrimination
Consider an experiment in which a participant has to make simple discriminatory deci-sions of varying difficulty. Each trial is assigned a parameter, c , that describes the physicaldistance between stimuli (e.g., in the Bahrami et al. experiment c was the difference incontrast between Gabor patches). Negative c describes a situation in which the right choiceis the first of the pair, whereas positive c describes the opposite situation. The absolutevalue of c reflects the difficulty of a given trial. The lower the value, the more difficult isthe resulting trial. From now on, we refer to the parameter describing physical difference as stimulus c . In the case of Bahrami et al. experimental setup, it can be interpreted as thetwo-interval stimulus with the difference of contrasts equal to c .By knowing the choices of a certain decision-making agent (in our case either a singleparticipant or a group making the decision together) for a range of stimuli, we can construct3 mathematical description of the agent’s performance on the task.For each agent, we can then determine his or her psychometric function: the probabilityof the agent choosing the second answer as a function of the stimulus, P ( c ) . An idealresponder would be described by the Heaviside step function: P ( c ) = 0 for all negativestimuli, and P ( c ) = 1 for all positive stimuli (i.e., choosing the second interval if and only if c > ).Because responders make errors, the actual decision rule and probability are different.One way to describe such a response is derived from signal detection theory (Sorkin et al.,2001). According to it, for a stimulus c , a participant perceives stimulus x , which is anormally distributed random variable centered around c + b and with variance σ , and decidesbasing on the sign of x . Two models described in this paper (Weighted Confidence Sharingand Direct Signal Sharing) use this mechanism explicitly. The modified realistic decisionrule of an agent states that if the observed stimuli x is negative, an agent decides to selectthe first patch (therefore interpreting the difference in contrast as negative), in the case ofpositive value, the second option is selected.In particular, psychometric curves which are cumulative of the normal distribution: P ( c ) = H (cid:0) c + bσ (cid:1) , where (1) H ( x ) = 1 √ π (cid:90) x −∞ exp (cid:0) − t / (cid:1) dt, (2)result in a good fit for the experimental data (Bahrami et al., 2010). The parameter σ canbe interpreted as the participant’s uncertainty about the decision. The parameter b is thebias (offset); it represents a tendency to choose a particular answer, see Fig. 1. The P ( c ) function, defined as above, can be viewed as a convolution of the step function (the correctanswer) and the Gaussian distribution (the discriminative error). (cid:45) (cid:45) (cid:45) (cid:45) stimulus0.20.40.60.81.0P (cid:72) c (cid:76) (cid:45) probability of choosing the second optionslope biasW Figure 1: Plot of the psychometric function, with shown slope s and positive bias b . For our purposes, we assume that bias is much smaller than the characteristic widthparameter, i.e., | b | (cid:28) σ . 4onsequently, σ becomes the main determinant of the effectiveness of discrimination. Itis convenient to choose the maximal slope of the psychometric function s = 1 √ πσ , (3)as the primary measure of the responding agent’s effectiveness.Now, we can proceed to extending the Bahrami et al. (2010) models. We would like toknow how the performance of a group of n people depends upon their individual cognitiveperformances. Therefore, we need to solve the explicit formulas for the propagation ofslopes and biases when combining several responders within each of the different models ofcommunication: s model = s model ( s , b , . . . , s n , b n ) , (4) b model = b model ( s , b , . . . , s n , b n ) . (5)Each model is described by the shared decision function P model = f [ P , . . . , P n ] , (6)where f is a functional. For all but two models that we investigate, P model ( c ) = f [ P ( c ) , . . . , P n ( c )] ,that is, the dependence is pointwise (i.e., result for a given c requires only knowing individual P i ( c ) for the same c ).We can obtain the effective slope (4) and bias (5) using straightforward formulas thatinvolve taking the derivative of the psychometric function with respect to the stimulus: s model = P (cid:48) model ( c ) | c = − b model ≈ P (cid:48) model ( c ) | c =0 (7) b model = (cid:20) b for which P model ( − b ) = 12 (cid:21) ≈ P model (0) − P (cid:48) model ( c ) | c =0 , (8)where assuming (1) the approximation for the relative error for both s and b is of order O ( s b ) (or equivalently, O ( b σ ) ), where O ( · ) stands for big O notation. The derivation is inAppendix A. Note that if P model ( c ) is a cumulative Gaussian function (as in (1)), then theformulas for slope (3) and (7) are equivalent. However, it can be used as a definition of theslope and the bias in the general case of an arbitrary communication strategy P model ( c ) , evenif (1) does not hold. Bear in mind that for practical applications, we expect P model ( c ) to beclose enough to the cumulative Gaussian function. Moreover, when there are no biases, forall decision-making considered in this paper, the maximal slope is at c = 0 .A question arises about the relation between the psychometric curve parameters and theexpected rate of errors. To assess the average amount of incorrect answers we could expectfrom a responder, we introduced the following quantity, W ( σ, b ) = (cid:90) −∞ P ( c ) dc + (cid:90) ∞ [1 − P ( c )] dc (9) = (cid:114) π σ exp (cid:16) − b σ (cid:17) + b (cid:2) H (cid:0) bσ (cid:1) − (cid:3) , (10)5here we integrated the error function (Abramowitz and Stegun, 1965). For a uniformdistribution and range of stimuli, ( − r, r ) for r (cid:29) ( σ + | b | ) , the rate of the incorrect responsesis given by W ( σ, b ) / (2 r ) . The average number of wrong answers is always reduced whenlowering either width or bias, regardless of the other parameter’s value. This fact furtherjustifies the choice of the slope as the proper effectiveness measure. When there is no bias,(10) simplifies to W ( σ,
0) = 2 /s ; thus, the rate of the incorrect responses is / ( rs ) .
3. Information-sharing models
In this section, we discuss different models of information sharing for n participants. Itis important to underline that the models incorporate the process of perceiving (what thesubjects may know), the state of mind (what the subjects know), and the communicationand the decision-making process (usually Bayes-optimal). We briefly define the assumptionsof each model and justify it in psychological terms. We give results in terms of the effectivepsychometric function, P model ( c ) , the effective slope, s model , and sometimes the effective bias, b model (as for a few models the bias is poorly defined). Whenever calculations of P model ( c ) are not straightforward, we give some insight into the underlying mathematics.We investigate the following models: • • • • • • The trial decision of a random group member is taken as the group decision.
Motivation.
Random Responder serves as one of the reference models, and it is not expectedto be fulfilled in most of realistic settings. Random factors determine the collective decision,i.e., communication is seen as ineffective within framework of this model. Sometimes thedecision is not based on any evidence and people may have very misleading impressions oftheir own accuracy. Additionally, their decisions may depend more upon a group member’scharisma or persuasive skills than his or her psychometric skills. In the work of Bahramiet al. (2010), this model is called ’Coin flip’. 6 esults. P RR ( c ) = 1 n n (cid:88) i =1 P i ( c ) (11)After the differentiation, one obtains the slope (7) and the bias (8): s RR ≈ s + . . . + s n n (12) b RR ≈ s b + . . . + s n b n s + . . . + s n (13)The relative error both for s RR and b RR is O ( s b ) + . . . + O ( s n b n ) . Note that P RR ( c ) is notnormal (1). Each participant makes her or his own decision. The majority vote determines thedecision of the group. In a case of equal votes for two outcomes, a coin is flipped.
Motivation.
People may have no access to their accuracy (or they cannot communicate itreliably); thus, a good strategy is to take voting as the final consensus result.
Results. P V ot ( c ) = (cid:98) n − (cid:99) (cid:88) k =1 (cid:88) (cid:126)i [1 − P i ( c )] · · · [1 − P i k ( c )] P i k +1 ( c ) · · · P i n ( c ) (14) + (cid:88) (cid:126)i [1 − P i ( c )] · · · (cid:104) − P i n/ ( c ) (cid:105) P i n/ ( c ) · · · P i n ( c ) if n is even , where sum over (cid:126)i denotes sum over every permutation of participants. We obtain (derivationin Appendix B) s V ot ≈ s + . . . + s n n × (cid:40) n n (cid:0) nn/ (cid:1) if n is even n n − (cid:0) ( n − n − / (cid:1) if n is odd (15) ≈ (cid:113) π × √ n × s + . . . + s n n (16) b V ot ≈ s b + . . . + s n b n s + . . . + s n (17)The P V ot ( c ) is not normal (1). The relative error both for s V ot and b V ot is O ( s b ) + . . . + O ( s n b n ) . Note that the addition of an odd member to a group does not increase its averageperformance. The formula (16) is an asymptotic expression for large n , which utilizes theWallis formula. For n = 2 , the Random Responder and Voting models yield the same results.7 .3. Best DecidesModel. The most accurate member of the group makes the decision. This model is called
Behavior and Feedback in Bahrami et al. (2010). In this model, we will focus on the casewith no bias, b = 0 . Nonzero bias would make the result difficult to state in explicit form;see (10) for further explanation. Motivation.
In some experimental settings, members of the group can determine, who is themost accurate (e.g., when feedback is present). Group members can then let that individualmake the final decision. Studies by Henry (1995) suggest that, at least in some types oftasks, participants can identify the most proficient member, so our assumption is plausible.As in the previous models, there is no (effective) communication between the members ofthe group.
Results. P BD ( c ) = P member with the highest s ( c ) (18) s BD = max( s , . . . , s n ) (19)When biases are large, the group psychometric function is that of the most effective par-ticipant (i.e., one with the lowest W ( σ i , b i ) (10)), P BD ( c ) = P i ( c ) . This strategy is mostbeneficial for a group with very diverse individual performances. Group members share their relative confidences z i = x i /σ i . The group decisiondepends on the sign of (cid:80) ni =1 z i , i.e., for the negative they choose the first option and for thepositive they choose the second. This model requires each P i ( c ) to be normal (1). Motivation.
The value x i is the stimulus perceived by i -th participant and has a distributionwith density P (cid:48) i ( c ) , as it is in Sorkin et al. (2001). We assume the confidence to be acontinuous variable. The true stimulus c is, of course, common for all participants in a giventrial. The relative confidence is equivalent to a z -score, if the participant is unbiased (i.e.,it is related to the probability that the participant is right). Put differently, participantsknow their z -scores on a given trial but are unaware of their own parameters s and b . Thismodel was first introduced by Bahrami et al. (2010). It is possible that, in an experimentaltrial, each participant can estimate and effectively communicate their relative confidences,by using a coarse real-world approximation of one’s z -score, e.g., ’I lean towards 1st’, or ’Iam almost sure it is the 2nd’ ? . The study by Bahrami et al. (2010) suggests that this modelmost accurately describes dyad performance.Given relative confidences (cid:126)z = ( z , . . . , z n ) , the group has to determine whether to choosethe first or the second option. If there are only two participants with different opinions, theone with the stronger confidence (for a given trial) decides. This can be written as follows:the group chooses the first option if z + z ≤ , the second option otherwise, yielding anoptimal strategy (Bahrami et al., 2010). In the general case of n participants, we use the8ayes optimal reasoning. We calculate the probability that the stimulus is positive (andthus the second answer is correct) given the z -scores provided by each participant: p ( c > | (cid:126)z ) = (cid:90) ∞ c =0 p ( c | (cid:126)z ) dc = (cid:82) ∞ p ( (cid:126)z | c ) p ( c ) dc (cid:82) ∞−∞ p ( (cid:126)z | c ) p ( c ) dc , (20)where p ( c ) is the probability of a discrimination task with c . The probability of observing z i -score, given stimulus c , is P (cid:48) i ( c − σ i z i ) . Thus p ( (cid:126)z | c ) = P (cid:48) ( c − x ) · . . . · P (cid:48) n ( c − x n ) . (21)Let us assume that the displayed stimulus has a uniform distribution, i.e., that p ( c ) isconstant (not going into mathematical nuances). To define the decision function, we need toknow when p ( c > | (cid:126)z ) ≥ / or, in other words, when the probability that the second answeris correct is greater than / . As (21) is a Gaussian function of c , finding its maximum leadsto the condition x σ + . . . + x n σ n ≥ , (22)or equivalently, using the slope parameter, s z + . . . + s n z n ≥ . (23)Thus, when the condition holds, choosing the second option is the Bayes optimal choice.Unfortunately, in this model we only have access to values of (cid:126)z , not to individual perfor-mances. To obtain the precise answer, we need to know the whole distribution of σ i (or s i ).Instead, we can use the approximate condition for the choice of the second option, z + . . . + z n ≥ , (24)to obtain a lower bound on the performance. The condition is exact for participants withequal performances (and should be close to the optimal if the values of σ i do not varymuch). This equation can be seen as a type of a weighted voting, where weights depend onsubjective confidences, but not on individual performances. Members do not know their own— or their peers’ — performance scores, so there is no justification for assigning more orless weight to a particular member throughout the experiment. The only thing that mattersis each member’s confidence in the present trial. Results.
To calculate P W CS ( c ) , we need to compute, given stimulus c , the probability ofobtaining set (cid:126)z with a positive sum (24). Thus P W CS ( c ) = (cid:90) x /σ + ... + x n /σ n ≥ exp (cid:20) − ( c + b − x ) σ + (25) − . . . − ( c + b n − x n ) σ n (cid:21) dx · · · dx n (2 π ) n/ σ · · · σ n = H (cid:104) √ πs W CS ( c + b W CS ) (cid:105) , (26)9here the integration is based upon the fact that a sum of Gaussian random variables z i isa Gaussian random variable (Piau, 2011). The resulting parameters are: s W CS = √ n × s + . . . + s n n , (27) b W CS = s b + . . . + s n b n s + . . . + s n . (28)Again, note that the above result for s W CS is the lower boundary value for optimalBayesian reasoning, exact only for n = 2 (due to symmetry) and a group of participantswith the same performance. By knowing the exact distribution of individual performances,we can obtain a better (or at least the same) group performance. Then, instead of thesummation of individual z -scores (24), one will get a more complicated formula for thedecision. Group members share both their perceived stimuli x i and their σ i . The group de-cision depends on the sign of (cid:80) ni =1 x i /σ i . This model requires each P i ( c ) to be normal(1). Motivation.
As for the WCS, we assume that the value x i is the stimulus perceived by i -thparticipant and has a distribution with the density P (cid:48) i ( c ) , as it is in (Sorkin et al., 2001).The group possesses complete knowledge about the characteristics of its members and theirperceptions, so its effectiveness is hindered only by the skill of the participants, not bycommunication. This model constitutes the upper bound for group performance, providedthat the stimuli are fully defined by their stimulus values (and perceived according to thediscussed model). In the case of a more complex, non-perceptive task, it is possible for agroup to exceed this bound (Hill, 1982). For example, this could occur when participants’skills complement each other. People know the strength of the stimuli but also their ownsensitivity. If the feedback is provided, we can plot x versus c to get σ . Results.
The final group decision follows the standard derivation of n classifiers collectingindependent results with normal distribution (e.g., Sorkin et al. (2001) and Bahrami et al.(2010)): P DSS ( c ) = 1 normalization (cid:90) c −∞ P (cid:48) ( x ) · . . . · P (cid:48) n ( x ) dx (29) s DSS = (cid:113) s + . . . + s n = √ n × (cid:114) s + . . . + s n n (30) b DSS = s b + . . . + s n b n s + . . . + s n (31)Note that, regardless of the distribution of the individual performances, the group perfor-mance outscores both Best Decides and Weighted Confidence Sharing.10 .6. Truth WinsModel. We assume that on each trial each member is in one of the two states: either theyknow the right answer or they are aware of their own ignorance. In the latter case, a randomguess is made. It is sufficient to have a single group member perceive the stimuli correctlyto get the correct group answer. We assume no bias, as there is no possible way to treat itconsistently and it introduces false convictions.
Motivation.
For so-called Eureka problems, the signal-theoretic limit can be exceeded (Hill,1982). The key is that the answer to such a problem has the property of demonstrability: itallows a single member who has the correct answer to easily convince the rest of the groupof its correctness (Laughlin et al., 1975). People know if they see the ’right’ stimuli (andall errors are due to guessing, not to false observations). This model has received muchattention in group decision theory, e.g., in Davis (1973). It is appropriate in situations whenthe correctness of a solution can be demonstrated. However, we do not expect this model tobe applicable to tasks similar to that of Bahrami et al. (2010). This model serves as a controland an explicit example of a result beyond one provided by the Direct Signal Sharing model.We included it with the aim of generalizing the models to different decision situations.
Results.
The probability that the responder knows with certainty the right answer is R ( c ) = | P ( c ) − | . (32)That is, we have a reversed formula saying that, when a responder knows the answer withprobability R ( c ) , the responder answers correctly with probability R ( c ) + (1 − R ( c )) / (asthere is the chance to answer correctly by a random guess). The probability that at leastone person knows the correct answer is R T W ( c ) = 1 − [1 − R ( c )] · . . . · [1 − R n ( c )] . (33)Consequently, P T W ( c ) = sign ( c ) R T W ( c ) + 12 (34) s T W = n × s + . . . + s n n , (35)where the slope is a result of straightforward differentiation (7).This model yields much better results than other models; note, however, that the absenceof false observations is a strong requirement. Other models have to operate without thisassumption. Note that the P T W ( c ) is not normal.
4. Aggregation of information in hierarchical schemes
So far, we have assumed that information from all participants is simultaneously col-lected and used in the group decision. One may argue that this is unrealistic for human11ommunication in groups of more than a few persons. We, therefore, propose hierarchicalmodels (schemes) in which only small subgroups can communicate at a particular time. Eachof these subgroups reaches its own decision, in a manner described by one of the modelsintroduced in the previous section. Hence, the subgroup can be regarded as a decision-making agent, described by a slope and a bias. The subgroup can then communicate withother subgroups or individual members, which results in larger groups being created, untilall information is gathered and the final decision is made.The results of employing a multi-level decision system can significantly deviate from whatsimultaneous information collection predicts. For instance, in a two-level voting system,which has been widely studied in the context of election results (Davis, 1973; Laughlin et al.,1975) the final outcome depends heavily upon the distribution of votes in the subgroups,sometimes allowing minority groups to overcome the majority, sometimes exaggerating thepower of the majority. It is thus interesting to study the possible effects of such hierarchicalsystems.We propose the following model for communication of n participants:1. In the beginning there are n agents.2. Each turn only g (for our purpose: or ) agents (groups or individuals) share theirinformation according to a chosen model. These agents are then merged into one agent(defined by s model ( s , . . . , s g ) ).In other words, a group of people who shared information, is treated as a single agent in thenext turn. There are two free parameters: • The model used to combine members’ parameters into group parameters. • How the groups are formed, i.e., the way to determine which agents should interact ingiven turn.Let us consider the following ways in which groups can form (see Fig. 2 for the diagramof the two first schemes): • g agents from the groups with the least number ofparticipants interact. • g − agents join to the group with largest number ofparticipants (that is, there is only one group to which each turn g − agents join). • g random agents interact.Above, by participants we understand the total number of individuals that were merged intoan agent.For some models, the way in which groups are formed is irrelevant for obvious reasons.This is the case for Random Responder, Best Decides, Direct Signal Sharing and TruthWins. The result is always the same and is equivalent to the simplest situation withoutany hierarchy. The models, which are affected to some degree, are as follows: WeightedConfidence Sharing and Voting. 12hallow Scheme Deep Scheme Figure 2: Diagram of the interaction ordering for aggregation schemes for g = 2 : Shallow Scheme — eachturn two agents from the least numerous groups interact, Deep Scheme — each turn a single participantjoins the previously formed group. Note that, in principle, agents do not know their own slopes, so the order of interactionscannot depend upon the individual (or group) s i . However, as both s W CS and s V ot dependlinearly on s i , averaging over every permutation of participants yields a result that is pro-portional to the arithmetical mean of s i , or (cid:104) s (cid:105) . Consequently, to investigate the influence ofthe hierarchical information-aggregation, it is sufficient to treat each participant as if his/herperformance is equal to (cid:104) s (cid:105) .For convenience, we consider a more general model with the parameter (the amplificationmultiplier) depending on g (the group size) as follows: s a g ( s , . . . , s g ) = a g s + . . . + s g g . (36)This generalization describes both WCS ( a = √ , a = √ , . . . ) and Voting ( a = 3 / , . . . ), and it allows us to give results in an elegant general form. Our justification for the Shallow hierarchy is the following: people may locally find theirpartners and then make a collective decision. Then, iteratively, groups of the same (orsimilar) size make the collective decision.The analysis is simple when the number of participants is a power of g , i.e., n = g k ,where k is a natural number.Then, every several elementary steps the number of agents isreduced by the factor of g , and agents’ slopes are multiplied by the factor a g . In the end,we get s a g ,Shallow,g = ( a g ) k (cid:104) s (cid:105) = n log g ( a g ) (cid:104) s (cid:105) . (37)In particular, for Weighted Confidence Sharing (i.e. a g = √ g ), we reach the saturation s W CS,Shallow,g = √ n (cid:104) s (cid:105) . (38)Thus, the aggregation process does not introduce a decrease in the group performance whenit is compared to collecting all information at once. The formula (38) holds only for n thatis a power of k . However, for different n s the formula works as a very good approximation.13ee Fig. 3 for the numerical results. The relation (i.e., that for groups of size n = g k wereach the efficiency of model without aggregation or s a g ,Shallow,g = s a g ,Shallow ) is true forevery model described by (36) with a g = g α for any α .In the Voting model we need to consider the aggregation in a group of at least three (i.e., g = 3 and a g = 3 / ). Otherwise, it is equivalent to the Random Responder model. For n being the power of three we get, s V ot,Shallow,g =3 = n log / (cid:104) s (cid:105) ≈ n . (cid:104) s (cid:105) , (39)which works as a good approximation also for the general odd n . For every even n , thereis at least one process with two parties, which significantly decreases the total performance(as voting for two participants reduces to a coin flip). In this case, there is a single group to which single agents join one after another. Theresulting slope is as for the Weighted Confidence Sharing model: s W CS,Deep,g =2 = 2 − ( n − / (cid:104) s (cid:105) + n − (cid:88) i =1 − i/ (cid:104) s (cid:105) = (cid:16) √ − − n/ (cid:17) (cid:104) s (cid:105) (40)and for the Voting model for an odd n and aggregation of three s V ot,Deep,g =3 = 2 ( n − / (cid:104) s (cid:105) + 2 ( n − / (cid:88) i =1 − i (cid:104) s (cid:105) = (cid:0) − − ( n − / (cid:1) (cid:104) s (cid:105) (41)We see that the Deep hierarchy is very inefficient. The multiplier of (cid:104) s (cid:105) converges to aconstant. This leads to the conclusion that simultaneous aggregation (i.e., Shallow hierarchy)is not only more natural but also much more efficient.To obtain the asymptotic value of s a g ,Deep,g , we can consider an equilibrium situationwherein g − individuals join the group, which has already reached the limit s a g ,Deep,g = a g (cid:18) g − g (cid:104) s (cid:105) + 1 g s a g ,Deep,g (cid:19) . (42)This leads to: s a g ,Deep,g = g − g/a g − (cid:104) s (cid:105) . (43) What happens between the Shallow hierarchy and the Deep hierarchy? If the groupsmerge at random, is the final s closer to the most efficient aggregation scheme, or to non-scaling (e.g., adding a few members at a time)? The answer, not surprisingly, lies in betweenthese two extremes. 14e parameterize time with t starting from . Each turn g agents merge into one of theslope (36). The current number of agents is described by n t = n − ( g − t . We investigatehow the distribution of slopes ρ t ( s ) evolves with time, which reads ρ t +1 ( s ) − ρ t ( s ) = (44) − g ρ t ( s ) n t + (cid:90) ρ t ( s ) n t · · · ρ t ( s g ) n t δ ( s model ( s , · · · , s g ) − s ) ds · · · ds g , where δ is the Dirac delta, i.e., a distribution such that (cid:82) ∞−∞ f ( x ) δ ( x − x ) dx = f ( x ) .The difference in distributions ρ t +1 ( s ) − ρ t ( s ) involves two processes. The first expressionmeans that we take g random agents. These agents interact and are removed from thedistribution. The second expression means that, for every possible group of g agents (withslopes s , . . . , s n ), a new agent is created with the slope s model ( s , . . . , s g ) .Note that we use integrals, but sum over a finite set will give the same result. Theparameter we are most concerned with is the mean slope, that is (cid:104) s (cid:105) t = n − t (cid:90) sρ t ( s ) ds. (45)We multiply (44) by s and integrate (cid:82) · ds . In our case, (36), this gives a relatively simpleresult: n t +1 (cid:104) s (cid:105) t +1 = n t (cid:104) s (cid:105) t − g (cid:104) s (cid:105) t + a g (cid:104) s (cid:105) t or (cid:104) s (cid:105) t = n − ( g − t + a g − n − ( g − t (cid:104) s (cid:105) t − . (46)To obtain the final result, we need to calculate (cid:104) s (cid:105) t max at the point of time when only oneagent remains. We consider t max = ( n − / ( g − to be an integer (e.g., for g = 3 weneed to consider an odd number of participants, for g = 2 there are no restrictions). Then,remembering that (cid:104) s (cid:105) = (cid:104) s (cid:105) and n = n , we get s a g ,Random,g = t max (cid:89) t =1 (cid:18) n − ( g − t + a g − n − ( g − t (cid:19) (cid:104) s (cid:105) (47) = Γ (cid:16) g − (cid:17) Γ (cid:16) a g g − (cid:17) Γ (cid:16) n g − + a g − g − (cid:17) Γ (cid:16) n g − (cid:17) (cid:104) s (cid:105) (48) ≈ Γ (cid:16) g − (cid:17) Γ (cid:16) a g g − (cid:17) ( g − ( a g − / ( g − × n ( a g − / ( g − × (cid:104) s (cid:105) (49)where Γ( x ) is the Euler gamma function, and we applied the Stirling approximation. For g = 2 , we obtain the neat result s a g ,Random,g =2 ≈ a ) n a − (cid:104) s (cid:105) , (50)15n particular, for the Weighted Confidence Sharing model ( a = √ ) we get s W CS,Random,g =2 ≈ . n . (cid:104) s (cid:105) , (51)whereas for the Voting model for g = 3 (and odd number of participants) we get s V ot,Random,g =3 ≈ . n . (cid:104) s (cid:105) . (52)In Fig. 3 , we present plots for Weighted Confidence Sharing in aggregation groups of two,and Voting in groups of three. We use both analytical approximations and numerical results. (cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:230)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:224)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)
10 20 30 40 50 60 n2345678multiplier (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230)(cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)(cid:236)
10 20 30 40 50 60 n1234multiplier
Weighted Confidence Sharing, g = 2 Voting, g = 3 Figure 3: Plot of numerically obtained multipliers of (cid:104) s (cid:105) for models with aggregation of information.Weighted Confidence Sharing with g = 2 for aggregation hierarchies: Shallow (circles), Deep (diamonds) andRandom (squares). Voting with g = 3 , and only for odd number of participants, for aggregation hierarchies:Shallow (circles), Deep (diamonds) and Random (squares). The lines are the respective analytical resultsfrom Sec. 4. The numerical results for the Random hierarchy are taken from one shot, i.e., they are notaveraged.
5. Discussion on results and comparison of models
For each investigated model, we arrived at the formula for the slope of the group as afunction of individual slopes, s model ( s , . . . , s n ) = multiplier model ( n ) × mean model ( s , . . . , s n ) . (53)Explicit results can be found in Tab. 1 and Fig. 4. Note that the formula is a product oftwo quantities — performance as a function of the group size (i.e., the multiplier), and themean of the individual slopes (if the better-performing contribute more to the outcome).For equally skilled participants, only the multiplier matters, whereas for a group of peoplewith high variance in performance, the type of mean is crucial.We not only solved the problem for a particular list of models, but we also constructeda general framework for the collaborative solving of a two-choice task, i.e., the group per-formance can be written as s model ( s , . . . , s n ) = d × n α × (cid:18) s p + . . . + s pn n (cid:19) /p , (54)16here parameters d , α and p can be fitted for any experimental data, even data not coveredby the models we investigated. Note that for p = 1 we arrive at the arithmetic mean, for p = 2 we arrive at the quadratic mean, and p → ∞ we arrive at the maximum. For themodels we investigated, (54) is either an exact solution (RR, WCS, BD, DSS, TH) or a goodapproximation (Voting, information aggregation schemes). If the result is exact, then d = 1 (to be consistent with the case of n = 1 ).For a given list of slopes ( s , . . . , s n ) , it is possible to write relations with the performances(slopes) for different models which read as follows: s RR ≤ s V ot < s
W CS ≤ s DSS ≤ s T W . (55)An average-performing participant is expected to benefit from participating in a joint task,unless the responder is chosen at random (in which case there is neither a gain nor a loss).It is somewhat more difficult to compare the Best Decides model to the other models, as ithighly depends on the distribution of the participants’ skills. We can write s RR < s BD < s DSS ≤ s T W . (56)However, how does the Best Decides model relate to the Voting and the Weighted ConfidenceSharing models? The answer lies in the comparison of the most skilled participant with theaverage performance, i.e., max ( s ) / (cid:104) s (cid:105) . If this ratio is greater than ≈ . √ n , the Best Decidesmodel outperforms the Voting. If the ratio is greater that √ n , Best Decides outperformsthe WCS as well. For example, when there is one expert (with s exp > among s non − exp = 1 )among the total number of n participants, then only when s exp > √ n + 1 it is better for agroup to use the Best Decides strategy.Model s ( s , s ) s ( s , s , s ) Mean MultiplierRR s + s s + s + s arithmetic 1Vot s + s s + s + s arithmetic ≈ . √ n BD max( s , s ) max( s , s , s ) maximum WCS s + s √ s + s + s √ arithmetic √ n DSS (cid:112) s + s (cid:112) s + s + s quadratic √ n TW s + s s + s + s arithmetic n Table 1: Models summary for the six considered models of Sec.3. For each model there is given explicitformula for two and three members. In each model the s model has the general form multiplier × mean. For schemes of aggregation (Tab. 2), we obtained two interesting results. First, mostof the models we investigated are not affected by the gradual aggregation of information.Second, for models that are affected, the optimal solution is to aggregate information inthe smallest possible groups, i.e., in g = 2 for Weighted Confidence Sharing and g = 3 forVoting.It is possible that participants’ strategies vary from trial to trial. In such situations, the17 (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230) (cid:230)(cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:224) (cid:236) (cid:236) (cid:236) (cid:236) (cid:236) (cid:236) (cid:236) (cid:236) (cid:236) (cid:236) (cid:242) (cid:242) (cid:242) (cid:242) (cid:242) (cid:242) (cid:242) TW (cid:236) WCS (cid:144)
DSS (cid:224)
Vot (cid:230) RR (cid:144) BD Figure 4: Plot summarizing multipliers for different models.
Model g Shallow hierarchy Random hierarchy Deep hierarchyVot 3 n . . n . . Vot 4 n . . n . . Vot 5 n . . n . . WCS 2 n . . n . . WCS 3 n . . n . . WCS 4 n . . n . . WCS 5 n . . n . . Table 2: Summary of information-aggregation results (see Sec. 4) in groups of g agents for affected models,i.e., Voting and Weighted Confidence Sharing. For each model there are provided asymptotic multipliers forthree different information-aggregation hierarchies. In each model the s model has the form multiplier timesarithmetic mean. Note that for Voting grouping in g = 4 is very ineffective (as, in fact, it effectively usesthe opinions of three out of four participants). Also note that, asymptotically, the most effective approach(i.e., the best for very large groups) for the Shallow and Deep aggregation schemes is to gather informationin the smallest possible groups of agents (i.e., in g = 3 for Voting and g = 2 for WCS). w model ), that is P eff ( c ) = (cid:88) models w model P model ( c ) , (57) s eff = (cid:88) models w model s model . (58)To distinguish between models, the sole analysis of the group performance might not beenough, as (psychologically) different models of problem-solving can yield the same per-formance. One can test modified schemes that put additional constraints on participants’interactions to investigate communication directly. For example, contact with other mem-bers could be limited to voice or text chat, or no feedback may be provided. In addition,participants might be asked to express their confidence explicitly on a Likert scale. How-ever, further experimental work should be carried out to clarify if and when confidence issubjectively accessible and can be communicated explicitly, and when it can be read fromparticipants’ behaviors. Preliminary results ( ? ) seem to suggest that the latter is common.Also the amount of feedback could range from full information about the stimulus to simpleinformation about accuracy, to no feedback at all. As a reference, it may serve to examineSocial Decision Scheme Theory (Davis, 1973), wherein the group decision is considered tobe a function of individual choices, regardless of skills, confidences or the difficulty of thetask.In all the models, interaction is beneficial for the overall performance, except for in theRandom Responder model (where the performance is the same as the averaged performanceof each individual). It is possible that beyond a certain critical size, groups start to performworse (Grofman, 1978). The models we consider do not predict such a collapse, as they arebased on information sharing and do not incorporate phenomena related to motivation andsocial or technical ability to work in groups.
6. Conclusion
In the paper, we examined mathematical models for solving a two-choice discriminativetask by a group of participants. We were interested in how group performance dependedupon the performance of the individuals, their ways of communication and their modes ofdecision aggregation. As a measure of performance, we used the slope of the psychometricfunction (3), which indicates how performance changes with the difficulty of the task. Thehigher the slope of s is, the better the performance of the individual (or the group).We analyzed a number of possible models of decision integration in a joint task. Aswe moved from 2-person to n -person groups, we also had to take into account patterns ofinteraction among members. Obviously, the choice of the way in which aggregate decisions ofgroup members are made is not always unconstrained. Some of the models, it seems, can beadopted in almost all group decision situations (such as the Random Responder model andthe Voting model). Regardless of the properties of the stimuli, people can make their owndecisions and vote. For the Best Decides model, we need to assume that the group possess19nformation about the members’ performances (e.g., from the feedback). Other models (i.e.,the Weighted Confidence Sharing, the Direct Signal Sharing and the Truth Wins models)make direct assumptions about the problem structure or the information that can be shared.Consequently, they can be considered only in particular tasks, in which a certain level ofconfidence in an individual’s own answer can be reached. Our list of models is by no meansexhaustive.We need to be aware of the fact that the presented models are valid only for our specificsituation (collaborative decisions in a two-choice perceptive task wherein difficulty can besmoothly adjusted). Other tasks may be analyzed within the same paradigm, such as inte-grating information in an individual’s mind. Several exposures to the same stimulus by asingle person, perhaps using different senses or with different noise levels, would be anothersubject for further investigation. Such an approach is presented in experiments on sensoryintegration, e.g., by Ernst and Banks (2002), which serve as one of the motivations for theBahrami et al. (2010) models. Perhaps collaborative decisions in other two-choice tasks(e.g., verbal or mathematical decisions) could also be treated in a similar fashion. However,for many other settings, more advanced models are needed, e.g., ones that take into accountmore choices or the dynamic interaction between solving a problem in an individual’s mindand communicating that decision to the other participants. Nevertheless, we believe thatthe first step should be to experimentally verify the predicted results of this paper (with anemphasis on the scaling of the performance), before proceeding to more advanced theoreticalmodels. Acknowledgements
The work was supported by EC EuroUnderstanding grant
DRUST to JRL, SpanishMINCIN project FIS2008-00784 (TOQATA) and ICFO PhD scholarship to PM, and thePolish Ministry of Education and Science (grants: N301 159735, N518 409238) to DP.
References
Abramowitz, M., Stegun, I. A., 1965. Handbook of Mathematical Functions: with Formulas, Graphs, andMathematical Tables. Dover PublicationBahrami, B., Olsen, K., Latham, P. E., Roepstorff, A., Rees, G., Frith, C. D., Aug. 2010. OptimallyInteracting Minds. Science 329 (5995), 1081–1085. 10.1126/science.1185718.Clark, A., Aug. 2006. Language, embodiment, and the cognitive niche. Trends in cognitive sciences 10 (8),370–4. 10.1016/j.tics.2006.06.012.Corfman, K. P., Kahn, B. E., Jan. 1995. The influence of member heterogeneity on dyad judgment: Are twoheads better than one? Marketing Letters 6 (1), 23–32. 10.1007/bf00994037.Davis, J. H., 1973. Group decision and social interaction: A theory of social decision schemes. PsychologicalReview 80 (2), 97–125. 10.1037/h0033951.Ernst, M. O., Banks, M. S., Jan. 2002. Humans integrate visual and haptic information in a statisticallyoptimal fashion. Nature 415 (6870), 429–33. 10.1038/415429a.Grofman, B., 1978. Judgmental competence of individuals and groups in a dichotomous choice situation: Isa majority of heads better than one? 10.1080/0022250x.1978.9989880.Hastie, R., Kameda, T., Apr. 2005. The robust beauty of majority rules in group decisions. Psychologicalreview 112 (2), 494–508. 10.1037/0033-295x.112.2.494. enry, R., May 1995. Improving Group Judgment Accuracy: Information Sharing and Determin-ing the Best Member. Organizational Behavior and Human Decision Processes 62 (2), 190–197.10.1006/obhd.1995.1042.Hill, G. W., 1982. Group versus individual performance: Are N+1 heads better than one? PsychologicalBulletin 91 (3), 517–539. 10.1037/0033-2909.91.3.517.Hutchins, E., Lintern, G., 1995. Cognition in the Wild. Vol. 262082314. MIT press Cambridge, MA.Kerr, N. L., Tindale, R. S., Jan. 2004. Group performance and decision making. Annual review of psychology55, 623–55. 10.1146/annurev.psych.55.090902.142009.Kirsh, D., Jan. 2006. Distributed cognition: A methodological note. Pragmatics & Cognition 14 (2), 249–262.10.1075/pc.14.2.06kirLaughlin, P. R., Kerr, N. L., Davis, J. H., Halff, H. M., Marciniak, K. A., 1975. Group size, member ability,and social decision schemes on an intellective task. Journal of Personality and Social Psychology 31 (3),522–535. 10.1037/h0076474.Piau, D., 2011. A simpler solution of the integral (cid:82) x + ... + x n ≥ a exp (cid:2) − π (cid:0) x + . . . + x n (cid:1)(cid:3) dx · · · dx n . Mathe-matics - Stack Exchange. http://math.stackexchange.com/q/61215 (version: 2011-09-01)Sorkin, R. D., Hays, C. J., West, R., 2001. Signal-detection analysis of group decision making. PsychologicalReview 108 (1), 183–203. 10.1037/0033-295x.108.1.183. Appendix A. Approximations P ( c ) can be expanded in Taylor series of c around c = − b . P ( c ) = P [ − b + ( c + b )] (A.1) = P ( − b ) + ( c + b ) P (cid:48) ( − b ) + ( c + b ) P (cid:48)(cid:48) ( − b ) + ( c + b ) P (cid:48)(cid:48)(cid:48) ( − b ) + . . . (A.2)where P ( i ) ( − b ) can be found explicitly using (1), P ( i ) ( c ) = 1 σ i H ( i ) ( c + bσ ) . (A.3)In particular H (0) = 1 / , H (cid:48) (0) = 1 / √ π , H (cid:48)(cid:48) (0) = 0 , H (cid:48)(cid:48)(cid:48) (0) = − / √ π .Consequently, P ( c ) = 12 + ( c + b ) √ πσ + O (cid:2) ( c + bσ ) (cid:3) , (A.4)that is, the approximation error of taking the linear approximation is of the order ( c + b ) /σ as the quadratic term vanishes. Plugging c = 0 we obtain P (0) = 12 + b √ πσ + O (cid:2) ( bσ ) (cid:3) (A.5) = 12 + sb + O [( sb ) ] (A.6)and similarly, the derivative of (A.4) in is P (cid:48) ( c ) | c =0 = 1 √ πσ + 1 √ πσ O (cid:2) ( bσ ) (cid:3) (A.7) = s (cid:2) O ( s b ) (cid:3) . (A.8)21he last equation gives the approximate equation for slope (7). Another expression P (0) − / P (cid:48) ( c ) | c =0 = b + bO [( sb ) ]1 + O [( sb ) ] = b (cid:2) O ( s b ) (cid:3) (A.9)yields in the approximate equation for bias (8). Appendix B. Voting P V ot ( c ) = (cid:98) n − (cid:99) (cid:88) k =1 (cid:88) (cid:126)i [1 − P i ( c )] · · · [1 − P i k ( c )] P i k +1 ( c ) · · · P i n ( c ) (B.1) + (cid:88) (cid:126)i [1 − P i ( c )] · · · (cid:104) − P i n/ ( c ) (cid:105) P i n/ ( c ) · · · P i n ( c ) if n is even , After plugging the linearization (A.4) in the above, and using µ i = s i ( b i + c ) , each part hasthe form of (cid:2) − µ i + O ( µ i ) (cid:3) · · · (cid:2) − µ i k + O ( µ i k ) (cid:3) (B.2) × (cid:104) + µ i k +1 + O ( µ i k +1 ) (cid:105) · · · (cid:2) + µ i n + O ( µ i n ) (cid:3) (B.3) = n − n − ( µ i + . . . + µ i k ) + n − (cid:0) µ i k +1 + . . . + µ i n (cid:1) (B.4) + O ( µ ) + . . . + O ( µ n ) (B.5)After applying permutations to the main part (i.e., without the error estimation) we get n (cid:18) nk (cid:19) + 12 n − (cid:18) nk (cid:19) [ − k + ( n − k )] µ + . . . + µ n n (B.6) = 12 n (cid:18) nk (cid:19) + n n − (cid:20) − (cid:18) n − k − (cid:19) + (cid:18) n − k (cid:19)(cid:21) µ + . . . + µ n n , (B.7)which is easy to be summed. The first component sums to / . In the second, binomialcoefficients cancel pairwise, except for (cid:0) n − − (cid:1) = 0 and (cid:0) n − (cid:98) ( n − / (cid:99) (cid:1) leaving only (cid:0) n − k (cid:1) for k = (cid:98) ( n − / (cid:99) . Consequently, when n is odd, one gets P V ot,odd ( c ) = 12 + n n − (cid:18) n − n − / (cid:19) µ + . . . + µ n n + O ( µ ) + . . . + O ( µ n ) (B.8)and for even nP V ot,even ( c ) = 12 + n n − (cid:18) n − n − / (cid:19) µ + . . . + µ n n + O ( µ ) + . . . + O ( µ n ) ..