aa r X i v : . [ s t a t . O T ] M a r A comparative study of scoring systems by simulations
L´aszl´o Csat´o ∗ Institute for Computer Science and Control (SZTAKI)E¨otv¨os Lor´and Research Network (ELKH)Laboratory on Engineering and Management IntelligenceResearch Group of Operations Research and Decision SystemsCorvinus University of Budapest (BCE)Department of Operations Research and Actuarial SciencesBudapest, Hungary “ Wer all seine Ziele erreicht, hat sie wahrscheinlich zu niedrig gew¨ahlt. ” (Herbert von Karajan) Abstract
Scoring rules aggregate individual rankings by assigning some points to each posi-tion in each ranking such that the total sum of points provides the overall ranking ofthe alternatives. They are widely used in sports competitions consisting of multiplecontests. We study the tradeoff between two risks in this setting: (1) the threat ofearly clinch when the title has been clinched before the last contest(s) of the com-petition take place; (2) the danger of winning the competition without finishing firstin any contest. In particular, four historical points scoring systems of the FormulaOne World Championship are compared with the family of geometric scoring rulesthat has favourable axiomatic properties. The rules used in practice are found tobe competitive or even better. The current scheme seems to be a reasonable com-promise in optimising the above goals. Our results shed more light on the evolutionof the Formula One points scoring systems and contribute to the issue of choosingthe set of point values.
Keywords : OR in sports; competition design; Formula One; rank aggregation; scor-ing system
MSC class : 62F07, 68U20, 91B14
JEL classification number : C44, C63, Z20 ∗ E-mail: [email protected] “ Those who have achieved all their aims probably set them too low. ”Source:
Introduction
Several sporting competitions consist of a series of races. In each race, a certain number ofpoints are assigned to the contestants based on their ranking, which are summed up acrossthe races. Finally, the contestant with the highest score becomes the champion. Thisaggregation method is usually called points scoring system . Examples include motorsport(e.g. Formula One), road bicycle racing (e.g. Points classification in the Tour de France),or winter sports (e.g. Biathlon World Cup). Consequently, the choice of points scoringsystem is a fundamental question of tournament design (Csat´o, 2021).Celebrated results of social choice reveal that using such a scoring rule offers the onlyreasonable way to aggregate individual rankings into an overall ranking (Smith, 1973;Young, 1975). Therefore, the meticulous analysis of points scoring systems is vital not onlyfor sports but for many other research fields like decision making, game theory, machinelearning, market design, or political science. Naturally, there exist numerous axiomaticcharacterisations of scoring rules (Nitzan and Rubinstein, 1981; Chebotarev and Shamis,1998; Merlin, 2003; Llamazares and Pe˜na, 2015; Kondratev et al., 2019). However, theyusually result in extremities or a parametric family of rules.The operations research literature extensively deals with the issue of aggregating in-dividual rankings, too. Stein et al. (1994) show how stochastic dominance can be ap-plied to determine a candidate who would win using any convex scoring function andto obtain a partial ordering under a class of scoring functions. Churilov and Flitman(2006) aim to design an objective impartial system of analysing the Olympics results.Llamazares and Pe˜na (2013) propose a model to evaluate each candidate according tothe most favorable weighting vector for him/her. Aledo et al. (2018) want to find a com-plete consensus ranking from a matrix of preferences using evolution strategies. Kaiser(2019) investigates the stability of historical Formula One rankings when the parametersare slightly changed. Turning to theoretical results, the Borda count and the Condorcetprocedure have been proved recently to be equivalent to the comparison of the randomvariables associated with the candidates in terms of expected values and statistical prefer-ence, respectively (Montes et al., 2020). Herrero and Villar (2021) provide an evaluationprotocol combining the methods of Borda and Condorcet to get cardinal ratings out ofordinal information.We study the issue of assigning point values for different ranking places from a sportingperspective. Inspired by real-world examples in Section 2, the current paper explores thetradeoff between two risks: (1) the threat of early clinch when the title has been clinchedbefore the last contest(s) of the competition take place; (2) the danger of winning thecompetition without finishing first in any contest. Every decision-maker should find acompromise between these two menaces since associating larger weights to the top ranks ceteris paribus reduces the danger of winning without finishing first but increases thethreat of early clinch.Some points scoring systems will be compared with respect to these two criteria viasimulations such that the probability of both events is quantified on the basis of FormulaOne World Championship results. The historical scoring rules are found to be competitiveor even better than the family of geometric scoring rules, proposed by Kondratev et al.(2019) using an axiomatic game theoretical approach. The current points scoring systemof Formula One turns out to be a good compromise for balancing the two threats.The paper is structured as follows. Section 2 presents two motivating examples. Sec-tion 3 discusses the methodological background and underlying data. The results are2rovided in Section 4, while Section 5 offers a concise summary and concluding remarks.
Our starting point is an unavoidable tradeoff between two objectives, illustrated by casestudies from the history of sports.Table 1: The 2002 Formula One World Championship (a) Points scoring system
Position 1 2 3 4 5 6Points 10 6 4 3 2 1 (b) Race results: ranks and total scores
Driver Race
Total score
Schumacher, M. 1 3 1 1 1 1 2 1 2 1 1 1 2 1 2 2 1
Barrichello — — — 2 — 2 7 3 1 2 — 4 1 2 1 1 2 Montoya 2 2 5 4 2 3 — — — 3 4 2 11 3 — 4 4 Schumacher, R. — 1 2 3 11 4 3 7 4 8 5 3 3 5 — 16 11 Coulthard — — 3 6 3 6 1 2 — 10 3 5 5 4 7 3 —
In the 2002 Formula One World Championship, 17 races were organisedwhere the top five drivers scored points according to Table 1.a. The outcome of theWorld Drivers’ Championship is provided in Table 1.b.
Michael Schumacher clinched thetitle after he scored 96 points in the first 11 races, while the actual runner-up
Juan-PabloMontoya scored only 34 points. Consequently, Schumacher had an advantage of 62 points,more than the maximal prize in the remaining six races.Obviously, Example 2.1 presents an unfavourable situation: who will be interested inwatching the last race(s) if the champion is already known? This risk is called the threatof early clinch . Example 2.2.
In the 1999 Grand Prix motorcycle racing, 16 races took place such thatthe top fifteen riders scored points according to Table 2.a. The outcome of the 125cccategory is provided in Table 2.b.
Emilio Alzamora was declared the champion despitenot winning any race. Contrarily, both the second and the third contestants won fiveraces.Example 2.2 outlines another situation that rule-makers probably do not like. Thespectator of any sporting event wants to see a dramatic struggle for the first spot insteadof risk avoidance by the runner-up. The possible situation of being the champion withoutwinning a single race is called the danger of winning without finishing first .Example 2.2 has not inspired any reform, the Motorcycle Grand Prix applies the samepoints scoring system since 1993. On the other hand, Formula One has introduced a newrule from 2003 by giving 10, 8, 6, 4, 3, 2, 1 points for the first eight drivers in each race,3able 2: The 1999 Motorcycle Grand Prix—125cc (a) Points scoring system
Position 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Points 25 20 16 13 11 10 9 8 7 6 5 4 3 2 1 (b) Race results: ranks and total scores
Rider Race
Total score
Alzamora 2 3 3 3 6 2 4 3 2 6 4 2 15 — 3 2
Melandri — — — 6 2 3 8 5 1 1 1 — 1 3 2 1
Azuma 1 1 1 4 7 — 1 1 6 12 10 — 5 14 6 —
Locatelli 18 — 5 1 1 6 3 4 4 — 11 8 6 4 8 3
Ueda — — 8 5 3 4 2 2 5 2 5 3 — — 1 —
Scalvini 3 12 4 7 — 10 5 6 7 4 19 1 4 1 — 7
Vincent 4 — 10 2 5 1 7 9 10 10 3 4 — 2 13 — respectively. Under the latter scheme, Michael Schumacher would have scored 102 pointsin the first 11 races, and Montoya would have scored 50. Therefore, the runner-up stillwould have had some (albeit marginal) chance to grab the championship in the last sixraces: Michael Schumacher would have clinched the title only after 12 races.Compared to the 2002 scoring rule, the 2003 system never provides fewer but some-times gives more points to all drivers except for the first. Thus it has reduced the threat ofearly clinch—and has simultaneously enhanced the danger of winning without being first.Clearly, there exists a bargain between excitement at the level of individual races andover the whole season, which could be a potential explanation for the continuous changesin the Formula One points scoring systems (Haigh, 2009; Kaiser, 2019; Kondratev et al.,2019).
In order to evaluate different points scoring systems concerning the threat of early clinchand the danger of winning without finishing first, one needs hypothetical individual rank-ings to be aggregated. For this purpose, we start from the race results of thirteen FormulaOne seasons between 2007 and 2019. Their main features are summarised in Table 3.The drivers are ranked by their number of points scored at the end of a season. Tiesare broken according to the greater number of first places, followed by the greater numberof second places, and so on until a winner emerges. If the procedure fails to produce aresult, the organiser nominates the winner according to such criteria as it thinks fit (FIA,2019, Article 7.2). In our simulations, only the two main ranking criteria (first: greaternumber of points scored; second: greater number of race wins) are implemented, with theremaining ties resolved through a drawing of lots.The outcome of a season is described as follows. Each driver is identified by its finalstanding in the World Drivers’ Championship. Every race is represented by the positionsof the drivers in this order. For instance, the vector [2 , ,
1] for a race means that the4able 3: The characteristics of Formula One seasons in our datasetSeason Drivers Races Clinched Margin2007 20 17 17 12008 19 18 18 12009 20 17 16 112010 18 19 19 42011 18 19 15 1222012 18 20 20 32013 18 19 16 1552014 17 19 19 672015 18 19 16 592016 19 21 21 52017 19 20 18 462018 20 21 19 882019 19 21 19 87
Drivers: Number of drivers who finished in the top ten positions at the end of atleast one race; Races: Number of races in the season; Clinched: Number of racesafter which the title of the Drivers’ Championship was secured; Margin: Theadvantage of the world champion over the runner-up at the end of the season.2007: The records of
Robert Kubica and
Sebastian Vettel were unified (the latterdriver substituted the former in the seventh race); the records of
Scott Speed and
Sebastian Vettel were unified (the latter driver substituted the former in the lastseven races); the records of
Alexander Wurz and
Kazuki Nakajima were unified(the latter driver substituted the former in the last race).2009: The records of
Timo Glock and
Kamui Kobayashi were unified (the latterdriver substituted the former in the last two races).At the second race of the Malaysian Grand Prix, half points were awarded be-cause less than 75% of the scheduled distance was completed due to heavy rain.2010: The records of
Petro de la Rosa and
Nick Heidfeld were unified (the latterdriver substituted the former in the last five races).2011: The records of
Nick Heidfeld and
Bruno Senna were unified (the latterdriver substituted the former in the last eight races).2014: Double points were awarded in the final race of Abu Dhabi Grand Prix.Hence 50 points could have been collected by winning here, which explains theseemingly high margin even though the title was clinched only in the last race.2016: The records of
Fernando Alonso and
Stoffel Vandoorne were unified (thelatter driver substituted the former in the second race).2019: One additional point was awarded for the fastest lap if the driver wasclassified in the top ten. later world champion finished as the second, the later runner-up obtained the third spot,and the race was won by the driver who earned the third place at the end of the season.A race is simulated in two ways:•
Method 1 : One race is drawn randomly with replacement from the set of all racesincluded in the underlying dataset.•
Method 2 : Two races are drawn randomly with replacement from the set of allraces included in the underlying dataset. Provisional spots are chosen by a cointoss from one of these two races, independently for each driver—thus provisionalspots will usually not provide an appropriate ranking of the drivers. The final5able 4: Possible race results in Example 3.1 under Method 2 (a) Provisional spots and the corresponding race results
Provisional spots Possible race results[1 , ,
2] [1 , , , ,
3] [1 , ,
3] [1 , , , ,
2] [1 , ,
3] [2 , , , ,
2] [1 , ,
2] [2 , , , ,
3] [1 , ,
3] [2 , , , ,
3] [1 , ,
3] [1 , , , ,
2] [2 , ,
3] [3 , , , ,
3] [2 , , (b) Race results and their probabilities Race result Probability[1 , ,
3] 1 /
16 + 1 /
16 + 1 /
16 + 1 /
16 = 4 /
16 = 1 / , ,
2] 1 / /
16 + 1 /
16 + 1 /
16 = 5 / , ,
3] 1 /
16 + 1 /
16 + 1 /
16 + 1 / / , ,
1] 1 / , ,
2] 1 / , ,
1] 0positions are obtained by ordering the drivers according to their provisional spotssuch that all ties are broken randomly.The motivation behind Method 1 is straightforward. Method 2 can be justified bythe following reasoning. An observed race represents only a single realisation of severalstochastic variables, therefore using only historical results might miss some reasonablescenarios. For instance, if a driver has finished either in the first or the third positionbased on the data, Method 1 will never generate an outcome where this driver is thesecond, even though that seems to be a potential “state of nature”.
Example 3.1.
Assume that there are three drivers and two races in the underlyingdataset, A = [1 , ,
2] and B = [2 , , A or B . In the case of Method 2, there are eight (2 ) possible lists of provisionalspots because the number of drivers is three, each of them occurring with probability 1 / A nor in race B butthis event has a 25% chance to happen under Method 2.A season that contains a given number of races is attained by generating the appro-priate number of individual race results independently from each other.Eight points scoring rules, shown in Table 5, are considered:6able 5: Points scoring systems of the analysisPosition Rule S S S S G G G G
41 25 10 9 10 10 12.58 42.62 181.592 18 6 6 8 9 11.03 32.01 112.873 15 4 4 6 8 9.55 23.86 69.924 12 3 3 5 7 8.14 17.58 43.075 10 2 2 4 6 6.80 12.76 26.306 8 1 1 3 5 5.53 9.04 15.817 6 0 0 2 4 4.310 6.187 9.2568 4 0 0 1 3 3.1525 3.99 5.169 2 0 0 0 2 2.05 2.3 2.610 1 0 0 0 1 1 1 1•
System S
1: the official Formula One points scoring system since 2010. Twocomplications were the double points awarded in the last race of the 2014 seasonand one additional point for the fastest lap in 2019 if the driver was classified inthe top ten. However, they do not appear in our simulations.•
System S
2: the official points scoring system of Formula One between 1991 and2002.•
System S
3: the official Formula One points scoring system between 1961 and 1990.Although not all results counted in the drivers’ championship (see https://en.wikipedia.org/wiki/List_of_Formula_One_World_Championship_points_scoring_systems ),this restriction is not implemented in our simulations.•
System S
4: the official points scoring system of Formula One between 2003 and2009.•
Systems G – G
4: according to Kondratev et al. (2019, Theorem 2), consistencyfor adding or removing a unanimous winner or loser pins down the one-parameterfamily geometric scoring rules with the set of scores 0, 1, 1 + p , 1 + p + p , . . .Again, points are awarded to the top ten spots, thus the score s j of the j thposition is ( p − j − / ( p −
1) for all 1 ≤ j ≤
10 if p >
1, which—becausethe scores are not distinguished up to scaling and translation—is equivalent toˆ s j = p − j . Four different values of the parameter have been chosen. G p = 1. G p = 1 . G p = 1 .
3, and G p = 1 . Full dataset : all races of every season between 2010 and 2019.•
Small margin dataset : all races of the seasons 2007, 2008, 2009, 2010, 2012, and2016.The full dataset contains ten recent seasons, where the rules were almost the same (scoringsystem S Positions S S S S G G G G Figure 1: The points scoring systems of Table 4
Scores for first place are normalised to 100. not change substantially during these years. As the column entitled “Margin” in Table 3uncovers, the small margin dataset consists of the seasons when the difference betweenthe first two drivers was marginal at the end, thus the competition was balanced andopen until the last race(s). Note that in 2014, the championship was open in the lastrace despite the seemingly high margin of 67 points as the winner got 50 points there and
Lewis Hamilton led by 17 points over
Nico Rosberg before this race.According to our simulations, winning without finishing first in any race rarely occursif a season is generated on the basis of the above data, which is called the original scenario in the following. It is probably caused by the steep points scoring systems used between2007 and 2019 ( S S
4) that provide powerful incentives for taking risks in order towin a race. But this motivation is not so strong if a relatively flat scoring rule is appliedto mitigate the threat of early clinch. Therefore, an alternative set of season results called risk averse scenario is generated to quantify the danger of winning without finishing first:after a whole season is simulated, all first spots of the world champion—who is determinedby system S Example 3.2.
Assume that a season consists of two races, A = [1 , ,
3] and B = [2 , , , ,
3] and [2 , , A are reversed but the outcome of race B does not change because the first driver is onlythe runner-up there. The first driver could not remain the champion (without finishingfirst in any race) under any points scoring system of Table 4 since the prize for a first anda third position is never less than the prize for two second spots, and the tie-breaking ruleis the number of races won. 8o summarise, the algorithm above aims to maximise the probability of winningwithout finishing first in a greedy way by assuming that the original world championdoes not take any risk to win a race.A simulation run contains the following phases:1. Choice of the underlying dataset: full or small margin.2. Choice of the race generation procedure: method 1 or method 2.3. Calculating original season results: generation of n races independently.4. Identification of the champion under the eight scoring rules listed in Table 4.5. Checking whether the champion has won at least one race.6. Determining the last race when the title has not already been secured, that is,checking whether the difference between the scores of the champion and the actualrunner-up (who is not necessarily the final runner-up) after m races is smallerthan the product of n − m and the score awarded to the first spot, or they areequal but the difference between the number of race wins for the champion andfor the actual runner-up is at most n − m . One of these conditions holds if andonly if the title is still not clinched after m races.7. Calculating risk averse season results on the basis of the original season resultsobtained in Step 3.8. Repeating Steps 4–6 for the risk averse scenario obtained in Step 7.The above process is carried out 100 thousand times independently in each simulation.Naturally, this approach for simulating individual rankings has several limitations.However, it should be kept in mind that—in contrast to some statistical studies (Graves et al.,2003; Henderson and Kirrane, 2018)—our aim is not to predict Formula One results andestimate the chance of a driver to win. For the evaluation of different points scoring sys-tems, essentially any reasonable model can be taken to determine the rankings (Appleton,1995; Csat´o, 2019). Nonetheless, the qualitative findings would be more reliable than theexact numerical values, and the calculations below are mainly for comparative purposes. A race can be called uninteresting if the title has already been clinched by a contestant.Figure 2 plots the probability that the champion did not win any race during a seasonconsisting of 20 races as the function of the average number of these uninteresting races.Since there is practically no difference between methods 1 and 2, only the latter will beconsidered in the following.According to Figure 2, the two goals can be optimised only at the expense of eachother as expected. Although geometric scoring rules have strong theoretical foundationsand are not vulnerable to removing unanimous winners or losers (Kondratev et al., 2019),they are poor alternatives to the historical points scoring systems: rule S G
3, and S G
4. Even though schemes S S . . . . . . . . . . . . . S S S S G G G G S S S S G G G G Average number of uninteresting races (original scenario) P r o b a b ili t y t h a tt h ec h a m p i o nd i dn o t w i n a n y r a ce ( r i s k a v e r s e s ce n a r i o ) Method 1: Full dataset Method 1: Small margin datasetMethod 2: Full dataset Method 2: Small margin dataset
Figure 2: The tradeoff between the threat of early clinch andthe danger of winning without finishing first (20 races per season)this has a significant effect on both of our measures. For example, S S S → S → S → S
1. The change from S S S S
1, which is about halfway between S S S S S S
1, effective from 2010. Perhaps the ad-ministrators recognised that the danger of winning without finishing first can besubstantially decreased at a small sacrifice in the threat of early clinch.• Compared to S
1, system S S S S S S G G G G . . . Full dataset . . . . . S S S S G G G G . . . . . Small margin dataset
Average number of uninteresting races (left scale)Probability that at least three races are uninteresting (right scale) . . . . . . Figure 3: The threat of early clinch (20 races per season, method 2, original scenario)that the champion is decided before the last three races out of the total 20. It reinforcesour previous findings, the geometric scoring rule G S S
3, while G S S S
1, the champion is known with a 30%chance in the last three races, and the probability remains around 10% if the season ishighly competitive. Even though the threat of early clinch cannot be eliminated, schemes G G a) Average number of uninteresting races (original scenario) Number of races per seasonFull dataset . Number of races per seasonSmall margin dataset(b) Probability that the champion did not win any race (risk averse scenario) . . . Number of races per seasonFull dataset . . . Number of races per season
Small margin dataset S S S S G G G G Figure 4: The threat of early clinch and the danger of winning without finishing firstas the function of the number of races per season (method 2)with the length of the season. On the other hand, the danger of winning without finishingfirst decreases if there are more races, with the possible exception of the “balanced” scoringrules G G
2. However, since even a completely risk averse contestant may win a raceout of many merely by chance, our risk averse scenario is farther from reality if there aremore races in the season.
This research has attempted to help to understand the properties of scoring rules, ex-tensively applied to derive an aggregate ranking from individual rankings, from a sport-ing perspective. In particular, we have focused on the tradeoff between two risks in achampionship composed of multiple contests: the threat of early clinch (when the title12s secured before all contests are over, therefore the last contest(s) become uninterest-ing) and the danger of winning without finishing first (when the overall winner doesnot win any contest). The probability of both events has been measured using FormulaOne World Championship results. The historical points scoring systems are found to becompetitive or even better than the family of geometric scoring rules, recommended byKondratev et al. (2019) based on an axiomatic game theoretical approach. Finally, thecurrent points scoring system of Formula One is shown to be a good compromise forbalancing the two perils.There are straightforward directions for future research. The robustness of the simu-lations can be improved by considering other datasets. The danger of winning withoutfinishing first can be quantified in alternative ways. Other aspects of adopting a pointsscoring system can be built into the framework. For instance, our focus has been limitedto winning without finishing first, while geometric rules with a high parameter p encour-age to risk taking at any position. Hopefully, our work will turn out to be only the firststep in the comparison of scoring rules via simulations. Acknowledgements
This paper could not have been written without my father (also called
L´aszl´o Csat´o ), whohas coded the simulations in Python.We are grateful to
Aleksei Y. Kondratev and
Josep Freixas for inspiration.
Aleksei Y. Kondratev , D´ora Gr´eta Petr´oczy , and
Mike Yearworth provided valuable com-ments and suggestions on an earlier draft.
D´ora Gr´eta Petr´oczy and
Gerg˝o T´ov´ari helped in data collection.We are indebted to the Wikipedia community for contributing to our research by sum-marising important details of the sports competitions discussed in the paper.The research was supported by the MTA Premium Postdoctoral Research Program grantPPD2019-9/2019.
References
Aledo, J. A., G´amez, J. A., and Rosete, A. (2018). Approaching rank aggregation problemsby using evolution strategies: The case of the optimal bucket order problem.
EuropeanJournal of Operational Research , 270(3):982–998.Appleton, D. R. (1995). May the best man win?
Journal of the Royal Statistical Society:Series D (The Statistician) , 44(4):529–538.Chebotarev, P. Yu. and Shamis, E. (1998). Characterizations of scoring methods forpreference aggregation.
Annals of Operations Research , 80:299–332.Churilov, L. and Flitman, A. (2006). Towards fair ranking of Olympics achievements:The case of Sydney 2000.
Computers & Operations Research , 33(7):2057–2082.Csat´o, L. (2019). A simulation comparison of tournament designs for the World Men’sHandball Championships.
International Transactions in Operational Research , in press.DOI: 10.1111/itor.12691. 13sat´o, L. (2021).
Tournament Design: How Operations Research Can Improve SportsRules . Palgrave Pivots in Sports Economics. Palgrave Macmillan, Cham, Switzerland.FIA (2019). . 12 March. .Graves, T., Reese, C. S., and Fitzgerald, M. (2003). Hierarchical models for permuta-tions: Analysis of auto racing results.
Journal of the American Statistical Association ,98(462):282–291.Haigh, J. (2009). Uses and limitations of mathematics in sport.
IMA Journal of Manage-ment Mathematics , 20(2):97–108.Henderson, D. A. and Kirrane, L. J. (2018). A comparison of truncated and time-weightedPlackett–Luce models for probabilistic forecasting of Formula One results.
BayesianAnalysis , 13(2):335–358.Herrero, C. and Villar, A. (2021). Group decisions from individual rankings: The Borda–Condorcet rule.
European Journal of Operational Research , 291(2):757–765.Kaiser, B. (2019). Strategy and paradoxes of Borda count in Formula 1 racing.
Decyzje ,6(31):115–132.Kondratev, A. Y., Ianovski, E., and Nesterov, A. S. (2019). How should we score athletesand candidates: geometric scoring rules. Manuscript. arXiv: 1907.05082.Llamazares, B. and Pe˜na, T. (2013). Aggregating preferences rankings with variableweights.
European Journal of Operational Research , 230(2):348–355.Llamazares, B. and Pe˜na, T. (2015). Scoring rules and social choice properties: somecharacterizations.
Theory and Decision , 78(3):429–450.Merlin, V. (2003). The axiomatic characterizations of majority voting and scoring rules.
Math´ematiques et Sciences Humaines , 41(161):87–109.Montes, I., Rademaker, M., P´erez-Fern´andez, R., and De Baets, B. (2020). A corres-pondence between voting procedures and stochastic orderings.
European Journal ofOperational Research , 285(3):977–987.Nitzan, S. and Rubinstein, A. (1981). A further characterization of Borda ranking method.
Public Choice , 36(1):153–158.Smith, J. H. (1973). Aggregation of preferences with variable electorate.
Econometrica ,26(1):1027–1041.Stein, W. E., Mizzi, P. J., and Pfaffenberger, R. C. (1994). A stochastic dominance ana-lysis of ranked voting systems with scoring.
European Journal of Operational Research ,74(1):78–85.Young, H. P. (1975). Social choice scoring functions.