Randomization and Fair Judgment in Law and Science
Julio Michael Stern, Marcos Antonio Simplicio, Marcos Vinicius M. Silva, Roberto A. Castellanos Pfeiffer
RRandomization and Fair Judgmentin Law and Science
Julio Michael Stern ∗ , Marcos Antonio Simplicio † ,Marcos Vinicius M. Silva ‡ , Roberto A. Castellanos Pfeiffer § To be published at Festschrift volume forFrancisco Antonio D´oria’s 75th birthday
Abstract
Randomization procedures are used in legal and statistical applica-tions, aiming to shield important decisions from spurious influences. Thisarticle gives an intuitive introduction to randomization and examinessome intended consequences of its use related to truthful statistical in-ference and fair legal judgment. This article also presents an open-codeJava implementation for a cryptographically secure, statistically reliable,transparent, traceable, and fully auditable randomization tool.
Keywords:
Randomization; Truthful inference; Fair judgment; Judicialautonomy and independence. κληρω νυν π(cid:15)παλασθ(cid:15) διαµπ(cid:15)ρ(cid:15)ς oς κ(cid:15) λαχησιν : Let the lot be shaken for all of you,and see who is chosen.
Iliad, VII, 171. !ד£רְפfiי Mיִמ„וצְַע Nיֵבוּ ל·רוֹגַּה תיִבּ(cid:160)שׁfiי Mי¢נ‡י(cid:159)דִמ
Casting the dice puts judgment quarrels to rest and keepsdistinct essential powers duly separated.
Proverbs 18:18. ∗ Institute of Mathematics and Statistics of the University of Sao Paulo [email protected] † Polytechnic School of the University of Sao Paulo [email protected] ‡ Polytechnic School of the University of Sao Paulo [email protected] § Law School of the University of Sao Paulo, [email protected] a r X i v : . [ s t a t . O T ] A ug igure 1: The Cardsharps (1594), by Michelangelo Caravaggio
Francisco Antonio D´oria has had a consistent interest in randomness and chaosand, together with his collaborators, has investigated fundamental aspects ofsuch phenomena. This article is our a contribution to the Festschrift celebratingDoria’s 75th birthday.This article analyses some pragmatical aspects of applying randomization inempirical science and law, considers some philosophical implications or premisesjustifying or motivating these applications, and offers some tools that promotegood randomization practices.
The Cardsharps (1594) marks the beginningof the independent career of the great Italian master Michelangelo Merisi daCaravaggio (1571-1610). This painting displays a wealthy but innocent lookingboy playing cards with his opponent, a cardsharp, that cheats in two ways: Onthe one hand, the cardsharp hides in his belt spurious cards that he intends touse in illegitimate ways; on the other hand, a sinister looking and strategicallypositioned accomplice gives him access to privileged and undue information.Finally, the cardsharp carries a dagger, hinting at the dangers lurking in thisenvironment of misrepresentation and deception.Caravaggio gives a beautiful depiction of some themes discussed in thisarticle. First, the social importance of activities involving randomization, thatis, the random setting of some variable of interest, like the drawing of dice2r, in this painting, the distribution of playing cards. Second, it suggests thequestion – Why to randomize? that is – Why should a rational agent abdicatethe opportunity of making a deterministic choice introducing, on purpose, arandom component in making a decision? If so – What is the role playedby randomization? Finally – How to randomize? that is – What dangerscould jeopardize a randomization process? and, if necessary – How to shield orimmunize the process against these dangers?In order to answer these questions, we have to pay attention to some top-ics in Statistics, Computer Science and Cryptography; in addition, we have toexamine some details concerning the design of empirical trials or the operationof legal systems. In this article, we investigate each one of the questions justraised, looking for an intuitive understanding of the role(s) played by random-ization.In the final sections of the paper, we present an easy to use, open-code,traceable, auditable, secure, and statistically sound randomization toll that isready for use in empirical trials and legal applications. This kind of secure ran-domization tool can prevent the possibility of misrepresentation and deception,as depicted in the painting by Caravaggio. Moreover, even in situations whereno misdeeds actually occur, the use of such a tool can be beneficial by fosteringpublic confidence in the soundness of important decisions, by strengtheningthe resilience of public institutions, and by favoring the peaceful resolution ofconflicts.
Gambling and lotteries exchange billions of Dollars every day worldwide. Hence,ensuring honesty and transparency in these activities should already be consid-ered a meritory task. However, since ancient times, sortition (i.e., selection bylottery) is used for many other purposes. In the Iliad, one of the oldest textsof western culture (aprox. 1200BC), the Argonauts (crew of the ship Argo)selected a man to execute a dangerous task by sortition – see this paper’s firstopening quotation. In the same manner, modern societies often resort to sor-tition for drafting. Figure 2 displays some photographs related to compulsoryenlistment for service in the USA, namely, military drafting during the Civil(left) and the Vietnam (center) wars, and selection for jury duty (right).In order to gain public trust, the sortitions for the Vietnam war were con-ducted in public view: Balls with calendar dates were placed in a transparenturn and some anniversary dates were then picked, giving the (un)lucky winnersthe opportunity to serve their country in the battlefield. A post hoc statisticalanalysis of these drawings revealed a significant bias favoring latter days of thecalendar, corresponding to the last balls placed inside the urn, an unexpectedeffect of an ill-conceived randomization process that generated misunderstand-3igure 2: Draft lotteries in war and peaceing, frustration and conflict.Figure 2 (right) shows a letter calling a citizen for jury duty in the USA.In this process, an eligible citizen was chosen at random by running a com-puter program. Post hoc analyses of this randomization process revealed nosignificant bias or any other statistical anomaly. Nevertheless, the code of thesecomputer programs were never made public, making the randomization processopaque and non-verifiable, thus generating mistrust and resentment.Finally, in the world of science, good clinical trials are conducted by (double)blind and random attribution of patients to two or more distinct treatments.The objective of such a trial is to find out if a new or alternative treatment issignificantly better than the old or standard one, according to well-establishedstatistical criteria. In this situation, some frequently asked questions are: Whyshould a patient’s treatment be selected at random? Why not give him or herthe freedom to chose his or her proffered treatment? Why hide from a patientinformation about his or her own treatment?
Imagine a clinical trial where patients are free to choose a treatment accordingto their own will. Among patients, there will be rich and poor, people withdifferent degrees of instruction, people with better or worst networks for sup-port, etc. Obviously, rich, well educated, and well connected patients will havebetter access to good information and advice and, therefore, will be prone tomake better decisions. Moreover, these same patients likely have better over-all living conditions and, therefore, even with the same treatment, might havea better chance of recovery. Hence, this freedom of choice would automati-cally introduce confounding effects : After the trial is over, we would not beable to (completely) discern the beneficial and adverse consequences of distincttreatments from consequences of preexisting conditions.Similar unwanted interference is generated by the placebo and nocebo con-4igure 3: Essentially different powers: Economy and Justicefounding effects. If a patient knows to be receiving either a new, experimentaland possibly wonderful drug, or else an old and possibly not very effectivedrug, his or her moral may be, respectively, lifted or depressed. That, in turn,may affect his or her overall health and chance of recovery. This is why, in agood clinical trial, treatment information is denied (blinding or censorship) topatients, and commonly also to their direct caretakers (double-blinding).There are in the medical literature plenty of examples of clinical trials thatcame to wrong conclusions in consequence of such confounding effects. Thebest known antidotes against these confounding effects rely in some form ofrandomization. The idea is to chose a patient’s treatment based on a randomvariable that is independent of any potentially confounding variable. In sodoing, the random element in the choice of treatment has the effect of breakingcausal links that should not interfere with the experiment, allowing the trialto adequately focus on the causal links of interest – see Stern (2008) and Pearl(2009) for further details.Finally, let us consider the use of randomization in the legal system, like theselection of jurors or judge(s) for a given case. Figure 3 displays two picturesfrom ancient Egypt. On the left, a stone carving of approx. 2400BC shows twomerchants using a two-pan balance to correctly measure amounts of goods fora fair commercial transaction. On the right, the Hunefer papirus of approx.1275BC shows the scale used by Maat, the goddess of justice, where the heartand the (de)merits of a man are measured. It should be clear that these twoscales are essentially distinct – they belong to distinct contexts. The figureat the center suggests the possibility of “mixing” these two essences: PerhapsMaat could make a more benevolent assessment in the scale of justice if she,or her priests, received goods of commercial valuable... There we have, oncemore, a confounding effect, characterized by spurious influences between powersbelonging to essentially distinct systems: in this case, the economic system andthe justice system.How to avoid such confounding effects caused by spurious influences, fos-tering autonomous decisions in a strong and independent justice system? Sur-prisingly, the Hebrew bible already offers very good advice at this respect, as5tated in the second opening quotation of this paper. Interestingly, the Hebrewroot !Mצע , etzem , whose literal meaning is bone, also generates words meaningessence (the etymological origin of the English word), strength, power and themodern Hebrew word for independence.Judges, even if perfectly honest, do not come to court as a blank slate, norshould they. Every judge has his or her own history of decisions and opinions.Hence, if the selection of judges could be influenced by the litigants or otherinterested parties, the richer, better informed, well connected, or otherwisemore powerful parties would likely have an advantage in directing the case toa judge sympathetic to their arguments. For this reason, in many moderndemocracies, the distribution of a new judicial case must take the form of arandom choice among the available judges or courts qualified to judge it. Previous sections discussed several applications of randomization and explana-tions of why to use it. This section describes some desirable characteristics ofsuch a randomization process, including:Statistical honesty: In a set of sortitions (random selections) in the system,the probability of any group of outcomes should be exactly as prescribedby the established rules.Cryptographic security: The outcome of the sortition should be unpredictable;moreover, no external agent should be able to influence the randomizationprocess, even if the agent knows in detail the randomization mechanismbeing used and has state of the art knowledge of all relevant technologiesinvolved.Transparency: All relevant information about the sortition process must be ofpublic knowledge, including any pertinent detail about the randomizationmechanisms being used.Auditability: All relevant occurrences of an actual randomization processmust be traceable and auditable. Furthermore, it should not be possibleto conceal any improper use of the randomization system.The first two requirements are of technical nature, stipulating that, in therandomization process, we should use “honest dice that cannot be tamperedwith”, or else a more convenient device, like a computational algorithm, thatadequately mimics all the relevant characteristics of “honest dice”. For moretechnical discussions on theses characteristics, see Marcondes et al (2019), Saaand Stern (2019) and Silva et al. (2020).In order to emphasize the importance of the last two requirements, let usdiscuss a form of cheating known as rerandomization . In this kind of cheating,6he agent responsible for a sortition has the privilege of using the randomizationmechanism out of public scrutiny, examine the outcome, and chose at willeither to make this process and its outcome public, or to hide this first tryand randomize a second time, as if the first try never happened. Imagine forexample the classic process of picking a ball from a transparent urn. However,instead of making a live presentation, the sortition ceremony is recorded forbroadcasting at a later time. A dishonest agent could repeat and record theprocess twice, and only release the recording that best fits his or her goals, as ifit were the only recording ever made. It should be clear that the repeated useof this subterfuge gives the agent in charge some latitude to pick and choose,biasing the final outcome according to his or her convenience.
Authority, Transparency and Understanding
Why is transparency even required in a randomization process? Would it no bepossible, or even easier, to anchor the credibility of the process on a principleof authority ? If a given authority is responsible for a randomization process,doesn’t the requirement of transparency imply an implicit doubt? If so, doesn’tthe requirement of transparency imply disrespect for the same authority?These are basic questions in philosophy of law, that can only be answeredin a context that specifies the fundamental values and goals chosen by a givensociety. Niklas Luhmann (1985, 1989), a celebrated scholar in philosophy of law,postulates that the fundamental goal of the justice system is – “the congruentgeneralization of normative expectations”. That is, the final objective of thelegal system is the construction of an harmonious society, where citizens havea coherent view of what constitutes a good set of rules for social behavior(normative expectations). Moreover, a legal system should provide mechanismsthat stimulate citizens to conform to normative expectations and inhibit theirtransgression.This conception of law requires from every citizen a well founded trust thatthe justice system is efficient and fair, preferably obtained by conscious under-standing of laws and regulations and their forms of implementation. Moreover,a justice system conceived according to such principles is weak or fragile if sus-tained on blind faith on ad hoc authority, but strong and resilient if sustainedby a conscious, engaged, and participative community. The articles of Silva etal. (2020) and Stern (2018) expand these ideas.
In this section we discuss intuitive ideas for how to implement an honest ran-domization device that all interested parties can trust. In following sections ofthis article we offer a viable technological solution to the problem of random-7igure 4: Modulo 12 or Clock Arithmeticization that satisfies all requirements stipulated in previous sections, followingthe general ideas hereby discussed.Modular arithmetic is an integer arithmetic system in which numbers “wraparound” after reaching a maximum value, m , called the modulus. A familiarexample is the standard reading of a clock. After noon (12 o’clock), we restartcounting from 1 , , ... (p.m.). Notice that, in Figure 5, the position correspond-ing to noon (or midnight) is marked either by the modulus value, m = 12, or bythe value zero – that is mathematically more convenient. In general, for positiveintegers, n and m >
1, we define n mod m (read n modulo m ) as the remainderof the division of n by m . For example, see Figure 5 (left): 13 mod 12 = 1,14 mod 12 = 2, ... 23 mod 12 = 11, and 24 mod 12 = 12 mod 12 = 0. Figure 5(right) illustrates the modular arithmetic operation (9 + 4) mod 12 = 1.Now imagine we have a game using a roulette or wheel of fortune, see Figure5, with k participants, also known as the stakeholders , all of them wanting theprivilege of spinning the roulette, and not trusting anyone else to do the job.How can we break this deadlock?We can solve the aforementioned impasse using the following protocol:1. Provide each stakeholder with a well-balanced roulette, marked accordingto the numbers set { , , . . . ( m − } ;2. Ask each stakeholder to spin his roulette honestly , that is, with a notfully controlled and strong enough initial impulse so to produce any ofthe possible outcomes, in { , , . . . ( m − } , with the same probability,(1 /m ). Moreover ask each stakeholder to use his roulette independently ,that is, to do so without sharing any information with other stakeholdersor interested parties;3. Collect and add, using modulo m arithmetic, the results produced byeach one of the k stakeholders in order to produce the final result: n f =( n + n , . . . + n k ) mod m .We can guarantee that the final result, n f , produced by this protocolis equivalent to an “honest roulette”, as long as at least one (any one) of8igure 5: Rolling Dice and Spinning Roulettesthe k stakeholders does his job as required. This guarantee is a corollaryof the following theorem: Let x and y be independent random variables in { , , . . . ( m − } . Then, if any one of these random variables, x or y , is uni-formly distributed, so is z = ( x + y ) mod m . Imagine, for example, that variable x is not random at all, but rather a known constant, c , namely, the initial stateof the roulette. Furthermore, imagine that y is independent of x and uniformlydistributed in { , , . . . ( m − } . Under these conditions, the theorem statesthat the final state of the roulette, z = c + y is uniformly distributed, corre-sponding to the intuitive idea that, when using a well-constructed roulette, astrong enough impulse will produce a final outcome that “forgets” the initialstate of the roulette. For further details and formal mathematical analyses, seeScozzafava (1993).In many applications in statistics, clinical trials, and complex sortitions, weneed a random variable x uniformly distributed in the interval [0 ,
1[ of the realline. In computational procedures, this continuous variable can be approxi-mated by a fraction n/m , where m in a large integer, and n ∈ { , , , . . . ( m − } . This fraction can be translated to standard floating point notation, andthen be further transformed into random variables with several probabilitydistributions of interest in statistical modeling, see Hamersley (1964), Ripley(1987). Such uniform or non-uniform random variables can, in turn, be used indynamic clinical trials, haphazard intentional sampling, adaptive sampling pro-cedures, and other complex applications of interest in statistical modeling anddecision science, see for example Fossaluza (2015) and Lauretto et al. (2012,2017) and the bibliography therein.In the sequel, we describe a software implementing the protocol outlinedin this section, including all necessary precautions in order to guarantee cryp-tographical security. In this software, every stakeholder is required to input arandom number n between 0 and m = 9 , , One crucial requirement of the random drawing approach described in Section5 is that the the roulettes are run independently. Otherwise, a dishonest stake-holder S i could wait until the results of all roulettes are revealed, and thenrun his/her own roulette for manipulating the final outcome of the drawing:for example, suppose that the sum of the contributions from all stakeholdersexcept S i is n = 3 for, say, m = 12; after learning this value of n , S i could forcehis/her own roulette to give n i = 2, thus obtaining n f = n + n i mod m = 5 asthe final (manipulated) outcome of the drawing.To ensure this independence property, Silva et al. (2020) builds upon theproperties of hash-based bit-commitment mechanisms. Intuitively, a hash func-tion H is a cryptography construct analogous to fingerprinting for humans, asillustrated in Figure 7 – we refer the reader to Beutelspacher (1994), Bultel etal. (2017) and Fellows and Koblitz (1994) for intuitive introductions on keyideas of cryptology, an to Rogaway and Shrimpton (2004) for a concise butformal explanation of properties of cryptographic hash functions. Specifically,given the fingerprint for an unknown human being, it takes a lot of computa-tional power to look all around the world for the owner of that fingerprint; onthe other hand, given a fingerprint and the corresponding human, it is quiteeasy to check whether or not they match, and it is hard to find two differentpeople with the same fingerprint (even considering identical twins).Similarly, suppose that someone computes the hash of a number n , i.e., avalue h = H ( n ), which acts as a ”fingerprint” for n ; then, if only h is revealed,10 = 5 h = ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d fingerprinting hashing fingerprinting n ≠ 5 hashing Figure 7: Hash functions and their similarity with human fingerprinting.but n is kept secret, there is no simple mechanism for finding the value of n . Of course, one could test every possible value of n , checking if a guess n i is such that H ( n i ) = h , just like finding the owner of a fingerprint givenonly the fingerprint itself. However, the computational effort for performingsuch brute-force attack would be very large. Actually, in practice the cost forhash functions would be even larger than searching for a human who ownsa fingerprint: while there are a few billions of humans in the world, a hashfunction can be used in such a manner that the number of tests would be aslarge as the number of atoms in the whole planet Earth! For this purpose, itsuffices to combine the value of n with a large and unpredictable (e.g., random)mask r when computing the hash, i.e., to make h = H ( r, n ). As a result, evenif there are only a few possible values of n to be tested, determining whetheror not a guess n i is correct would require testing all possible values of r too.Therefore, it suffices to use a large-enough mask r (e.g., 256-bits) to ensurethat any attempt of determining n via brute force would be computationallyinfeasible.When both n and h are revealed, on the other hand, it is easy to verifywhether or not they match: it suffices to compute H ( n ) directly, and check ifthe output of this computation is identical to the provided value of h . However,like different humans should not have the same fingerprint, it is computationallyhard to find two distinct values of n (say, n and n ) that have the same hash h .Hence, once h = H ( n ) is revealed, one can say that the person who revealedit is “committed” to revealing n , i.e., it would be hard to trick someone intobelieving that h was computed from any other input n (cid:54) = n .Such properties are used by Silva et al. (2020) to build a two-phase proce-dure for ensuring the fairness of random draws:11. Commitment phase: first, each stakeholder S i runs a roulette (honestlyor not), getting a value n i as result. Then, S i computes the hash of n i ,denoted h i , and reveals only h i to the other stakeholders, keeping n i itselfsecret. This prevents S i from learning the roulette results from his/herpeers, and vice-versa.2. Reveal phase: only after all hashes are received, every stakeholder S i reveals its own n i . The outcome of the drawing is then computed locallyby S i by adding every n i together using modular arithmetic as explainedin Section 5. In this case, even if S i is malicious and tries to delay therevelation of n i until it learns the partial outcome of the drawing fromthe values revealed by his/her peers, it would be already too late: afterrevealing h i in the commitment phase, S i has no choice but to revealthe already chosen n i , rather than some other value that might lead to amore desirable (but unfair) drawing outcome. Java
Implementation
We developed a simple Java library for implementing the protocol describedin Silva et. al (2020), and made it available under the MIT License at https://doi.org/10.24433/CO.6108166.v1
This library can, thus, be freely adaptedfor the needs of specific application scenario. To help in this task, we alsoprovide a simple proof-of-concept graphical interface for testing purposes, whichis depicted in Figure 8. More precisely, this figure shows:(a) A simple configuration interface for drawing a number among m =10,000,000 candidates, i.e., from 0 to 9,999,999. The number of stake-holders participating in the drawing and additional metadata related toit can also be defined.(b) A snapshot of the Commitment phase, as seen by Stakeholder S in adrawing involving 5 stakeholders. In this snapshot, S is then free tochoose a number n to commit, which is combined with a random maskfor better security against brute force attacks. Meanwhile, S , S and S have already sent the hashes h , h and h of their own commitments, n , n and n , respectively; as a result, these stakeholders cannot modifythe chosen values n , n and n anymore.(c) A snapshot of the Reveal phase, as seen by Stakeholder S . The figureshows that S is the only one who has not yet revealed the chosen valuefor n , while all of his/her peers have already revealed n , n , n and n . Nevertheless, S can only reveal the correct n (and correspondingmask), since the revealed value must match the committed value h .12 a) Random drawing with 5 stakeholders( S , S , S , S and S ) and modulus value m = 10 , , S , S and S after commitment, asseen by S . Value committed by S is h = . . . (c) S , S , S and S in reveal phase. Par-tial result as seen by S . (d) Drawing result: 1,610,027 + 5,871,032+ 6,029,108 + 7,664,824 + 5,757,989 mod10,000,000 = 6,932,980. Figure 8: Proof-of-concept Java implementation: screenshots(d) The completion of the protocol, when one of the eligible numbers (namely,6,932,980) is picked with uniform probability based on all stakeholders’contributions n , n , n , n and n .13 Final Remarks
Previous articles of this research group have explored the need of randomiza-tion procedures in legal systems, like the random assignment (distribution) oflegal cases to individual judges or courts, the sortition of jurors for a given case,etc., see Marcondes et al. (2019), Saa and Stern (2019), Silva et al. (2020).Moreover, these papers provide extensive discussions on how to build honest(statistically non-biased) and cryptographically secure procedures and proto-cols, on the sociological and political importance of using fully transparentand auditable procedures, and on the positive effects of using procedures fullycompliant with the aforementioned desiderata in the constitution of strong andautonomous legal institutions.Finally, breaking away from vicious old habits can always be stimulatedby respectful criticism, by firm encouragement, and by making available userfriendly tools that facilitate the adoption of virtuous new habits without theimposition of additional difficulties beyond the already heavy load of overcom-ing corporate inertia. This paper provides such a tool, fully compliant with alltechnical desiderata, user friendly, written in freely available and open sourcecode. The authors hope it will be soon put to use by Brazilian legal institutionsand, if necessary, stand ready to help in this endeavor.
Acknowledgments
This work was supported by: Ripple’s University Blockchain Research Ini-tiative; CNPq (Brazilian National Council for Scientific and TechnologicalDevelopment – grants PQ 307648/2018-4 and 301198/2017-9); and FAPESP(S˜ao Paulo Research Foundation, grants CEPID-CeMEAI 2013/07375-0 andCEPID-Shell-RCGI 2014/50279-4). The authors are grateful for suggestionsreceived from participants of the Interdisciplinary Colloquium on ProbabilityTheory, held on October 10, 2019 at IEA-USP (Institute of Advanced Studiesof the University of Sap Paulo), for early conversations with Julio Adolfo Zu-con Trecenti from ABJ (Brazilian Jurimetrics Association), and for the mobileinterface design conceived by Giovanni A. dos Santos and Joao Paulo A. S. E.Lins. The authors are grateful for the invitation of Jean-Yves Beziau, fromABF (Brazilian Academy of Philosophy), and for the effort of Jos´e Ac´acio deBarros and D´ecio Krause, organizers of the Festschrift celebrating Doria’s 75thbirthday.
References - A. Beutelspacher (1994).
Cryptology . Mathematical Association of America.- X. Bultel, J. Dreier, P. Lafourcade, M. More (2017). How to explain modern securityconcepts to your children.
Cryptologia , 41, 5, 422447. - M. R. Fellows, N. Koblitz (1994). Kid Crypto. Congressus Numerantium , 99, 941.- J.M. Hammersley and D.C. Handscomb (1964).
Monte Carlo Methods . Methuen.- V. Fossaluza, M.S. Lauretto, C.A.B. Pereira, and J.M. Stern (2015). Combiningoptimization and randomization approaches for the design of clinical trials.
SpringerProceedings in Mathematics and Statistics , 118, 173184. doi:10.1007/978-3-319-12454-4 14 - M.S. Lauretto, F. Nakano, C.A.B. Pereira, J.M. Stern (2012). Intentional Sam-pling by Goal Optimization with Decoupling by Stochastic Perturbation.
AmericanInstitute of Physics Conference Proceedings , 1490, 189-201. doi:10.1063/1.4759603 - M.S. Lauretto, R.B. Stern, K.L. Morgam, M.H. Clark, J.M. Stern (2017). Hap-hazard Intentional Allocation and Rerandomization to Improve Covariate Balance inExperiments.
American Institute of Physics Conference Proceedings , 1853, 050003,18. doi:10.1063/1.4985356 - N. Luhmann (1985).
A Sociological Theory of Law . Routledge.- N. Luhmann (1989).
Ecological Communication . The Univ. of Chicago Press.- M. Naor, M. Yung (1990). Public-key cryptosystems provably secure against chosenciphertext attacks.
Proceedings of the 22nd Annual ACM Symposium on Theory ofComputing , May 13-17, Baltimore, MD, p. 427-37.- D. Marcondes, C. Peixoto, and J.M. Stern (2019). Assessing randomness in caseassignment: The case study of the Brazilian Supreme Court.
Law, Probability andRisk , 18, 2/3, 97114. doi:10.1093/lpr/mgz006 - J. Pearl (2009).
Causality: Models, Reasoning, and Inference . Cambridge Univ.Press.- B. Ripley (1987).
Stochastic Simulation . Wiley.- P. Rogaway, T. Shrimpton (2004). Cryptographic hash-function basics: Definitions,implications, and separations for preimage resistance, second-preimage resistance,and collision resistance.
Lecture Notes in Computer Science , v.3017 p.371-88.- O.T. Saa, J.M. Stern (2019). Auditable Blockchain Randomization Tool.
Proceed-ings , 33, 1,, 17.117.6. doi:10.3390/proceedings2019033017 - P. Scozzafava (1993). Uniform distribution and sum modulo m of independentrandom variables.
Statistics and Probability Letters , 18, 4, 313-314.- M.V.M. Silva, M.A. Simplicio, R.A.C. Pfeiffer, J.M. Stern (2020).
A Fair, Traceable,Auditable and Participatory Randomization Tool for Legal Systems . arXiv:2006.02956 - J.M. Stern (2008). Decoupling, Sparsity, Randomization, and Objective BayesianInference. Cybernetics and Human Knowing , 15, 2, 49-68.- J.M. Stern (2011). Symmetry, Invariance and Ontology in Physics and Statistics.
Symmetry , 3, 3, 611-635. doi:10.3390/sym3030611 - J.M. Stern (2018). Verstehen (causal/interpretative understanding), Erkl¨aren (law-governed description/prediction), and Empirical Legal Studies.
JITE - Journal ofInstitutional and Theoretical Economics , 174, 1, 105-114. doi:10.1628/093245617X15120238641866doi:10.1628/093245617X15120238641866