Parallel Equivalence Class Sorting: Algorithms, Lower Bounds, and Distribution-Based Analysis
William E. Devanny, Michael T. Goodrich, Kristopher Jetviroj
PParallel Equivalence Class Sorting: Algorithms,Lower Bounds, and Distribution-Based Analysis
William E. Devanny Dept. of Computer ScienceUniv. of California, IrvineIrvine, CA 92697 USA [email protected]
Michael T. GoodrichDept. of Computer ScienceUniv. of California, IrvineIrvine, CA 92697 USA [email protected]
Kristopher JetvirojDept. of Computer ScienceUniv. of California, IrvineIrvine, CA 92697 USA [email protected]
Abstract
We study parallel comparison-based algorithms for finding all equiv-alence classes of a set of n elements, where sorting according to sometotal order is not possible. Such scenarios arise, for example, in applica-tions, such as in distributed computer security, where each of n agentsare working to identify the private group to which they belong, withthe only operation available to them being a zero-knowledge pairwise-comparison (which is sometimes called a “secret handshake”) that revealsonly whether two agents are in the same group or in different groups. Weprovide new parallel algorithms for this problem, as well as new lowerbounds and distribution-based analysis. In the
Equivalence Class Sorting problem, we are given a set, S , of n elementsand an equivalence relation, and we are asked to group the elements of the setinto their equivalence classes by only making pairwise equivalence tests (e.g.,see [12]). For example, imagine a convention of n political interns where eachperson at the convention belongs to one of k political parties, such as Republi-can, Democrat, Green, Labor, Libertarian, etc., but no intern wants to openly William E. Devanny was supported by an NSF Graduate Research Fellowship under grantDGE-1321846. a r X i v : . [ c s . D S ] M a y xpress his or her party affiliation unless they know they are talking with some-one of their same party. Suppose further that each party has a secret handshakethat two people can perform that allows them to determine whether they are inthe same political party (or they belong to different unidentified parties). Weare interested in this paper in the computational complexity of the equivalenceclass sorting problem in distributed and parallel settings, where we would liketo minimize the total number of parallel comparison rounds and/or the totalnumber of comparisons needed in order to classify every element in S .An important property of the equivalence class sorting problem is that it isnot possible to order the elements in S according to some total ordering thatis consistent with the equivalence classes. Such a restriction could come froma general lack of such an ordering or from security or privacy concerns. Forexample, consider the following applications: • Generalized fault diagnosis . Suppose that each of n different computersare in one of k distinct malware states, depending on whether they havebeen infected with various computer worms. Each worm does not wishto reveal its presence, but it nevertheless has an ability to detect whenanother computer is already infected with it (or risk autodetection by anexponential cascade, as occurred with the Morris worm [15]). But a wormon one computer is unlikely to be able to detect a different kind of wormon another computer. Thus, two computers can only compare each otherto determine if they have exactly the same kinds of infections or not. Thegeneralized fault diagnosis problem, therefore, is to have the n computersclassify themselves into k malware groups depending on their infections,where the only testing method available is for two computers to performa pairwise comparison that tells them that they are either in the samemalware state or they are in different states. This is a generalization of theclassic fault diagnosis problem, where there are only two states, “faulty”or “good,” which is studied in a number of interesting papers, includingone from the very first SPAA conference (e.g., see [4–6, 10, 17, 18]). • Group classification via secret handshakes . This is a cryptographic ana-logue to the motivating example given above of interns at a political con-vention. In this case, n agents are each assigned to one of k groups, suchthat any two agents can perform a cryptographic “secret handshake” pro-tocol that results in them learning only whether they belong to the samegroup or not (e.g., see [7,11,20,22]). The problem is to perform an efficientnumber of pairwise secret-handshake tests in a few parallel rounds so thateach agent identifies itself with the others of its group. • Graph mining . Graph mining is the study of structure in collections ofgraphs [8]. One of the algorithmic problems in this area is to classify whichof a collection of n graphs are isomorphic to one another (e.g., see [16]).That is, testing if two graphs are in the same group involves performing agraph isomorphism comparison of the two graphs, which is a computation2hat tends to be nontrivial but is nevertheless computationally feasible insome contexts (e.g., see [3]).Note that each of these applications contains two important features that formthe essence of the equivalence class sorting problem:1. In each application, it is not possible to sort elements according to a knowntotal order, either because no such total order exists or because it wouldbreak a security/privacy condition to provide such a total order.2. The equivalence or nonequivalence between two elements can be deter-mined only through pairwise comparisons.There are nevertheless some interesting differences between these applica-tions, as well, which motivate our study of two different versions of the equiv-alence class sorting problem. Namely, in the first two applications, the com-parisons done in any given round in an algorithm must be disjoint, since theelements themselves are performing the comparisons. In the latter two applica-tions, however, the elements are the objects of the comparisons, and we could,in principle, allow for comparisons involving multiple copies of the same elementin each round. For this reason, we allow for two versions of the equivalence classsorting problem: • Exclusive-Read (ER) version. In this version, each element in S can beinvolved in at most a single comparison of itself and another element in S in any given comparison round. • Concurrent-Read (CR) version. In this version, each element in S can beinvolved in multiple comparisons of itself and other elements in S in anycomparison round.In either version, we are interested in minimizing the number of parallel com-parison rounds and/or the total number of comparisons needed to classify everyelement of S into its group.Because we expect the number parallel comparison rounds and the totalnumber of comparisons to be the main performance bottlenecks, we are inter-ested here in studying the equivalence class sorting problem in Valiant’s parallelcomparison model [21], which only counts steps in which comparisons are made.This is a synchronous computation model that does not count any steps donebetween comparison steps, for example, to aggregate groups of equivalent ele-ments based on comparisons done in previous steps. In addition to the references cited above that motivate the equivalence classsorting problem or study the special case when the number of groups, k , is two,Jayapaul et al. [12] study the general equivalence class sorting problem, albeitstrictly from a sequential perspective. For example, they show that one can3olve the equivalence class sorting problem using O ( n /(cid:96) ) comparisons, where (cid:96) is the size of the smallest equivalence class. They also show that this problemhas a lower bound of Ω( n /(cid:96) ) even if the value of (cid:96) is known in advance.The equivalence class sorting problem is, of course, related to comparison-based algorithms for computing the majority or mode of a set of elements, forwhich there is an extensive set of prior research (e.g., see [1, 2, 9, 19]). None ofthese algorithms for majority or mode result in efficient parallel algorithms forthe equivalence class sorting problem, however. In this paper, we study the equivalence class sorting (ECS) problem from aparallel perspective, providing a number of new results, including the following:1. The CR version of the ECS problem can be solved in O ( k + log log n )parallel rounds using n processors, were k is the number of equivalenceclasses.2. The ER version of the ECS problem can be solved in O ( k log n ) parallelrounds using n processors, were k is the number of equivalence classes.3. The ER version of the ECS problem can be solved in O (1) parallel roundsusing n processors, for the case when (cid:96) is at least λn , for a fixed constant0 < λ ≤ .
4, where (cid:96) is the size of the smallest equivalence class.4. If every equivalence class is of size f , then solving the ECS problemrequires Ω( n /f ) total comparisons. This improves a lower bound ofΩ( n /f ) by Jayapaul et al. [12].5. Solving the ECS problem requires Ω( n /(cid:96) ) total comparisons, where (cid:96) isthe size of the smallest equivalence class. This improves a lower bound ofΩ( n /(cid:96) ) by Jayapaul et al. [12].6. In Section 4, we study how to efficiently solve the ECS problem when theinput is drawn from a known distribution on equivalence classes. In thissetting, we assume n elements have been sampled and fed as input to thealgorithm. We establish a relationship between the mean of the distri-bution and the algorithm’s total number of comparisons, obtaining upperbounds with high probability for a variety of interesting distributions.7. We provide the results of several experiments to validate the results fromSection 4 and study how total comparison counts change as parameters ofthe distributions change.Our methods are based on several novel techniques, including a two-phasedcompounding-comparison technique for the parallel upper bounds and the useof a new coloring argument for the lower bounds.4 Parallel Algorithms
In this section, we provide efficient parallel algorithms for solving the equivalenceclass sorting (ECS) problem in Valiant’s parallel model of computation [21]. Wefocus on both the exclusive-read (ER) and concurrent-read (CR) versions of theproblem, and we assume we have n processors, each of which can be assignedto one equivalence comparison test to perform in a given parallel round. Note,therefore, that any lower bound, T ( n ), on the total number of comparisonsneeded to solve the ECS problem (e.g., as given by Jayapaul et al. [12] and aswe discuss in Section 3), immediately implies a lower bound of Ω( T ( n ) /n ) forthe number of parallel rounds of computation using n processors per round. Forinstance, these lower bounds imply that the number of parallel rounds for solvingthe ECS problem with n processors must be Ω( n/(cid:96) ) and Ω( k ), respectively,where k is the number of equivalence classes and (cid:96) is the size of the smallestequivalence class.With respect to upper bounds, recall that Jayapaul et al. [12] studied theECS problem from a sequential perspective. Unfortunately, their algorithmcannot be easily parallelized, because the comparisons performed in a “round”of their algorithm depend on the results from other comparisons in that sameround. Thus, new parallel ECS algorithms are needed. In this subsection, we describe CR and ER algorithms based on knowledge ofthe number of groups, k .If two sets of elements are sorted into their equivalence classes, merging thetwo answers into the answer for the union requires at most k equivalence testsby simply performing a comparison between every pair of equivalence class onefrom the first answer and one from the second. This idea leads to the followingalgorithm, which uses a two-phased compounding-comparison technique to solvethe ECS problem:1. Initialize a list of n answers containing the individual input elements.2. While the number of processors per answer is less than 4 k , merge pairsof answers by performing k tests.3. While there is more than one answer, let ck be the number of processorsavailable per answer and merge c answers together by performing at most (cid:0) c (cid:1) k tests between each of the answers.We analyze this algorithm in the following two lemmas and we illustrate itin Figure 1. Lemma 1.
The first while loop takes O ( k ) rounds to complete.Proof. In each round the number of equivalence classes in an answer at mostdoubles until it reaches the upper bound of k . In loop iteration i ≤ (cid:100) log k (cid:101) , the5 umber of answers Processorsper answer Answer size Roundsneeded nn/ ∼ n/k
124 1 ≤ ≤ ∼ k ≤ k ≤ k kk/ k/ ≤ k/ ∼ k/ ∼ n/k ∼ n/ k ∼ k. . .. . .. . .. . . ... ... ... ... ...... ... ... ...1 ≤ k ∼ k ∼ n/k Answerreductionfactor22......... 1 ≤ k ∼ k ∼ n/ k ≤ k ≥ k ≤ n/ k ≤ k ≥ k ≤ n/ k ... ... ... ... ... k n ≤ n/ k ≥ k ≤ k ∼ ... ... 2 ≤ k ∼ k / ∼ n/k ≤ n/ i k ≥ i k ≤ k ∼ i First while loop phaseSecond while loop phase
Figure 1: A visualization of the parallel algorithm with a table on the rightkeeping track of relevant numbers for each loop iteration.6nswers are size at most 2 i and there are 2 i processors per answer. Therefore ittakes at most 2 i rounds to merge two answers. The number of rounds to reachthe (cid:100) log k (cid:101) loop iteration is O ( k ). For loop iterations (cid:100) log k (cid:101) < i < (cid:100) log k (cid:101) ,the answers are size at most k , but there are still at most 2 i processors peranswer. The number of rounds needed for these iterations is also O ( k ), as itforms a geometric sum that adds up to be O ( k ). This part of the algorithm isillustrated in the bottom half of Figure 1. Lemma 2.
The second while loop takes O (log log n ) rounds to complete.Proof. When entering the second while, there are more processors per answerthan needed to merge just two answers at a time. If an answer has access to ck processors, then a group of (cid:0) c (cid:1) answers can merge into one answer in a singleround. This means that if there are n/ ( ck ) answers at the start of a round,then we merge groups of c / n/ ( c k/ c ≥ i of the second while loop, there are at most n/ (2 i k ) answers. Andso the second while loop will terminate after O (log log n ) rounds with the singleanswer for the entire input. This is illustrated in the top half of Figure 1.Combining these two lemmas, we get the following. Theorem 1.
The CR version of the equivalence class sorting problem on n elements and k equivalence classes can be solved in O ( k + log log n ) parallelrounds of equivalence tests, using n processors in Valiant’s parallel comparisonmodel.Proof. Lemmas 1 and 2.We also have the following.
Theorem 2.
The ER version of the equivalence class sorting problem on n elements and k equivalence classes can be solved in O ( k log n ) parallel rounds ofequivalence tests, using n processors in Valiant’s parallel comparison model.Proof. Merging two answers for the ER version of the ECS problem model willalways take at most k rounds. Repeatedly merging answers will arrive at oneanswer in log n iterations. So equivalence class sorting can be done in O ( k log n )parallel rounds of equivalence tests. In this subsection, we describe ER algorithms based on knowledge of (cid:96) , the sizeof the smallest equivalence class. We assume in this section that (cid:96) ≥ λn , for someconstant λ >
0, and we show how to solve the ECS problem in this scenariousing O (1) parallel comparison rounds. Our methods are generalizations ofprevious methods for the parallel fault diagnosis problem when there are onlytwo classes, “good” and “faulty” [4–6, 10]. Let us assume, therefore, that thereare at least 3 equivalence classes.We begin with a theorem from Goodrich [10].7 heorem 3 (Goodrich [10]) . Let V be a set of n vertices, and let < γ, λ < .Let H d = ( V, E ) be a directed graph defined by the union of d independentrandomly-chosen Hamiltonian cycles on V (with all such cycles equally likely).Then, for all subsets W of V of λn vertices, H d induces at least one stronglyconnected component on W of size greater than γλn , with probability at least − e n [(1+ λ ) ln 2+ d ( α ln α + β ln β − (1 − λ ) ln(1 − λ ))]+ O (1) , where α = 1 − − γ λ and β = 1 − γ λ . In the context of the present paper, let us take γ = 1 /
4, so α = 1 − (3 / λ and β = 1 − (5 / λ . Let us also assume that λ ≤ .
4, since we are consideringthe case when the number of equivalence classes is at least 3; hence, the smallestequivalence class is of size at most n/ λ .So instead we use the following inequalities, which hold for x in the range [0 , . − x − x − x ≤ ln(1 − x ) ≤ − x − x − x . These bounds allow us to bound the main term, t , in the above probabilityof Theorem 3 (for γ = 1 /
4) as follows: t = α ln α + β ln β − (1 − λ ) ln(1 − λ )= (1 − λ ) ln(1 − λ ) + (1 − λ ) ln(1 − λ ) − (1 − λ ) ln(1 − λ ) ≤ (1 − λ ) (cid:32) − λ − (cid:18) λ (cid:19) − (cid:18) λ (cid:19) (cid:33) + (1 − λ ) (cid:32) − λ − (cid:18) λ (cid:19) − (cid:18) λ (cid:19) (cid:33) − (1 − λ ) (cid:18) − λ − λ − λ (cid:19) ≤ − λ + 19256 λ − λ , which, in turn, is at most − λ , for 0 < λ ≤ .
4. Thus, since this bound is negative for any constant 0 < λ ≤ . d to be a constant (depending on λ ) so that Theorem 3 holds withhigh probability.Our ECS algorithm, then, is as follows: That is, H d is defined by the union of cycles determined by d random permutations ofthe n vertices in V , so H d is, by definition, a simple directed graph.
8. Construct a graph, H d , as in Theorem 3, as described above, with d set toa constant so that the theorem holds for the fixed λ in the range (0 , . H d is a union of d Hamiltonian cycles. Thus, let us perform allthe comparisons in H d in 2 d rounds. Furthermore, we can do this set ofcomparisons even for the ER version of the problem. Moreover, since d is O (1), this step involves a constant number of parallel rounds (of O ( n )comparisons per round).3. For each strongly connected component, C , in H d consisting of elementsof the same equivalence class, compare the elements in C with the otherelements in S , taking | C | at a time. By Theorem 3, | C | ≥ λn/
8. Thus,this step can be performed in O (1 /λ ) = O (1) rounds for each connectedcomponent; hence it requires O (1) parallel rounds in total. Moreover, afterthis step completes, we will necessarily have identified all the members ofeach equivalence class.We summarize as follows. Theorem 4.
Suppose S is a set of n elements, such that the smallest equivalenceclass in S is of size at least λn , for a fixed constant, λ , in the range (0 , . . Thenthe ER version of the equivalence class sorting problem on S can be solved in O (1) parallel rounds using n processors in Valiant’s parallel comparison model. This theorem is true regardless of whether or not λ is known. If the valueof λ is not known, it is possible to repeatedly run the ECS algorithm startingwith an arbitrary constant of 0 . λ and halving the constant whenever thealgorithm fails. Once the value is less than the unknown λ , the algorithm willsucceed and the number of rounds will be independent of n and a function ofonly the constant λ .As we show in the next section, this performance is optimal when (cid:96) ≥ λn ,for a fixed constant λ ∈ (0 , . The following lower bound questions were left open by Jayapaul et al. [12]: • If every equivalence class has size f , the total number of comparisonsneeded to solve the equivalence class sorting problem Θ( n /f ) or Θ( n /f )? • Is the total number of comparisons for finding an element in the smallestequivalence class Θ( n /(cid:96) ) or Θ( n /(cid:96) )?Speaking loosely these lower bounds can be thought of as a question of howdifficult it is for an element to locate its equivalence class. The Θ( n /f ) and9 yx ≡ y ? x yx y TrueFalse
Figure 2: We test if x and y are in the same equivalence class. If they are, theirvertices are contracted together. If they are not, an edge is added.Θ( n /(cid:96) ) bounds can be interpreted as saying the average element needs to com-pare to at least one element in most of the other equivalence classes before itfinds an equivalent element. Because there must be (cid:0) x (cid:1) comparisons between x equivalence classes, the Θ( n /f ) and Θ( n /(cid:96) ) bounds say we do not needtoo many more comparisons then the very minimal number needed just to dif-ferentiate the equivalence classes. It seems unlikely that so few comparisons arerequired and we prove that this intuition is correct by proving lower bounds ofΩ( n /f ) and Ω( n /(cid:96) ) comparisons.Note that these lower bounds are on the total number of comparisons neededto accomplish a task, that is they bound the work a parallel algorithm wouldneed to perform. By dividing by n , they also give simple bounds on the numberof rounds needed in either the ER or CR models.With respect to such lower bound questions as these, let us maintain the stateof an algorithm’s knowledge about element relationships in a simple graph. Ateach step, the vertex set of this graph is a partition of the elements where eachset is a partially discovered equivalence class for S . Thus, each element in S isassociated with exactly one vertex in this graph at each step of the algorithm,and a vertex can have multiple elements from S associated with it. If a pairof elements was compared and found to not be equal, then there should be anedge in between the two vertices containing those elements. So initially thegraph has a vertex for each element and no edges. When an algorithm testsequivalence for a pair of elements, then, if the elements are not equivalent, theappropriate edge is added (if it is absent) and, if the elements are equivalent,the two corresponding vertices are contracted into a new vertex whose set isthe union of the two. A depiction of this is shown in Figure 2. An algorithmhas finished sorting once this graph is a clique and the vertex sets are thecorresponding equivalence classes.An equitable k -coloring of a graph is a proper coloring of a graph such thatthe size of each color class is either (cid:98) n/k (cid:99) or (cid:100) n/k (cid:101) . A weighted equitable k -coloring of a vertex weighted graph is a proper coloring of a graph such that10 Figure 3: On the left we have a graph with an equitable 3-coloring and on theright we have a graph with a weighted equitable 3-coloring.the sum of the weight in each color class is either (cid:98) n/k (cid:99) or (cid:100) n/k (cid:101) . Examples ofthese can be seen in Figure 3.An adversary for the problem of equivalence class sorting when every equiva-lence class has the same size f (so f divides n ) must maintain that the graph hasa weighted equitable n/f -coloring where the weights are the size of the vertexsets. The adversary we describe here will maintain such a coloring and addi-tionally mark the elements and the color classes in a special way. It proceedsas follows.First, initialize an arbitrary equitable coloring on the starting graph thatconsists of n vertices and no edges. For each comparison of two elements done bythe adversary algorithm, let us characterize how we react based on the followingcase analysis: • If either of the elements is unmarked and this comparison would increaseits degree to higher than n/ f , then mark it as having “high” elementdegree. • If either element is still unmarked, they currently have the same color,and there is another unmarked vertex such that it is not adjacent to avertex with the color involved in the comparison and no vertex with itscolor is adjacent to the unmarked vertex in the comparison (i.e. we canhave it swap colors with one of the vertices in the comparison), then swapthe color of that element and the unmarked element in the comparison. • If either element is still unmarked, they currently have the same color,and there is no other unmarked vertex with a different unmarked colornot adjacent to the color of the two elements being compared, then markall elements with the color involved in the comparison as having “high”color degree and mark the color as having “high” degree. • At this point, either both elements are marked and we answer based ontheir color, or one of the elements is unmarked and they have differentcolors, so we answer “not equal” to the adversary algorithm.At all times, the vertices that contain unmarked elements all have weightone, because the adversary only answers equivalent for comparisons once bothvertices are marked. When a color class is marked, all elements in that colorclass are marked as having “high” color degree. A few of the cases the adversarygoes through are depicted in Figure 4.11 ? ≡ ? ≡ ? ≡ ? ≡ ? ≡ ?Marking an element with high element degree:Swapping two colors:Marking blue with high color degree: Figure 4: Three cases of how the adversary works to mark vertices and swapcolors. The dashed line indicates the two elements being compared. Markedvertices are denoted with stars.
Lemma 3. If n/ elements are marked during the execution of an algorithm,then Ω( n /f ) comparisons were performed.Proof. There are three types of marked vertices: those with “high” elementdegree marks, those with “high” color degree marks, and those with both marks.The color classes must have been marked as having “high” degree when acomparison was being performed between two elements of that color class andthere were no unmarked color candidates to swap colors with. Because one ofthe elements in the comparison had degree less than n/ f , only a quarter of theelements have a color class it cannot be swapped with. So if there were at least n − n/ n/ n/ f compar-isons each. So if i color classes were marked and j elements were only markedwith “high” element degree, then the marked elements must have been a part12f a test at least ni/ nj/ f ≥ ( i + j/f ) n/ if + j ≥ n/
8, thenat least n / f equivalence tests were performed. Theorem 5.
If every equivalence class has the same size f , then sorting requiresat least Ω( n /f ) equivalence comparisons.Proof. When an algorithm finishes sorting, each vertex will have weight f andso the elements must all be marked. Thus, by Lemma 3, at least Ω( n /f )comparisons must have been performed.We also have the following lower bound as well. Theorem 6.
Finding an element in the smallest equivalence class, whose sizeis (cid:96) , requires at least Ω( n /(cid:96) ) equivalence comparisons.Proof. We use an adversary argument similar to the previous one, but we startwith (cid:96) vertices colored a special smallest class color (scc) and seperate theremaining n − (cid:96) vertices into (cid:98) ( n − (cid:96) ) / ( (cid:96) + 1) (cid:99) color classes of size n (cid:98) ( n − (cid:96) ) / ( (cid:96) +1) (cid:99) or n (cid:98) ( n − (cid:96) ) / ( (cid:96) +1) (cid:99) + 1.There are two changes to the previous adversary responses. First, the degreerequirement for having “high” degree is now n/ (cid:96) . Second, if an scc element isabout to be marked as having “high” degree, we attempt to swap its color withany valid unmarked vertex. Otherwise, we proceed exactly as before.If an algorithm attempts to identify an element as belonging to the smallestequivalence class, no scc elements are marked, and there have been fewer than n/ n/ n/ n /(cid:96) )equivalence tests are needed. In this subsection, we study a version of the equivalence class sorting problemwhere we are given a distribution, D , on a countable set, S , and we wish toenumerate the set in order of most likely to least likely, s , s , s , . . . . Forexample, consider the following distributions: • Uniform: In this case, D is a distribution on k equivalence classes, witheach equivalence class being equally likely for every element of S .13 Geometric: Here, D is a distribution such that the i th most probableequivalence class has probability p i (1 − p ). Each element “flips” a biasedcoin where “heads” occurs with probability p until it comes up “tails.”Then that element is in equivalence class i if it flipped i heads. • Poisson: In this case, D is model of the number of times an event occursin an interval of time, with an expected number of events determined by aparameter λ . Equivalence class i is defined to be all the samples that havethe same number of events occurring, where the probability of i eventsoccurring is λ i e − λ i ! . • Zeta: This distribution, D , is related to Zipf’s law, and models whenthe sizes of the equivalence classes follows a power law, based on a pa-rameter, s >
1, which is common in many real-world scenarios, such asthe frequency of words in natural language documents. With respect toequivalence classes, the i th equivalence class has probability i − s ζ ( s ) , where ζ ( s ) is Riemann zeta function (which normalizes the probabilitiesto sum to 1).So as to number equivalence classes from most likely to least likely, as i =0 , , . . . , define D N to be a distribution on the natural numbers such thatPr x ∼ D N [ x = i ] = Pr y ∼ D [ y = s i ] . Furthermore, so as to “cut off” this distribution at n , define D N ( n ) to be adistribution on the natural numbers less than or equal to n such that, for 0 ≤ i < n , Pr x ∼ D N ( n ) [ x = i ] = Pr y ∼ D N [ y = i ]and Pr x ∼ D N ( n ) [ x = n ] = Pr y ∼ D N [ y ≥ n ] . That is, we are “piling up” the tail of the D N distribution on n .The following theorem shows that we can use D N ( n ) to bound the numberof comparisons in an ECS algorithm when the equivalence classes are drawnfrom D . In particular, we focus here on an algorithm by Jayapaul et al. [12] forequivalence class sorting, which involves a round-robin testing regiment, suchthat each element, x , initiates a comparison with the next element, y , with anunknown relationship to x , until all equivalence classes are known. Theorem 7.
Given a distribution, D , on a set of equivalence classes, then n elements who have corresponding equivalence class independently drawn from D can be equivalence class sorted using a total number of comparisons stochasticallydominated by twice the sum of n draws from the distribution D N ( n ) . roof. Let V i denote the random variable that is equal to the natural numbercorresponding to the equivalence class of element i in D N ( n ). We denote thenumber of elements in equivalence class i as Y i . Let us denote the number ofequivalence tests performed by the algorithm by Jayapaul et al. [12] using therandom variable, R .By a lemma from [12], for any pair of equivalence classes, i and j , the round-robin ECS algorithm performs at most 2 min( Y i , Y j ) equivalence tests in total.Thus, the total number of equivalence tests in our distribution-based analysisis upper bounded by R ≤ ∞ (cid:88) i =0 i − (cid:88) j =0 Y i , Y j )= 2 n (cid:88) i =0 i − (cid:88) j =0 min( Y i , Y j ) + 2 ∞ (cid:88) i = n +1 i − (cid:88) j =0 min( Y i , Y j ) ≤ n (cid:88) i =0 i − (cid:88) j =0 Y i + 2 ∞ (cid:88) i = n +1 nY i ≤ (cid:32) n (cid:88) i =0 iY i + ∞ (cid:88) i = n +1 nY i (cid:33) = 2 n (cid:88) i =1 V i The second line in the above simplification is a simple separation of thedouble summation and the third line follows because (cid:80) i − j =0 min( Y i , Y j ) is zeroif Y i is zero and at most n , otherwise. So the total number of comparisons inthe algorithm is bounded by twice the sum of n draws from D N ( n ).Given this theorem, we can apply it to a number of distributions to showthat the total number of comparisons performed is linear with high probability. Theorem 8. If D is a discrete uniform, a geometric, or a Poisson distributionon a set equivalence classes, then it is possible to equivalence class sort usinglinear total number of comparisons with exponentially high probability.Proof. The sum of n draws from D N ( n ) is stochastically dominated by the sumof n draws from D N . Let us consider each distribution in turn. • Uniform: The sum of n draws from a discrete uniform distribution isbounded by n times the maximum value. • Geometric: Let p be the parameter of a geometric distribution and let X = (cid:80) n − i =0 X i where the X i are drawn from Geom ( p ), which is, of course,related to the Binomial distribution, Bin ( n, p ), where one flips n coinswith probability p and records the number of “heads.” Then, by a Chernoff15ound for the geometric distribution (e.g., see [14]),Pr[ X − (1 /p ) n > k ] = Pr[ Bin ( k + (1 /p ) n, p ) < n ] ≤ e − ( pk + n − n )2 k +(1 /p ) n Pr[
X > (2 /p ) n ] ≤ e − np • Poisson: Let λ be the parameter of a Poisson distribution and let Y = (cid:80) n − i =0 Y i where the Y i are drawn from P oisson ( λ ). Then, by a Chernoffbound for the Poisson distribution (e.g., see [14]),Pr[ Y > ( λ ( e −
1) + 1) n ] = Pr[ e Y > e ( λ ( e − n ] ≤ ( E [ e Y i ]) n e ( λ ( e − n = e λ ( e − n e ( λ ( e − n = e − n So, in each case with exponentially high probability, the sum of n draws from thedistribution is O ( n ) and the round-robin algorithm does O ( n ) total equivalencetests.We next address the zeta distribution. Theorem 9.
Given a zeta distribution with parameter s > , n elements whohave corresponding equivalence class independently drawn from the zeta distri-bution can be equivalence class sorted in O ( n ) work in expectation.Proof. When s >
2, the mean of the zeta distribution is ζ ( s − ζ ( s ) , which is a constant. So the sum of n draws from the distribution is expected tobe linear. Therefore, the expected total number of comparisons in the round-robin algorithm is linear.Unfortunately, for zeta distributions it is not immediately clear if it is possi-ble to improve the above theorem so that total number of comparisons is shownto be linear when 2 ≥ s > s cause the runtime to behave. Likewise, our high-probability bounds on the totalnumber of comparisons in the round-robin algorithm for the other distibutionsinvites experimental analysis as well. In this section, we report on experimental validatations of the theorems from theprevious section and investigations of the behavior of running the round-robin16lgorithm on the zeta distribution. For the uniform, geometric, and Poisson dis-tributions, we ran ten tests on sizes of 10 ,
000 to 200 ,
000 elements incrementingin steps of 10 , s < ,
000 to 20 ,
000 in incrementsof 1 , k = 10 , , p = , , Poisson: λ = 1 , , s = 1 . , . , , . s < s = 1 . s = 1 . , . s = 2, the data points vary by as much as 10%. In this paper we have studied the equivalence class sorting problem, from a par-allel perspective, giving several new algorithms, as well as new lower bounds anddistribution-based analysis. We leave as open problems the following interestingquestions: • Is it possible to find all equivalance classes in the ER version of the ECSproblem in O ( k ) parallel rounds, for k ≥
3, where k is the number ofequivalence classes? Note that the answer is “yes” for k = 2, as it followsfrom previous results for the parallel fault diagnosis problem [4–6]. • Is it possible to bound the number of comparisons away from O ( n ) forthe zeta distribution when s < • Is it possible to prove a high-probability concentration bound for the zetadistribution, similar to the concentration bounds we proved for other dis-tributions?
Acknowledgments
This research was supported in part by the National Science Foundation undergrant 1228639 and a gift from the 3M Corporation. We would like to thank17avid Eppstein and Ian Munro for several helpful discussions concerning thetopics of this paper.
References [1] L. Alonso and E. M. Reingold. Analysis of Boyer and Moore’s MJRTYalgorithm.
Information Processing Letters , 113(13):495–497, 2013.[2] L. Alonso, E. M. Reingold, and R. Schott. Determining the majority.
Information Processing Letters , 47(5):253 – 255, 1993.[3] L. Babai, P. Erd¨os, and S. M. Selkow. Random graph isomorphism.
SIAM Journal on Computing , 9(3):628–635, 1980.[4] R. Beigel, W. Hurwood, and N. Kahale. Fault diagnosis in a flash. In , pages571–580, 1995.[5] R. Beigel, S. R. Kosaraju, and G. F. Sullivan. Locating faults in aconstant number of parallel testing rounds. In , pages 189–198, 1989.[6] R. Beigel, G. Margulis, and D. A. Spielman. Fault diagnosis in a smallconstant number of parallel testing rounds. In , pages 21–29, 1993.[7] C. Castelluccia, S. Jarecki, and G. Tsudik. Secret handshakes fromCA-oblivious encryption. In P. J. Lee, editor,
Advances in Cryptology -ASIACRYPT , pages 293–307. Springer, 2004.[8] D. J. Cook and L. B. Holder.
Mining Graph Data . John Wiley & Sons,2006.[9] D. Dobkin and J. I. Munro. Determining the mode.
Theoretical ComputerScience , 12(3):255–263, 1980.[10] M. T. Goodrich. Pipelined algorithms to detect cheating in long-term gridcomputations.
Theoretical Computer Science , 408(2–3):199–207, 2008.[11] S. Jarecki and X. Liu. Unlinkable secret handshakes and key-privategroup key management schemes. In J. Katz and M. Yung, editors, , pages270–287. Springer, 2007.[12] V. Jayapaul, J. I. Munro, V. Raman, and S. R. Satti. Sorting andselection with equality comparisons. In F. Dehne, J.-R. Sack, andU. Stege, editors, , pages 434–445. Springer, 2015.1813] L. Kozma. Useful inequalities cheat sheet, 2016. .[14] M. Mitzenmacher and E. Upfal.
Probability and Computing: RandomizedAlgorithms and Probabilistic Analysis . Cambridge University Press, 2005.[15] H. Orman. The Morris worm: A fifteen-year perspective.
IEEE Security& Privacy , 1(5):35–43, 2003.[16] S. Parthasarathy, S. Tatikonda, and D. Ucar. A survey of graph miningtechniques for biological datasets. In C. C. Aggarwal and H. Wang,editors,
Managing and Mining Graph Data , pages 547–580. Springer, 2010.[17] A. Pelc and E. Upfal. Reliable fault diagnosis with few tests.
Combinatorics, Probability and Computing , 7:323–333, 1998.[18] F. P. Preparata, G. Metze, and R. T. Chien. On the connectionassignment problem of diagnosable systems.
IEEE Trans. on ElectronicComputers , EC-16(6):848–854, 1967.[19] M. E. Saks and M. Werman. On computing majority by comparisons.
Combinatorica , 11(4):383–387, 1991.[20] A. Sorniotti and R. Molva. A provably secure secret handshake withdynamic controlled matching.
Computers & Security , 29(5):619–627, 2010.[21] L. G. Valiant. Parallelism in comparison problems.
SIAM Journal onComputing , 4(3):348–355, 1975.[22] S. Xu and M. Yung. K-anonymous secret handshakes with reusablecredentials. In11th ACM Conf. on Computer and CommunicationsSecurity (CCS)