Optimal construction of a layer-ordered heap
Jake Pennington, Patrick Kreitzberg, Kyle Lucke, Oliver Serang
OOptimal construction of a layer-ordered heap ∗ Jake Pennington † Patrick Kreitzberg † Kyle Lucke ‡ Oliver Serang ‡ § August 18, 2020
Abstract
The layer-ordered heap (LOH) is a simple, recently pro-posed data structure used in optimal selection on X + Y ,the algorithm with the best known runtime for selectionon X + X + · · · + X m , and the fastest method in prac-tice for computing the most abundant isotope peaks ina chemical compound. Here, we introduce a few algo-rithms for constructing LOHs, analyze their complex-ity, and demonstrate that one algorithm is optimal forbuilding a LOH of any rank α . These results are shownto correspond with empirical experiments of runtimeswhen applying the LOH construction algorithms to acommon task in machine learning. ∗ Supported by grant number 1845465 from the National Sci-ence Foundation. † University of Montana Department of Mathematics ‡ University of Montana Department of Computer Science § Corresponding Author, Email: [email protected]
Copyright c (cid:13)(cid:13)
Copyright c (cid:13)(cid:13) a r X i v : . [ c s . D S ] A ug Introduction
Layer-ordered heaps (LOHs) are used in algorithmsthat perform optimal selection on A + B [8], algorithmswith the best known runtime for selection on X + X + · · · + X m [6], and the fastest known method forcomputing the most abundant isotopes of a compound[7]. A LOH of rank α consists of (cid:96) layers of values L ≤ L ≤ L ≤ · · · ≤ L (cid:96) − , where each L i is anunordered array and the ratio of the sizes, | L i +1 || L i | , tendsto α as the index, i , tends to infinity. One possible wayto achieve this is to have the exact size of each layer, | L i | , be p i − p i − where p i , the i th pivot, is calculatedas p i = (cid:108)(cid:80) ij =0 α j (cid:109) . The size of the last layer is thedifference between the size of the array and the lastpivot. Figure 1 depicts a LOH of rank α = 2 .Soft heaps [5] are qualitatively similar in thatthey achieve partial ordering in theoretically efficienttime; however, the disorder from a soft heap occursin a less regular manner, requiring client algorithmsto cope with a bounded number of “corrupt” items.Furthermore, they are less efficient in practice becauseof the discontiguous data structures and the greatercomplexity of implementation.Throughout this paper, the process of constructinga LOH of rank α from an array of length n will bedenoted “LOHification.” While LOHify with α = 1 is equivalent to comparison sort and α (cid:29) can beperformed in O ( n ) [6], the optimal runtime for anarbitrary α is unknown. Likewise, no optimal LOHifyalgorithm is known for arbitrary α .Here we derive a lower bound runtime for LOHi-fication, describe a few algorithms for LOHification,prove their runtimes, and demonstrate optimality of onemethod for any α . We then demonstrate the practicalperformance of these methods on a non-parametric statstest. ≤ ≤ ≤ L L L L Figure 1:
A LOH of rank 2
Pivot indices are shadedin gray. Notice that the last layer is not full.
In this sec-tion we will prove an asymptotic lower bound on thecomplexity of constructing a LOH in terms of n and α by first proving bounds on variables and then usingthose to bound the process as a whole. Lemma 2.1. (Upper bound on number of layers)
An upper bound on the number of layers, (cid:96) , in a LOHof n elements is log α ( n · ( α −
1) + 1) + 1 .Proof.
Because the final pivot can be no more than n ,the size of our array, we have the following inequality: (cid:38) (cid:96) − (cid:88) i =0 α i (cid:39) ≤ n (cid:96) − (cid:88) i =0 α i ≤ nα (cid:96) − − α − ≤ nα (cid:96) − − ≤ n · ( α − α (cid:96) − ≤ n · ( α −
1) + 1 (cid:96) − ≤ log α ( n · ( α −
1) + 1) (cid:96) ≤ log α ( n · ( α −
1) + 1) + 1
Lemma 2.2. (Lower bound on number of layers)
A lower bound on the number of layers, (cid:96) , in a LOH of n elements is log α ( n · ( α −
1) + 1) .Proof.
Because an additional pivot (after the finalpivot) must be more than n , the size of our array, wehave the following inequality: (cid:38) (cid:96) − (cid:88) i =0 α i (cid:39) > n (cid:96) − (cid:88) i =0 α i ≥ n , because n is discrete; α (cid:96) − α − ≥ nα (cid:96) − ≥ n · ( α − α (cid:96) ≥ n · ( α −
1) + 1 (cid:96) ≥ log α ( n · ( α −
1) + 1) (cid:96) > log α ( n · ( α − Lemma 2.3. (Asypmtotic number of layers)
The number of layers as n grows is asymptotic to log α ( n · ( α −
1) + 1) .Proof.
For α = 1 , the number of layers is n. lim α → log α ( n · ( α −
1) + 1)
Copyright c (cid:13)(cid:13)
Copyright c (cid:13)(cid:13) lim α → log( n · ( α −
1) + 1)log( α )= lim α → (cid:16) nn · ( α − (cid:17)(cid:0) α (cid:1) by L’Hôpital’s rule = lim α → n · αn · ( α −
1) + 1= n For α > we know log α ( n · ( α −
1) + 1) ≤ (cid:96) ≤ log α ( n · ( α −
1) + 1) + 1 . lim n →∞ log α ( n · ( α −
1) + 1) + 1log α ( n · ( α −
1) + 1)= lim n →∞ log α ( n · ( α −
1) + 1)log α ( n · ( α −
1) + 1) + 1log α ( n · ( α −
1) + 1)= 1 + 0= 1
Lemma 2.4. (Upper bound on size of layers)
Anupper bound on the size of layer i is | L i | ≤ (cid:100) α i (cid:101) .Proof. | L i | , as defined above, can be calculated by: | L i | = p i − p i +1 = i (cid:88) j =0 α j − i − (cid:88) j =0 α j ≤ (cid:100) α i (cid:101) + i − (cid:88) j =0 α j − i − (cid:88) j =0 α j ≤ (cid:100) α i (cid:101) Here wewill show that, for any α > , LOHification is in Ω (cid:16) n log( α − ) + n · α · log( α ) α − (cid:17) . Theorem 2.1. ∀ α > , LOHification ∈ Ω (cid:16) n log( α − ) + n · α · log( α ) α − (cid:17) Proof. If α = 1 , we are sorting, which is known to bein Ω( n log( n )) . Hence, for the following derivation, weshall assume that α > . From n ! possible unsorted ar-rays, LOHification produces one of | L | ! · | L | ! · · · | L (cid:96) − | ! possible valid results; hence, using an optimal decisiontree, r ( n ) is in Ω (cid:16) log (cid:16)(cid:0) n | L | , | L | ,..., | L (cid:96) − | (cid:1)(cid:17)(cid:17) ; hence, r ( n ) ∈ Ω (cid:32) log (cid:32) n ! (cid:81) (cid:96) − i =0 ( | L i | !) (cid:33)(cid:33) = Ω (cid:32) n log( n ) − (cid:96) − (cid:88) i =0 log( | L i | !) (cid:33) = Ω (cid:32) n log( n ) − (cid:96) − (cid:88) i =0 log( (cid:100) α i (cid:101) !) (cid:33) by Lemma .
4= Ω (cid:32) n log( n ) − (cid:96) − (cid:88) i =0 (cid:100) α i (cid:101) · log( (cid:100) α i (cid:101) ) (cid:33) (since log( n !) ∈ Θ( n log( n )) ) = Ω (cid:32) n log( n ) − (cid:96) − (cid:88) i =0 α i · log( α i ) (cid:33) by Lemma A.
1= Ω (cid:32) n log( n ) − (cid:96) − (cid:88) i =0 i · α i log( α ) (cid:33) = Ω (cid:32) n log( n ) − log( α ) · (cid:96) − (cid:88) i =0 i · α i (cid:33) = Ω ( n log( n ) − log( α ) · (cid:18) α (cid:96) +1 · ( (cid:96) −
1) + α − α (cid:96) · (cid:96) ( α − (cid:19)(cid:19) = Ω ( n log( n ) − log( α ) · (cid:18) ( α log α ( n · ( α − · log α ( n · ( α −
1) + 1) − α − + α ( α − − α log α ( n · ( α − · log α ( n · ( α −
1) + 1)( α − (cid:19)(cid:19) by Lemma .
3= Ω ( n log( n ) − log( α ) · (cid:18) ( n · ( α −
1) + 1) · α · (log α ( n · ( α −
1) + 1) − α − + α ( α − − ( n · ( α −
1) + 1) · log α ( n · ( α −
1) + 1)( α − (cid:19)(cid:19) = Ω ( n log( n ) − (( n · ( α −
1) + 1) · α · (log( n · ( α −
1) + 1) − log( α ))( α − + α log( α )( α − − ( n · ( α −
1) + 1) log( n · ( α −
1) + 1)( α − (cid:19)(cid:19) = Ω( n log( n ) − (cid:18) ( n · ( α −
1) + 1) · α log( n · ( α −
1) + 1)( α − Copyright c (cid:13)(cid:13)
1) + 1)( α − Copyright c (cid:13)(cid:13) ( n · ( α −
1) + 1) · α log( α )( α − + α log( α )( α − − ( n · ( α −
1) + 1) · log( n · ( α −
1) + 1)( α − (cid:19)(cid:19) = Ω( n log( n ) − (cid:18) ( n · ( α −
1) + 1) · log( n · ( α −
1) + 1) · ( α − α − + α log( α )( α − − ( n · ( α −
1) + 1) · α log( α )( α − (cid:19)(cid:19) = Ω( n log( n ) − ( n · ( α −
1) + 1) · log( n · ( α −
1) + 1) α − − α log( α )( α − + ( n · ( α −
1) + 1) · α log( α )( α − (cid:19) = Ω( n log( n ) − n · log( n · ( α −
1) + 1) − log( n · ( α −
1) + 1) α − − α log( α )( α − + n · α log( α ) α − α log( α )( α − (cid:19) = Ω( n log( n ) − n · log( n · ( α −
1) + 1) − log( n · ( α −
1) + 1) α − n · α log( α ) α − (cid:19) = Ω (cid:18) n log (cid:18) nn · ( α −
1) + 1 (cid:19) − log( n · ( α −
1) + 1) α − n · α log( α ) α − (cid:19) ⊆ Ω (cid:18) n log (cid:18) nn · ( α −
1) + 1 (cid:19) + n · α log( α ) α − (cid:19) by Lemma A.
2= Ω (cid:18) n log (cid:18) α − (cid:19) + n · α log( α ) α − (cid:19) by Lemma A. In some applications, it may be useful to have abound on LOHification that includes α = 1 . Theorem 2.2. ∀ α ≥ , LOHification ∈ Ω( n log( nn · ( α − ) + n · α · log( α ) α − ) Proof.
From our proof in Theorem . , we know thatLOHification with α > is in Ω( n log( nn · ( α − ) + n · α · log( α ) α − ) . Because LOHification with α = 1 is sorting,which is in Ω( n log( n )) , it just remains to show that ourbound is Ω( n log( n )) at α = 1 . Ω (cid:18) lim α → n log (cid:18) nn · ( α −
1) + 1 (cid:19) + n · α log( α ) α − (cid:19) = Ω (cid:18) n log( n ) + lim α → n · α log( α ) α − (cid:19) = Ω (cid:18) n log( n ) + lim α → n · (log( α ) + 1)1 (cid:19) by L’Hôpital’s rule = Ω ( n log( n ) + n )= Ω( n log( n )) We have now established LOHification to be in Ω( n log( nn · ( α − ) + n · α · log( α ) α − ) for any α at least 1.In the following sections, we will explore different algo-rithms for LOHification, their complexity, and for whatvalues of α they are optimal. Sorting in Θ( n log( n )) trivially LOHifies an array (sortingcan be done using any LOHification method by setting α = 1 ); note that this also guarantees the LOHproperty for any α ≥ , because any partitioning oflayers in a sorted array will have L i ≤ L i +1 . Hence,LOHification is in O ( n log( n )) . α = 1 indicateseach layer has | L i | = 1 , meaning an ordering over allelements; this means that sorting must be performed.Thus, for α = 1 , sorting is optimal. Furthermore, wecan find an α ∗ where sorting is optimal for all α ≤ α ∗ .Doing this, we find that, for any constant, C > ,sorting is optimal for α ∗ ≤ Cn . Theorem 2.3.
For any constant,
C > , sorting isoptimal for α ≤ (cid:0) Cn (cid:1) := α ∗ Proof.
Because decreasing α can only increase the num-ber of layers (and therefore the work), its suffices toshow that sorting is optimal at α ∗ = (cid:0) Cn (cid:1) .r( n ) ∈ Ω (cid:18) n log (cid:18) α ∗ − (cid:19) + n · α ∗ · log( α ∗ ) α ∗ − (cid:19) = Ω (cid:32) n log (cid:32) (cid:0) Cn (cid:1) − (cid:33) + n · (cid:0) Cn (cid:1) · log( (cid:0) Cn (cid:1) ) (cid:0) Cn (cid:1) − (cid:33) = Ω (cid:32) n log (cid:16) nC (cid:17) + ( n + C ) · log (cid:0) Cn (cid:1) Cn (cid:33) = Ω( n log( n ) − n log( C )+ ( n + C · n ) · log (cid:0) Cn (cid:1) C (cid:33) = Ω (cid:18) n · log( n ) + ( n + n ) · log (cid:18) Cn (cid:19)(cid:19) Copyright c (cid:13)(cid:13)
Because decreasing α can only increase the num-ber of layers (and therefore the work), its suffices toshow that sorting is optimal at α ∗ = (cid:0) Cn (cid:1) .r( n ) ∈ Ω (cid:18) n log (cid:18) α ∗ − (cid:19) + n · α ∗ · log( α ∗ ) α ∗ − (cid:19) = Ω (cid:32) n log (cid:32) (cid:0) Cn (cid:1) − (cid:33) + n · (cid:0) Cn (cid:1) · log( (cid:0) Cn (cid:1) ) (cid:0) Cn (cid:1) − (cid:33) = Ω (cid:32) n log (cid:16) nC (cid:17) + ( n + C ) · log (cid:0) Cn (cid:1) Cn (cid:33) = Ω( n log( n ) − n log( C )+ ( n + C · n ) · log (cid:0) Cn (cid:1) C (cid:33) = Ω (cid:18) n · log( n ) + ( n + n ) · log (cid:18) Cn (cid:19)(cid:19) Copyright c (cid:13)(cid:13) Ω( n · log( n ) + o ( n · log( n ))) Lemma A. ⊆ Ω( n · log( n )) Therefore;LOH ∈ Θ( n · log( n )) ∀ α ≤ (cid:18) Cn (cid:19) Because sorting is optimal for these values of α , weknow that, for all α at most a ∗ = (cid:0) Cn (cid:1) , LOHificationis in Θ( n log( nn · ( α − ) + n · α · log( α ) α − ) . Next we will lookat LOHification methods that are based on selection. LOHscan be constructed using one-dimensional selection(one-dimensional selection can be done in linear timevia median-of-medians [4]). In this section, we will de-scribe a LOHification algorithm that selects away layersfrom the end of the array, prove its complexity, and findfor which values of α it is optimal. This algorithm repeatedly performs a linear-time one-dimensional selection on the value at the firstindex (were the array in sorted order) in L (cid:96) − , then theLOH is partitioned about this value. This is repeatedfor L (cid:96) − , L (cid:96) − , and so on until the LOH has beenpartitioned about the minimum value in each layer. Wewill prove that this algorithm is in Θ (cid:16) α · nα − (cid:17) . Lemma 2.5.
Selecting away the layer with the greatestindex is in Ω (cid:16) α · nα − (cid:17) Proof.
By using a linear time one-dimensional selection,we can see that the runtime for selecting away the layerwith the greatest index is: r ( n ) ∈ Θ (cid:96) − (cid:88) i =0 n − (cid:96) − (cid:88) j = (cid:96) − i | L j | ⊆ Ω (cid:96) − (cid:88) i =0 n − (cid:96) − (cid:88) j = (cid:96) − i (cid:100) α j (cid:101) ⊆ Ω (cid:96) − (cid:88) i =0 n − (cid:96) − (cid:88) j = (cid:96) − i ( α j + 1) = Ω (cid:96) − (cid:88) i =0 n − i − (cid:96) − (cid:88) j = (cid:96) − i α j = Ω (cid:32) n · (cid:96) − (cid:96) − (cid:96) − α − · (cid:32) (cid:96) − (cid:88) i =0 ( α (cid:96) − α (cid:96) − i ) (cid:33)(cid:33) = Ω (cid:18) n · (cid:96) − (cid:96) − (cid:96) − α − · ( (cid:96) − α (cid:96) +1 − (cid:96) · α (cid:96) + αα − (cid:19) = Ω (cid:18) n · (cid:96) − (cid:96) − (cid:96) − α − · ( α − · (cid:96) · α (cid:96) − α · ( α (cid:96) − α − (cid:19) = Ω (cid:18) n · (cid:96) − (cid:96) − (cid:96) − (cid:96) · α (cid:96) α − α · ( α (cid:96) − α − (cid:19) = Ω (cid:18) (cid:96) · (cid:18) n − (cid:96) − − α (cid:96) α − (cid:19) + α · ( α (cid:96) − α − (cid:19) ⊆ Ω (cid:18) (cid:96) · (cid:18) n − (cid:96) − − n · ( α −
1) + 1 α − (cid:19) + α · n · ( α − α − (cid:19) by Lemma .
2= Ω (cid:18) (cid:96) · (cid:18) − (cid:96) − − α − (cid:19) + α · nα − (cid:19) = Ω (cid:18) α · nα − − (cid:18) (cid:96) − (cid:96) (cid:96)α − (cid:19)(cid:19) ⊆ Ω (cid:18) α · nα − − (cid:18) (log α ( n · ( α −
1) + 1)) − log α ( n · ( α − (cid:19) − (cid:18) log α ( n · ( α − α − (cid:19)(cid:19) by Lemma . and Lemma . ⊆ Ω (cid:18) α · nα − (cid:19) Lemma A. and Lemma A. Theorem 2.4.
Selecting away the layer with the great-est index is in Θ (cid:16) α · nα − (cid:17) Proof.
Using a linear time one-dimensional selection, wecan see that the runtime for selecting away the layerwith the greatest index is: r ( n ) ∈ Θ (cid:96) − (cid:88) i =0 n − (cid:96) − (cid:88) j = (cid:96) − i | L j | ⊆ O (cid:96) − (cid:88) i =0 n − (cid:96) − (cid:88) j = (cid:96) − i α j = O (cid:32) n · (cid:96) − α − · (cid:32) (cid:96) − (cid:88) i =0 ( α (cid:96) − α (cid:96) − i ) (cid:33)(cid:33) Copyright c (cid:13)(cid:13)
Using a linear time one-dimensional selection, wecan see that the runtime for selecting away the layerwith the greatest index is: r ( n ) ∈ Θ (cid:96) − (cid:88) i =0 n − (cid:96) − (cid:88) j = (cid:96) − i | L j | ⊆ O (cid:96) − (cid:88) i =0 n − (cid:96) − (cid:88) j = (cid:96) − i α j = O (cid:32) n · (cid:96) − α − · (cid:32) (cid:96) − (cid:88) i =0 ( α (cid:96) − α (cid:96) − i ) (cid:33)(cid:33) Copyright c (cid:13)(cid:13) O (cid:18) n · (cid:96) − α − · ( (cid:96) − α (cid:96) +1 − (cid:96) · α (cid:96) + αα − (cid:19) = O (cid:18) n · (cid:96) − α − · ( α − · (cid:96) · α (cid:96) − α · ( α (cid:96) − α − (cid:19) = O (cid:18) n · (cid:96) − (cid:96) · α (cid:96) α − α · ( α (cid:96) − α − (cid:19) = O (cid:18) (cid:96) · (cid:18) n − α (cid:96) α − (cid:19) + α · ( α (cid:96) − α − (cid:19) ⊆ O (cid:18) (cid:96) · (cid:18) n − n · ( α −
1) + 1 α − (cid:19) + α · n · ( α − α − (cid:19) by Lemma . O (cid:18) (cid:96) · (cid:18) − α − (cid:19) + α · nα − (cid:19) = O (cid:18) α · nα − − (cid:18) (cid:96)α − (cid:19)(cid:19) ⊆ O (cid:18) α · nα − − (cid:18) log α ( n · ( α − α − (cid:19)(cid:19) by Lemma . ⊆ O (cid:18) α · nα − (cid:19) by Lemma A. therefore by Lemma . ; r ( n ) ∈ Θ (cid:18) α · nα − (cid:19) Weshall also assume that α > as sorting is optimal for α = 1 . We will prove that this method is optimal for allvalues of α at least two, but not for all values of α lessthan two. Theorem 2.5.
Iterative selection is optimal for all α ≥ Proof.
LOHification is trivially done in Ω( n ) , as that isthe cost to load the data. As α increases, the numberof layers (hence the work) can only decrease, thus itsuffices to show iterative selection is optimal at α = 2 . r ( n ) ∈ O (cid:18) · n − (cid:19) Theorem . ∈ O ( n ) therefore;LOH ∈ Θ( n ) ∀ α ≥ Lemma 2.6.
Iterative selection is sub-optimal for α = α ∗ = 1 + Cn where C is any constant > . Proof. r ( n ) ∈ Θ (cid:18) α ∗ · nα ∗ − (cid:19) Theorem .
4= Θ (cid:32) (cid:0) Cn (cid:1) · n (cid:0) Cn (cid:1) − (cid:33) = Θ (cid:32) n + C (cid:0) Cn (cid:1) (cid:33) = Θ (cid:18) n + C · nC (cid:19) ⊆ Θ( n ) ⊆ ω ( n · log( n )) Theorem 2.6.
Iterative selection is sub-optimal for < α < Proof.
For this derivation, we shall look at the runtimeof iterative selection as a function of α defined by f ( α ) = α · nα − . We can see that f (cid:48) ( α ) = − n ( α − is negativefor all α > , thus it is decreasing on the interval α in (1 , ∞ ) . Because decreasing α can only increasethe number of layers (hence the runtime), we knowthe runtime is sub-optimal for α ≤ α ∗ by Lemma . .Because f ( α ) = α · nα − is continuous and decreasing onthe interval α in (1 , ∞ ) and sub-optimal at α = α ∗ ; it issub-optimal for α ∗ ≤ α < α (cid:48) where α (cid:48) is the first valueof α , greater than 1, for which f ( α ) = α · nα − is optimal.We can find α (cid:48) by solving: α (cid:48) · nα (cid:48) − n log (cid:18) α (cid:48) − (cid:19) + n · α (cid:48) · log ( α (cid:48) ) α (cid:48) − Which can be simplified to: α (cid:48) α (cid:48) − (cid:18) α (cid:48) − (cid:19) + α (cid:48) · log ( α (cid:48) ) α (cid:48) − We see that α (cid:48) = 2 is our solution. Therefore, iterativeselection is sub-optimal for < α < . For this algorithm, we first calculate the pivotindices in O ( n ) . Then, we perform a linear-time one-dimensional selection on the layers up to the medianpivot. We then recurse on the sub-problems until thearray is LOHified. Because one-dimensional selection isin Θ( n ) , the cost of every layer in the recursion is in Θ( n ) . Because splitting at the median pivot creates abalanced-binary recursion tree, the cost of the algorithm Copyright c (cid:13)(cid:13)
For this derivation, we shall look at the runtimeof iterative selection as a function of α defined by f ( α ) = α · nα − . We can see that f (cid:48) ( α ) = − n ( α − is negativefor all α > , thus it is decreasing on the interval α in (1 , ∞ ) . Because decreasing α can only increasethe number of layers (hence the runtime), we knowthe runtime is sub-optimal for α ≤ α ∗ by Lemma . .Because f ( α ) = α · nα − is continuous and decreasing onthe interval α in (1 , ∞ ) and sub-optimal at α = α ∗ ; it issub-optimal for α ∗ ≤ α < α (cid:48) where α (cid:48) is the first valueof α , greater than 1, for which f ( α ) = α · nα − is optimal.We can find α (cid:48) by solving: α (cid:48) · nα (cid:48) − n log (cid:18) α (cid:48) − (cid:19) + n · α (cid:48) · log ( α (cid:48) ) α (cid:48) − Which can be simplified to: α (cid:48) α (cid:48) − (cid:18) α (cid:48) − (cid:19) + α (cid:48) · log ( α (cid:48) ) α (cid:48) − We see that α (cid:48) = 2 is our solution. Therefore, iterativeselection is sub-optimal for < α < . For this algorithm, we first calculate the pivotindices in O ( n ) . Then, we perform a linear-time one-dimensional selection on the layers up to the medianpivot. We then recurse on the sub-problems until thearray is LOHified. Because one-dimensional selection isin Θ( n ) , the cost of every layer in the recursion is in Θ( n ) . Because splitting at the median pivot creates abalanced-binary recursion tree, the cost of the algorithm Copyright c (cid:13)(cid:13) s in Θ( n · d ) where d is the depth of the recursion tree.Because the number of pivots in each recursive call isone less than half of the number of pivots in the parentcall, we have d = log ( (cid:96) ) . Hence: r ( n ) ∈ Θ( n · log( (cid:96) ))= Θ( n · log(log α ( n · ( α −
1) + 1)))= Θ (cid:18) n · log (cid:18) log( n · ( α −
1) + 1)log( α ) (cid:19)(cid:19) Here we will show that thismethod is optimal for the values of α where sorting isoptimal, i.e. ≤ α ≤ α ∗ = 1 + Cn for any constant, C > . Then, however, we will show that it is notoptimal for some interval between α ∗ and two. Lemma 2.7.
Selecting to divide remaining pivot indicesin half is optimal for α = α ∗ = 1 + Cn for any constant, C > .Proof. r ( n ) ∈ Θ (cid:18) n · log (cid:18) log( n · ( α ∗ −
1) + 1)log( α ∗ ) (cid:19)(cid:19) = Θ (cid:32) n · log (cid:32) log (cid:0) n · (cid:0)(cid:0) Cn (cid:1) − (cid:1) + 1 (cid:1) log (cid:0) Cn (cid:1) (cid:33)(cid:33) = Θ (cid:32) n · log (cid:32) log( C ))log (cid:0) Cn (cid:1) (cid:33)(cid:33) = Θ( n · log( n )) Lemma A. Lemma 2.8.
Selecting to divide remaining pivot indicesin half is sub-optimal for α = 2 Proof. r ( n ) ∈ Θ (cid:18) n · log (cid:18) log( n · (2 −
1) + 1)log(2) (cid:19)(cid:19) = Θ( n · log(log( n + 1))) ⊆ Θ( n · log(log( n ))) ⊆ ω ( n ) Theorem 2.7.
Selecting to divide remaining pivot in-dices in half is sub-optimal for some interval in α ∗ <α ≤ Proof.
For this derivation, we shall look at the runtimeof dividing the remaining pivot indices in half as a function of α defined by f ( α ) = n · log (cid:16) log( n · ( α − α ) (cid:17) .By Lemma . and Lemma . , f ( α ) is optimal at α ∗ and sub-optimal at . Because f (cid:48) ( α ) = n · log( α ) · (cid:16) n · α · log( α ) − ( n · ( α − · log( n · ( α − n · ( α − · log ( α ) · α (cid:17) log( n · ( α −
1) + 1) is negative for large n and α > , the algorithm performsbetter as α increases. Because it is sub-optimal at α = 2 there must be an interval in ( α ∗ , where f ( α ) is sub-optimal. For this implementation of thealgorithm, we start by computing the pivots and thenperforming a linear-time selection algorithm on thepivot closest to the true median of the array to partitionthe array into two parts. We then recurse on the partsuntil all layers are generated. In this section, we willdescribe the runtime recurrence in detail, and thenprove that this method has optimal performance at any α . Let n s and n e bethe starting and ending indices (respectively) of our(sub)array. Let m ( n s , n e ) be the number of pivots be-tween n s and n e (exclusive). Let x ( n s , n e ) be the indexof the pivot closest to the middle of the (sub)arraystarting at n s and ending at n e . Then the runtime ofour algorithm is r (0 , n ) where r ( n s , n e ) = , n s ≥ n e , m ( n s , n e ) = 0 n e − n s + r ( n s , x ( n s , n e ) − r ( x ( n s , n e ) + 1 , n e ) , elseThe recurrence for this algorithm is solved by nei-ther the master theorem [3] nor the more general Akra-Bazzi method [1]. Instead, we will bound the runtime bybounding how far right we go in the recursion tree, t max ,and using this to find the deepest layer, d ∗ for which allbranches have work. Because performing two selectionsis in O ( n ) , we will bound the size of the recursions byhalf of the parent by selecting on the pivots on both sidesof the true median (if the true median is a pivot we justpay for it twice). From there, the bound on the runtimecan be computed as O ( d ∗ · n ) + O (cid:16)(cid:80) log( n ) d = d ∗ (cid:80) t max t =1 n d (cid:17) .This scheme is depicted in Figure 2. Copyright c (cid:13)(cid:13)
1) + 1) is negative for large n and α > , the algorithm performsbetter as α increases. Because it is sub-optimal at α = 2 there must be an interval in ( α ∗ , where f ( α ) is sub-optimal. For this implementation of thealgorithm, we start by computing the pivots and thenperforming a linear-time selection algorithm on thepivot closest to the true median of the array to partitionthe array into two parts. We then recurse on the partsuntil all layers are generated. In this section, we willdescribe the runtime recurrence in detail, and thenprove that this method has optimal performance at any α . Let n s and n e bethe starting and ending indices (respectively) of our(sub)array. Let m ( n s , n e ) be the number of pivots be-tween n s and n e (exclusive). Let x ( n s , n e ) be the indexof the pivot closest to the middle of the (sub)arraystarting at n s and ending at n e . Then the runtime ofour algorithm is r (0 , n ) where r ( n s , n e ) = , n s ≥ n e , m ( n s , n e ) = 0 n e − n s + r ( n s , x ( n s , n e ) − r ( x ( n s , n e ) + 1 , n e ) , elseThe recurrence for this algorithm is solved by nei-ther the master theorem [3] nor the more general Akra-Bazzi method [1]. Instead, we will bound the runtime bybounding how far right we go in the recursion tree, t max ,and using this to find the deepest layer, d ∗ for which allbranches have work. Because performing two selectionsis in O ( n ) , we will bound the size of the recursions byhalf of the parent by selecting on the pivots on both sidesof the true median (if the true median is a pivot we justpay for it twice). From there, the bound on the runtimecan be computed as O ( d ∗ · n ) + O (cid:16)(cid:80) log( n ) d = d ∗ (cid:80) t max t =1 n d (cid:17) .This scheme is depicted in Figure 2. Copyright c (cid:13)(cid:13) max d * Recursion Treelog(n) TopBottomNo work done
Figure 2:
The recursion tree for partitioning onthe pivot closest to the center of the array.
Thework at “Top” is in O ( n · d ∗ ) and the work done at“Bottom” is in O (cid:16)(cid:80) log( n ) d = d ∗ (cid:80) t max t =1 n d (cid:17) . For the derivation ofthe following bounds, we shall assume that α > . Lemma 2.9.
The number of pivots between any twopoints is m ( n s , n e ) ≤ log α (cid:16) n e · ( α − n s − · ( α − (cid:17) .Proof. By our definition, the i th pivot, p i , occurs at p i = (cid:108)(cid:80) ij =0 α j (cid:109) = (cid:108) α i +1 − α − (cid:109) . Let n s be the start of our(sub)array and n e be the end of our (sub)array. Thenthe number of pivots, p e , occurring before n e is boundby the inequality: n e ≥ (cid:24) α p e +1 − α − (cid:25) n e ≥ α p e +1 − α − n e · ( α − ≥ α p e +1 − n e · ( α −
1) + 1 ≥ α p e +1 log α ( n e · ( α −
1) + 1) ≥ p e + 1log α ( n e · ( α −
1) + 1) − ≥ p e Similarly, the number of pivots, p s , occurring before n s is bound by the inequality: n s ≤ (cid:24) α p s +1 − α − (cid:25) n s ≤ α p s +1 − α − n s − ≤ α p s +1 − α − n s − · ( α − ≤ α p s +1 − n s − · ( α −
1) + 1 ≤ α p s +1 log α (( n s − · ( α −
1) + 1) ≤ p s + 1log α (( n s − · ( α −
1) + 1) − ≤ p s By combining these two inequalities, we can find anupper bound on the number of pivots in the (sub)array, m ( n s , n e ) : m ( n s , n e ) ≤ (log α ( n e · ( α −
1) + 1) − − (log α (( n s − · ( α −
1) + 1) − m ( n s , n e ) ≤ log α ( n e · ( α −
1) + 1) − log α (( n s − · ( α −
1) + 1) m ( n s , n e ) ≤ log α (cid:18) n e · ( α −
1) + 1( n s − · ( α −
1) + 1 (cid:19)
Forthe following bounds, we will assume that α > . Let d be the depth of our current recursion (indexed at 0)and t be how far right in the tree we are at our currentrecursion (indexed at 1). To get an upper bound onthe recurrence, we will compute the cost of selecting forboth the first index before the true middle and the firstindex after the true middle. We will then treat the truemiddle as x ( n s , n e ) for our recursive calls. Under theserestrictions, n s = n · ( t − d and n e = n · t d for a given t and d . Knowing this, we can calculate bounds for m ( n s , n e ) in terms of t and d . m ( n s , n e ) ≤ log α (cid:18) n e · ( α −
1) + 1( n s − · ( α −
1) + 1 (cid:19) by Lemma . ≤ log α (cid:32) n · t d · ( α −
1) + 1( n · ( t − d − · ( α −
1) + 1 (cid:33) ≤ log α (cid:18) n · t · ( α −
1) + 2 d ( n · ( t − − d ) · ( α −
1) + 2 d (cid:19) We can then use this to calculate t , in terms of α , n and d for which m ( n s , n e ) < . This will give us abound on how far right we go in the recursion tree. log α (cid:18) n · t · ( α −
1) + 2 d ( n · ( t − − d ) · ( α −
1) + 2 d (cid:19) < (cid:18) n · t · ( α −
1) + 2 d ( n · ( t − − d ) · ( α −
1) + 2 d (cid:19) < αn · t · ( α −
1) + 2 d < α · ( n · ( t − − d ) · ( α − α · d n · t · ( α − < α · ( n · ( t − − d ) · ( α − α − · d Copyright c (cid:13) · t < α · ( n · ( t − − d ) + 2 d n · t < α · n · t − α · n − α · d + 2 d t < α · t − α − ( α − · d nt − α · t < − α − ( α − · d nt · ( α − > α + ( α − · d nt > αα − d n Because d ≤ n at any layer of the recursion, t max = αα − + 1 . Using this, we can define r ∗ , an upper boundon our runtime recurrence where r (0 , n ) ≤ r ∗ (1 , and r ∗ ( t, d ) = t > t max d > n n d + r (2 · t − , d + 1) + r (2 · t, d + 1) else . Theorem 2.8.
For α > , partitioning on the pivotclosest to the center of the array is in O (cid:16) n log (cid:16) αα − (cid:17)(cid:17) Proof.
Let d ∗ be the largest d for which all branchesat layer d have work. Because t max = αα − + 1 , d ∗ = log ( αα − + 1) . This yields: r ( n ) ≤ r ∗ ( n ) ∈ O log( n ) (cid:88) d = d ∗ t max (cid:88) t =1 n d + O ( n · d ∗ ) ∈ O log( n ) (cid:88) d = d ∗ αα − · n d + O (cid:18) n · log (cid:18) αα − (cid:19)(cid:19) ∈ O n · αα − · log( n ) (cid:88) d = d ∗ d + O (cid:18) n · log (cid:18) αα − (cid:19)(cid:19) ∈ O (cid:18) n · αα − · (cid:16) − d ∗ − − log( n ) (cid:17)(cid:19) + O (cid:18) n · log (cid:18) αα − (cid:19)(cid:19) ∈ O (cid:18) n · αα − · (cid:18) · α − · α − − n (cid:19)(cid:19) + O (cid:18) n · log (cid:18) αα − (cid:19)(cid:19) ∈ O (cid:18) · n · α · α − − αα − (cid:19) + O (cid:18) n · log (cid:18) αα − (cid:19)(cid:19) ∈ O ( n ) + O (cid:18) n · log (cid:18) αα − (cid:19)(cid:19) ∈ O (cid:18) n · log (cid:18) αα − (cid:19)(cid:19) Theorem 2.9.
For α = 1 , partitioning on the pivotclosest to the center of the array is optimal.Proof. Because we are sorting in this case, it sufficesto show that this method is in O ( n log( n )) . Let d ∗ bethe largest d for which all branches at that layer havework. Because α = 1 , all branches have work. Thus d ∗ = log ( n ) . This yields: r ( n ) ≤ r ∗ ( n ) ∈ O ( n · d ∗ ) ∈ O ( n log( n )) Lemma 2.10. partitioning on the pivot closest to thecenter of the array is optimal for ≤ α ≤ α ∗ = 1 + Cn for any constant, C > .Proof. By Theorem . , this method sorts an array in O ( n log( n )) . Because a sorted array is also a LOH ofany order and LOHification of order α ∗ = 1 + Cn for anyconstant, C > , is in Ω( n log( n )) by Theorem . ; thismethod is optimal for ≤ α ≤ α ∗ . Lemma 2.11. partitioning on the pivot closest to thecenter of the array is optimal for α ≥ Proof. r ( n ) ∈ O (cid:18) n · log (cid:18) αα − (cid:19)(cid:19) = O ( n ) ⊆ Θ( n ) Theorem 2.10. partitioning on the pivot closest to thecenter of the array is optimal for all α ≥ Proof.
By Lemma . and Lemma . , it suffices toshow that partitioning on the pivot closest to the centerof the array is optimal for α ∗ < α < . Suppose α ∗ < α < . Then: r ( n ) ∈ Ω (cid:18) n log (cid:18) α − (cid:19) + n · α · log( α ) α − (cid:19) by Theorem . Copyright c (cid:13)(cid:13)
By Lemma . and Lemma . , it suffices toshow that partitioning on the pivot closest to the centerof the array is optimal for α ∗ < α < . Suppose α ∗ < α < . Then: r ( n ) ∈ Ω (cid:18) n log (cid:18) α − (cid:19) + n · α · log( α ) α − (cid:19) by Theorem . Copyright c (cid:13)(cid:13) Ω (cid:18) n log (cid:18) α − (cid:19) + n (cid:19) ⊆ Ω (cid:18) n log (cid:18) α − (cid:19)(cid:19) By Theorem . , we have: r ( n ) ∈ O (cid:18) n · log (cid:18) αα − (cid:19)(cid:19) ∈ O (cid:18) n · log (cid:18) α − (cid:19) + n · log( α ) (cid:19) ∈ O (cid:18) n · log (cid:18) α − (cid:19)(cid:19) hence; r ( n ) ∈ Θ (cid:18) n · log (cid:18) α − (cid:19)(cid:19) Partitioningon the pivot closest to the center of the array is optimalfor all α ≥ by Theorem . . We can combine thiswith Theorem . to determine that LOHification is in: Θ (cid:18) n log (cid:18) nn · ( α −
1) + 1 (cid:19) + n · α · log( α ) α − (cid:19) For this implementation of thealgorithm, we partition on a random element, recordthe index of this element in an auxiliary array and thenrecurse on the left side until the best element is selected.While this method is probabilistic with a worst caseconstruction in O ( n ) , it performs well in practice andhas a linear expected construction time. Quick LOHify can be thought of as a Quick-Selectionwith k = 1 and a constant number of operations perrecursion for the auxiliary array. By this, we know theexpected runtime to be in Θ( n ) . A direct proof is alsoprovided. Theorem 2.11.
The expected runtime for Quick LO-Hify is in Θ( n ) Proof.
The runtime is proportional to the number ofcomparisons. Suppose x i is the i th element in the sortedarray and assume without loss of generality that i < j .We compare x i and x j only when one of these valuesis the pivot element. This makes the greatest possibleprobability that these elements are compared j as jis the minimum range that contains these elements.The expected number of comparisons can be found bysumming this probability over all pairs of elements. This yields: E = n − (cid:88) i =0 n − (cid:88) j = i +1 j = 2 · n − (cid:88) i =0 n − (cid:88) j = i +1 j = 2 · + + · · · + n − + + + · · · + n − + + · · · + n − + . . . + ...+ n − = 2 · ( n − · n − ∈ Θ( n ) α of Quick LOHify Unlike otherconstructions of a LOH, an α is not specified whenperforming Quick LOHify nor is it guaranteed to be thesame across different runs. We can, however, determinethat the expected value of α to be in Θ(log( n )) . Theorem 2.12.
The expected α for Quick LOHify is in Θ(log( n )) Proof.
The average α , α (cid:48) , can be computed as theaverage ratio of the last two layers. This can be foundby dividing the sum of all ratios by the number of waysto choose the pivots. This yields: α (cid:48) = 1 (cid:0) n (cid:1) · n − (cid:88) i =0 n − (cid:88) j = i +1 n − jj − i = 2 n − n · n − (cid:88) i =0 n − (cid:88) j = i +1 n − jj − i = 2 n − n · n − (cid:88) i =0 n − i − (cid:88) k =1 n − i − kk = 2 n − n · n − (cid:88) i =0 (cid:32) n − i − (cid:88) k =1 n − ik − (cid:33) = 2 n − n · n − (cid:88) i =0 (cid:32)(cid:32) n − i − (cid:88) k =1 n − ik (cid:33) − ( n − i − (cid:33) = 2 n − n · n − (cid:88) i =0 (cid:32) ( n − i ) · (cid:32) n − i − (cid:88) k =1 k (cid:33) − n + i + 2 (cid:33) = 2 n − n · n − (cid:88) i =0 (( n − i ) · H n − i − − n + i + 2) Copyright c (cid:13) n − n · n − (cid:88) i =0 ( n · H n − i − − i · H n − i − − n + i + 2)= 2 n − n · (cid:32)(cid:32) n · n − (cid:88) i =0 ( H n − i − ) (cid:33) − (cid:32) n − (cid:88) i =0 ( i · H n − i − ) (cid:33) − ( n − · n ) + (cid:32) n − (cid:88) i =0 i (cid:33) + (2 · n − (cid:33) = 2 n − n · (cid:32)(cid:32) n · n − (cid:88) k =1 H k (cid:33) − (cid:32) n − (cid:88) k =1 ( n − k − · H k (cid:33) − ( n − · n ) + n − · n + 22 + (2 · n − (cid:19) = 2 n − n · (cid:32)(cid:32) n − (cid:88) k =1 ( k + 1) · H k (cid:33) + − n + 5 n − (cid:33) † = 2 n − n · (cid:18) ( n + n ) · H n − n − · n − n + 5 · n − (cid:19) = 2 · n · H n + 2 · n · H n − · n + 7 · n − · n − · n ∈ Θ(log( n )) † Simplified with Wolfram Mathematica
Sum[Sum[(k +1)/i, i, 1, k], k, 1, -1 + n]
We compare the runtimes of various LOHify algo-rithms to compute the most permissive score thresholdat which a given false discovery rate (FDR) [2] τ oc-curs. This is traditionally accomplished by sorting thescored hypotheses (which are labeled as TP or FP) bestfirst and then advancing one element at a time, updat-ing the FDR to the current F DR = F P
F P + T P at thethreshold, finding the worst score at which
F DR ≤ τ occurs.The LOH method behaves similarly, but they com-pute optimistic bounds on the FDRs in each layer (ifall TPs in the layer come first) and pessimistic bounds(if all FPs in the layer come first). When these boundsinclude τ , the layer is recursed on, until the size of thelist is in O (1) .Table 1 demonstrates the performance benefit andthe influence of α on practical performance. α = 6 . (where applicable) n SORT SLWGI SDRPIH PPCCA QUICK Size = α SLWGI SDRPIH PPCCA1.05 29.9283 16.8750 14.48781.1 19.3675 14.8553 12.90621.5 8.92916 11.0468 8.122652.0 8.24106 9.24010 8.212613.0 5.73344 7.94349 5.600234.0 3.83187 6.35753 3.840146.0 3.60989 6.64702 3.551668.0 4.62627 6.90307 4.5759
Table 1:
Runtimes (seconds) of different LOHifi-cation methods with various α . Reported runtimesare averages over 10 iterations. SORT is sorting, SLWGIis selecting the layer with the greatest index, SDRPIHis selecting to divide the remaining pivot indices in half,PPCCA is partitioning on the pivot closest to the cen-ter of the array, and QUICK is Quick-LOHify. Quick-LOHify generates its own partition indices, which arenot determined by α . Due to the Ω( n log( n )) bound on comparison-basedsorting, ordering values using only pairwise comparisonis generally considered to be an area for little practicalperformance benefit; however, LOHs have been usedto replace sorting in applications where sorting is alimiting factor. Optimal LOHify for any α and thepractically fast Quick-LOHify variant are useful toreplace sorting in applications such as finding the mostabundant isotopes of a compound [7] (fast in practicewith < α (cid:28) ) and finding the score at which a desiredFDR threshold occurs (fast in practice with an α (cid:29) ). Copyright c (cid:13) eferences [1] M. Akra and L. Bazzi,
On the Solution of LinearRecurrence Equations , Computational Optimizationand Applications, 10(2) (2000), pp. 195–210.[2] Y. Benjamini, and Y. Hochberg,
Controlling the falsediscovery rate: a practical and powerful approach tomultiple testing , Journal of the Royal Statistical Soci-ety B, 57 (1995), pp. 289–300.[3] J. L. Bently, D. Haken, and J. B. Saxe,
A GeneralMethod for Solving Divide-and-Conquer Recurrences ,SIGACT News, 12(3) (1980), pp. 36–44.[4] M. Blum, R. W. Floyd, V. R. Pratt, R. L. Rivest, andR. E. Tarjan,
Time bounds for selection , Journal ofComputer and System Sciences, 7(4) (1973), pp. 448–461.[5] B. Chazelle,
The soft heap: an approximate priorityqueue with optimal error rate , Journal of the ACM(JACM), 47(6) (200), pp. 1012–1027.[6] P. Kreitzberg, K. Lucke, and O. Serang
Selection on X + X + · · · + X m with layer-ordered heaps , arXivpreprint arXiv:1910.11993, (2019), Not yet submitted.[7] P. Kreitzberg, J. Pennington, K. Lucke, O. Serang, Fastexact computation of the k most abundant isotope peakswith layer-ordered heaps , Analytical Chemistry, (JustAccepted) (2020), PMID: 32663022.[8] O. Serang,
Optimal selection on X + Y simplified withlayer-ordered heaps , arXiv preprint arXiv:2001.11607,(2020)
Copyright c (cid:13) ppendixA Lemmas used in methods
Lemma A.1. ∀ α > , (cid:100) α i (cid:101) · log( (cid:100) α i (cid:101) ) ∼ α i · log( α i ) Proof. α i · log( α i ) ≤ (cid:100) α i (cid:101)· log( (cid:100) α i (cid:101) ) ≤ ( α i +1) · log( α i +1)lim i →∞ ( α i + 1) · log( α i + 1) α i · log( α i )= lim i →∞ α i log( α i + 1) α i log( α i ) + log( α i + 1) α i log( α i )= lim i →∞ log( α i + 1)log( α i ) + log( α i + 1) α i log( α i ) which, by L’Hôpital’s rule; = lim i →∞ α i log( α ) α i +1 log( α ) + α i · log( α ) α i +1 α i · log( α ) · (log( α i ) + 1)= lim i →∞ α i α i + 1 + 1( α i + 1) · (log( α i ) + 1)= lim i →∞ − α i + 1= 1 Lemma A.2. ∀ α > , log( n · ( α − α − ∈ o ( n ) Proof. lim n →∞ log( n · ( α − α − n = lim n →∞ log( n · ( α −
1) + 1) n · ( α − n →∞ α − α − · ( n · ( α −
1) + 1) by L’Hôpital’s rule = lim n →∞ n · ( α −
1) + 1= 0
Lemma A.3. ∀ α > , n log( nn · ( α − ) ∼ n log( α − ) Proof. lim n →∞ n log (cid:16) nn · ( α − (cid:17) n log (cid:16) α − (cid:17) = lim n →∞ log (cid:16) nn · ( α − (cid:17) log (cid:16) α − (cid:17) = 1log( α − ) · (cid:18) lim n →∞ log (cid:18) nn · ( α −
1) + 1 (cid:19)(cid:19) = 1log( α − ) · (cid:18) log (cid:18) lim n →∞ nn · ( α −
1) + 1 (cid:19)(cid:19) = 1log( α − ) · log( 1 α − by L’Hôpital’s rule = 1 Lemma A.4.
For any constant,
C > , ( n + n ) · log(1+ Cn ) ∈ o ( n · log( n )) Proof. lim n →∞ ( n + n ) · log(1 + Cn ) n · log( n )= lim n →∞ ( n + 1) · log(1 + Cn )log( n )= lim n →∞ n · log(1 + Cn )log( n ) + log(1 + n )log( n )= lim n →∞ n · log(1 + Cn )log( n )= lim n →∞ log(1 + Cn ) − Cn + C ( Cn ) by L’Hôpital’s rule = lim n →∞ log(1 + Cn )( Cn ) − nn + C = lim n →∞ − ( Cn + C · n ) − ( Cn ) − by L’Hôpital’s rule = lim n →∞ n n + C · n −
1= lim n →∞ nn + C − by L’Hôpital’s rule = 1 − by L’Hôpital’s rule = 0 Lemma A.5. ∀ α > , (cid:16) (log α ( n · ( α − − log α ( n · ( α − (cid:17) ∈ o (cid:16) α · nα − (cid:17) Proof. lim n →∞ (cid:16) (log α ( n · ( α − − log α ( n · ( α − (cid:17)(cid:16) α · nα − (cid:17) = lim n →∞ α − α )) · (log( n · ( α −
1) + 1)) · α · n − α − α ) · log( n · ( α −
1) + 1)2 · α · n Copyright c (cid:13)(cid:13)
1) + 1)2 · α · n Copyright c (cid:13)(cid:13) lim n →∞ · ( α − (log( α )) · log( n · ( α − n · ( α − − (cid:24)(cid:24)(cid:24)(cid:24)(cid:24)(cid:24)(cid:24)(cid:24)(cid:88)(cid:88)(cid:88)(cid:88)(cid:88)(cid:88)(cid:88)(cid:88) ( α − log( α ) · n · ( α − · α by L’Hôpital’s rule = lim n →∞ ( α − α · (log( α )) · log( n · ( α −
1) + 1) n · ( α −
1) + 1= ( α − α · (log( α )) · (cid:18) lim n →∞ log( n · ( α −
1) + 1) n · ( α −
1) + 1 (cid:19) = ( α − α · (log( α )) · (cid:18) lim n →∞ n · ( α −
1) + 1 (cid:19) by L’Hôpital’s rule = 0
Lemma A.6. ∀ α > , (cid:16) log α ( n · ( α − α − (cid:17) ∈ o (cid:16) α · nα − (cid:17) Proof. lim n →∞ (cid:16) log α ( n · ( α − α − (cid:17)(cid:16) α · nα − (cid:17) = lim n →∞ log α ( n · ( α − α · n = lim n →∞ α · log( α ) · log( n · ( α − n = 1 α · log( α ) · (cid:18) lim n →∞ log( n · ( α − n (cid:19) = 1 α · log( α ) · (cid:18) lim n →∞ αn · ( α − (cid:19) by L’Hôpital’s rule = 0 Lemma A.7. n · log (cid:18) log( C )log ( Cn ) (cid:19) ∈ Θ( n · log( n )) Proof. lim n →∞ n · log (cid:18) log( C )log ( Cn ) (cid:19) n · log( n )= lim n →∞ log (cid:18) log( C )log ( Cn ) (cid:19) log( n )= lim n →∞ (cid:18) Cn · ( n + C ) · log ( Cn ) (cid:19)(cid:0) n (cid:1) by L’Hôpital’s rule = lim n →∞ n + C ) · log (cid:0) Cn (cid:1) = lim n →∞ (cid:16) n + C (cid:17) log (cid:0) Cn (cid:1) = lim n →∞ (cid:16) − n +2 · C · n + C (cid:17)(cid:16) − Cn + C · n (cid:17) by L’Hôpital’s rule = lim n →∞ n + C · nC · n + 2 · C · n + C = 1 C by L’Hôpital’s rule Copyright c (cid:13)(cid:13)