Generalized Non-adaptive Group Testing
11 Generalized Non-adaptive Group Testing
Xiwei Cheng , Sidharth Jaggi , , Qiaoqiao Zhou The Chinese University of Hong Kong, University of Bristol
Abstract
In the problem of classical group testing one aims to identify a small subset (of expected size d ) diseased individuals/defectiveitems in a large population (of size n ) based on a minimal number of suitably-designed group non-adaptive tests on subsets ofitems, where the test outcome is governed by an “OR” function, i.e., the test outcome is positive iff the given test containsat least one defective item. Motivated by physical considerations we consider a generalized scenario (that includes as specialcases multiple other group-testing-like models in the literature) wherein the test outcome is governed by an arbitrary monotone (stochastic) test function f ( · ) , with the test outcome being positive with probability f ( x ) , where x is the number of defectivestested in that pool. For any monotone test function f ( · ) we present a non-adaptive generalized group-testing scheme that identifiesall defective items with high probability. Our scheme requires at most O ( d log( n )) tests for any monotone test function f ( · ) ,and at most O ( d log( n )) in the physically relevant sub-class of sensitive test functions (and hence is information-theoreticallyorder-optimal for this sub-class).A short video describing this paper (narrated by the first author) is at [1]. I. I
NTRODUCTION
Group testing [2] is the non-linear sparse recovery process of identifying a small subset of defective items from a largerset of items based on a series of judiciously designed tests. Each test is carried out on a subset of items simultaneously, andeach binary outcome indicates whether or not the test includes at least one defective item. In other words, the test outcomeis specified by the “OR” function. In designing the testing scheme, it is desirable to minimize the number of tests whilestill enabling high probability of correct identification of the subset of defective items. The group testing paradigm findsapplications in a wide variety of contexts, including biology [4], pattern finding [5], wireless communications [6], [7], andtesting for diseases recently COVID-19 testing [8].Many variants of the classical group testing paradigm have already been considered in the literature. For example, Dam-aschke [9] considered threshold test functions : the test outcome is negative if the number of defectives in a test is no larger thanthe lower threshold (cid:96) ; positive if no smaller than the upper threshold u ; and arbitrary (negative or positive) otherwise. Let n and d be the number of all items and the number of defective items, respectively. For u = (cid:96) + 1 , [9] proposed an adaptive algorithmwith the number of tests scales as O (cid:0) ( d + u ) log n (cid:1) to exactly identify the defectives. However, for u > (cid:96) + 1 , they provedthat the defectives cannot be exactly identified, but O (cid:0) ( dn b + d u ) log n (cid:1) adaptive tests suffice to identify the defectives if up to ( u − b ) − (cid:96) misidentifications are allowed (here b > is an arbitrary constant). Chen and Fu [10] proposed a non-adaptivealgorithm for which the number of tests scales as O (cid:0) ed u +1 log( nd ) (cid:1) if up to u − (cid:96) − misidentifications and e erroneous testsare allowed. Subsequently, Cheraghchi [11] showed that it can be reduced to O (cid:0) d u − (cid:96) +1 log d log( nd ) (cid:1) . More recently, for thespecial case u = (cid:96) + 1 , [12] reduced it further to O (cid:16) d log( nd ) (cid:17) when u is asymptotically close to d . The works of [13],[14] sought to find schemes that admit low decoding complexity. Chan et al. [15] studied stochastic threshold group testing.They introduced two stochastic variants of the threshold test function: Bernoulli gap stochasticity and linear gap stochasticity.For Bernoulli gap stochasticity, the test outcome is equally likely to be negative or positive whenever the number of defectivesin a test is in the interval ( (cid:96), u ) . For linear gap stochasticity, the probability of having positive outcome increases linearly asthe number of defectives ranges from (cid:96) to u . By allowing vanishing error probability (cid:15) , they proposed a two-stage adaptivealgorithm with . e d log( n ) + O (cid:0) d log( (cid:15) ) (cid:1) number of tests and a non-adaptive algorithm with O (cid:16) log( (cid:15) ) d √ (cid:96) log( n ) (cid:17) number of tests for Bernoulli gap stochasticity, and a non-adaptive algorithm with O (cid:0) ( u − (cid:96) − d log( n ) (cid:1) + O (cid:0) d log( (cid:15) ) (cid:1) number of tests for linear gap stochasticity. Recently, for Bernoulli gap stochasticity, Reisizadeh et al. [16] improved thenumber of tests required to O (cid:0) √ ud log n (cid:1) .In this paper, motivated by physical considerations such as the effect of dilution on the chemistry of group tests, we formulateand analyze group testing with a general monotonically increasing stochastic test function f ( · ) (i.e., x ≥ y ⇒ f ( x ) ≥ f ( y ) )that takes as input the number of defective items in a test and outputs the probability of the given test having a positiveoutcome. We present a non-adaptive algorithm that can identify the defectives with high probability. Our first main result,Theorem 1, shows that the number of tests required for high probability recovery scales as O (cid:0) d log (cid:0) n(cid:15) (cid:1)(cid:1) for any monotonetest function. By exploiting the structure of the test function and a more sophisticated analysis, we are able to improve thisbound to O (cid:0) d log (cid:0) n(cid:15) (cid:1)(cid:1) (for so-called sensitive test functions) or O (cid:0) d log (cid:0) n(cid:15) (cid:1)(cid:1) depending on the shape of the test function. A significant part of the group-testing literature focuses on zero-error recovery – see for instance the survey in [3]. However, even in the context of classicalgroup-testing more stringent recovery criterion comes at the cost of requiring a number of tests than high-probability recovery requires. Further, in the contextof this work, where test outcomes are probabilistic in nature, zero-error recovery is impossible, hence we focus on high-probability recovery. a r X i v : . [ c s . I T ] F e b Since the information-theoretic lower bound of (1 − (cid:15) ) (cid:0) d log (cid:0) nd (cid:1)(cid:1) tests in [17] continues to hold for our setting, our algorithmsare order-optimal for sensitive test functions, and no more than a factor of O ( d ) off for general test functions. In principle,our schemes generalize to other variants of group testing, such as semiquantitative group testing [18], and group testing withinhibitors [19].The paper is organized as follows. We formulate the generalized non-adaptive group testing problem in Section II. Section IIIpresents our proposed non-adaptive algorithm. In Section IV, we describe the intuition behind the algorithm. Then in SectionsV and VI respectively, we state the main results of this work. Section VII gives proof sketches of our main results whilethe full proofs are deferred to Section VIII and Appendix. Finally, Section IX contains the simulation result of the algorithmconsidered. II. P ROBLEM F ORMULATION
A set N := { , . . . , n } of n items contains a subset D (cid:40) N of defective items – elements in N \ D are called non-defective .We follow the “probabilistic group-testing model” [3], [7], [20], [21] and assume that each item is defective independentlywith probability p . That is, the expected number of defective items d := E ( |D| ) satisfies d = np , and in settings of interest d is typically much smaller than n , so we assume that d = o ( n ) . The identity of D is unknown a priori – the goal of grouptesting is to correctly identify D through a minimal series of group tests on subsets of items. In “classical” group testing atest outcome is negative if every item in the pool is non-defective, and is positive if at least one item is defective. As such thismay be viewed as a disjunctive measurement, i.e., viewing each item as a or a depending on whether it is non-defective ordefective, each test performs an OR of the items in its pool. A canonical setting in which this measurement model is pertinentis when a small number of individuals in a large population are diseased but only a small number of testing kits are available;in this case samples from different individuals may be “pooled” together in different combinations and the set of test outcomesanalyzed jointly to infer D .Instantiating disjunctive tests which are sensitive to even a single defective in a testing pool may be tricky, for instance dueto the impact of dilution on the chemistry used in pooled tests [22]. Our primary contribution in this work is to consider avery general class of (probabilistic) measurement functions f ( · ) : Z + → [0 , . The input, say x , to the measurement function f ( x ) is the number of defective items x in a given pool, and the value of f ( x ) is the probability that the given test results in apositive test outcome. In this work we restrict ourselves to the natural class of measurement monotone measurement functions,i.e., x ≥ y ⇒ f ( x ) ≥ f ( y ) . Monotone measurement functions subsume many existing models of group-testing as specialcases. Let x be the number of defective items in a test. For instance, when f ( x ) = (cid:26) x = 0 , x ≥ , (1)this reduces to the problem of classical group testing. Observe that when f ( x ) = x ≤ (cid:96), x − (cid:96)u − (cid:96) (cid:96) < x < u, x ≥ u, (2)for some integers < (cid:96) < u < d , this reduces to the “linear gap” stochastic group testing examined by [15]. As the classicalgroup testing model (1) is well-understood, in this paper, we further restrict ourselves to monotone f ( · ) such that f ( x ) = 0 x ≤ , ∈ [0 ,
1] 1 < x < d, = 1 x ≥ d. (3)Here, we focus on non-adaptive group testing, where the set of items being tested in each test is independent of the outcomeof any other test. The probability of error of any non-adaptive algorithm is defined as the probability that the estimateddefective set ˆ D differs from the true D . We require the probability of error is bounded from above by some (cid:15) . That is, Pr( ˆ
D (cid:54) = D ) ≤ (cid:15) It is worth mentioning that the information-theoretic lower bound of (1 − (cid:15) ) (cid:0) d log (cid:0) nd (cid:1)(cid:1) tests required for (1 − (cid:15) ) -probabilityrecovery in [17] continues to hold for the above setting. Our focus in this work is on the upper bound. That is, we come upwith a non-adaptive algorithm that can recover the defective items with vanishing probability of error for any given monotonetest function f ( · ) satisfying (3). Another model of group-testing assumes that |D| ≤ d . In many group-testing models similar results are obtainable under both probabilistic and combinatorialmodels, so for ease of analysis we choose to focus on the probabilistic model in this work. The adaptive version of group-testing has also been extensively studied – see for instance the survey in [3]. However, since non-adaptive tests allow fortest-parallelization, and also make it easier to design hardware to perform the tests (unlike adaptive test designs, where the composition of (at least some)tests may depend on prior test outcomes), we restrict our attention in this work to non-adaptive designs.
Model parameters N The set of all items. n The total number of items, and n = |N | . p The probability of each item being defective, and p = o (cid:0) n (cid:1) . D The unknown subset of defective items. d The expected number of defective items, with d = np . f ( · ) The test function , a monotone function indicating the probability of f ( x ) that a test pool with x defectives has a positive outcome. (cid:15) The pre-specified error probability. T The number of tests. Algorithmic parameters s A pool-size parameter: each test includes ( s + 1) items. d s An index ranging from to s specifying the number of defectives among a given set of s items. ( L , U ) The values of d s such that f ( d s ) ≈ and f ( d s ) ≈ respectively, as defined in (4). k The “largest slope” value of d s in the range [ L , U ] , as defined in (5). k A “high sensitivity” value of d s in the range [0 , d ] , such that f ( k + 1) − f ( k ) ∈ Θ(1) , as defined in (7). h ( · ) A function that is the dividing line of f ( x ) ∈ o (1) and f ( x ) ∈ ω (1) , as defined in (2). ( L , U ) A pair of parameters in the range of Θ( g ( d )) , as defined in (12). P ( − , s ) The probability that the test has a positive outcome when i is non-defective, as defined in (14). P (+ , s ) The probability that the test has a positive outcome when i is defective, as defined in (14). ∆( s ) The difference between the probabilities of positive test outcomes conditioned on item i being defective or not, defined in (15). P min ( s ) Minimal test sensitivity defined in (16). m Test participation parameter defined in (17).
TABLE I: Table of frequently used notationIII. T
EST DESIGN AND DECODING
We now present our non-adaptive test design, and the corresponding decoding algorithm. We design T non-adaptive tests,where the value of T is a design parameter specified later in Theorems 1 and 2. Test pool size:
A critically important parameter in our test designs is s , which determines the size of each pool as s + 1 .Each test pool is designed by randomly drawing s + 1 items from N without replacement, with each test pool designedindependently. The parameter s is chosen differently in our two main results, Theorems 1 and 2. To disambiguate between thevalues of s used, we use s in Theorem 1, and s a , s b for the two cases in Theorem 2.1) Choice of s in Theorem 1: In Theorem 1, we choose s according to the “maximal sensitivity” of the test function f ( · ) .Since f ∈ [0 , and is monotone, then ∃ L s.t. f ( L ) < , f ( L + 1) ≥ , ∃ U s.t. f ( U ) < , f ( U + 1) ≥ . (4)It follows from (3) that ≤ L ≤ U < d . Let k denote the point that has the “largest slope” in [ L , U ] , namely, k ∈ [ L ,U ] s.t. ∀ x ∈ [ L , U ] ,f ( k + 1) − f ( k ) ≥ f ( x + 1) − f ( x ) . (5)We then choose the pool-size parameter s as s = (cid:22) k p (cid:23) . (6)2) Choice of s in Theorem 2: In Theorem 2 we choose s in a somewhat more sophisticated manner, taking into account the“shape” of the test function f ( · ) . There are two possible cases: • Case a):
If the test function f ( · ) has at least one region where it has high sensitivity/large slope, then s is chosenaccordingly. In particular, test function f ( · ) is said to be sensitive if ∃ k ≤ d s.t. ( f ( k + 1) − f ( k )) ∈ Θ(1) . (7)In this case we choose the pool-size parameter s a as s a = (cid:22) k p (cid:23) . (8)To illustrate, consider the example f ( x ) = (cid:26) x ≤ , x ≥ . (9)See Figure 1a for illustration. Clearly, by definition, this function satisfies (7) with k = 4 . Hence, according to (8),we shall choose s a = (cid:106) p (cid:107) for this function. • Case b):
The test function f ( · ) does not have any region with high sensitivity/large slope. That is, the function isincreasing slowly on the whole [0 , d ] . In particular, ( f ( x + 1) − f ( x )) ∈ o (1) , ∀ x ≤ d. (10)Then, from (3), we know there exists some function h ( d ) ≤ d such that f ( x ) (cid:26) ∈ o (1) x ∈ o ( h ( d )) , ∈ Θ(1) x ∈ ω ( h ( d )) . (11)Thus, ∃ c ∈ Θ( h ( d )) such that f ( c ) = w , for some w ∈ Θ(1) . Since f ∈ [0 , and is monotone, we have ∃ L s.t. f ( L ) < w , f ( L + 1) ≥ w , ∃ U s.t. f ( U ) < w , f ( U + 1) ≥ w . (12)Clearly, from (3), we have L ≥ . Meanwhile, L (cid:54) = 1 , because, otherwise, f satisfies (7), contradicting our assumption(10). Therefore, we have U ≥ L ≥ . We will then choose some s b ∈ (cid:104) L p , U p (cid:105) that satisfies certain conditions asthe pool-size parameter. Since the specification involves some parameters introduced later, it is deferred to Lemma 4in Section VI.To illustrate, consider the example f ( x ) = x ≤ d , d x − d < x < d , x ≥ d . (13)which is represented in figure 2a. This function satisfies (10) since f ( x + 1) − f ( x ) ≤ d , ∀ x ≤ d. For this function, we have h ( d ) = d and f ( x ) = (cid:26) x ∈ o ( d ) , x ∈ ω ( d ) . We can choose c = d since f ( d ) = 1 . According to (12), L = (cid:4) d (cid:5) and U = (cid:4) d (cid:5) . The pool-size parameter ischosen as s b = (cid:106) d p (cid:107) ∈ (cid:104) L p , U p (cid:105) , which satisfies the conditions in Lemma 4. Parameters for the decoding rule:
Given these tests and their outcomes we now specify the decoding algorithm we useto produce an estimate ˆ D of the defective set D . Before presenting the algorithm, let us first introduce some definitions andnotation. Definition 1: Test positivity probability:
For any item i in a test containing s other items, the quantity P ( − , s ) denotes the probabilitythat the test has a positive outcome conditioned on the event that item i is non-defective. Analogously, P (+ , s ) , denotesthe probability that the test has a positive outcome conditioned on the event that item i is defective. Mathematically, P ( − , s ) := s (cid:88) j =0 (cid:18) sj (cid:19) p j (1 − p ) s − j f ( j ) , and P (+ , s ) := s (cid:88) j =0 (cid:18) sj (cid:19) p j (1 − p ) s − j f ( j + 1) . (14)2) Item test sensitivity:
For any item i in a test, its test sensitivity ∆( s ) is defined as the difference between the probabilitiesof positive test outcomes conditioned on item i being defective or not. Mathematically, ∆( s ) := P (+ , s ) − P ( − , s ) . (15)3) Minimal test sensitivity:
A parameter that will be useful in our code design and analysis is the minimal test sensitivity P min ( s ) , defined as P min ( s ) := min ( P ( − , s ) , − P (+ , s )) . (16)4) Test participation parameter:
It also helps to define the test participation parameter m as in (17) below. As shown inLemma 5 in Section VIII-A, with high probability each item in N participates in at least m tests. m := (cid:38) . P min ( s ) (∆( s )) log (cid:18) en(cid:15)P min ( s ) (cid:19)(cid:39) . (17) Number of tests:
Finally, we set the number of tests as T = (cid:24) dsp (cid:18) m + 1 .
39 log (cid:18) n(cid:15) (cid:19)(cid:19)(cid:25) . (18) Decoding rule:
We are now ready to describe our decoding rule, which proceeds by separately estimating whether or noteach item i ∈ N is defective or not.Each item i ∈ N that appears in less than m tests is automatically declared to be non-defective. Otherwise, we randomlychoose m tests from the set of all tests item i participates in, and denote by m + ( respectively m − ) the number of tests withpositive (respectively negative) outcomes among this set. We then classify i as follows: i = (cid:40) non-defective if m + m ≤ P ( − ,s )1 − ∆( s ) , defective if m + m > P ( − ,s )1 − ∆( s ) . (19)Finally, we show in Section VIII that Lemma 1:
The above test design has probability of error at most (cid:15) . Remark:
Note that the value of s (and hence the values of subsidiary parameters such as m in our test design) differ inTheorems 1 and 2, resulting in differing values of T in these two theorems.IV. I NTUITION
We give here some high-level intuition behind the proposed algorithm. Given any item i in a test of size s + 1 , the expectednumber of defectives among the s other items is sp . Then the probability that the given test has a positive outcome isconcentrated around f ( sp ) if item i is non-defective and around f ( sp + 1) if item i is defective. This means the fraction ofpositive test outcomes among the tests containing item i is around f ( sp ) (respectively f ( sp + 1) ) if item i is non-defective(respectively defective). Therefore, we can identify the status of item i according to the fraction of positive test outcomesamong the tests that it participates in.For accurate identification, we prefer the gap between f ( sp ) and f ( sp + 1) to be as large as possible, which translates tothe choice of pool-size parameter s . In Theorem 1, we simply choose sp (therefore s ) to be the best one among points whosevalues of f ( · ) are between / and / . In Theorem 2, by inspecting the structure of the function f ( · ) , we choose s in asomewhat refined manner. If the function f ( · ) has at least one region where it increases very fast, we can directly choose sp to be the point in that region. Otherwise, the function is increasing slowly in the whole region. In this case, we first find aregion where the function increases relatively fast and then choose sp to be a particular point where the gap is no smaller thana given lower bound. V. T HEOREM s as in (5) and (6), in Appendix A we bound the item test sensitivity ∆( s ) in (15), and minimal test sensitivity P min ( s ) in (16) as follows. Lemma 2: ∆( s ) > U + 1 − L ) √ k , P min ( s ) > . (20)Substituting these bounds into (17) and thence into (18) and using the fact that both U + 1 − L ≤ d and k ≤ d , combinedwith Lemma 1 leads to our first main result: Theorem 1:
For any sufficiently large n , sufficiently small (cid:15) > , and p ∈ o (1) , there exists a non-adaptive group testingdesign with T ∈ O (cid:16) d log (cid:16) n(cid:15) (cid:17)(cid:17) (21)that ensures a probability of error at most (cid:15) . More precisely, any T satisfying T ≥ (cid:24) . d log (cid:18) en(cid:15) (cid:19) + 1 . d log (cid:18) n(cid:15) (cid:19)(cid:25) (22)ensures a probability of error at most (cid:15) . VI. T
HEOREM Case a):
Given that the test function satisfies (7), and given our choice of the pool-size parameter s a as in (8), in AppendixB we can bound the item test sensitivity ∆( s a ) in (15) and minimal test sensitivity in P min ( s a ) in (16) as follows. Lemma 3:
Let δ = f ( k + 1) − f ( k ) , we have ∆( s a ) > δ √ k , P min ( s a ) > δ . (23)Substituting these bounds into (17) and thence into (18) and using the fact that k ≤ d , combined with Lemma 1 impliesthat any T satisfying T ≥ (cid:24) . δ d log (cid:18) enδ(cid:15) (cid:19) + 1 . d log (cid:18) n(cid:15) (cid:19)(cid:25) (24)ensures a probability of error at most (cid:15) . Since δ ∈ Θ(1) , it follows that T ∈ O (cid:16) d log (cid:16) n(cid:15) (cid:17)(cid:17) . (25) Case b):
Given that the test function satisfies (10), and given the definitions of L , U and w in (12), we show there existspool-size parameter s b ∈ (cid:104) L p , U p (cid:105) whose item test sensitivity ∆( s b ) in (15) and minimal test sensitivity in P min ( s b ) in (16)satisfy the following property. The proof is rather involved and relegated to the Appendix C. Lemma 4: ∃ s b ∈ (cid:104) L p , U p (cid:105) such that ∆( s b ) > w U + p ) , P min ( s b ) > w . (26)Now, using s b as the pool-size parameter, and substituting (26) into (17) and thence into (18) and using the fact that ps b ≥ L , combined with Lemma 1 yields: any T satisfying T ≥ (cid:24) . d ( U + p ) w L log (cid:18) enw(cid:15) (cid:19) + 1 . d log (cid:18) n(cid:15) (cid:19)(cid:25) (27)ensures a probability of error at most (cid:15) . Since L , U ∈ Θ( g ( d )) , w ∈ Θ(1) and g ( d ) ≤ d , we obtain T ∈ O (cid:16) d log (cid:16) n(cid:15) (cid:17)(cid:17) . (28)Summarizing the two cases, we immediately obtain our second main result: Theorem 2:
For any sufficiently large n , sufficiently small (cid:15) > , and p ∈ o (1) , there exists a non-adaptive group testingdesign with probability of error at most (cid:15) ,i) If ∃ k ≤ d s.t. ( f ( k + 1) − f ( k )) ∈ Θ(1) , and T ∈ O (cid:16) d log (cid:16) n(cid:15) (cid:17)(cid:17) , or more explicitly, any T satisfying (24);ii) If (cid:64) k ≤ d s.t. ( f ( k + 1) − f ( k )) ∈ Θ(1) , or in other words, ( f ( k + 1) − f ( k )) ∈ o (1) , ∀ k ≤ d , and T ∈ O (cid:16) d log (cid:16) n(cid:15) (cid:17)(cid:17) , or more explicitly, any T satisfying (27). VII. P ROOF S KETCHES
Proof sketch of Lemma 1:
The proof breaks into two parts. First, in Lemma 5, we compute the probability that an arbitraryitem takes less than m tests, which, by using the Chernoff bound [23], is no larger than (cid:15) n . Second, conditioning on the eventthat each item participates in at least m tests, in Lemma 6 we compute the probability of misidentification (false defective orfalse non-defective). Using Stirling’s approximation [24] and suitable approximations, this probability is again no larger than (cid:15) n . Combining these two parts with a union bound and using Bernoulli’s inequality [25] yields the desired result. Proof sketch of Lemma 2:
In Lemma 8, we use Stirling’s approximation to derive lower and upper bounds on the quantity (cid:0) (cid:98) dsp (cid:99) d s (cid:1) p d s (1 − p ) (cid:98) dsp (cid:99)− d s . Then the bound on ∆( s ) follows by using this lower bound along with the definitions of L , U and k .In order to establish the bound on P min ( s ) , we provide lower and upper tail bounds on (cid:80) j (cid:0) (cid:98) dsp (cid:99) j (cid:1) p j (1 − p ) (cid:98) dsp (cid:99)− j in Lemmas9 and 10, respectively. To obtain these two bounds, we first bound the ratio between successive values of (cid:0) (cid:98) dsp (cid:99) j (cid:1) p j (1 − p ) (cid:98) dsp (cid:99)− j ; Then we can bound (cid:80) j (cid:0) (cid:98) dsp (cid:99) j (cid:1) p j (1 − p ) (cid:98) dsp (cid:99)− j by the sum of a geometric sequence. The bound on P min ( s ) can now be obtainedby using these tails bound and the monotonicity property of f ( · ) . Proof sketch of Lemma 3:
The proof of Lemma 3 is similar to that of Lemma 2 except that the test function f ( · ) now issensitive, leading to a tighter bound. Proof sketch of Lemma 4:
The proof of bounding P min ( s b ) is similar to that of Lemma 2. The main idea of findinga s b ∈ [ L p , U p ] with ∆( s b ) satisfying (26) is as follows: We add up all ∆( s ) for s ∈ [ L p , U p ] , then there must exist one s ∈ [ L p , U p ] whose ∆( s ) is no smaller than the mean value. The difficulty lies in deriving a reasonable lower bound on thesum of ∆( s ) . Towards this end, in Lemma 11 we establish a lower bound on (cid:98) U p (cid:99) (cid:80) s = (cid:98) L p (cid:99) (cid:0) sd s (cid:1) p d s (1 − p ) s − d s . This is done byusing the monotonicity of the Binomial distribution to bound the desired quantity by the sum of two geometric sequences.VIII. P ROOF OF L EMMA m tests, which can be made sufficiently small. Second, assuming that each item takes at least m tests, we computethe probability of misidentification, which again can be made sufficiently small.We shall use the following well-known Stirling’s approximation [24] for the factorial function. Fact 1 (Stirling’s approximation [24]): √ π n n + e − n ≤ n ! ≤ e n n + e − n (29) A. Probability that each item participates in at least m tests.Lemma 5: With probability at least − (cid:15)/ over the test design, each item i ∈ N participates in at least m tests. Proof:
Consider an arbitrary item i ∈ N . Let T i denote the number of tests that item i takes. Since we randomly choose ( s + 1) items from N for each test, we know from (18) that the expected number of tests item i involving is E ( T i ) = ( s + 1) Tn ≥ ( s + 1) dn ps (cid:18) m + 2 ln (cid:18) n(cid:15) (cid:19)(cid:19) > m + 2 ln (cid:18) n(cid:15) (cid:19) . (30)Let E i be the event that item i takes less than m tests. By the Chernoff bound [23], we have Pr( E i ) = Pr (cid:18) T i < m = (cid:18) − E ( T i ) − m E ( T i ) (cid:19) E ( T i ) (cid:19) < exp (cid:32) − (cid:18) E ( T i ) − m E ( T i ) (cid:19) E ( T i )2 (cid:33) = exp (cid:18) − E ( T i )2 + m − m E ( T i ) (cid:19) < exp (cid:18) − E ( T i )2 + m (cid:19) < exp (cid:18) − ln (cid:18) n(cid:15) (cid:19)(cid:19) = (cid:15) n . (31)Then the probability that all items take at least m tests can be bounded as n (cid:89) i =1 (1 − Pr( E i )) > (cid:16) − (cid:15) n (cid:17) n > − (cid:15) n · n = 1 − (cid:15) (32)where the last inequality follows from the Bernoulli’s inequality [25]. B. Probability that all items are correctly identified.Lemma 6:
Conditioning on the event that each item participates in at least m tests, with probability at least − (cid:15) over thetest design, all items are correctly identified. Proof:
To begin with, assume that each item participates in at least m tests. As described before, we will randomly choose m tests and identify the item via (19). The error comes from two parts:1) Item i is non-defective, but is identified as defective;2) Item i is defective, but is identified as non-defective.We will bound and separately as follows.
1) Item i is non-defective, but is identified as defective: For this case, each test outcome is positive with probability P ( − , s ) ,and m + m > P ( − , s )1 − ∆( s ) . (33)Let P + e denote the probability of this false defective. For notational simplicity, let ρ = P ( − , s ) , ∆ = ∆( s ) , and define θ := ρ − ∆ , θ := eπ (cid:115) m ( m − (cid:100) θ m (cid:101) ) (cid:100) θ m (cid:101) . (34)Now P + e can be bounded as P + e = m (cid:88) m + = (cid:98) θ m (cid:99) +1 (cid:18) mm + (cid:19) ρ m + (1 − ρ ) m − m + (35) ≤ m (cid:88) m + = (cid:100) θ m (cid:101) (cid:18) mm + (cid:19) ρ m + (1 − ρ ) m − m + (36) ≤ ( m − (cid:100) θ m (cid:101) + 1) (cid:18) m (cid:100) θ m (cid:101) (cid:19) ρ (cid:100) θ m (cid:101) (1 − ρ ) m −(cid:100) θ m (cid:101) (37) ≤ m − (cid:100) θ m (cid:101) ) ( em m + e − m ) ρ (cid:100) θ m (cid:101) (1 − ρ ) m −(cid:100) θ m (cid:101) π ( (cid:100) θ m (cid:101) ) (cid:100) θ m (cid:101) + ( m − (cid:100) θ m (cid:101) ) ( m −(cid:100) θ m (cid:101) )+ e − m (38) = θ (cid:18) ρm (cid:100) θ m (cid:101) (cid:19) (cid:100) θ m (cid:101) (cid:18) (1 − ρ ) mm − (cid:100) θ m (cid:101) (cid:19) m −(cid:100) θ m (cid:101) (39) ≤ θ (cid:18) ρmθ m (cid:19) θ m (cid:18) (1 − ρ ) mm − (cid:100) θ m (cid:101) (cid:19) m −(cid:100) θ m (cid:101) (40) = θ (cid:18) ρθ (cid:19) θ m (cid:18) (cid:100) θ m (cid:101) − ρmm − (cid:100) θ m (cid:101) (cid:19) m −(cid:100) θ m (cid:101) ≤ θ (cid:18) ρθ (cid:19) θ m e (cid:100) θ m (cid:101)− ρm (41) ≤ θ (cid:18) ρθ (cid:19) θ m e θ m +1 − ρm (42) = e θ (cid:0) (1 − ∆) e ∆ (cid:1) θ m . (43)Here: • (35) follows by adding up all the possibilities satisfying (33), which are from (cid:98) θ m (cid:99) + 1 to m . • (36) follows since (cid:100) θ m (cid:101) ≤ (cid:98) θ m (cid:99) + 1 . • (37) can be justified as follows: First, note that the summands in (36) are Binomial mass function with expected valueequals to ρm . Next, from (34), we have ρm ≤ θ m ≤ (cid:100) θ m (cid:101) , which means the summand is decreasing in m + . Therefore,(37) follows by enlarging all summands to the first one m + = (cid:100) θ m (cid:101) . • (38) follows from Stirling’s approximation in (29), and the fact that m − (cid:100) θ m (cid:101) ≥ since θ < implied by P (+ , s ) < . • (39) follows by employing (34). • (40) follows by noting that (cid:18) ρm (cid:100) θ m (cid:101) (cid:19) (cid:100) θ m (cid:101) ≤ (cid:18) ρm (cid:100) θ m (cid:101) (cid:19) θ m ≤ (cid:18) ρmθ m (cid:19) θ m since ρ ≤ θ by (34). • (41) follows by using the standard identity x ≤ e x coupled with the fact that m − (cid:100) θ m (cid:101) > . • (42) follows by noting that (cid:100) θ m (cid:101) ≤ θ m + 1 . • (43) follows by using (34).To proceed, we make use of the following simple identity. Lemma 7: (1 − ∆) e ∆ ≤ − ∆ Proof:
Let ξ ( x ) := 1 − x − (1 − x ) e x , x ∈ [0 , ∞ ] . Taking the derivative of ξ ( x ) , we obtain ξ (cid:48) ( x ) = x ( e x − ≥ .Hence ξ ( x ) ≥ ξ (0) = 0 , which implies (1 − ∆) e ∆ ≤ − ∆ and the assertion follows. With this, we can further bound P + e as follows P + e ≤ e θ (cid:18) − ∆ (cid:19) θ m (44) ≤ e (cid:114) mP min ( s ) (cid:18) − ∆ (cid:19) P min ( s ) m (45) ≤ e (cid:114) mP min ( s ) exp (cid:18) − ∆ P min ( s ) m (cid:19) (46) ≤ (cid:15) n · (cid:15)P min ( s )4 en ∆ · (cid:115) ln (cid:18) en(cid:15)P min ( s ) (cid:19) (47) ≤ (cid:15) n , (48)where • (44) follows from (43) by invoking Lemma 7; • (45) is because θ ≥ ρ = P ( − , s ) ≥ P min ( s ) , and θ ≤ (cid:115) m θ m = (cid:114) mθ ≤ (cid:114) mP min ( s ) ; • (46) follows by using the standard inequality (1 − x ) < e − x , ∀ x ∈ (0 , • (47) can be argued as follows: Denoting the R.H.S. of (46) by ζ ( m ) , we have ζ (cid:48) ( m ) = e (cid:0) − ∆ P min ( s ) m (cid:1) (cid:112) mP min ( s ) exp (cid:18) − ∆ P min ( s ) m (cid:19) In light of (17), we deduce − ∆ P min ( s ) m ≤ − (cid:18) en(cid:15)P min ( s ) (cid:19) ≤ Thus, ζ (cid:48) ( m ) ≤ , which, when combined with (17), yields ζ ( m ) ≤ ζ (cid:16) P min ( s ) ∆ ln (cid:16) en(cid:15)P min ( s ) (cid:17)(cid:17) . Simplifying this givesus (47). • (48) follows from the assumption that n is sufficiently large, more precisely, en ∆ (cid:15)P min ( s ) ≥ (cid:115) ln (cid:18) en(cid:15)P min ( s ) (cid:19) ⇔ n ≥ (cid:15)P min ( s )4 e · exp (cid:32) − W − (cid:0) − (cid:1) (cid:33) where W − denotes the Lambert W function [26].
2) Item i is defective, but is identified as non-defective: The calculation is similar to the above case. In this case, eachoutcome is positive with probability of P (+ , s ) , and m + m ≤ P ( − , s )1 − ∆( s ) (49)or equivalently m − m ≥ − P ( − , s )1 − ∆( s ) = 1 − P (+ , s )1 − ∆( s ) (50)Let P − e denote the probability of this false non-defective. For ease of presentation, let (cid:37) = 1 − P (+ , s ) , which is the probabilitythat the test containing i has a negative outcome conditioned on the event that item i is defective, and define θ (cid:48) := (cid:37) − ∆ , θ (cid:48) := eπ (cid:115) m ( m − (cid:100) θ (cid:48) m (cid:101) ) (cid:100) θ (cid:48) m (cid:101) . Then we can bound P − e as P − e = m (cid:88) m − = (cid:100) θ (cid:48) m (cid:101) (cid:18) mm − (cid:19) (cid:37) m − (1 − (cid:37) ) m − m − ≤ (cid:15) n (51)Inequality (51) can be argued in a similar manner as that of false defective estimates in (36)-(48).From (48) and (51), we can conclude that the probability of misidentification is smaller than (cid:15) n , regardless of item i isdefective or not. Therefore, when all items take at least m tests, the probability that all items are correctly identified is boundedbelow by (cid:16) − (cid:15) n (cid:17) n ≥ − (cid:15) n n = 1 − (cid:15) . (52)which finishes the proof of Lemma 6.Finally, combining Lemmas (5) and (6), the probability that all items are correctly identified is bounded from below by (cid:16) − (cid:15) (cid:17) = 1 − (cid:15) + (cid:15) > − (cid:15) . (53)This proves Lemma 1. IX. S IMULATION R ESULTS
In this section, we perform computer simulations to evaluate the performance of the proposed schemes. We input • number of items n ; • number of defectives d ; • test function f ; • number of tests T .We then run the proposed test design and decoding algorithm multiple times to evaluate the probability of successful recon-struction. Testing:
For given ( n, d, f, T ) , we randomly generate an array a of length n with ( n − d )
0s and d s accordingly, and randomly generate a n × T matrix M such that each column contains ( n − s −
0s and ( s + 1) M corresponds to a distinct item,and each column corresponds to a distinct test. Finally, we compute b = ( b , . . . , b T ) = aM and generate the test outcomesaccording to f ( b i ) , i = 1 . . . , T . Decoding:
Our simulations differ from the analysis in Theorems 1 and 2 in the following manner – we shall use all thetests that item i participates in, rather than only choosing m of them. More precisely, we denote the number of tests that item i participates in by t i and the number of positive tests within the t i tests by t + i . We then classify i as follows: ˆ a i = (non-defective) if t + i t i ≤ P ( − ,s )1 − ∆( s ) (defective) if t + i t i > P ( − ,s )1 − ∆( s ) (54)The test succeeds if ˆ a = a , and fails otherwise. A. Simulation result for Case i) in Theorem 2
Consider the function f ( · ) defined in (9) and illustrated in Figure 1a. This function belongs to Case i) in Theorem 2. Nowaccording to (8), the pool-size parameter is chosen as s = (cid:4) nd (cid:5) . In the waterfall plot in Figure 1b, n = 6000 , d = 30 ,the x -axis plots the number of tests T ranging from T min = (cid:98) . d log( n ) (cid:99) to T max = (cid:98) . d log( n ) (cid:99) with step size ∆ T = (cid:4) T max − T min (cid:5) , and the y -axis plots the probability of successful reconstruction calculated by trials for each test T = T min + j · ∆ T , j ∈ { , , . . . , } . When T (cid:39) . d log( n ) , the probability of successful reconstruction exceeds . . Inthe heat-map in Figure 1c, n = 6000 , the x -axis denotes the expected number of defectives d ranging from to , and the y -axis denotes the number of tests T as a multiple of d log( n ) . In the heat-map in Figure 1d, d = 30 , the x -axis correspondsto the number of items n ranging from to , and the y -axis corresponds to the number of tests T as a multiple of d log( n ) . In both Figures 1c and 1d, each pixel is coloured according to the probability of successful reconstruction calculatedby trials for each test T – the lighter the colour, the higher the probability of reconstruction success. For each value of d (respectively n ) in Figure 1c (respectively Figure 1d), the corresponding red dot in that column represents the number of testsfor which this probability first equals . . The horizontal blue dashed line indicates that when T (cid:39) . d log( n ) (respectively . d log( n ) ), the probability of successful reconstruction generally exceeds . . The computing resource we use is an Intel Core i7-9750H CPU. The reason for this choice is due to the relatively small value of d in our simulations, the number of times an item is tested is not concentrated tightlyenough and becomes a dominant source of error. f ( x ) x . d (a) Example test function defined in (9) : The test outcome is positive if and onlyif at least items in a pool are defective. (b) The x -axis plots the number of tests T as a multiple of d log( n ) , and the y -axis plots the probability of successful reconstruction, for fixed n = 6000 , d =30 . When T (cid:39) . d log( n ) , the probability of successful reconstructionexceeds . . (c) The x -axis corresponds to the expected number of defectives d ranging from to , and the y -axis corresponds to the number of tests T as a multipleof d log( n ) –the number of items n is fixed to be . Each pixel is colouredaccording to the probability of successful reconstruction– the lighter the colour,the higher the probability of reconstruction success. For each value of d , thecorresponding red dot in that column represents the number of tests for whichthis probability first equals . . The horizontal blue dashed line indicates thatwhen T (cid:39) . d log( n ) , the probability of successful reconstruction generallyexceeds . . (d) The x -axis denotes the number of items n ranging from to ,and the y -axis denotes the number of tests T as a multiple of d log( n ) –theexpected number of defectives d equals . Each pixel is coloured according tothe probability of successful reconstruction – the lighter the colour, the higherthe probability of reconstruction success. For each value of n , the correspondingred dot in that column represents the number of tests for which this probabilityfirst equals . . The horizontal blue dashed line indicates that when T (cid:39) . d log( n ) , the probability of successful reconstruction generally exceeds . . Fig. 1:
Simulation result for Case i) in Theorem 2.B. Simulation result for Case ii) in Theorem 2
We now consider the function f ( · ) defined in (13) and illustrated in Figure 2a. This function belongs to Case ii) in Theorem2. As discussed before, the pool-size parameter is chosen as s = (cid:4) n (cid:5) . In the waterfall plot in Figure 2b, n = 6000 , d = 30 ,the x -axis plots the number of tests T ranging from T (cid:48) min = (cid:4) . d log( n ) (cid:5) to T (cid:48) max = (cid:4) . d log( n ) (cid:5) with step size ∆ (cid:48) T = (cid:4) T max − T min (cid:5) , and the y -axis plots the probability of successful reconstruction calculated by trials for each test T = T (cid:48) min + j · ∆ (cid:48) T , j ∈ { , , . . . , } . When T (cid:39) . d log( n ) , the probability of successful reconstruction exceeds . .In the heat-map figure 2c, n = 6000 , the x -axis denotes the expected number of defectives d ranging from to , and the y -axis denotes the number of tests T as a multiple of d log( n ) . In the heat-map figure 2d, d = 30 , the x -axis correspondsto the number of items n ranging from to , and the y -axis corresponds to the number of tests T as a multiple of d log( n ) . In both figures 2c and 2d, each pixel is coloured according to the probability of successful reconstruction calculatedby trials for each test T – the lighter the colour, the higher the probability of reconstruction success. For each value of d (respectively n ) in figure 2c (respectively figure 2d), the corresponding red dot in that column represents the number of tests forwhich this probability first equals . . The horizontal blue dashed line indicates that when T (cid:39) . d log( n ) (respectively f ( x ) x . d d d (a) Example test function f defined in (13) : The probability that test outcome ispositive increases linearly if the number of defective items in a pool is between d and d . (b) The x -axis plots the number of tests T as a multiple of d log( n ) , and the y -axis plots the probability of successful reconstruction, for fixed n = 6000 and d = 30 . When T (cid:39) . d log( n ) , the probability of successful reconstructionexceeds . . (c) The x -axis corresponds to the expected number of defectives d ranging from to , and the y -axis corresponds to the number of tests T as a multipleof d log( n ) , for fixed number of items n = 6000 . Each pixel is colouredaccording to the probability of successful reconstruction– the lighter the colour,the higher the probability of reconstruction success. For each value of d , thecorresponding red dot in that column represents the number of tests for whichthis probability first equals . . The horizontal blue dashed line indicates thatwhen T (cid:39) . d log( n ) , the probability of successful reconstruction generallyexceeds . . (d) The x -axis denotes the number of items n ranging from to ,and the y -axis denotes the number of tests T as a multiple of d log( n ) , forfixed expected number of defectives d = 30 . Each pixel is coloured accordingto the probability of successful reconstruction – the lighter the colour, the higherthe probability of reconstruction success. For each value of n , the correspondingred dot in that column represents the number of tests for which this probabilityfirst equals . . The horizontal blue dashed line indicates that when T (cid:39) . d log( n ) , the probability of successful reconstruction generally exceeds . . Fig. 2:
Simulation result for Case ii) in Theorem 2. . d log( n ) ), the probability of successful reconstruction generally exceeds . .A PPENDIX
A. Proof of Lemma 2
To prove Lemma 2, we will make use of the following three technical results in Lemmas 8, 9 and 10. Lemma 8 usesStirling’s approximation to bound from above and below the quantity (cid:0) (cid:98) dsp (cid:99) d s (cid:1) p d s (1 − p ) (cid:98) dsp (cid:99)− d s ; and Lemmas 9 and 10 providelower and upper tail bounds on (cid:80) j (cid:0) (cid:98) dsp (cid:99) j (cid:1) p j (1 − p ) (cid:98) dsp (cid:99)− j . Lemma 8:
For any positive integer d s and < p ≤ , √ π (1 − p ) e √ d s ≤ (cid:18) (cid:98) d s p (cid:99) d s (cid:19) p d s (1 − p ) (cid:98) dsp (cid:99)− d s ≤ e p π (cid:112) d s (1 − p ) Proof:
Let s := (cid:106) d s p (cid:107) . Using the upper and lower bounds on n ! in (29), we have (cid:18) sd s (cid:19) p d s (1 − p ) s − d s ≥ √ πe √ s (cid:112) d s ( s − d s ) (cid:18) psd s (cid:19) d s (cid:18) s − pss − d s (cid:19) s − d s ≥ √ πe √ d s (cid:18) − pd s (cid:19) d s ≥ √ π (1 − p ) e √ d s where the second inequality follows by noting that d s − p < ps ≤ d s since d s p − < s ≤ d s p ; the last inequality follows fromthe Bernoulli’s inequality. Similarly, we also have (cid:18) sd s (cid:19) p d s (1 − p ) s − d s ≤ e π √ s (cid:112) d s ( s − d s ) (cid:18) psd s (cid:19) d s (cid:18) d s − pss − d s (cid:19) s − d s ≤ e π √ s (cid:112) d s ( s − sp − p ) (cid:18) ps − d s (cid:19) s − d s ≤ e p π (cid:112) d s (1 − p ) where the second inequality follows from d s − p < ps ≤ d s ; the last inequality follows from the fact that x ≤ e x , ∀ x ≥ .Combining the two bounds gives the desired result. Lemma 9:
For any < p ≤ . and any positive integer d s , we have d s − (cid:80) j =0 (cid:0) (cid:98) dsp (cid:99) j (cid:1) p j (1 − p ) (cid:98) dsp (cid:99)− j > . Proof:
Let s := (cid:98) d s p (cid:99) . We consider two cases, depending on whether d s is at most , or more than .Case 1: When d s ≤ , d s − (cid:88) j =0 (cid:18) sj (cid:19) p j (1 − p ) s − j ≥ (cid:18) s (cid:19) (1 − p ) s ≥ (1 − p ) p ≥ (1 − . = 0 . . . . > . Here, the penultimate inequality is because (1 − p ) p is a deceasing function on (0 , .Case 2: When d s ≥ , we consider integers j in the range j ∈ [ d s − √ d s , d s − . For notational convenience, we denote (cid:0) sj (cid:1) p j (1 − p ) s − j by Γ( j ) . Then we can bound the ratio between successive values of Γ( j ) as Γ( j )Γ( j + 1) = (cid:0) sj (cid:1) p j (1 − p ) s − j (cid:0) sj +1 (cid:1) p j +1 (1 − p ) s − j − = ( j + 1)(1 − p )( s − j ) p ≥ ( d s − √ d s )(1 − p ) ps − ( d s − √ d s ) p ≥ ( d s − √ d s )(1 − p ) d s − ( d s − √ d s ) p = 1 − √ d s − ( √ d s − p ≥ − √ d s (1 − p ) It follows that Γ( j ) ≥ (cid:16) − √ d s (1 − p ) (cid:17) d s − j Γ( d s ) . With this and the bound on Γ( d s ) obtained in Lemma 8, we have d s − (cid:88) j =0 (cid:18) sj (cid:19) p j (1 − p ) s − j = d s − (cid:88) j =0 Γ( j ) ≥ d s − (cid:88) j = (cid:100) d s −√ d s (cid:101) Γ( j ) ≥ (cid:98) √ d s (cid:99) (cid:88) j =1 (cid:18) − √ d s (1 − p ) (cid:19) j Γ( d s )= 1 − √ d s (1 − p ) − (cid:16) − √ d s (1 − p ) (cid:17) (cid:98) √ d s (cid:99) +1 − (cid:16) − √ d s (1 − p ) (cid:17) · Γ( d s ) ≥ − √ d s (1 − p ) − e − (cid:98) √ ds (cid:99) +1 √ ds (1 − p ) √ d s (1 − p ) · Γ( d s ) ≥ − √ d s (1 − p ) − e − (cid:98) √ ds (cid:99) +1 √ ds (1 − p ) √ d s (1 − p ) · √ π (1 − p ) e √ d s ≥ (1 − . (cid:16) −
12 (1 − . − e − (cid:17) √ πe = 0 . . . . > . (55)Hence Lemma 9 holds. Lemma 10:
For any < p ≤ . and any positive integer d s , we have (cid:98) dsp (cid:99) (cid:80) j = d s +1 (cid:0) (cid:98) dsp (cid:99) j (cid:1) p j (1 − p ) (cid:98) dsp (cid:99)− j > . Proof:
At a high level, the proof of this lemma is similar to that of Lemma 9 above, with different approximations neededat various points, so we present the proof in some detail. As before, s := (cid:106) d s p (cid:107) . We now consider two cases, depending onwhether d s is at most , or more than .Case 1: When d s ≤ , s (cid:88) j = d s +1 (cid:18) sj (cid:19) p j (1 − p ) s − j = 1 − d s (cid:88) j =0 (cid:18) sj (cid:19) p j (1 − p ) s − j ≥ − d s (cid:88) j =0 s j j ! p j (1 − p ) s − d s (56) ≥ − d s (cid:88) j =0 d sj e − sp + d s p j ! (57) ≥ − e − . d s +0 . d s (cid:88) j =0 d sj j ! (58) ≥ . when d s ∈ { , , } (59)Inequality (56) follows by noting that (cid:0) sj (cid:1) ≤ s j j ! and that (1 − p ) s − j ≤ (1 − p ) s − d s for d s ≥ j . Inequality (57) follows by notingthat d s ≥ sp since s = (cid:106) d s p (cid:107) , and by using the standard inequality (1 − p ) ≤ e − p coupled with the fact that s − d s ≥ (since d s and s are both integers, and since s = (cid:106) d s p (cid:107) and p ≤ . ). Inequality (58) follows by noting that yp ≥ x − p ≥ x − . since s = (cid:106) d s p (cid:107) and p ≤ . . Inequality (59) follows by noting that (58) evaluates respectively to . . . . , . . . . and . . . . when d s ranges from to .Case 2: When d s ≥ , we consider integers j in the range j ∈ (cid:2) d s + 1 , d s + √ d s (cid:3) . Recall that Γ( j ) was defined as (cid:0) sj (cid:1) p j (1 − p ) s − j . The definition of s as (cid:106) d s p (cid:107) implies that d s − p < ps , and hence we can bound the ratio between successivevalues of Γ( j ) as Γ( j + 1)Γ( j ) = ( s − j ) p ( j + 1)(1 − p ) ≥ d s − p − ( d s + √ d s ) p ( d s + √ d s + 1)(1 − p )= 1 − ( √ d s + 1)( d s + √ d s + 1)(1 − p ) ≥ − √ d s (1 − p ) Thus, by the same sequence of inequalities as in (55), we have that s (cid:88) j = d s +1 (cid:18) sj (cid:19) p j (1 − p ) s − j ≥ (cid:98) d s + √ d s (cid:99) (cid:88) j = d s +1 Γ( j ) ≥ (cid:98) √ d s (cid:99) (cid:88) j =1 (cid:18) − √ d s (1 − p ) (cid:19) j Γ( d s ) > . (60)yielding the desired result.Now, we are ready to prove Lemmas 2. By the definitions of L , U and k in (4) and (5), respectively, we get f ( k + 1) − f ( k ) ≥ f ( U + 1) − f ( L ) U + 1 − L > U + 1 − L ) (61)Using this observation along with Lemma 8, we can then write ∆( s ) = s (cid:88) j =0 (cid:18) s j (cid:19) p j (1 − p ) s − j ( f ( j + 1) − f ( j )) ≥ (cid:18) s k (cid:19) p k (1 − p ) s − k ( f ( k + 1) − f ( k )) > √ π (1 − p ) e √ k · U + 1 − L ) > U + 1 − L ) √ k (62)Employing Lemmas 9 and 10 along with the assumption that f ( · ) is monotonically increasing, we can separately bound P ( − , s ) and − P (+ , s ) as follows: P ( − , s ) = s (cid:88) j =0 (cid:18) s j (cid:19) p j (1 − p ) s − j f ( j ) ≥ s (cid:88) j = k +1 (cid:18) s j (cid:19) p j (1 − p ) s − j f ( k + 1) ≥ · s (cid:88) j = k +1 (cid:18) s j (cid:19) p j (1 − p ) s − j > · .
04 = 175 (63) − P (+ , s ) = 1 − s (cid:88) j =0 (cid:18) s j (cid:19) p j (1 − p ) s − j f ( j + 1)= s (cid:88) j =0 (cid:18) s j (cid:19) p j (1 − p ) s − j (1 − f ( j + 1)) ≥ k − (cid:88) j =0 (cid:18) s j (cid:19) p j (1 − p ) s − j (1 − f ( k )) ≥ · k − (cid:88) j =0 (cid:18) s j (cid:19) p j (1 − p ) s − j > · .
04 = 175 (64)Substituting (63) and (64) into (16), we obtain P min ( s ) ≥ as desired. B. Proof of Lemma 3
The proof of Lemma 3 is similar to that of Lemma 2 in Appendix A. As in the statement of the lemma, let δ = f ( k +1) − f ( k ) . It follows from (7) that δ ∈ Θ(1) . Using Lemma 8, it can be shown that ∆( s a ) = s a (cid:88) j =0 (cid:18) s a j (cid:19) p j (1 − p ) s a − j ( f ( j + 1) − f ( j )) ≥ (cid:18) s a k (cid:19) p k (1 − p ) s a − k ( f ( k + 1) − f ( k )) ≥ √ π (1 − p ) e √ k · δ> δ √ k (65)Using Lemmas 9 and 10 together with the monotonicity of f ( · ) , we will bound P ( − , s a ) and − P (+ , s a ) separately asfollows: P ( − , s a ) = s a (cid:88) j =0 (cid:18) s a j (cid:19) p j (1 − p ) s a − j f ( j ) ≥ s a (cid:88) j = k +1 (cid:18) s a j (cid:19) p j (1 − p ) s a − j f ( k + 1) > . · ( δ + f ( k )) ≥ . · δ = δ − P (+ , s a ) = 1 − s a (cid:88) j =0 (cid:18) s a j (cid:19) p j (1 − p ) s a − j f ( j + 1)= s a (cid:88) j =0 (cid:18) s a j (cid:19) p j (1 − p ) s a − j (1 − f ( j + 1)) ≥ k − (cid:88) j =0 (cid:18) s a j (cid:19) p j (1 − p ) s a − j (1 − f ( k )) > . · ( f ( k + 1) − f ( k ))= 0 . · δ = δ It follows from (16) that P min ( s a ) = min ( P ( − , s a ) , − P (+ , s a )) ≥ δ , (66)thus completing the proof of Lemma 3. C. Proof of Lemma 4
The idea of finding a s b with ∆( s b ) satisfying (26) is as follows: We will add up all ∆( s ) for integers s in the range s ∈ [ L p , U p ] , then there must exist one ∆( s ) which is no smaller than the mean value.In the proof of Lemma 4, the following auxiliary inequality will be instrumental. Lemma 11:
For integers d s in the range d s ∈ [ L , U ] , we have (cid:98) U p (cid:99) (cid:88) s = (cid:98) L p (cid:99) (cid:18) sd s (cid:19) p d s (1 − p ) s − d s ≥ (cid:106) U p (cid:107) − (cid:106) L p (cid:107) + 140( U + p ) Proof:
See Appendix D.We start by bounding the sum of ∆( s ) for integers s ∈ (cid:104) L p , U p (cid:105) as follows: (cid:98) U p (cid:99) (cid:88) s = (cid:98) L p (cid:99) ∆( s )= (cid:98) U p (cid:99) (cid:88) s = (cid:98) L p (cid:99) s (cid:88) d s =0 (cid:18) sd s (cid:19) p d s (1 − p ) s − d s ( f ( d s + 1) − f ( d s )) (67) = s (cid:88) d s =0 ( f ( d s + 1) − f ( d s )) (cid:98) U p (cid:99) (cid:88) s = (cid:98) L p (cid:99) (cid:18) sd s (cid:19) p d s (1 − p ) s − d s (68) ≥ U (cid:88) d s = L ( f ( d s + 1) − f ( d s )) (cid:98) U p (cid:99) (cid:88) s = (cid:98) L p (cid:99) (cid:18) sd s (cid:19) p d s (1 − p ) s − d s (69) ≥ (cid:106) U p (cid:107) − (cid:106) L p (cid:107) + 140( U + p ) U (cid:88) d s = L ( f ( d s + 1) − f ( d s )) (70) = (cid:106) U p (cid:107) − (cid:106) L p (cid:107) + 140( U + p ) ( f ( U + 1) − f ( L )) > (cid:106) U p (cid:107) − (cid:106) L p (cid:107) + 140( U + p ) · w (71)where (67) follows from the definition of ∆( s ) in (15) (see also (14)); (68) follows by exchanging the order of summation;(69) follows by only taking the summands of (68) from d s = L to d s = U ; (70) follows from Lemma 11 in Appendix; (71)follows from (12).Then, there exists an integer s b ∈ (cid:104) L p , U p (cid:105) such that ∆( s b ) ≥ (cid:106) U p (cid:107) − (cid:106) L p (cid:107) + 1 · (cid:106) U p (cid:107) − (cid:106) L p (cid:107) + 140( U + p ) · w w U + p ) (72)It only remains to prove P min ( s b ) ≥ w . By Lemmas 9 and 10, we can separately bound P ( − , s b ) and − P (+ , s b ) asfollows: P ( − , s b ) = s b (cid:88) j =0 (cid:18) s b j (cid:19) p j (1 − p ) s b − j f ( j ) ≥ s b (cid:88) j = (cid:98) ps b (cid:99) +1 (cid:18) s b j (cid:19) p j (1 − p ) s b − j f ( (cid:4) ps b (cid:5) + 1) > . · f ( L + 1) ≥ . · w w − P (+ , s b ) = 1 − s b (cid:88) j =0 (cid:18) s b j (cid:19) p j (1 − p ) s b − j f ( j + 1)= s b (cid:88) j =0 (cid:18) s b j (cid:19) p j (1 − p ) s b − j (1 − f ( j + 1)) ≥ (cid:100) ps b (cid:101) − (cid:88) j =0 (cid:18) s b j (cid:19) p j (1 − p ) s b − j (1 − f ( (cid:100) ps (cid:101) )) > . · (1 − f ( U )) > . · w w Therefore, from (16), we eventually arrive at P min ( s b ) = min( P ( − , s b ) , − P (+ , s b )) ≥ w . (73)This proves Lemma 4. D. Proof of Lemma 11
Consider the following sampling scenario: There is an infinite deck of cards, and each card has a “star” with probability p , and does not has a “star” with probability (1 − p ) . We will keep drawing cards one by one from the deck until we obtainexactly ( s + 1) “star” cards. Consider the event that we draw ( s + 1) cards from the deck in total. This means there are d s “star” cards in the first s cards, and the ( s + 1) -th card is the ( s + 1) -th “star” card. The probability of this event, denoted by ˆΓ( s ) , can be computed as ˆΓ( s ) := (cid:18) sd s (cid:19) p d s (1 − p ) s − d s p (74)Since the expectation of s is d s p , we know ˆΓ( s ) is increasing , when s ∈ (cid:16) , d s p (cid:17) decreasing , when s ∈ (cid:16) d s p , ∞ (cid:17) (75)Now the proof of Lemma 11 is tantamount to showing that (cid:98) U p (cid:99) (cid:88) s = (cid:98) L p (cid:99) ˆΓ( s ) ≥ p (cid:16)(cid:106) U p (cid:107) − (cid:106) L p (cid:107) + 1 (cid:17) U + p ) , ∀ d s ∈ [ L , U ] (76)which will be proved in the remaining subsection.First, it follows from (75) that (cid:98) dsp (cid:99) − (cid:80) s = (cid:98) L p (cid:99) ˆΓ( s ) (cid:106) d s p (cid:107) − (cid:106) L p (cid:107) ≥ (cid:98) dsp (cid:99) − (cid:80) s =0 ˆΓ( s ) (cid:106) d s p (cid:107) ≥ (cid:98) dsp (cid:99) − (cid:80) s =0 ˆΓ( s ) U p + 1 which implies (cid:98) dsp (cid:99) − (cid:88) s = (cid:98) L p (cid:99) ˆΓ( s ) ≥ (cid:106) d s p (cid:107) − (cid:106) L p (cid:107) U p + 1 dsp − (cid:88) s =0 ˆΓ( s ) (77) Given that the expectation of s is d s p , intuitively we expect that most of the ˆΓ( s ) terms should fall in the range of s ∈ (cid:104) , d s p (cid:105) , and that ˆΓ( s ) in s ∈ (cid:104) L p , U p (cid:105) is a “large” part of the probability mass. If so, then U /p (cid:88) s = L /p ˆΓ( s ) ≥ U − L U · U /p (cid:88) s =0 ˆΓ( s ) ≈ U − L U . However, this bound is hard to prove. Therefore, we use the bound (76) instead. Similarly, we have (cid:98) U p (cid:99) (cid:80) s = (cid:98) dsp (cid:99) +1 ˆΓ( s ) (cid:106) U p (cid:107) − (cid:106) d s p (cid:107) ≥ (cid:98) U dsp (cid:99) (cid:80) s = (cid:98) dsp (cid:99) +1 ˆΓ( s ) (cid:106) U + d s p (cid:107) − (cid:106) d s p (cid:107) ≥ (cid:98) U dsp (cid:99) (cid:80) s = (cid:98) dsp (cid:99) +1 ˆΓ( s ) U p + 1 where the last inequality is because (cid:106) U + d s p (cid:107) ≤ U p + (cid:106) d s p (cid:107) + 1 . As a consequence, we thus obtain (cid:98) U p (cid:99) (cid:88) s = (cid:98) dsp (cid:99) +1 ˆΓ( s ) ≥ (cid:106) U p (cid:107) − (cid:106) d s p (cid:107) U p + 1 (cid:98) U dsp (cid:99) (cid:88) s = (cid:98) dsp (cid:99) +1 ˆΓ( s ) (78)Next, we will bound the ratio between successive values of ˆΓ( s ) , which, in turn, helps bound the sum of ˆΓ( s ) as that of ageometric sequence. For the sake of notational brevity, we define λ := (cid:112) (1 − p ) d s µ := λp ( d s − λ )(1 − p ) Then, for any integer s in the range s ∈ (cid:104) d s − λp − , d s p (cid:105) , we have ˆΓ( s )ˆΓ( s + 1) = (cid:0) sd s (cid:1) p d s (1 − p ) s − d s p (cid:0) s +1 d s (cid:1) p d s (1 − p ) s +1 − d s p = s + 1 − d s ( s + 1)(1 − p )= 11 − p − d s ( s + 1)(1 − p ) ≥ − p − d sd s − λp (1 − p )= 1 − µ (79)Upon leveraging Lemma 8, we immediately see that ˆΓ (cid:18)(cid:22) d s p (cid:23)(cid:19) ≥ √ π (1 − p ) e √ d s · p ≥ p √ d s > p b + p ) (80)For d s ≥ L ≥ and p ≤ , we have a series of inequalities: d s λ = (cid:115) d s − p ≥ µ = λp ( d s − λ )(1 − p ) ≤ λ · λ · = 310 µ · (cid:22) d s p (cid:23) > µλp − µ = λ − pλ − p > p µ √ d s = ( d s − λ )(1 − p )3 λ √ d s ≥ ( d s − λ )(1 − p )3 d s ≥ d s · . d s ≥ . (81)Using these inequalities along with (79) implies (cid:98) dsp (cid:99) − (cid:88) s = (cid:98) ds − λp (cid:99) ˆΓ( s ) ≥ (cid:98) λp (cid:99) − (cid:88) j =1 (1 − µ ) j · ˆΓ (cid:18)(cid:22) d s p (cid:23)(cid:19) = 1 − µ − (1 − µ ) (cid:98) λp (cid:99) − (1 − µ ) · ˆΓ (cid:18)(cid:22) d s p (cid:23)(cid:19) > − µ − exp (cid:16) − µ · (cid:106) λp (cid:107)(cid:17) µ · p √ d s > (cid:18) − − e (cid:19) · . > (82) On the other hand, for any integer s ∈ (cid:104) d s p − , d s + λp − (cid:105) , we also have ˆΓ( s + 1)ˆΓ( s ) = (cid:0) s +1 d s (cid:1) p d s (1 − p ) s +1 − d s p (cid:0) sd s (cid:1) p d s (1 − p ) s − d s p = ( s + 1)(1 − p ) s + 1 − d s = 1 − p + d s (1 − p ) s + 1 − d s ≥ − p + d s (1 − p ) d s + λp − d s = 1 − λp (1 − p ) d s + λ ≥ − µ (83)Similar to the derivation of (82), using (81), we obtain from (83) that (cid:98) ds + λp (cid:99) (cid:88) s = (cid:98) dsp (cid:99) +1 ˆΓ( s ) ≥ (cid:98) λp (cid:99) − (cid:88) j =1 (1 − µ ) j · ˆΓ (cid:18)(cid:22) d s p (cid:23)(cid:19) > (84)Finally, combining (77), (78), (80), (82) and (84), we conclude that (cid:98) U p (cid:99) (cid:88) s = (cid:98) L p (cid:99) ˆΓ( s ) = (cid:98) dsp (cid:99) − (cid:88) s = (cid:98) L p (cid:99) ˆΓ( s ) + ˆΓ (cid:18)(cid:22) d s p (cid:23)(cid:19) + (cid:98) U p (cid:99) (cid:88) s = (cid:98) dsp (cid:99) +1 ˆΓ( s ) ≥ (cid:106) d s p (cid:107) − (cid:106) L p (cid:107) U p + 1 (cid:98) dsp (cid:99) − (cid:88) s =0 ˆΓ( s ) + ˆΓ (cid:18)(cid:22) d s p (cid:23)(cid:19) + (cid:106) U p (cid:107) − (cid:106) d s p (cid:107) U p + 1 (cid:98) U dsp (cid:99) (cid:88) s = (cid:98) dsp (cid:99) +1 ˆΓ( s ) ≥ (cid:106) d s p (cid:107) − (cid:106) L p (cid:107) U p + 1 (cid:98) dsp (cid:99) − (cid:88) (cid:98) ds − λp (cid:99) ˆΓ( s ) + ˆΓ (cid:18)(cid:22) d s p (cid:23)(cid:19) + (cid:106) U p (cid:107) − (cid:106) d s p (cid:107) U p + 1 (cid:98) ds + λp (cid:99) (cid:88) s = (cid:98) dsp (cid:99) +1 ˆΓ( s ) > (cid:106) d s p (cid:107) − (cid:106) L p (cid:107) U p + 1 ·
140 + 1 U p + 1 ·
140 + (cid:106) U p (cid:107) − (cid:106) d s p (cid:107) U p + 1 · p (cid:16)(cid:106) U p (cid:107) − (cid:106) L p (cid:107) + 1 (cid:17) b + p ) thereby completing the proof of Lemma 11. R The Annals of Mathematical Statistics , vol. 14, no. 4, pp. 436–440, 1943.[3] D. Du, F. K. Hwang, and F. Hwang,
Combinatorial group testing and its applications . World Scientific, 2000, vol. 12.[4] H. Q. Ngo and D.-Z. Du, “A survey on combinatorial group testing algorithms with applications to DNA library screening,”
Discrete mathematicalproblems with medical applications , vol. 55, pp. 171–182, 2000.[5] A. J. Macula and L. J. Popyack, “A group testing method for finding patterns in data,”
Discrete applied mathematics , vol. 144, no. 1-2, pp. 149–157,2004.[6] T. Berger, N. Mehravari, D. Towsley, and J. Wolf, “Random multiple-access communication and group testing,”
IEEE Transactions on Communications ,vol. 32, no. 7, pp. 769–779, Jul. 1984.[7] J. Wolf, “Born again group testing: Multiaccess communications,”
IEEE Transactions on Information Theory , vol. 31, no. 2, pp. 185–191, 1985.[8] C. Gollier and O. Gossner, “Group testing against covid-19,”
Covid Economics , vol. 2, 2020.[9] P. Damaschke, “Threshold group testing,” in
General theory of information transfer and combinatorics . Springer, 2006, pp. 707–718.[10] H.-B. Chen and H.-L. Fu, “Nonadaptive algorithms for threshold group testing,”
Discrete Applied Mathematics , vol. 157, no. 7, pp. 1581–1585, 2009.[11] M. Cheraghchi, “Improved constructions for non-adaptive threshold group testing,” in
International Colloquium on Automata, Languages, andProgramming . Springer, 2010, pp. 552–564.[12] G. De Marco, T. Jurdzi´nski, D. R. Kowalski, M. R´o˙za´nski, and G. Stachowiak, “Subquadratic non-adaptive threshold group testing,”
Journal of Computerand System Sciences , vol. 111, pp. 42–56, 2020.[13] T. V. Bui, M. Kuribayashi, M. Cheraghchi, and I. Echizen, “Efficiently decodable non-adaptive threshold group testing,”
IEEE Transactions on InformationTheory , vol. 65, no. 9, pp. 5519–5528, 2019.[14] T. V. Bui, M. Cheraghchi, and I. Echizen, “Improved non-adaptive algorithms for threshold group testing with a gap,” in
Proc. IEEE Int. Symp. Inf.Theory (ISIT) , LA, CA, USA, Jun. 2020, pp. 1414–1419.[15] C. L. Chan, S. Cai, M. Bakshi, S. Jaggi, and V. Saligrama, “Stochastic threshold group testing,” in
Proc. IEEE Inf. Theory Workshop (ITW) , Sevilla,Spain, Sep. 2013, pp. 1–5.[16] A. Reisizadeh, P. Abdalla, and R. Pedarsani, “Sub-linear time stochastic threshold group testing via sparse-graph codes,” in
Proc. IEEE Inf. TheoryWorkshop (ITW) . Guangzhou, China: IEEE, Nov. 2018, pp. 1–5.[17] C. L. Chan, S. Jaggi, V. Saligrama, and S. Agnihotri, “Non-adaptive group testing: Explicit bounds and novel algorithms,”
IEEE Transactions onInformation Theory , vol. 60, no. 5, pp. 3019–3035, 2014.[18] A. Emad and O. Milenkovic, “Semiquantitative group testing,”
IEEE Transactions on Information Theory , vol. 60, no. 8, pp. 4614–4636, 2014.[19] A. Ganesan, S. Jaggi, and V. Saligrama, “Learning immune-defectives graph through group tests,”
IEEE Transactions on Information Theory , vol. 63,no. 5, pp. 3010–3028, 2017.[20] M. Sobel and P. A. Groll, “Group testing to eliminate efficiently all defectives in a binomial sample,”
Bell System Technical Journal , vol. 38, no. 5, pp.1179–1252, 1959.[21] T. Li, C. L. Chan, W. Huang, T. Kaced, and S. Jaggi, “Group testing with prior statistics,” in
Proc. IEEE Int. Symp. Inf. Theory (ISIT) . Honolulu, HI,USA: IEEE, Jun./Jul. 2014, pp. 2346–2350.[22] S. A. Zenios and L. M. Wein, “Pooled testing for hiv prevalence estimation: exploiting the dilution effect,”
Statistics in Medicine , vol. 17, no. 13, pp.1447–1467, 1998.[23] H. Chernoff et al. , “A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations,”
The Annals of Mathematical Statistics ,vol. 23, no. 4, pp. 493–507, 1952.[24] N. G. D. Bruijn,
Asymptotic Methods Analysis . Chelmsford, MA, USA: Courier Corporation, 1981.[25] G. B. Folland,
Real analysis: modern techniques and their applications . John Wiley & Sons, 1999, vol. 40.[26] R. M. Corless, G. H. Gonnet, D. E. Hare, D. J. Jeffrey, and D. E. Knuth, “On the lambertw function,”