Admissible ways of merging p-values under arbitrary dependence
AAdmissible ways of merging p-values under arbitrarydependence
Vladimir Vovk ∗ Bin Wang † Ruodu Wang ‡ July 29, 2020
Abstract
Methods of merging several p-values into a single p-value are important in theirown right and widely used in multiple hypothesis testing. This paper is the first to sys-tematically study the admissibility (in Wald’s sense) of p-merging functions and theirdomination structure, without any assumptions on the dependence structure of theinput p-values. As a technical tool we use the notion of e-values, which are alternativesto p-values recently promoted by several authors. We obtain several results on therepresentation of admissible p-merging functions via e-values and on (in)admissibilityof existing p-merging functions. By introducing new admissible p-merging functions,we show that some classic merging methods can be strictly improved to enhance powerwithout compromising validity under arbitrary dependence.
Keywords: p-values, duality, multiple hypothesis testing, admissibility, e-values
A common task in multiple testing of a single hypothesis and testing multiple hypothesesis to combine several p-values into one p-value. If one assumes independence (or anotherspecific dependence structure) among p-values testing a scientific hypothesis H , then thecombined p-value is effectively testing a composition of H and the independence assump-tion. A rejection obtained from such a test may be due to statistical evidence against eitherindependence or the scientific hypothesis of interest (or both). As we typically only haveone realization of a bunch of p-values, it is not possible to identify the source of rejection.Hence, such a method cannot be justified unless convincing evidence of independence issupplied; however, as argued by Efron [2010, p. 50], neither independence nor positive re-gression dependence, which is often assumed in literature, is realistic in large-scale inference.Therefore, it is important to consider merging methods that are valid without any depen-dence assumption. In general, dropping the assumption of independence makes the problemof merging p-values more difficult: see, e.g., Vovk and Wang [2020b, Section 1]. ∗ Department of Computer Science, Royal Holloway, University of London, Egham, Surrey, UK. E-mail:[email protected]. † RCSDS, NCMIS, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing,China. Email: [email protected]. ‡ Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, Ontario, Canada.E-mail: [email protected]. a r X i v : . [ m a t h . S T ] J u l here are several valid merging methods for arbitrary dependence structure among p-values, which of course come at a cost of power. The most well-known one is arguablythe Bonferroni correction, which uses the minimum of p-values times the number of tests.Several other methods include those of R¨uger [1978] and Hommel [1983] based on orderstatistics of p-values, and those of Vovk and Wang [2019] based on generalized means ofp-values; see Section 3 for details of these merging methods. These methods include versionsof the method of Simes [1986] and the harmonic mean of Wilson [2019] that are valid underarbitrary dependence.The main objective of this paper is to study the domination structure among validfunctions for merging arbitrarily dependent p-values, henceforth p-merging functions . Ap-merging function is admissible if it is not strictly dominated by any other p-mergingfunction. Ideally, ceteris paribus, only admissible p-merging functions should be used, asother methods can be strictly improved. It turns out that admissibility and dominationstructure among p-merging functions give rise to highly non-trivial mathematical challenges.We are mainly interested in homogeneous and symmetric p-merging functions, as most p-merging functions used in practice are of this kind.Let us briefly summarize our main contributions. First, the merging function of Simes[1986] (valid under the assumption of independence) is the minimum of all symmetric p-merging functions (Theorem 6). Second, we give two representation results (Theorems 9and 10) of admissible p-merging functions which are naturally connected to e-values [Vovkand Wang, 2020a, Shafer, 2019, Gr¨unwald et al., 2020], our important technical tool, via aduality argument. Third, we provide an analytical condition for a calibrator to induce anadmissible p-merging function (Theorem 15). Fourth, we proceed to show that the classicp-merging functions of Hommel [1983] and the averaging functions of Vovk and Wang [2019]can be strictly improved to their more powerful versions (Theorem 16 and 20), whereas theorder statistics of R¨uger [1978] are generally admissible after a trivial modification (Theorem18). Various other smaller results on properties and comparisons of p-merging functions areobtained during our scientific journey.Our study gives rise to new merging methods which are free of any dependence assump-tion and more powerful than the ones in the existing literature. These p-merging functionscan be directly applied to any procedures for multiple hypothesis testing, such as thoseof Genovese and Wasserman [2004] and Goeman and Solari [2011]. The two most impor-tant new p-merging functions are H ∗ K , strictly dominating the function of Hommel [1983]and F ∗− ,K , strictly dominating the harmonic merging function of Vovk and Wang [2019].The Hommel and the harmonic p-merging functions have been shown to be special amongtwo general families (see Section 4 of Chen et al. [2020]) with wide applications, attractiveproperties, and good empirical performance (e.g., Wilson [2020]).Several mathematical results in this paper are quite sophisticated and surprising. InTheorem 16, we find the unexpected result that H ∗ K is always admissible among symmet-ric p-merging functions and it is admissible (among all p-merging functions) for non-primenumbers K of the input p-values, but not admissible in general for prime K . For a given p-merging function, it is generally difficult to prove or disprove its admissibility, or to constructa dominating admissible p-merging function. The proofs of our results rely on recent tech-niques in robust risk aggregation and dependence modeling. In particular, advanced resultson joint mixability in Wang and Wang [2011, 2016] play a crucial role in proving Theorem15, and many other results in the paper require complicated constructions of specific de-pendence structure among p-variables. Some open questions are presented in concludingSection 11 for the interested reader. 2 P-merging functions and basic properties
Without loss of generality we fix an atomless probability space (Ω , A , Q ) (see, e.g., F¨ollmerand Schied [2011, Proposition A.27] or Vovk and Wang [2020a, Appendix D]). A p-variable is a random variable P : Ω → [0 , ∞ ) satisfying Q ( P ≤ (cid:15) ) ≤ (cid:15) for all (cid:15) ∈ (0 , . The set of all p-variables is denoted by P Q . Throughout, K ≥ p-merging function of K p-values is an increasing Borel function F : [0 , ∞ ) K → [0 , ∞ ) suchthat F ( P , . . . , P K ) ∈ P Q whenever P , . . . , P K ∈ P Q . (Notice that the joint distributionof P , . . . , P K ∈ P Q can be arbitrary.) A p-merging function F is symmetric if F ( p ) isinvariant under any permutation of p , and it is homogeneous if F ( λ p ) = λF ( p ) for all λ ∈ (0 ,
1] and p with F ( p ) ≤
1. All p-merging functions that we encounter in this paper arehomogeneous and symmetric. Although we allow the domain of F to be [0 , ∞ ) K in orderto simplify presentation, the informative part of F is its restriction to [0 , K . Throughout, is the K -vector of zeros, is the K -vector of ones, and all vector inequalities and theoperation ∧ of taking the minimum of two vectors are component-wise.We say that a p-merging function F dominates a p-merging function G if F ≤ G . Thedomination is strict if, in addition, F ( p ) < G ( p ) for at least one p ∈ [0 , K . We say that ap-merging function is admissible if it is not strictly dominated by any p-merging function.Analogously, we can define admissibility within smaller classes of p-merging functions, suchas the class of symmetric p-merging functions. Finally, a p-merging function F is said to be precise if sup P ∈P KQ Q ( F ( P ) ≤ (cid:15) ) = (cid:15) for all (cid:15) ∈ (0 , . In other words, (cid:15) by (cid:15) , F attains the largest possible probability allowed for F ( P ) to be ap-value. Precise p-merging functions are the main object studied by Vovk and Wang [2019],where p-values are combined via averaging.We collect some basic properties of admissible p-merging functions, which will be usefulin our analysis later. In particular, an admissible p-merging function is always precise andlower semi-continuous, the limit of p-merging functions is again a p-merging function, andany p-merging function is dominated by an admissible p-merging function. The proofs ofthese results are put in Appendix A.1. Proposition 1.
An admissible p-merging function is always precise.
For an increasing Borel function F : [0 , ∞ ) K → [0 , ∞ ), its lower semicontinuous version F (cid:48) is given by F (cid:48) ( p ) := lim λ ↑ F ( λ p ) , p ∈ [0 , ∞ ) K . (1)Clearly, F (cid:48) is increasing, lower semicontinuous, and F (cid:48) ≤ F . Moreover, we define the zero-one adjusted version (cid:101) F of F by (cid:101) F ( p ) := (cid:40) F ( p ∧ ) ∧ p ∈ (0 , ∞ ) K . (2) Proposition 2. If F is a p-merging function, then both its lower semicontinuous version F (cid:48) in (1) and its zero-one adjusted version (cid:101) F in (2) are p-merging functions. In particular,an admissible p-merging function is always lower semicontinuous, takes value on [0 , ∞ ) K \ (0 , ∞ ) K , and satisfies F ( p ) = F ( p ∧ ) ∧ for all p ∈ [0 , ∞ ) K . Proposition 3.
The point-wise limit of a sequence of p-merging functions is a p-mergingfunction.
Combining the above results, we are able to show that any p-merging function is domi-nated by an admissible one.
Proposition 4.
Any p-merging function is dominated by an admissible p-merging function.Remark . Using the same proof as for Proposition 4, we can show that any symmetric p-merging function is dominated by a p-merging function that is admissible among symmetricp-merging functions. The same holds true if “symmetric” is replaced by “homogeneous” or“symmetric and homogeneous”.
Similarly to Vovk and Wang [2020a], we pay special attention to the two most natural fami-lies of p-merging functions: the family based on order statistics introduced by R¨uger [1978],henceforth the
O-family , where “O” stands for “order”, and the new family introduced byVovk and Wang [2019], henceforth the
M-family , where “M” stands for “mean”. The O-family is parameterized by k ∈ { , . . . , K } , and its k th element is the function (shown byR¨uger [1978] to be a p-merging function) G k,K : ( p , . . . , p K ) (cid:55)→ Kk p ( k ) ∧ , (3)where p ( k ) is the k th order statistic of p , . . . , p K . The M -family is parameterized by r ∈ [ −∞ , ∞ ], and its element with index r has the form F r,K : ( p , . . . , p K ) (cid:55)→ ( b r,K M r,K ( p , . . . , p K )) ∧ , (4)where M r,K ( p , . . . , p K ) := (cid:18) p r + · · · + p rK K (cid:19) /r and b r,K ≥ F r,K a precise merging function (its value willbe specified in Section 8.1). The average M r,K is also defined for r ∈ { , ∞ , −∞} as thelimiting cases of (4), which correspond to the geometric average, the maximum, and theminimum, respectively. The members of both families are precise p-merging functions.The initial and final elements of the M- and O-families coincide: the initial element isthe Bonferroni p-merging function G ,K = F −∞ ,K : ( p , . . . , p K ) (cid:55)→ K min( p , . . . , p K ) ∧ , and the final element is the maximum p-merging function G K,K = F ∞ ,K : ( p , . . . , p K ) (cid:55)→ max( p , . . . , p K ) . Another important p-merging function is that of Hommel [1983], given by H K := (cid:32) K (cid:88) k =1 k (cid:33) K (cid:94) k =1 G k,K . H K (or H K ∧
1, since a truncation at 1 is trivial) is a precise p-mergingfunction and it equals a constant (cid:96) K := (cid:80) Kk =1 k − times the function S K := K (cid:94) k =1 G k,K = 1 (cid:96) K H K , used by Simes [1986]. The Simes function S K is a valid merging function for independentp-variables (or under some other dependence assumptions, as in, e.g., Sarkar [1998]).Admissibility of the above p-merging functions will be studied in Sections 7 and 8. In thecase of inadmissibility, a function can be strictly improved to another p-merging functionwithout losing validity (Proposition 4). We will explicitly construct new merging functionsthat strictly dominate the existing ones. In the two special cases, the Bonferroni p-mergingfunction is shown to be admissible in Vovk and Wang [2020a, Proposition 6.1]. On thecontrary, the maximum p-merging function G K,K ( F ∞ ,K ) is not admissible for any K ≥ p , . . . , p K ) (cid:55)→ p . Nevertheless, after atrivial modification, G K,K is admissible within the class of symmetric p-merging functions;see Theorem 18 in Section 7.Next, we present a result showing that the Simes function S K has a very special rolein the context of p-merging, as it is a lower bound for any symmetric p-merging functions.Therefore, S K ( p , . . . , p K ) can be seen as the best achievable p-value obtained via symmetricmerging of p , . . . , p K , although the function S K itself is not a valid p-merging function. Theorem 6.
The Simes function S K is the minimum of all symmetric p-merging functions.Proof. Take any symmetric p-merging function F and p = ( p , . . . , p K ). Let α := S K ( p ) /K .Note that Kα ≤ p ( k ) ≥ kα for each k = 1 , . . . , K . By the symmetry of F , F ( p ) = F ( p (1) , . . . , p ( K ) ) ≥ F ( α, α, . . . , Kα ) =: β. Let Π be the set of all permutations of the vector ( α, α, . . . , Kα ), and µ be the discreteuniform distribution over Π. Take a random vector ( P , . . . , P K ) following the distribution Kαµ +(1 − Kα ) δ (1 ,..., . For each k , the distribution of P k is given by (cid:80) Kk =1 αδ kα +(1 − Kα ) δ ,and hence P k is a p-variable. Since F is a p-merging function, we have β ≥ Q ( F ( P , . . . , P K ) ≤ β ) ≥ Q (( P , . . . , P K ) ∈ Π) =
Kα.
This implies F ( p ) ≥ Kα = S K ( p ), and hence S K dominates all symmetric p-mergingfunctions. Finally, the statement of S K as a minimum follows from S K = (cid:86) Kk =1 G k,K ,noting that each G k,K is a symmetric p-merging function.In the main part of the paper we will be mainly interested in the case K >
2. The case K = 2 is very different but simple; it is treated separately in Appendix B. In this case, theBonferroni p-merging function is the only admissible symmetric p-merging function. As a prelude to studying the problem of merging p-values, we will discuss the notion ofe-values and the much easier problem of merging p-values into an e-value [Vovk and Wang,5020a, Appendix G]. As already mentioned, in this paper we are only interested in e-valuesas a technical tool.An e-variable is a non-negative extended random variable E : Ω → [0 , ∞ ] with E Q [ E ] ≤
1. A calibrator (or, more fully, “p-to-e calibrator”) is a decreasing function f : [0 , ∞ ) → [0 , ∞ ] satisfying f = 0 on (1 , ∞ ) and (cid:82) f ( x ) d x ≤
1. A calibrator transforms any p-variableto an e-variable. It is admissible if it is upper semicontinuous, f (0) = ∞ , and (cid:82) f ( x ) d x = 1(equivalently [Vovk and Wang, 2020a, Propositions 2.1 and 2.2], it is not strictly dominated,in a natural sense, by any other calibrator).A function F : [0 , ∞ ) K → [0 , ∞ ] is a p-to-e merging function if F ( P , . . . , P K ) is ane-variable for any p-variables P , . . . , P K . A p-to-e merging function F dominates a p-to-e-merging function G if F ≥ G , and the domination is strict if F (cid:54) = G ; F is admissible if it isnot strictly dominated by any other p-to-e merging function.Below, ∆ K is the standard K -simplex, that is, ∆ K := { ( λ , . . . , λ K ) ∈ [0 , K : λ + · · · + λ K = 1 } , and we always write p := ( p , . . . , p K ).It is clear that a convex mixture of e-variables is an e-variable. (In this sense convexmixture is an “e-merging function”. In the symmetric case arithmetic average essentiallydominates any other e-merging function [Vovk and Wang, 2020a, Proposition 3.1].) There-fore, for any calibrators f , . . . , f K and any ( λ , . . . , λ K ) ∈ ∆ K , the function F ( p ) := λ f ( p ) + · · · + λ K f K ( p K ) (5)is a p-to-e merging function.The following corollary of a duality theorem for optimal transport says that this proce-dure of p-to-e merging is general. Theorem 7.
For any calibrators f , . . . , f K and any ( λ , . . . , λ K ) ∈ ∆ K , (5) is a p-to-e merging function. Conversely, any p-to-e merging function is dominated by the p-to-emerging function (5) for some calibrators f , . . . , f K and some ( λ , . . . , λ K ) ∈ ∆ K .Proof. The non-trivial statement is the second one. Let F be a p-to-e merging function.Denote by F the set of decreasing real functions on [0 , ∞ ), and define the operator (cid:76) as (cid:32) K (cid:77) k =1 g k (cid:33) ( x , . . . , x K ) := K (cid:88) k =1 g k ( x k ) , ( g , . . . , g K ) ∈ F K , ( x , . . . , x K ) ∈ [0 , ∞ ) K . Using a classic duality theorem (see, e.g., Theorem 2.3 of R¨uschendorf [2013]), we havemin (cid:40) K (cid:88) k =1 (cid:90) g k ( x ) d x : ( g , . . . , g K ) ∈ F K , K (cid:77) k =1 g k ≥ F (cid:41) = sup P ∈P KQ E Q [ F ( P )] ≤ . Choose g , . . . , g K at which the minimum is attained. It is clear that we can define calibrators f , . . . , f K and ( λ , . . . , λ K ) ∈ ∆ K in such a way that λ k f k ≥ g k for all k , and then F willbe dominated by the p-to-e merging function (5).In the context of merging p-values, the most interesting special case of Theorem 7 iswhere F is constant in some region and zero outside the region. Corollary 8.
The class of admissible p-to-e merging functions coincides with the class offunctions (5) , f , . . . , f K ranging over the admissible calibrators and ( λ , . . . , λ K ) over ∆ K .Proof. Combine Theorem 7 with Vovk and Wang [2020a, Proposition G.2].6
Rejection regions of admissible p-merging functions
A p-merging function can be characterized by its rejection regions. The rejection region ofa p-merging function F at level (cid:15) > R (cid:15) ( F ) := (cid:8) p ∈ [0 , ∞ ) K : F ( p ) ≤ (cid:15) (cid:9) . (6)If F is homogeneous, then R (cid:15) ( F ), (cid:15) ∈ (0 , R (cid:15) ( F ) = (cid:15)A for some A ⊆ [0 , ∞ ) K .Conversely, any increasing collection of Borel lower sets { R (cid:15) ⊆ [0 , ∞ ) K : (cid:15) ∈ (0 , } determines an increasing Borel function F : [0 , ∞ ) K → [0 ,
1] by the equation F ( p ) = inf { (cid:15) ∈ (0 ,
1) : p ∈ R (cid:15) } , (7)with the convention inf ∅ = 1. It is immediate that F is a p-merging function if and onlyif Q ( P ∈ R (cid:15) ) ≤ (cid:15) for all (cid:15) ∈ (0 ,
1) and P ∈ P KQ .The main result in this section is a representation of rejection regions of admissible p-merging functions. It turns out calibrating p-values is a useful technical tool for studyingsuch rejection regions. Theorem 9.
For any admissible homogeneous p-merging function F , there exist ( λ , . . . , λ K ) ∈ ∆ K and admissible calibrators f , . . . , f K such that R (cid:15) ( F ) = (cid:15) (cid:40) p ∈ [0 , ∞ ) K : K (cid:88) k =1 λ k f k ( p k ) ≥ (cid:41) for each (cid:15) ∈ (0 , . (8) Conversely, for any ( λ , . . . , λ K ) ∈ ∆ K and calibrators f , . . . , f K , (8) determines a homo-geneous p-merging function. Theorem 7 applied to the e-variable 1 R (cid:15) ( F ) /(cid:15) only gives ( λ , . . . , λ K ) ∈ ∆ K and cali-brators f (cid:15) , . . . , f (cid:15)K satisfying (cid:80) k λ k f (cid:15)k ( p k ) ≥ R (cid:15) ( F ) ( p ) /(cid:15) . Theorem 9 gives, in the case ofadmissible homogeneous F , much more: in fact, the family of calibrators f (cid:15)k can be chosenas f (cid:15)k ( p ) := f k ( p/(cid:15) ) /(cid:15) .If the homogeneous p-merging function F is symmetric, then f , . . . , f K , as well as λ , . . . , λ K , in Theorem 9 can be chosen identical. Theorem 10.
For any F that is admissible within the family of homogeneous symmetricp-merging functions, there exists an admissible calibrator f such that R (cid:15) ( F ) = (cid:15) (cid:40) p ∈ [0 , ∞ ) K : 1 K K (cid:88) k =1 f ( p k ) ≥ (cid:41) for each (cid:15) ∈ (0 , . (9) Conversely, for any calibrator f , (9) determines a homogeneous symmetric p-merging func-tion. For a decreasing function f : [0 , ∞ ) → [0 , ∞ ] and a p-merging function F taking values in[0 , f induces F if (9) holds; similarly, we say that λ , . . . , λ K and f , . . . , f K induce F if (8) holds. Theorems 9 and 10 imply that admissible p-merging functions areinduced by some admissible calibrators. Generally, the calibrator inducing a given p-mergingfunction may not be unique. In the following examples, p-merging functions are induced bycalibrators, although these p-merging functions are not necessarily admissible.7 xample 11. The p-merging function F := G k,K , k ∈ { , . . . , K } , is induced by thecalibrator ( K/k )1 [0 ,k/K ] . Example 12.
In the case K = 2, the p-merging function F : p (cid:55)→ M ,K ( p ∧ ) ∧ { min p > } = 2 M ,K ( p ) ∧ { min p > } is induced by the admissible calibrator f : x (cid:55)→ (2 − x ) + on (0 , ∞ ) and f (0) = ∞ . Thefunction F is the zero-one adjusted version (see Proposition 2) of the arithmetic mergingfunction, and it is dominated by the Bonferroni merging function. Hence, F is not admis-sible. Example 13.
One may also generate p-merging functions from (9) where f is not a cal-ibrator. For the arithmetic merging function F := 2 M ,K , equality (9) holds by choosingthe function f : x (cid:55)→ − x . Note that f is not a calibrator and it takes negative values for x >
1. For another example, we take F := F r,K for r < F ( (cid:15) p ) ≤ (cid:15) as b r,K ( K (cid:80) Kk =1 p rk ) /r ≤
1, we see that R (cid:15) ( F ) = (cid:15) (cid:40) p ∈ [0 , ∞ ) K : 1 K K (cid:88) k =1 b rr,K p rk ≥ (cid:41) , thus satisfying (9) with f : x (cid:55)→ b rr,K x r . Such f is generally not a calibrator (not evenintegrable for r ≤ − b r,K in Section 8.The requirement f (0) = ∞ for an admissible calibrator f implies that the combined test(9) gives a rejection as soon as one of the input p-values is 0, which is obviously necessaryfor admissibility (Proposition 2). Although many examples in the M- and O-families, inparticular F r,K for r > G k,K for k >
1, do not satisfy this, we can make the zero-oneadjustment (2), which does not affect the validity of the p-merging function by Proposition2. In the sequel, a calibrator will be specified by its values on (0 , f = 0 on (1 , ∞ ) forany calibrator f , and f (0) should be clear in each specific example (in particular f (0) = ∞ if f is admissible). The value f (0) does not affect the p-merging function determined by (9)as long as f (0) ≥ K . We have seen that p-merging functions induced by admissible calibrators via Theorems 9 and10 are not necessarily admissible (Example 12). In this section, we study sufficient conditionsfor admissibility based on calibrators. First, Theorems 9 and 10 give an immediate criterionfor checking the admissibility of an induced p-merging function.
Proposition 14.
Suppose that F is a p-merging function taking values in [0 , and satis-fying (9) for a decreasing function f . The following statements hold:(i) F is admissible among symmetric p-merging functions if and only if there is no cali-brator g such that (cid:40) p ∈ [0 , ∞ ) K : 1 K K (cid:88) k =1 f ( p k ) ≥ (cid:41) (cid:40) (cid:40) p ∈ [0 , ∞ ) K : 1 K K (cid:88) k =1 g ( p k ) ≥ (cid:41) . (10)8 ii) F is admissible if and only if there are no ( λ , . . . , λ K ) ∈ ∆ K and calibrators g , . . . , g K such that (cid:40) p ∈ [0 , ∞ ) K : 1 K K (cid:88) k =1 f ( p k ) ≥ (cid:41) (cid:40) (cid:40) p ∈ [0 , ∞ ) K : K (cid:88) k =1 λ k g k ( p k ) ≥ (cid:41) . (11) Proof.
We will only show the first statement, as the second one follows from essentiallythe same proof. It suffices to show that F is not admissible among symmetric p-mergingfunctions if and only if (10) holds for some calibrator g . First, if there exists such g , thenthe p-merging function based on the calibrator g strictly dominates F . Second, if F isnot admissible, using Proposition 4 and Remark 5, we know that there exists G ≤ F thatis admissible among symmetric p-merging functions. Note that G can be safely chosen ashomogeneous. Using Theorem 10, G is induced by a calibrator g . Since G strictly dominates F , we know that (10) holds.Note that (10) does not imply g ≥ f , making the existence of g often complicatedto analyze. Proposition 14 implies, in particular, that for any calibrator f , f ≤ K on(0 ,
1] is a necessary condition for the induced p-merging function to be admissible, becauseotherwise the function g : x (cid:55)→ f ( cx ) ∧ K where c := (cid:82) f ( x ) ∧ K d x < F . On the other hand, if f (1) >
0, then the calibrator g := ( f − f (1)) / (1 − f (1))1 [0 , induces the same p-merging function F . Hence, it sufficesto consider f with f ≤ K on (0 ,
1] and f (1) = 0.The main result of this section gives a sufficient condition for the admissibility of thecorresponding p-merging function. For a calibrator f , we define another calibrator g :[0 , ∞ ) → [0 , ∞ ], for some η ∈ [0 , /K ], via g : x (cid:55)→ f (cid:18) x − η − Kη (cid:19) { x ∈ ( η, − ( K − η ] } + K { x ∈ [0 ,η ] } . (12)It is straightforward to verify (cid:82) g ( x ) d x ≤
1, and g defined via (12) is a calibrator. Theorem 15.
Suppose that an admissible calibrator f is strictly convex or strictly concaveon (0 , , f (0+) ∈ ( K/ ( K − , K ] , and f (1) = 0 . The p-merging function induced by f , or g in (12) for any η ∈ [0 , /K ] , is admissible.Proof. We will prove the statement on f , and the statement on g would then follow fromLemma 28 in Appendix A.3, which says that if f induces an admissible p-merging function,then so does g in (12). We only show the case where f is strictly convex, as the case ofa strictly concave f follows from a symmetric argument; we remark that f (0+) ≤ K for aconvex f and f (0+) > K/ ( K −
1) for a concave f play the same role in the proof.Suppose for the purpose of contradiction that there exists a p-merging function G whichstrictly dominates F , that is, there exist p = ( p , . . . , p K ) ∈ [0 , K and α ∈ (0 ,
1) such that G ( p ) < α < F ( p ) <
1. Denote by a := lim t ↓ f ( t ) ≤ K . Clearly, a > ,
1] bounded by 2 integrates to 1. Hence, it suffices to assume K ≥ f is continuous and strictly decreasing on (0 , f − : (0 , a ) (cid:55)→ (0 , f , which is strictly decreasing and strictly convex. Let U be auniform random variable on [0 , h be the density function f ( U ). Note that h is astrictly decreasing density function. Since p / ∈ R α ( F ), we have (cid:80) Kk =1 f ( p k /α ) < K . Denote9y y k := f ( p k /α ), k = 1 , . . . , K . Note that y + · · · + y K < K and y k < a for each k . Takea small constant (cid:15) := 14 min (cid:40) K (cid:94) k =1 ( a − y k ) , a − , − K K (cid:88) k =1 y k (cid:41) > . For each k = 1 , . . . , K , h is strictly decreasing in [ y k + (cid:15), y k + 2 (cid:15) ] since y k + 2 (cid:15) ≤ a − (cid:15) .Define another density function v k = ( h − h ( y k + 2 (cid:15) ))1 [ y k + (cid:15),y k +2 (cid:15) ] with its mass m k := (cid:82) y k +2 (cid:15)y k + (cid:15) v k ( t )d t > µ ( v k ) smaller than y k + 2 (cid:15) .Write β := 1 − K ( µ ( v )+ · · · + µ ( v K )). Since µ ( v )+ · · · + µ ( v K ) < y + · · · + y K +2 K(cid:15) < K ,we have β >
0. Take another small constant θ := min (cid:40) K (cid:94) k =1 m k βa − , f − ( a − (cid:15) ) , (1 − α )( K − α (cid:41) > , and let m ∗ := (cid:82) θ f ( t )d t − θβ ≤ ( a − θβ ≤ K (cid:94) k =1 m k . We have (cid:90) θ f ( t )d t = 1 − (cid:90) θ f ( t )d t = 1 − θ − m ∗ β. Note that a > f ( θ ) ≥ a − (cid:15) > (cid:87) Kk =1 y k + 2 (cid:15) . For k = 1 , . . . , K , define a probability densityfunction h k = 11 − θ − m ∗ (cid:18) h (0 ,f ( θ )] − m ∗ v k m k (cid:19) , (13)which is supported in interval (0 , f ( θ )], and its mean µ ( h k ) satisfies µ ( h k ) = (cid:82) θ f ( t )d t − m ∗ µ ( v k )1 − θ − m ∗ = 1 − θ − m ∗ β − m ∗ µ ( v k )1 − θ − m ∗ . We have K (cid:88) k =1 µ ( h k ) = K (1 − θ − m ∗ β ) − m ∗ (cid:80) Kk =1 µ ( v k )1 − θ − m ∗ = K > f ( θ ) . Note that each of h , . . . , h K has a decreasing density in (0 , f ( θ )], and the sum of their meansis larger than f ( θ ), thus satisfying the condition of joint mixability in Wang and Wang[2016, Theorem 3.2]. Using that theorem, there exists a random vector X = ( X , . . . , X K )satisfying X k ∼ h k , k = 1 , . . . , K and X + · · · + X K = K .Take disjoint events A, B, C, B , . . . , B K independent of X such that Q ( A ) = (1 − θ − m ∗ ) α , Q ( B ) = m ∗ α , Q ( C ) = 1 − α − θα/ ( K −
1) and Q ( B ) = · · · = Q ( B K ) = θα/ ( K − P = ( P , . . . , P K ) by letting, for k = 1 , . . . , K , P k = αf − ( X k )1 A + p k B + K (cid:88) j =1 ,j (cid:54) = k θα B j + 1 B k + 1 C . (14)10he decomposition (13) gives, for each k = 1 , . . . , K , that Q ( f − ( X k )1 A + f − ( y k )1 B > x )(1 − θ ) α ≥ − x − θ for all x ∈ ( θ, f − ( X k )1 A + f − ( y k )1 B on A ∪ B is stochasticallylarger than the U[ θ, P k is stochastically larger than θαδ θα + (1 − θ ) α U[ θα, α ] + (1 − α ) δ , and hence P k is a p-variable.If A happens, then f ( P k /α ) = X k for each k , and (cid:80) Kk =1 f ( P k /α ) = (cid:80) Kk =1 X k = K . Ifany of B k happens, then (cid:80) Kk =1 f ( P k /α ) = ( K − f ( θ ) > ( K − a − (cid:15) ) > K . In bothcases, using (9), P ∈ R α ( F ) ⊆ R α ( G ). If B happens, then P = p ∈ R α ( G ). Therefore, Q ( P ∈ R α ( G )) ≥ Q ( A ) + Q ( B ) + K (cid:88) k =1 Q ( B k ) = α + θαK − > α, (15)a contradiction to G being a p-merging function. This shows that F is admissible.Rephrasing the condition on g in Theorem 15, we get a sufficient condition on an admis-sible calibrator f to ensure that the induced p-merging function is admissible:For some η ∈ [0 , K ) and τ := 1 − ( K − η : f = K on (0 , η ], f ( η +) ∈ ( KK − , K ], f is strictly convex or strictly concave on ( η, τ ], and f (1) = 0. (16)Notice that the condition f ( η +) ∈ ( KK − , K ] in (16) and Theorem 15 excludes the simplecase K = 2 (treated in Appendix B). One may try to relax the requirement that convexityor concavity be strict; we explain technical difficulties in Remark 31 in Appendix A.5 forthe interested reader.In the following few sections, we analyze admissibility of the Hommel function, membersof the O-family, and members of the M-family. In cases of non-admissibility, we construct adominating admissible p-merging function. It turns out that, except for the Bonferroni p-merging function, none of these p-merging functions has a calibrator satisfying the condition(16), and many of them can indeed be improved, either trivially or significantly. Theorem15 becomes very useful in the construction of admissible p-merging functions dominatingthe ones in the M-family. This section is dedicated to the admissibility of the Hommel function H K and the O-family ofp-merging functions ( G k,K ) k =1 ,...,K for a given K . The calibrators we see below are generallynot continuous, and hence they do not satisfy the condition in Theorem 15. Nevertheless,some alternative arguments will justify the (in-)admissibility of the induced functions.We first show that the Hommel function H K ∧ H ∗ K . Recall that H K is given by H K := (cid:96) K (cid:86) Kk =1 G k,K , where (cid:96) K := (cid:80) Ki =1 1 k . Our modification H ∗ K of the Hommel function will beinduced by the function f : [0 , ∞ ) → [0 , ∞ ) defined by f : x (cid:55)→ K { (cid:96) K x ≤ } (cid:100) K(cid:96) K x (cid:101) , (17)11igure 1: The Hommel ∗ calibrator (solid and black) and the Harmonic ∗ calibrator (dashedand blue), for K := 12which we call the Hommel ∗ calibrator and whose graph is shown in Figure 1 as the blackpiece-wise horizontal line. It is straightforward to check that f is decreasing, f (1) = 0, and (cid:82) f ( x ) d x = 1, and hence f is indeed a calibrator. Theorem 16.
The p-merging function H K ∧ is dominated (strictly if K ≥ ) by thep-merging function H ∗ K induced by the Hommel ∗ calibrator. Moreover, H ∗ K is always ad-missible among symmetric p-merging functions, and it is admissible if K is not a primenumber.Proof. Since f induces H ∗ K , by Theorem 10, H ∗ K is a p-merging function.Let us verify that H K ≥ H ∗ K . The rejection region of H ∗ K satisfies R (cid:15) ( H ∗ K ) = (cid:40) p ∈ [0 , ∞ ) K : K (cid:88) k =1 { (cid:96) K p k ≤ (cid:15) } (cid:100) K(cid:96) K p k /(cid:15) (cid:101) ≥ (cid:41) . (18)For any p ∈ [0 , ∞ ) K and (cid:15) >
0, if H K ( p ) ≤ (cid:15) , then there exists m = 1 , . . . , K such that { k : K(cid:96) K p k /m ≤ (cid:15) } ≥ m . It follows that K (cid:88) k =1 { (cid:96) K p k ≤ (cid:15) } (cid:100) K(cid:96) K p k /(cid:15) (cid:101) ≥ K (cid:88) k =1 m { K(cid:96) K p k /(cid:15) ≤ m } = 1 m { k : K(cid:96) K p k /m ≤ (cid:15) } ≥ . By (18), p ∈ R (cid:15) ( H ∗ K ), and thus H ∗ K ( p ) ≤ (cid:15) . This shows H K ≥ H ∗ K . It is easy to check thatthe reverse direction holds (i.e., H K = H ∗ K ) if and only if K ≤ H ∗ K . Set τ := 1 / ( K(cid:96) K ). Using Proposition 14,suppose, for the purpose of contradiction, that there exists a calibrator g satisfying (10).12or x ∈ (0 , Kτ ], set p = · · · = p m = x and p m +1 = · · · = p K >
1, where m := (cid:100) τ x (cid:101) . Since f ( x ) = K/m , we have (cid:80) Kk =1 f ( p k ) = K . Using (10), K ≤ (cid:80) Kk =1 g ( p k ) = mg ( x ), and thus g ( x ) ≥ K/m = f ( x ).Since x ∈ (0 , Kτ ] is arbitrary, we have (cid:82) Kτ g ( x )d x ≥ (cid:82) Kτ f ( x )d x = 1. As g is acalibrator, this means g = f almost everywhere on [0 , f is left-continuous,which further implies g ≤ f . Hence, both sides of (10) coincide, leading to a contradiction.Thus, H ∗ K is admissible among symmetric p-merging functions.Finally, we show that H ∗ K is admissible if K is not a prime number. Suppose thatthere exist ( λ , . . . , λ K ) ∈ ∆ K and calibrators g , . . . , g K satisfying (11). For each m, k =1 , . . . , K , set y m,k := λ k g k ( mτ ) and T m := (cid:80) Kk =1 y m,k .Fix any m = 1 , . . . , K . Let Π m be the set of all subsets of { , , . . . , K } of exactly m elements. There are (cid:0) Km (cid:1) elements (sets) in Π m . For any J ∈ Π m , take any β > p = ( p , . . . , p K ) be given by p k = mτ { k ∈ J } + β { k / ∈ J } , k = 1 , . . . , K . Since (cid:80) Kk =1 f ( p k ) = K , (11) implies 1 ≤ (cid:80) Kk =1 λ k g k ( mτ ) = (cid:80) k ∈ J y m,k . Therefore, (cid:18) Km (cid:19) ≤ (cid:88) J ∈ Π m (cid:88) k ∈ J y m,k = (cid:18) K − m − (cid:19) K (cid:88) k =1 y m,k = (cid:18) K − m − (cid:19) T m . This gives T m ≥ K/m .For x ∈ (( m − τ, mτ ] and each k , we have λ k g k ( x ) ≥ λ k g k ( mτ ) = y m,K , and hence λ k ≥ (cid:82) Kτ λ k g k ( x )d x ≥ τ (cid:80) Km =1 y m,k . Therefore, K (cid:88) m =1 T m = K (cid:88) m =1 K (cid:88) k =1 y m,k = K (cid:88) k =1 K (cid:88) m =1 y m,k ≤ τ K (cid:88) k =1 λ K = 1 τ = K (cid:88) m =1 Km . (19)Putting (cid:80) k ∈ J y m,k ≥ T m ≥ K/m and (19) together, we get T m = K/m for each m = 1 , . . . , K , and (cid:80) k ∈ J y m,k = 1 for each J ∈ Π m . This further implies y m,k = 1 /m forall m ≤ K − k . Note that the case of m = K is not concluded here since Π K onlyhas one element, and the analysis of this case requires K to not be a prime number. Write K = k k for some integers k , k ≥ I ∈ Π k and J ∈ Π k − such that I ∩ J = ∅ , by noting that k + k − < K .Let p = ( p , . . . , p K ) be given by p k = Kτ { k ∈ I } + k τ { k ∈ J } + β { k / ∈ I ∪ J } , k = 1 , . . . , K. We have (cid:80) Kk =1 f ( p k ) = k + ( k − K/k = K . By (11), we have1 ≤ K (cid:88) k =1 λ k g k ( p k ) = (cid:88) k ∈ I y K,k + ( k −
1) 1 k . Hence, (cid:80) k ∈ I y K,k ≥ k /K for any I ∈ Π k . On the other hand, (cid:80) Kk =1 y K,k = 1, which leadsto y K,k = 1 /K for all k = 1 , . . . , K . Therefore, we obtain y m,K = m for all m, k = 1 , . . . , K .This implies λ k ≥ (cid:90) Kτ λ k g k ( x )d x ≥ τ K (cid:88) m =1 y m,k = 1 K .
Since (cid:80) Kk =1 λ k = 1, we now know g k = f almost everywhere, which further implies g k ≤ f ,and λ k = 1 /K , k = 1 , . . . , K . Therefore, both sides of (11) coincide, which is a contradiction.Thus, H ∗ K is admissible if K is not a prime number.13sing Theorem 6, we have S K ≤ F ≤ H K for any symmetric p-merging function F dominating H K , including F = H ∗ K . Hence, the improvement of any F over H K , measuredby the ratio H K /F , should always be in [1 , (cid:96) K ]. The improvement ratio H K /H ∗ K will beanalyzed in Section 9.In Theorem 16, we obtain that H ∗ K is admissible if K is not a prime number. Quitesurprisingly, if K is a prime number, then H ∗ K may be strictly dominated by some non-symmetric p-merging functions. In the following simple example, we give the dominatingfunctions for K = 2 and K = 3. More complicated examples can be constructed forlarger prime numbers, although we do not know whether K being prime always impliesnon-admissibility of H ∗ K . Example 17.
In case K = 2, H ∗ : ( p , p ) (cid:55)→ (3 p (1) ) ∧ ( p (2) ) is strictly dominated by F : ( p , p ) (cid:55)→ (3 p ) ∧ ( p ), which is a (non-symmetric) p-merging function because for anyp-variables P , P and α ∈ (0 , Q ( F ( P , P ) ≤ α ) ≤ Q (cid:18) P ≤ α (cid:19) + Q (cid:18) P ≤ α (cid:19) ≤ α + 23 α = α. In case K = 3, H ∗ is induced by the calibrator 3 g on (0 ,
1] where g := 1 [0 , / + 12 1 (2 / , / + 13 1 (4 / , / . Let the function F be given by the rejection set, for (cid:15) ∈ (0 , R (cid:15) ( F ) = (cid:15) { p ∈ [0 , ∞ ) : g ( p ) + g ( p ) + g ( p ) ≥ } , where g := g + (4 / , / , g := g − (4 / , / , and g := g . By Theorem 9, F isa (non-symmetric) p-merging function. Direct calculation shows that F strictly dominates H ∗ .Example 17 also shows that H K ∧ K ≥
2, since it is eitherstrictly dominated by H ∗ K ( K ≥
4) or by the functions in Example 17 ( K = 2 , G K,K , each member of theO-family is admissible if we trivially modify it by zero-one adjustment, as in Proposition 2.Although G K,K fails to be admissible, it is admissible among symmetric p-merging functionsafter this modification.
Theorem 18.
The p-merging function p (cid:55)→ G k,K ( p ∧ ) ∧ { min( p ) > } = G k,K ( p ) ∧ { min( p ) > } is admissible for k = 1 , . . . , K − , and it is admissible among symmetric p-merging functionsfor k = K .Proof. As we see from Example 11, for each k = 1 , . . . , K , p (cid:55)→ G k,K ( p ) ∧ { min( p ) > } isinduced by f : x (cid:55)→ ∞ { x =0 } + ( K/k )1 { x ∈ (0 ,k/K ] } .First, fix m = 1 , . . . , K −
1. Using Proposition 14, suppose, for the purpose of contra-diction, that there exist ( λ , . . . , λ K ) ∈ ∆ K and calibrators g , . . . , g K satisfying (11). Foreach k = 1 , . . . , K , denote y k := λ k g k ( m/K ). Since 1 = (cid:82) g k ( x )d x ≥ mK g k ( m/K ), we have y k ≤ λ k K/m , which implies (cid:80) Kk =1 y k ≤ K/m .14et Π m be the set of all subsets of { , , . . . , K } of exactly m elements. There are (cid:0) Km (cid:1) elements (sets) in Π m . For any J ∈ Π m , take any β > p = ( p , . . . , p K ) begiven by p k = mK { k ∈ J } + β { k / ∈ J } , k = 1 , . . . , K . Since (cid:80) Kk =1 f ( p k ) = K , (11) implies1 ≤ (cid:80) Kk =1 λ k g k ( p k ) = (cid:80) k ∈ J y k . Therefore, (cid:18) Km (cid:19) ≤ (cid:88) J ∈ Π m (cid:88) k ∈ J y k = (cid:18) K − m − (cid:19) K (cid:88) k =1 y k ≤ (cid:18) K − m − (cid:19) Km = (cid:18) Km (cid:19) . This implies (cid:80) k ∈ J y k = 1 for each J ∈ Π m , and further y k = 1 /m for each k = 1 , . . . , K .Therefore, λ k ≥ (cid:82) m/K λ k g k ( x )d x ≥ mK y k = 1 /K . Since (cid:80) Kk =1 λ k = 1, we have g k = f almost everywhere, which further implies g k ≤ f , and λ k = 1 /K , k = 1 , . . . , K . There-fore, both sides of (11) coincide, which is a contradiction. Thus, G m,K ( p ) ∧ { min( p ) > } isadmissible for each m = 1 , . . . , K − m = K , suppose that there exists a calibrator g satisfying(10). Since f ( x ) = 1 for x ∈ (0 , (cid:80) Kk =1 f ( x ) = K , which gives K ≤ Kg ( x ), andthus g ( x ) ≥ K/m = f ( x ). We have (cid:82) m/K g ( x )d x ≥ (cid:82) m/K f ( x )d x = 1. As g is a calibrator,this means g = f almost everywhere and further implies g ≤ f . Therefore, both sides of(10) coincide, which is a contradiction. Thus, G K,K ( p ) ∧ { min( p ) > } is admissible amongsymmetric p-merging functions. In this section, we study admissibility and the domination structure among the M-family ofp-merging functions, which turn out to be drastically different from those of the O-family,as members in the M-family are generally not admissible, except for the cases of F −∞ ,K and F ∞ ,K covered in Theorem 18.To study functions F r,K = ( b r,K M r,K ) ∧ b r,K , which unfortunately do not always admit an analytical form. The values of b r,K are obtained in Vovk and Wang [2019] for the cases r ≥ / ( K −
1) (Proposition 3), r = 0 (Proposition 4), and r = − b −∞ ,K = K and b ∞ ,K = 1 aretrivial to check. Below, we complement these results by providing formulas of b r,K for all r ∈ R via an analytical equation. We fix some notation which will be useful throughout thissection. For a fixed K and r ∈ ( −∞ , / ( K − c r be the unique number c ∈ (0 , /K )solving the equation( K − − ( K − c ) r + c r = K (1 − ( K − c ) r +1 − c r +1 ( r + 1)(1 − Kc ) , if r (cid:54)∈ {− , } ;1 − KcKc (1 − ( K − c ) = log(1 /c − ( K − c ) , if r = − K (1 − Kc ) = log(1 /c − ( K − c ) , if r = 0 . The existence and uniqueness of the solution c to the above equation can be checked directly,and it is implied by Lemma 3.1 of Jakobsons et al. [2016] in a more general setting. Moreover,15et c r := 0 if r ≥ / ( K − d r := 1 − ( K − c r , r ∈ R . (20)The proofs of propositions in this section are put in Appendix A.4. Proposition 19.
For K ≥ and r ≥ / ( K − , we have b r,K = (( r + 1) ∧ K ) /r . For K ≥ and r ∈ ( −∞ , K − ) , we have b r,K = 1 /M r,K ( c r , d r , . . . , d r ) . Via well-known inequalities on generalized mean functions of Hardy et al. [1952], it isstraightforward to check, without using Proposition 19, that if r < s and rs >
0, then K /s − /r b r,K ≤ b s,K ≤ b r,K . (21)The relationship (21) conveniently gives, among other implications, the monotonicity of r (cid:55)→ b r,K and its continuity except at 0. The continuity at 0 can be verified via Proposition 19. As illustrated by the numerical examples in Vovk and Wang [2019] and Wilson [2020], themost useful cases of the M-family are those with r ≤
0. In particular, the harmonic mergingfunction F − ,K , which is a constant times the harmonic mean p-value of Wilson [2019](truncated to 1), has a special role among the M-family and it performs similarly to theHommel function; see Chen et al. [2020]. On the other hand, the members F r,K for r > r <
0, as the other cases are similar. Usingthe equality b rr,K = K ( c rr + ( K − d rr ) − in Proposition 19, the rejection region of F r,K for (cid:15) ∈ (0 ,
1) is given by (see Example 13), R (cid:15) ( F r,K ) = (cid:15) (cid:40) p ∈ [0 , ∞ ) K : (cid:80) Kk =1 p rk c rr + ( K − d rr ≥ (cid:41) = (cid:15) (cid:40) p ∈ [0 , ∞ ) K : K (cid:88) k =1 p rk − d rr c rr − d rr ≥ (cid:41) . The strictly convex function x (cid:55)→ K ( x r − d rr ) / ( c rr − d rr ) is generally not a calibrator. Never-theless, there is a simple modification which induces a p-merging function dominating F r,K .Define the following function f r : x (cid:55)→ K (cid:18) x r − d rr c rr − d rr ∧ (cid:19) + . We can check that each f r is a calibrator. Let F ∗ r be the p-merging function induced by f r ,that is, R (cid:15) ( F ∗ r ) = (cid:15) (cid:40) p ∈ [0 , ∞ ) K : K (cid:88) k =1 (cid:18) p rk − d rr c rr − d rr (cid:19) + ≥ (cid:41) , (cid:15) ∈ (0 , . (22)It is clear that F ∗ r dominates F r,K . Moreover, the calibrator f r satisfies (16) with η = c r ,which means that F ∗ r is admissible by Theorem 15. In this way, an admissible p-mergingfunction dominating F r,K is constructed.In the next result, we give a rigorous statement of the above idea for all r < K − F ∗ r have a very simple relationship to those of F r,K .Remember that the minimum ∧ of two vectors is understood component-wise.16 heorem 20. For K ≥ and r ∈ ( −∞ , K − , F r,K is strictly dominated by the p-mergingfunction F ∗ r,K defined via, for p ∈ (0 , ∞ ) K and (cid:15) ∈ (0 , , F ∗ r,K ( p ) ≤ (cid:15) ⇐⇒ F r,K ( p ∧ ( (cid:15)d r )) ≤ (cid:15) or min( p ) = 0 , (23) where d r is given in (20) . Moreover, F ∗ r,K is admissible unless r = 1 .Proof. We first address the case r < / ( K − r ∈ (0 , / ( K − R (cid:15) ( F r,K ) = (cid:15) (cid:40) p ∈ [0 , ∞ ) K : (cid:80) Kk =1 p rk c rr + ( K − d rr ≤ (cid:41) = (cid:15) (cid:40) p ∈ [0 , ∞ ) K : K (cid:88) k =1 p rk − d rr c rr − d rr ≥ (cid:41) . and R (cid:15) ( F ,K ) = (cid:15) (cid:40) p ∈ [0 , ∞ ) K : K (cid:88) k =1 log p k − log d log c − log d ≥ (cid:41) , which share a form very similar to the case r <
0. Define the functions f r : x (cid:55)→ K (cid:18) x r − d rr c rr − d rr ∧ (cid:19) + for r (cid:54) = 0 and f : x (cid:55)→ K (cid:18) log x − log d log c − log d ∧ (cid:19) + . We can check with Proposition 19 that (cid:90) d r c r x r − d rr c rr − d rr d x = 1 − Kc r c rr − d rr (cid:18) c rr + ( K − d rr K − d rr (cid:19) = 1 − Kc r K , which implies (cid:82) f r ( x ) d x = 1, and similarly for r = 0. Hence, f r is a calibrator, whichfurther satisfies (16). As we explained above for the case r <
0, the p-merging function F ∗ r induced by f r strictly dominates F r,K , and the admissibility of F ∗ r follows from Theorem15. Finally, comparing the conditions for p ∈ R (cid:15) ( F r,K ) and p ∈ R (cid:15) ( F ∗ r ), i.e., if r (cid:54) = 0, K (cid:88) k =1 ( p k /(cid:15) ) r − d rr c rr − d rr ≥ K (cid:88) k =1 (cid:18) ( p k /(cid:15) ) r − d rr c rr − d rr (cid:19) + ≥ , the only difference is that any value p k larger than d r (cid:15) is treated as d r (cid:15) by F ∗ r . This implies F ∗ r = F ∗ r,K for F ∗ r,K in (23). The case r = 0 is similar.Next, we prove the statement for r ∈ [1 / ( K − , K − b rr,K = r + 1. Hence, the rejection region of F r,K for (cid:15) ∈ (0 ,
1) is given by R (cid:15) ( F r,K ) = (cid:15) (cid:40) p ∈ [0 , ∞ ) K : r + 1 K K (cid:88) k =1 p rk ≤ (cid:41) = (cid:15) (cid:40) p ∈ [0 , ∞ ) K : 1 K K (cid:88) k =1 g r ( p k ) ≥ (cid:41) , where g r : x (cid:55)→ ( r + 1)(1 − x r ) /r . Let τ = r/ ( r + 1) ∈ [1 /K, − /K ). Define a function f r : x (cid:55)→ τ − (1 − x r ) + for x > f r (0) = K . It is clear that f r is a calibrator bychecking (cid:82) f r ( x ) d x = 1. Since f r ≥ g r , we know that the p-merging function F ∗ r inducedby f r dominates F r,K . The domination F ∗ r ≤ F r,K is strict since it is easy to find some p , . . . , p K ∈ (0 , ∞ ) such that (cid:80) Kk =1 f r ( p k ) ≥ K > (cid:80) Kk =1 g r ( p k ). Moreover, for r (cid:54) = 1, f r iseither strictly convex or strictly concave on (0 ,
1) satisfying (16), hence F ∗ r is admissible byTheorem 15. The statement F ∗ r = F ∗ r,K is analogous to the case r < / ( K − f r of F ∗ r,K is given by x (cid:55)→ K (cid:18) x r − d rr c rr − d rr ∧ (cid:19) + if r < / ( K −
1) and r (cid:54) = 0; x (cid:55)→ K (cid:18) log x − log d log c − log d ∧ (cid:19) + if r = 0; x (cid:55)→ K { x =0 } + r + 1 r (1 − x r ) + if r ∈ [1 / ( K − , K − . Remark . Although in different disguises, the calibrator f := f − of F ∗− ,K and theHommel ∗ calibrator (17) are indeed remarkably similar: on the set { x > < f ( x ) < K } ,one of them takes the form f ( x ) = a/x − b , and the other one takes the form f ( x ) = a/ (cid:100) bx (cid:101) for some suitably chosen values of a, b >
0. In other words, the calibrator of F ∗− ,K can beseen as a continuous version of that of H ∗ K . Both calibrators are shown in Figure 1, where f − is referred to as the Harmonic ∗ calibrator. In Section 10, we shall see that F ∗− ,K and H ∗ K perform very similarly in our simulation experiments.In the next proposition, we give an explicit formula for F ∗ r,K in Theorem 20. In whatfollows, p (1) , . . . , p ( K ) are always the order statistics of components of p , from the smallestto the largest, and p m := ( p (1) , . . . , p ( m ) ) is the vector of the m smallest components of p . Proposition 22.
For K ≥ and p ∈ [0 , ∞ ) K , we have, if r ∈ ( −∞ , / ( K − , F ∗ r,K ( p ) = (cid:32) K (cid:94) m =1 M r,m ( p m ) M r,m ( c r , d r , . . . , d r ) (cid:33) ∧ { p (1) > } , (24) and, if r ∈ [1 / ( K − , K − , with the convention · / ∞ , F ∗ r,K ( p ) = (cid:32) K (cid:94) m =1 M r,m ( p m )(1 − rK ( r +1) m ) + (cid:33) ∧ { p (1) > } . (25)The remaining functions F r,K for r ≥ K − F ∞ ,K , which will be discussed in Proposition 24 below. To summarize,except for the Bonferroni and the maximum p-merging functions, any other member of theM-family is not admissible among homogeneous symmetric p-merging functions. Neverthe-less, for r < K −
1, a simple modification in (23) leads to admissible p-merging functionsbased on the generalized mean, which has a stronger power than the original members ofthe M-family.The (in-)admissibility of F ∗ r,K for r = 1 cannot be studied via Theorem 15 since the cali-brator is neither strictly convex or strictly concave. A discussion of the technical challengesin this special case is provided in Remark 31 in Appendix A.5. Next, we study the domination structure within the M-family of p-merging functions F r,K ,which are generally not admissible. It turns out that most members of the family are notcomparable; however, for K = 2 or large r , there are some domination relationships amongthe members in the family. We note that M s,K and M r,K for r (cid:54) = s are not proportional to18ach other, and hence the relations of domination among members of the M-family are allstrict.The following proposition gives a simple comparison for aM r,K and bM s,K , where a, b are two positive constants, e.g., a = b r,K and b = b s,K in the M-family. Using this result,we can compare two p-merging functions that are not precise (but perhaps have simplerforms), such as the asymptotically precise p-merging functions in Vovk and Wang [2019]. Proposition 23.
For r < s , K ≥ and a, b ∈ (0 , ∞ ) , the following statements hold.(i) aM r,K dominates bM s,K if and only if a ≤ b .(ii) bM s,K dominates aM r,K if and only if rs > and aK − /r ≥ bK − /s . Proposition 23 immediately implies that the asymptotically precise p-merging functions( K → ∞ ) in Table 1 of Vovk and Wang [2019] do not dominate each other. Proposition 24.
Suppose r (cid:54) = s . If K = 2 , F r,K is dominated by F s,K if and only if ≤ r < s or s < r ≤ . If K ≥ , F r,K is dominated by F s,K if and only if K − ≤ r < s . As a consequence of Proposition 24, in addition to F ∞ ,K , the members F r,K for r < K − K ≥
3, and the members for r ∈ [ K − , ∞ ) are not.In the simple case K = 2, the only two admissible members in the M-family are F −∞ , and F ∞ , , and the arithmetic average F , is the worst, as it is strictly dominated by every othermember of the M-family. By focusing on some the most important cases, we calculate the following four ratios mea-suring the improvement of the dominating p-merging functions over the standard ones inTheorems 16 and 20,inf p ∈ (0 , K F ∗− ,K ( p ) F − ,K ( p ) , inf p ∈ (0 , K F ∗ ,K ( p ) F ,K ( p ) , inf p ∈ (0 , K F ∗ ,K ( p ) F ,K ( p ) , and inf p ∈ (0 , K H ∗ K ( p ) H K ( p ) . The results are summarized in the following proposition.
Proposition 25.
For K ≥ , we have inf p ∈ (0 , K F ∗ ,K ( p ) F ,K ( p ) = inf p ∈ (0 , K F ∗ ,K ( p ) F ,K ( p ) = 0 , inf p ∈ (0 , K F ∗− ,K ( p ) F − ,K ( p ) = 1 − ( K − c − , and min p ∈ (0 , K H ∗ K ( p ) H K ( p ) = min (cid:40) t > K (cid:88) k =1 { t ≥ k/K } (cid:100) k/t (cid:101) ≥ (cid:41) =: γ K . Moreover, c − ∼ / ( K log K ) and γ K ∼ / log K as K → ∞ .Proof. Let (cid:15) = ( (cid:15), . . . , (cid:15), ∈ R K and (cid:15) (cid:48) = ( (cid:15), , . . . , ∈ R K for some (cid:15) > F ,K ( (cid:15) ) ≥ /K and F ∗ ,K ( (cid:15) ) ≤ KK − (cid:15) ≤ (cid:15) . Hence, F ∗ ,K ( (cid:15) ) /F ,K ( (cid:15) ) → (cid:15) ↓
0. 19ii) By definition, F ,K ( (cid:15) (cid:48) ) = (cid:15) /K c for some constant c > F ∗ ,K ( (cid:15) (cid:48) ) ≤ (cid:15)c (cid:48) for someconstant c (cid:48) >
0. Hence, F ∗ ,K ( (cid:15) (cid:48) ) /F ,K ( (cid:15) (cid:48) ) → (cid:15) ↓ c := c − . For any p ∈ (0 , ∞ ) K , we have F ∗− ,K ( p ) F − ,K ( p ) = K (cid:94) m =1 M − ,m ( p m ) /M − ,m ( v m ( c )) M − ,K ( p ) /M − ,K ( v K ( c ))= K (cid:94) m =1 (cid:32) c − + ( m − − ( K − c ) − c − + ( K − − ( K − c ) − × (cid:80) Kk =1 p − k ) (cid:80) mk =1 p − k ) (cid:33) ≥ K (cid:94) m =1 − ( K − c + ( m − c − ( K − c + ( K − c = 1 − ( K − c. Taking p = (cid:15) (cid:48) and letting (cid:15) ↓ p and let α = (cid:86) Kk =1 p ( k ) /k . Without loss of generality, we assume αK(cid:96) K ≤ H ∗ K ( p ) ≤ H K ( p ) ≤
1. Since H ∗ K is homogeneous, symmetric and increasing,we have H ∗ K ( p ) ≥ H ∗ K ( α, α, . . . , Kα ) = αK(cid:96) K γ K = γ K H K ( p ) . (26)The minimum ratio H ∗ K ( p ) /H K ( p ) = γ K is attained by p = ( α, α, . . . , Kα ) for α ∈ (0 , /K(cid:96) K ].(v) We continue to write c = c − . Proposition 6 of Vovk and Wang [2019] gives that b − ,K ∼ log K , and with Proposition 19 we get c (1 − ( K − c ) ∼ / ( K log K ). Since c ∈ (0 , /K ), the above implies Kc → K → ∞ , and this further implies c ∼ / ( K log K ). Next, we look at the quantity y K := 1 γ K = max (cid:40) y ≥ K (cid:88) k =1 { y ≤ K/k } (cid:100) ky (cid:101) ≥ (cid:41) . Note that y (cid:48) := (cid:98) y K (cid:99) + 1 satisfies (cid:80) Kk =1 1 { y (cid:48)≤ K/k } (cid:100) ky (cid:48) (cid:101) <
1, and we get1 > K (cid:88) i =1 { y (cid:48) ≤ K/k } ky (cid:48) = 1 y (cid:48) (cid:96) (cid:98) K/y (cid:48) (cid:99) ≥ log K − log y (cid:48) y (cid:48) , where the last inequality is due to (cid:96) k ≥ log( k + 1) for all k ∈ N . Hence, y (cid:48) + log y (cid:48) > log K , which implies y (cid:48) > log K − log log K and thus y K ≥ (cid:98) log K − log log K (cid:99) . Onthe other hand, Theorem 6 implies that y K ≤ H K /S K = (cid:96) K ≤ log K + 1. Therefore, y K ∼ log K as K → ∞ .In Proposition 25, there is a sharp contrast between the greatest improvement of F ∗− ,K and that of H ∗ K over their standard counterparts: asymptotically as K → ∞ , F ∗− ,K canimprove F − ,K only by a factor of 1 − / log K →
1, while H ∗ K can improve H K by asignificant factor of 1 / log K →
0. This observation is interesting especially seeing that H K and F − ,K perform similarly in simulation scenarios (see, e.g., the simulation studies inWilson [2020] and Chen et al. [2020]). Moreover, since H K = (cid:96) K S K and γ K ∼ / log K ∼ /(cid:96) K , ∗ K performs similarly to the Simes function S K for some values of p , e.g., those with orderstatistics close to (1 , . . . , K ) times a constant (see (26)), a situation that likely happens ifthe p-values are generated iid from a flat density around 0. This is remarkable as we see inTheorem 6 that all symmetric p-merging functions are dominated by S K .
10 Numerical illustration
In this section, we compare the performance of p-merging functions via simulation. Forthis purpose, we will plot the survival functions of F ( P , . . . , P K ) where F is one of H K , H ∗ K , F − ,K , F ∗− ,K and S K . The Simes function S K is used as a lower bound because itis the minimum of all symmetric p-merging functions (Theorem 6). The random variables P , . . . , P K are generated from several settings. In each setting, the percentage of the alter-native hypothesis δ takes the value 1 (dense signal), 0 . .
05 (sparsesignal), and K = 40 (small number of tests) or K = 400 (large number of tests). Theempirical survival function of F ( P , . . . , P K ) is computed via an average of 10,000 repeti-tions of simulation. Since the value of the survival function at (cid:15) > K hypotheses at level (cid:15) >
0, a smaller survival function means a strongerpower as soon as there is any signal.In the experiments of Figure 2, we conduct correlated z-tests for the mean of normalsamples with variance 1, where ρ stands for the correlation between any two tests and µ stands for the deviation of the mean from the null hypothesis to the alternative hypothesis.The first column in Figure 2 represents the case of µ = 0; that is, the null hypothesis is alwaystrue. To make the plots more informative, we drop the part “ ∧
1” of the definitions of H K and F − ,K and redefine H ∗ K and F ∗− ,K by (9) with the same calibrators f but with (cid:15) rangingover (0 , ∞ ) (this is relevant only for the first column of Figure 2). In the experiments ofFigure 3, we directly specify the distributions of p-values, under the alternative hypothesis,as a Beta( α,
1) distribution with density h ( p ) = αp α − , p ∈ (0 , α ∈ (0 , α means a stronger evidence against thenull hypothesis. The strength of signal, represented by µ or α , is simply chosen in each caseto make the comparison among the curves of survival functions visible. As we are interestedin the relative performance of these methods, the precise values of the parameters are notimportant.Our first observation from Figures 2–3 is the stunning similarity between the performanceof H ∗ K (black) and F ∗− ,K (red); in most cases one could not separate the two (see Remark21). Moreover, the improvement of H ∗ K over H K (green) is the most significant whenthe signal is dense; on the other hand, the improvement of F ∗− ,K over F − ,K (blue) isrelatively small but usually visible when there is some signal. As K grows from 40 to 400,the improvements become more pronounced. These observations are consistent with thefindings in Section 9, and all curves lie above that of Simes (cyan) as implied by Theorem6. We do not expect any of the other curves to be very close to that of S K , as the price forthe validity guarantee without any dependence assumption is never cheap.
11 Concluding remarks
In this paper, we established a representation and some conditions for admissible p-mergingfunctions via calibrators. Several new p-merging functions, most notably H ∗ K and F ∗− ,K ,21 . . . . . . rho = 0 , mu = 0 , delta = 0 , K = 40 s u r v i v a l f un c t i on . . . . . . rho = 0 , mu = 1.5 , delta = 1 , K = 40 s u r v i v a l f un c t i on . . . . . . rho = 0 , mu = 2 , delta = 0.3 , K = 40 s u r v i v a l f un c t i on . . . . . . rho = 0 , mu = 4 , delta = 0.05 , K = 40 s u r v i v a l f un c t i on Simes Hommel Harmonic Hommel* Harmonic* . . . . . . rho = 0.5 , mu = 0 , delta = 0 , K = 40 s u r v i v a l f un c t i on . . . . . . rho = 0.5 , mu = 1.5 , delta = 1 , K = 40 s u r v i v a l f un c t i on . . . . . . rho = 0.5 , mu = 2 , delta = 0.3 , K = 40 s u r v i v a l f un c t i on . . . . . . rho = 0.5 , mu = 4 , delta = 0.05 , K = 40 s u r v i v a l f un c t i on Simes Hommel Harmonic Hommel* Harmonic* . . . . . . rho = 0 , mu = 0 , delta = 0 , K = 400 s u r v i v a l f un c t i on . . . . . . rho = 0 , mu = 2 , delta = 1 , K = 400 s u r v i v a l f un c t i on . . . . . . rho = 0 , mu = 2.5 , delta = 0.3 , K = 400 s u r v i v a l f un c t i on . . . . . . rho = 0 , mu = 3 , delta = 0.05 , K = 400 s u r v i v a l f un c t i on Simes Hommel Harmonic Hommel* Harmonic* . . . . . . rho = 0.5 , mu = 0 , delta = 0 , K = 400 s u r v i v a l f un c t i on . . . . . . rho = 0.5 , mu = 2 , delta = 1 , K = 400 s u r v i v a l f un c t i on . . . . . . rho = 0.5 , mu = 2.5 , delta = 0.3 , K = 400 s u r v i v a l f un c t i on . . . . . . rho = 0.5 , mu = 3 , delta = 0.05 , K = 400 s u r v i v a l f un c t i on Simes Hommel Harmonic Hommel* Harmonic*
Figure 2: Survival functions of F ( P , . . . , P K ) for correlated z-tests22 .0 0.2 0.4 0.6 0.8 1.0 . . . . . . alpha = 0.6 , delta = 1 , K = 40 s u r v i v a l f un c t i on . . . . . . alpha = 0.4 , delta = 1 , K = 40 s u r v i v a l f un c t i on . . . . . . alpha = 0.2 , delta = 0.3 , K = 40 s u r v i v a l f un c t i on . . . . . . alpha = 0.05 , delta = 0.05 , K = 40 s u r v i v a l f un c t i on Simes Hommel Harmonic Hommel* Harmonic* . . . . . . alpha = 0.6 , delta = 1 , K = 400 s u r v i v a l f un c t i on . . . . . . alpha = 0.4 , delta = 1 , K = 400 s u r v i v a l f un c t i on . . . . . . alpha = 0.3 , delta = 0.3 , K = 400 s u r v i v a l f un c t i on . . . . . . alpha = 0.2 , delta = 0.05 , K = 400 s u r v i v a l f un c t i on Simes Hommel Harmonic Hommel* Harmonic*
Figure 3: Survival functions of F ( P , . . . , P K ) for independent Beta p-valuesare proposed and shown to be admissible. As seen from our main results and their proofs,admissibility of p-merging functions is quite a sophisticated object.We mention a few open questions about H ∗ K and F ∗ r,K . First, our study is mainlyconfined to homogeneous p-merging functions. The homogeneity requirement in Theorem 9is essential to our proof, and it is unclear whether or how one could relax it. On the otherhand, we are not sure how non-homogeneous p-merging functions are useful in hypothesistesting. Second, it is unclear how the strict convexity in Theorem 15 can be relaxed;see discussions in Remark 31. As a consequence, we suspect, but could not prove, theadmissibility of F ∗ ,K for K ≥
3. This function is not admissible for K = 2; see Example12. Third, we do not know whether H ∗ K is always inadmissible for all prime numbers K (see Example 17 for the cases of K = 2 and K = 3). Fourth, the admissible p-mergingfunction which dominates a given p-merging function is not unique. We wonder whetherthere are other admissible p-merging functions which dominate H K and F − ,K , the twomost important inadmissible p-merging functions, that have analytical formulas as well assuperior statistical performance. Acknowledgments
V. Vovk’s research has been partially supported by Astra Zeneca and Stena Line. R. Wangis supported by the Natural Sciences and Engineering Research Council of Canada (RGPIN-2018-03823, RGPAS-2018-522590). 23 eferences
Yuyu Chen, Peng Liu, Ken Seng Tan, and Ruodu Wang. Trade-off between valid-ity and efficiency of merging p-values under arbitrary dependence. Technical ReportarXiv:2007.12366 [math.ST], arXiv.org e-Print archive, July 2020.Bradley Efron.
Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing,and Prediction . Cambridge University Press, 2010.Hans F¨ollmer and Alexander Schied.
Stochastic Finance: An Introduction in Discrete Time .De Gruyter, Berlin, third edition, 2011.Christopher Genovese and Larry Wasserman. A stochastic process approach to false discov-ery control.
Annals of Statistics , 32(3):1035–1061, 2004.Jelle J. Goeman and Aldo Solari. Multiple testing for exploratory research.
StatisticalScience , 26(4):584–597, 2011.Peter Gr¨unwald, Rianne de Heide, and Wouter M. Koolen. Safe testing. Technical ReportarXiv:1906.07801 [math.ST], arXiv.org e-Print archive, June 2020.G. H. Hardy, John E. Littlewood, and George P´olya.
Inequalities . Cambridge UniversityPress, Cambridge, second edition, 1952.Gerhard Hommel. Tests of the overall hypothesis for arbitrary dependence structures.
Bio-metrical Journal , 25:423–430, 1983.Edwards Jakobsons, Xiaoying Han, and Ruodu Wang. General convex order on risk aggre-gation.
Scandinavian Actuarial Journal , 2016:713–740, 2016.Bernhard R¨uger. Das maximale Signifikanzniveau des Tests “Lehne H ab, wenn k unter n gegebenen Tests zur Ablehnung f¨uhren”. Metrika , 25:171–178, 1978.Ludger R¨uschendorf.
Mathematical Risk Analysis . Springer, Heidelberg, 2013.Sanat K. Sarkar. Some probability inequalities for ordered MTP random variables: a proofof the Simes conjecture. Annals of Statistics , 26:494–504, 1998.Thomas Sellke, M. J. Bayarri, and James Berger. Calibration of p-values for testing precisenull hypotheses.
American Statistician , 55:62–71, 2001.Glenn Shafer. The language of betting as a strategy for statistical and scientific communi-cation. Technical Report arXiv:1903.06991 [math.ST], arXiv.org e-Print archive, March2019. To appear in the
Journal of the Royal Statistical Society: Series A (with discussion).R. John Simes. An improved Bonferroni procedure for multiple tests of significance.
Biometrika , 73:751–754, 1986.Vladimir Vovk and Ruodu Wang. Combining p-values via averaging. Technical Re-port arXiv:1212.4966 [math.ST], arXiv.org e-Print archive, October 2019. To appearin
Biometrika .Vladimir Vovk and Ruodu Wang. Combining e-values and p-values. Technical ReportarXiv:1912.06116 [math.ST], arXiv.org e-Print archive, May 2020a.24ladimir Vovk and Ruodu Wang. A class of ie-merging functions. Technical ReportarXiv:2007.06382 [math.ST], arXiv.org e-Print archive, July 2020b.Bin Wang and Ruodu Wang. The complete mixability and convex minimization problemswith monotone marginal densities.
Journal of Multivariate Analysis , 102(10):1344–1360,2011.Bin Wang and Ruodu Wang. Joint mixability.
Mathematics of Operations Research , 41(3):808–826, 2016.Ruodu Wang, Liang Peng, and Jingping Yang. Bounds for the sum of dependent risksand worst Value-at-Risk with monotone marginal densities.
Finance and Stochastics , 17:395–417, 2013.Daniel J. Wilson. The harmonic mean p-value for combining dependent tests.
Proceedingsof the National Academy of Sciences , 116:1195–1200, 2019.Daniel J. Wilson. Generalized mean p-values for combining dependent tests: comparison ofgeneralized central limit theorem and robust risk analysis.
Wellcome Open Research , 5(55):1–21, 2020.
A Technical details
A.1 Proofs of Propositions 1, 2, 3 and 4
Proof of Proposition 1.
Suppose that F is an admissible p-merging function and there exists b ∈ (0 ,
1) such that a := sup P ∈P KQ Q ( F ( P ) ≤ b ) < b. Define the increasing function h : [0 , ∞ ) → [0 , ∞ ) by h ( x ) = a { x ∈ [ a,b ] } + x { x (cid:54)∈ [ a,b ] } . Wecan check, for t ∈ [ a, b ],sup P ∈P KQ Q ( h ◦ F ( P ) ≤ t ) = sup P ∈P KQ Q ( F ( P ) ≤ b ) = a ≤ t, and for t (cid:54)∈ [ a, b ], sup P ∈P KQ Q ( h ◦ F ( P ) ≤ t ) = sup P ∈P KQ Q ( F ( P ) ≤ t ) ≤ t. Hence, h ◦ F is a p-merging function. The fact that ( h ◦ F ) ∧ F ∧ F . Therefore, we obtain sup P ∈P KQ Q ( F ( P ) ≤ t ) ≥ t, t ∈ (0 , F is a p-merging function, we havesup P ∈P KQ Q ( F ( P ) ≤ t ) = t, t ∈ (0 , , and thus F is precise. 25 roof of Proposition 2. Fix P = ( P , . . . , P K ) ∈ P KQ and α ∈ (0 , Q ( F (cid:48) ( P ) ≤ α ) ≤ α . For every λ ∈ (0 , A λ be an event independent of P with Q ( A λ ) = λ and define the random vector P λ = ( P λ , . . . , P λK ) via P λ = λ P if A λ occurs,and P λ = (1 , . . . ,
1) if A λ does not occur. For all λ ∈ (0 ,
1) and k = 1 , . . . , K , noting that Q ( P k ≤ α/λ ) ≤ α/λ , we have Q ( P λk ≤ α ) = λQ ( λP k ≤ α ) = λQ ( P k ≤ α/λ ) ≤ α. Thus, P λ ∈ P KQ and by the fact that F is a p-merging function, we have Q ( F ( P λ ) ≤ α ) ≤ α .Note that Q ( F ( P λ ) ≤ α ) ≥ Q ( A λ ) Q ( F ( P λ ) ≤ α | A λ ) = λQ ( F ( λ P ) ≤ α ) , from which we obtain Q ( F ( λ P ) ≤ α ) ≤ αλ . Since F is increasing, by (1), we have F (cid:48) ( P ) ≥ F ( λ P ) for all λ ∈ (0 , Q ( F (cid:48) ( P ) ≤ α ) ≤ Q ( F ( λ P ) ≤ α ) ≤ αλ . Since λ ∈ (0 ,
1) is arbitrary, we have Q ( F (cid:48) ( P ) ≤ α ) ≤ α , thus showing that F (cid:48) is a p-mergingfunction.For the statement on ˜ F , it is clear that Q (cid:0) P ∈ [0 , K \ (0 , K (cid:1) = Q (cid:32) K (cid:91) k =1 { P k = 0 } (cid:33) ≤ K (cid:88) k =1 Q ( P k = 0) = 0 . Therefore, the values of F on [0 , K \ (0 , K do not affect its validity as a p-merging function.To show the last statement, let F be an admissible p-merging function. Using the aboveresults, we obtain that F (cid:48) ≤ F is a p-merging function. Admissibility of F forces F = F (cid:48) ,implying that F is lower semicontinuous. Similarly, F = (cid:101) F , implying that F takes value 0on [0 , K \ (0 , K . Proof of Proposition 3.
Let ( F n ) n ∈ N be a sequence of p-merging functions which convergesto its point-wise limit F . For any P = ( P , . . . , P K ) ∈ P KQ , we know that F n ( P ) → F ( P ) indistribution. Using the Portmanteau theorem, we have for all α ∈ (0 , Q ( F ( P ) < α ) ≤ lim inf n →∞ Q ( F n ( P ) < α ) ≤ α. It follows that for any (cid:15) > α ∈ (0 , Q ( F ( P ) ≤ α ) ≤ α + (cid:15). Since α and (cid:15) are arbitrary, we know that F ( P ) is a p-variable, and F is a p-mergingfunction. Proof of Proposition 4.
Let R be the uniform probability measure on [0 , K . Fix a p-merging function F . Set F := F and let c i := sup G : G ≤ F i − (cid:90) R ( G ≤ (cid:15) ) d (cid:15), (27)26here i := 1 and G ranges over all p-merging functions dominating F i − . Let F i be ap-merging function satisfying F i ≤ F i − and (cid:90) R ( F i ≤ (cid:15) ) d (cid:15) ≥ c i − − i , (28)where i := 1. Continue setting (27) and choosing F i to satisfy (28) for i = 2 , , . . . . Set G := lim i →∞ F i . By Proposition 3, G is a p-merging function. Clearly, G dominates F and (cid:90) R ( G ≤ (cid:15) ) d (cid:15) = (cid:90) R ( H ≤ (cid:15) ) d (cid:15) for any p-merging function H dominating G .By Proposition 2, the zero-one adjusted version (cid:101) G of G is a p-merging function, and sois the lower semicontinuous version (cid:101) G (cid:48) of (cid:101) G . Clearly (cid:101) G (cid:48) = 0 on [0 , K \ (0 , K . Let us checkthat (cid:101) G (cid:48) is admissible. Suppose that there exists a p-merging function H such that H ≤ (cid:101) G (cid:48) and H (cid:54) = (cid:101) G (cid:48) on (0 , K . Fix such an H and a p ∈ (0 , K satisfying H ( p ) < (cid:101) G (cid:48) ( p ). Since (cid:101) G (cid:48) is lower semicontinuous and H is increasing, there exists λ ∈ (0 ,
1) such that
H < (cid:101) G (cid:48) onthe hypercube [ λ p , p ] ⊆ [0 , K , which has a positive R -measure. This gives (cid:90) R ( G ≤ (cid:15) ) d (cid:15) ≤ (cid:90) R ( (cid:101) G (cid:48) ≤ (cid:15) ) d (cid:15) < (cid:90) R ( H ≤ (cid:15) ) d (cid:15), a contradiction. A.2 Proofs of Theorems 9 and 10
Proof of Theorem 9.
Fix an arbitrary (cid:15) ∈ (0 , R (cid:15) ( F ) is a lower set, andit is closed due to Proposition 2. By Theorem 2.3 of R¨uschendorf [2013],min (cid:40) K (cid:88) k =1 (cid:90) g k ( x ) d x : ( g , . . . , g K ) ∈ F K , K (cid:77) k =1 g k ≥ R (cid:15) ( F ) (cid:41) = max P ∈P KQ Q ( P ∈ R (cid:15) ( F )) = (cid:15), where the last equality holds because F is precise (Proposition 1). Take ( g (cid:15) , . . . , g (cid:15)K ) ∈ F K such that (cid:76) Kk =1 g (cid:15)k ≥ R (cid:15) ( F ) and (cid:80) Kk =1 (cid:82) g (cid:15)k ( x ) d x = (cid:15) . Obviously we can choose each g (cid:15)k to be non-negative and left-continuous. Using the fact that R (cid:15) ( F ) is a closed lower set, wehave max P ∈P KQ Q ( P ∈ R (cid:15) ( F )) = (cid:15) = ⇒ max P ∈P KQ Q ( (cid:15) P ∈ R (cid:15) ( F )) = 1 . (29)Therefore, using duality again,min (cid:40) K (cid:88) k =1 (cid:15) (cid:90) (cid:15) g k ( x ) d x : ( g , . . . , g K ) ∈ F K , K (cid:77) k =1 g k ≥ R (cid:15) ( F ) (cid:41) = 1 , implying (cid:80) Kk =1 (cid:82) (cid:15) g (cid:15)k ( x ) d x ≥ (cid:15) . As g k ≥ k and (cid:80) Kk =1 (cid:82) g (cid:15)k ( x ) d x = (cid:15) , we know g (cid:15)k ( x ) = 0 for x > (cid:15) . 27efine the set A (cid:15) := { p ∈ [0 , ∞ ) K : (cid:80) Kk =1 g (cid:15)k ( p k ) ≥ } . Since (cid:76) Kk =1 g (cid:15)k ≥ R (cid:15) ( F ) , wehave R (cid:15) ( F ) ⊆ A (cid:15) . Note that A (cid:15) is a closed lower set. By Markov’s inequality,sup P ∈P KQ Q (cid:32) K (cid:77) k =1 g (cid:15)k ( P ) ≥ (cid:33) ≤ sup P ∈P Q K (cid:88) k =1 E Q [ g (cid:15)k ( P )] = (cid:15). Hence, we can define a function F (cid:48) : [0 , ∞ ) K → R via R (cid:15) ( F (cid:48) ) = A (cid:15) and R δ ( F (cid:48) ) = δ(cid:15) − A (cid:15) forall δ ∈ (0 , A (cid:15) , F (cid:48) is a valid homogeneous p-merging function.Moreover, F (cid:48) dominates F since R δ ( F ) ⊆ A δ for all δ ∈ (0 ,
1) due to homogeneity of F .The admissibility of F now gives F = F (cid:48) , and thus R (cid:15) ( F ) = A (cid:15) = (cid:15) (cid:40) p ∈ [0 , ∞ ) K : K (cid:88) k =1 g (cid:15)k ( (cid:15)p k ) ≥ (cid:41) for each (cid:15) ∈ (0 , . Note that A := (cid:15) − R (cid:15) ( F ) = (cid:15) − A (cid:15) does not depend on (cid:15) ∈ (0 , (cid:15) ∈ (0 , λ k := (cid:15) − (cid:82) (cid:15) g (cid:15) ( x ) d x and f k : (0 , ∞ ) → R , x (cid:55)→ g (cid:15)k ( (cid:15)x ) /λ k for each k = 1 , . . . , K (if λ k = 0,then let f k := 1), and further set f k (0) = ∞ . It is clear that for each k with λ k (cid:54) = 0, (cid:90) f k ( x ) d x = (cid:82) (cid:15)g (cid:15)k ( (cid:15)x ) d x (cid:82) g (cid:15)k ( x ) d x = (cid:82) (cid:15) g (cid:15)k ( x ) d x (cid:82) g (cid:15)k ( x ) d x = 1 . The conditions that f k is decreasing and left-continuous, (cid:82) f k ( x ) d x = 1, f k (0) = ∞ , and f k ( x ) = 0 for x > f k is an admissible calibrator. Therefore, (8) holds.For the last statement, let f , . . . , f K be calibrators and ( λ , . . . , λ K ) ∈ ∆ K . Note thatfor each (cid:15) ∈ (0 , R (cid:15) ( F ) = (cid:40) p ∈ [0 , ∞ ) K : K (cid:88) k =1 λ k f k (cid:16) p k (cid:15) (cid:17) ≥ (cid:41) , and since f ( x ) = 0 for x >
1, it holds K (cid:88) k =1 λ k (cid:90) f k (cid:16) x(cid:15) (cid:17) d x = K (cid:88) k =1 λ k (cid:90) /(cid:15) f k ( y ) d y = (cid:15) K (cid:88) k =1 λ k = (cid:15). Hence, Markov’s inequality givessup P ∈P KQ Q ( P ∈ R (cid:15) ( F )) = sup P ∈P KQ Q (cid:32) K (cid:77) k =1 λ k f k (cid:18) P (cid:15) (cid:19) ≥ (cid:33) ≤ (cid:15). Thus, (8) determines a homogeneous p-merging function.
Proof of Theorem 10.
The proof is similar to that of Theorem 9 and we only mention thedifferences. For the first statement, it suffices to notice two facts. First, if R (cid:15) is symmetric,then g (cid:15) , . . . , g (cid:15)K in the proof of Theorem 9 can be chosen as identical; for instance, one canchoose the average of them (see, e.g., Proposition 2.5 of R¨uschendorf [2013]). Second, thesymmetry of R (cid:15) ( F ) guarantees that F (cid:48) in the proof of Theorem 9 is symmetric, and henceit is sufficient to require the admissibility of F within homogeneous symmetric p-mergingfunctions in this proposition. The last statement in the proposition follows from Theorem9 by noting that (9) defines a symmetric rejection region.28 emark . In the converse statements of Theorems 9 and 10, a p-merging function in-duced by admissible calibrators is not necessarily admissible (see Example 12), althoughadmissibility is indispensable in the proof of the forward direction. Using (29) and a com-pactness argument, a necessary and sufficient condition for a calibrator f to induce a precisep-merging function (a weaker requirement than admissibility) via (9) is Q (cid:32) K K (cid:88) k =1 f ( P k ) = 1 (cid:33) = 1 for some P , . . . , P K ∼ U[0 , . (30)Condition (30) may be difficult to check for a given f in general. For a convex f , as shownby Wang and Wang [2011, Theorem 2.4], (30) holds if and only if f ≤ K on (0 , Remark . Similarly to (30), an equivalent condition for the p-merging function F in (8)to be precise is Q (cid:32) K (cid:88) k =1 λ k f k ( P k ) = 1 (cid:33) = 1 for some P , . . . , P K ∼ U[0 , . (31)Using the terminology of Wang and Wang [2016], (31) means that the distributions of λ k f k ( P k ), k = 1 , . . . , K , are jointly mixable. Assuming convexity of the calibrators, (31)has a similar equivalent condition [Wang and Wang, 2016, Theorem 3.2], and this result isessential to the proof of Theorem 15 below. A.3 A lemma used in the proof of Theorem 15
Lemma 28.
If the p-merging function induced by a calibrator f is admissible, then so isthe p-merging function induced by g in (12) for any η ∈ [0 , /K ] .Proof of the lemma. The case η = 0 is trivial since g = f . If η = 1 /K , then g induces theBonferroni p-merging function, which is admissible as shown in Proposition 6.1 of Vovk andWang [2020a]. Below we assume η ∈ (0 , /K ). Let F and G be the p-merging functionsinduced by f and g , respectively, and let G (cid:48) be a p-merging function dominating G . Supposefor the purpose of contradiction that there exists p ∈ [0 , ∞ ) K and α ∈ (0 ,
1) such that α p ∈ R α ( G (cid:48) ) and α p (cid:54)∈ R α ( G ). Clearly, no component of p can be in [0 , η ], and hence p ∈ ( η, ∞ ) K . Let p (cid:48) = ( p − η ) / (1 − Kη ). By the relationship between f and g , weknow α p (cid:48) (cid:54)∈ R α ( F ). Let A = R α ( F ) ∪ { α p (cid:48) } . Take any vector P of p-variables, and let ν be the distribution of α ((1 − Kη ) P + η ). Further, let Π be the set of all permutationsof the vector ( αη, , . . . ,
1) and µ be the discrete uniform distribution over Π. Clearly,Π ⊆ R α ( G ) ⊆ R α ( G (cid:48) ). Let P (cid:48) follow the distribution ( Kηα ) µ + Kη (1 − α ) δ + (1 − Kη ) ν .It is easy to verify that the components of P (cid:48) are p-variables. Note that if α P ∈ A , then α ((1 − Kη ) P + η ) ∈ ( R α ( G ) ∪ { α p } ) ⊆ R α ( G (cid:48) ). We have α ≥ Q ( P (cid:48) ∈ R α ( G (cid:48) )) = Kηα + (1 − Kη ) Q ( α ((1 − Kη ) P + η ) ∈ R α ( G )) ≥ Kηα + (1 − Kη ) Q ( P ∈ A ) . Hence, Q ( P ∈ A ) ≤ α . Since P is arbitrary, this implies that the rejection region of F atlevel α can be enlarged to A , a contradiction of the admissibility of F . Therefore, the above p does not exist, and G is admissible. 29 .4 Proofs of Propositions 19, 22, 23 and 24 Proof of Proposition 19.
The case for r ≥ / ( K − r = 0 and r = − r = − r = 0are simply rearrangement of the results mentioned above. It remains to show the rest cases.Let q and q be the essential infimum and the essential supremum of a random variable,respectively, and U ⊂ P Q be the set of U[0 ,
1] random variables. Note that R (cid:15) ( F r,K ) = (cid:26) p ∈ [0 , ∞ ) K : M r,K ( p ) ≤ (cid:15)b r,K (cid:27) = (cid:15) (cid:26) p ∈ [0 , ∞ ) K : M r,K ( p ) ≤ b r,K (cid:27) . From R (cid:15) ( F r,K ), in order for sup P ∈P KQ Q ( P ∈ R (cid:15) ( F r,K )) = (cid:15) , it is necessary and sufficient tochoose 1 b r,K = inf P ∈P KQ q ( M r,K ( P )) , Simple algebra gives, for r < b − r,K = (cid:18) K sup { q ( U r + · · · + U rK ) | U , . . . , U K ∈ U} (cid:19) /r , and for r > b − r,K = (cid:18) K inf { q ( U r + · · · + U rK ) | U , . . . , U K ∈ U} (cid:19) /r . The rest of the proof is a direct consequence of Lemma 29 below, which gives, for r < { q ( U r + · · · + U rK ) | U , . . . , U K ∈ U} = ( K − − ( K − c ) r + c r , and for r ∈ (0 , / ( K − { q ( U r + · · · + U rK ) | U , . . . , U K ∈ U} = ( K − − ( K − c ) r + c r , where c = c r . Therefore, b − r,K = M r,K ( c r , d r , . . . , d r ). Lemma 29.
For any increasing convex function f : [0 , → R satisfying either f (1) = ∞ or f (1) − f (0) > K (cid:82) ( f ( u ) − f (0)) d u where f (1) is the limit of f ( x ) as x ↑ , we have sup { q ( f ( U ) + · · · + f ( U K )) | U , . . . , U K ∈ U} = ( K − f (( K − c F ) + f (1 − c F ) , where c F is the unique solution c ∈ (0 , /K ) to the following equation ( K − F − (( K − c ) + F − (1 − c ) = K (cid:82) − c ( K − c F − ( y ) d y − Kc . (32)
Proof of the lemma.
The lemma is essentially Theorem 3.4 of Wang et al. [2013] applied tothe probability level α = 0, noting that any convex quantile function f can be approximatedby distributions with a decreasing density. 30 roof of Proposition 22. We use the calibrators f r mentioned after Theorem 20. We firstconsider r <
0. For m = 1 , . . . , K and p , . . . , p K >
0, let v m := ( c r , d r , . . . , d r ) ∈ R m , andwe have m (cid:88) k =1 p r ( k ) − d rr c rr − d rr ≥ ⇐⇒ M r,m ( p m ) ≤ M r,m ( c r , d r , . . . , d r ) = M r,m ( v m ) . Hence, K (cid:88) k =1 (cid:18) p r ( k ) − d rr c rr − d rr (cid:19) + ≥ ⇐⇒ K (cid:95) m =1 (cid:32) m (cid:88) k =1 p r ( k ) − d rr c rr − d rr (cid:33) ≥ ⇐⇒ K (cid:94) m =1 M r,m ( p m ) M r,m ( v m ) ≤ . Using its calibrator f r , for each (cid:15) ∈ (0 , F ∗ r ( p ) ≤ (cid:15) if and only if (cid:86) Km =1 M r,m ( p m ) M r,m ( v m ) ≤ (cid:15) , andhence (24) holds. The case r ∈ [0 , / ( K − r ≥ / ( K − m = 1 , . . . , K and p , . . . , p K >
0, we have1 K m (cid:88) k =1 τ − (1 − p r ( k ) ) ≥ ⇐⇒ M r,m ( p m ) ≤ − τ Km . Hence, m (cid:88) k =1 f r ( p r ( k ) ) ≥ K ⇐⇒ K (cid:95) m =1 (cid:32) m (cid:88) k =1 τ − (1 − p r ( k ) ) + (cid:33) ≥ K ⇐⇒ K (cid:94) m =1 M r,m ( p m )(1 − τ K/m ) + ≤ . Since F ∗ r is induced by f r , we have, for (cid:15) ∈ (0 , F ∗ r ( p ) ≤ (cid:15) if and only if (cid:86) Km =1 M r,m ( p m )(1 − τK/m ) + ≤ (cid:15) or p (1) = 0. Hence, (25) holds. Proof of Proposition 23. (i) To show the “if” statement, it suffices to note again that M r,K ( u ) ≤ M s,K ( u ) for all u ∈ (0 , ∞ ) K and the above inequality is strict unless u has only one positive component. [Hardy et al., 1952, Theorem 16]. Therefore, aM r,K (strictly) dominates bM s,K . To show the “only if” statement, we note that aM r,K cannot dominate bM s,K if a > b since M r,K and M s,K agree on vectors with equalcomponents.(ii) We first assume 0 < r < s . To show the “if” statement, it suffices to note again that K /r M r,K ( u ) ≥ K /s M s,K ( u ) for all u ∈ [0 , ∞ ) K and the above inequality is strictif u does not have equal components [Hardy et al., 1952, Theorem 19]. Therefore, bM s,K (strictly) dominates aM r,K . To show the “only if” statement, we note that, if aK − /r < bK − /s , F r,K (1 , , . . . ,
0) = aK − /r < bK − /s = F s,K (1 , , . . . , , and thus bM s,K cannot dominate aM r,K if aK − /r < bK − /s .We next assume r < s <
0. To show the “if” statement, we first note that, usingHardy et al. [1952, Theorem 19], for all u ∈ (0 , ∞ ] K , K /r M r,K (1 / u ) = 1 K − /r M − r,K ( u ) ≥ K − /s M − s,K ( u ) = K /s M s,K (1 / u ) , u is 0. Therefore, bM s,K strictly dominates aM r,K if aK − /r ≤ bK − /s . To show the “only if” statement,we note that, if aK − /r < bK − /s , we havelim (cid:15) ↓ aM r,K (1 , /(cid:15), . . . , /(cid:15) ) = aK − /r < bK − /s = lim (cid:15) ↓ bM s,K (1 , /(cid:15), . . . , /(cid:15) ) , and thus bM s,K cannot dominate aM r,K if aK − /r < bK − /s .Finally, we consider the case rs ≤
0. If r ≤ < s , then using simple properties of theaverages, we have M r,K (0 , , . . . ,
1) = 0 < (cid:18) K − K (cid:19) /s = M s,K (0 , , . . . , . If r < s = 0, we havelim (cid:15) ↓ (cid:15) M r,K ( (cid:15) K , , . . . ,
1) = lim (cid:15) ↓ (cid:18) M r,K ( (cid:15) K , , . . . , (cid:15) (cid:19) = lim (cid:15) ↓ (cid:18) (cid:15) Kr + K − K(cid:15) r (cid:19) /r = 0 , whereas lim (cid:15) ↓ (cid:15) M ,K ( (cid:15) K , , . . . ,
1) = lim (cid:15) ↓ (cid:15) M ,K ( (cid:15) K , , . . . ,
1) = 1 > . In either case, bM ,K cannot dominate aM r,K .Summarizing the above cases, bM ,K dominates aM r,K if and only if aK − /r ≥ bK − /s and rs > Proof of Proposition 24.
In this proof, we do not truncate our merging functions at 1. Thatis, we directly treat F r,K = b r,K M r,K without loss of generality, since the functions in theM-family are homogeneous. We say that two p-merging functions are not comparable ifneither of them dominates the other one.Using Table 1 of Vovk and Wang [2019] (or Appendix B), the case K = 2 follows directlyfrom Proposition 23 since b r, = 2 /r for all r ∈ [ −∞ ,
1] and b r, = 2 for r <
1. We nextstudy the case K ≥
3. Using Table 1 of Vovk and Wang [2019], b r,K = K /r for r ≥ K − F r,K is dominated by F s,K if K − ≤ r < s . We next show thatthis is the only possible domination between F r,K and F s,K .First, for r, s ∈ [( K − − , K − b r,K = (1 + r ) /r . Clearly, b r,K is strictlydecreasing in r , and hence Proposition 23 (i) implies that F r,K does not dominate F s,K for r < s . Moreover, we can calculate b r,K K − /r b s,K K − /s = (cid:0) rK (cid:1) /r (cid:0) sK (cid:1) /s = (cid:18) r s (cid:19) /s (cid:18) rK (cid:19) /r − /s < . Therefore, F s,K does not dominate F r,K either. We thus know that F s,K and F r,K are notcomparable in this case.Next, we consider s < r ≤ ( K − − . To show that F s,K and F r,K are not comparable,by (21) and Proposition 23, it suffices to show b r,K (cid:54) = b s,K and b r,K K − /r (cid:54) = b s,K K − /s .These can be shown by straightforward (although cumbersome) calculation from the explicitformulas in Proposition 19. An intuitive explanation is that the dependence structure of the32ector P r ∈ P KQ which gives the precise probability Q ( F r,K ( P r ) ≤ (cid:15) ) = (cid:15) is different across r ∈ ( −∞ , K −
1] (see, e.g., Wang et al. [2013]). This leads to Q ( F s,K ( P r ) ≤ (cid:15) ) < (cid:15) and Q ( F r,K ( P s ) ≤ (cid:15) ) < (cid:15) for s (cid:54) = r , and hence the two p-merging functions cannot be compared.The above arguments show that each F r,K , r < K − F s,K for s in a neighbourhood of r . Finally, using Lemma 30 below, we obtain that F r,K for r ≤ K − Lemma 30. If F r,K is not dominated by F s,K for any s in a neighbourhood of r , then F r,K is admissible within the M-family.Proof of Lemma 30. Since F r,K is not dominated by any F s,K for s in a neighbourhood of r , we obtain from Proposition 23 (i) that b r,K > b s,K for all s > r using monotonicity of b r,K in (21). Similarly, b r,K K − /r < b s,K K − /s for all s < r with rs >
0. Using Proposition23 (i) and (ii), we know that F r,K is not dominated by F s,K if rs >
0. Also, by Proposition23 (ii), F r,K is not dominated by F s,K if s < r and rs ≤
0. Therefore, F r,K is admissiblewithin the M-family. A.5 An additional technical remark
Remark . We discuss technical challenges arising in trying to relax the strict convexity (orstrict concavity) imposed in Theorem 15 and to prove the admissibility of F ∗ ,K in Theorem20 for K ≥
3. Recall in the proof of Theorem 15 that the density h is obtained from adistribution with quantile function f , and h is decreasing if f is convex. A crucial step in thisproof is to verify that the distributions with densities h , . . . , h K are jointly mixable, whichensures that in (14), if A happens, the vector ( P , . . . , P K ) /α = ( f − ( X ) , . . . , f − ( X K ))satisfies (cid:80) Kk =1 f ( P k ) ≥ K , so that ( P , . . . , P K ) ∈ R α ( F ). The densities h , . . . , h K areobtained from the density h by removing a tiny piece m ∗ v k /m k for each k ; see (13). Since m ∗ v k /m k is tiny, the resulting density is still decreasing (or increasing) if h is strictlydecreasing (or strictly increasing), and hence joint mixability can be obtained from Theorem3.2 of Wang and Wang [2016]. In case the convex function f is linear on some interval(which is the case for F ∗ ,K ), h is constant on this interval. After removing a tiny piece onthis interval from h , the resulting density is no longer monotone, and no result for jointmixability is available in this case. Proving joint mixability is known to be a very difficulttask, although we suspect that it holds true for the above special case (if a proof is available,it likely will require a new paper). Unfortunately, it seems to us that one could not avoidthis task for a generalization of Theorem 15, since showing (cid:80) Kk =1 f ( P k ) ≥ K for h withsome pieces removed is essential for constructing any counter-example, at least to the bestof our imagination. B The case K = 2 In the simple case K = 2, where the task is to merge two p-values, the class of admissiblep-merging functions admits an explicit description.For E ⊆ [0 , K , let us set P ( E ) := sup P ∈P KQ Q ( P ∈ E )and call P ( E ) the upper p-probability of E . In the case K = 2 upper p-probability admits asimple characterization. 33 emma 32. The upper p-probability of any nonempty Borel lower set E ⊆ [0 , is P ( E ) = 1 ∧ inf (cid:8) u + u : ( u , u ) ∈ [0 , \ E (cid:9) . (33) Proof.
Let E be a nonempty lower Borel set in [0 , ; suppose P ( E ) is strictly less than theright-hand side of (33). Let t be any number strictly between P ( E ) and the right-hand sideof (33). If P is concentrated on [( t, , (0 , t )] ∪ [( t, t ) , (1 , , (34)and each of its components is uniformly distributed on [0 , P ∈ E with probability atleast t since E contains [( t, , (0 , t )]. Therefore, P ( E ) ≥ t . This contradiction proves theinequality ≥ in (33).As for the opposite inequality, we will check P ( E ) ≤ inf (cid:8) u + · · · + u K : ( u , . . . , u K ) ∈ [0 , K \ E (cid:9) for an arbitrary K ≥
2. Let us assume that E does not contain the set of all ( u , . . . , u K ) with u + · · · + u K = 1 (the case when it does is trivial). Choose (cid:15) > p , . . . , p K ) ∈ [0 , K \ E such that t := p + · · · + p K ∈ [ (cid:15),
1] and E contains all ( u , . . . , u K ) ∈ [0 , K satisfying u + · · · + u K = t − (cid:15) . Since E is a lower set, we have E ⊆ K (cid:91) k =1 (cid:8) ( u , . . . , u K ) ∈ [0 , K : u k ≤ p k (cid:9) , and the subadditivity of P further implies P ( E ) ≤ K (cid:88) k =1 P (cid:0)(cid:8) ( u , . . . , u K ) ∈ [0 , K : u k ≤ p k (cid:9)(cid:1) = K (cid:88) k =1 p k = t ≤ inf (cid:8) u + · · · + u K : ( u , . . . , u K ) ∈ [0 , K \ E (cid:9) + (cid:15). It remains to notice that (cid:15) can be chosen arbitrarily small.There is a natural bijection between the admissible p-merging functions for K = 2 andincreasing right-continuous functions f : [0 , → [0 , epigraph boundary of such f isthe set of points ( u , u ) ∈ [0 , such that f ( u − ) ≤ u ≤ f ( u ), where f (0 − ) is understoodto be 0 and f (1) is understood to be 1. A diagonal curve is the epigraph boundary of someincreasing function. The admissible p-merging function corresponding to a diagonal curve A ⊆ [0 , is defined by F ( p , p ) := u + u , where ( u , u ) ∈ A is the largest point in A that does not exceed ( p , p ) in the component-wise order ( A is linearly ordered by thispartial order).In particular, the only symmetric admissible p-merging function for K = 2 is Bonferroni.It corresponds to the identity function f : u (cid:55)→ uu