Fortunato Pesarin

Finite-sample consistency of combination-based permutation tests with application to repeated measures designs

In several application fields, e.g. genetics, image and functional analysis, several biomedical and social experimental and observational studies, etc. it may happen that the number of observed variables is much larger than that of subjects. It can be proved that, for a given and fixed number of subjects, when the number of variables increases and the noncentrality parameter of the underlying population distribution increases with respect to each added variable, then power of multivariate permutation tests based on Pesarins combining functions [Pesarin, F. (2001), Multivariate Permutation Tests with Applications in Biostatistics, New York: Wiley, Chichester] is monotonically increasing. These results confirm and extend those presented by [Blair, Higgins, Karniski and Kromrey (1994), ‘A Study of Multivariate Permutation Tests which May Replace Hotellings T 2 Test in Prescribed Circumstances’, Multivariate Behavioral Research 29, 141–163]. Moreover, they allow us to introduce the property of finite-sample consistency for those kinds of combination-based permutation tests. Sufficient conditions are given in order that the rejection rate converges to one, for fixed sample sizes at any attainable α -values, when the number of variables diverges. A simulation study and a real case study are presented.

A review and some new results on permutation testing for multivariate problems

In recent years permutation testing methods have increased both in number of applications and in solving complex multivariate problems. When available permutation tests are essentially of an exact nonparametric nature in a conditional context, where conditioning is on the pooled observed data set which is often a set of sufficient statistics in the null hypothesis. Whereas, the reference null distribution of most parametric tests is only known asymptotically. Thus, for most sample sizes of practical interest, the possible lack of efficiency of permutation solutions may be compensated by the lack of approximation of parametric counterparts. There are many complex multivariate problems, quite common in empirical sciences, which are difficult to solve outside the conditional framework and in particular outside the method of nonparametric combination (NPC) of dependent permutation tests. In this paper we review such a method and its main properties along with some new results in experimental and observational situations (robust testing, multi-sided alternatives and testing for survival functions).

A resampling procedure for nonparametric combination of several dependent tests

This paper deals with nonparametric methods for combining dependent permutation or randomization tests. Particularly, they are nonparametric with respect to the underlying dependence structure. The methods are based on a without replacement resampling procedure (WRRP) conditional on the observed data, also called conditional simulation, which provide suitable estimates, as good as computing time permits, of the permutational distribution of any statistic. A class C of combining functions is characterized in such a way that all its members, under suitable and reasonable conditions, are found to be consistent and unbiased. Moreover, for some of its members, their almost sure asymptotic equivalence with respect to best tests, in particular cases, is shown. An applicational example to a multivariate permutationalt-paired test is also discussed.

Losses, Hostility, and depression.

Forty outpatients with a primary unipolar major depression of recent onset and a matched control group of employees were administered the hostility scale of the Kellner Symptom Questionnaire, depression was rated with the Hamilton Rating Scale for Depression, and life events were recorded in a structured interview. In a subgroup of depressives who had not reported losses, there was an association of hostility and depression, whereas no such association was found in depressives who had reported losses. The findings are consistent with those of previous studies showing that hostility is largely limited to a subgroup of depressives. In a substantial proportion of the depressives, depression and hostility are independent affects.

On a Nonparametric Combination Method for Dependent Permutation Tests with Applications

This paper deals with a combination method for dependent permutation tests, which is nonparametric with respect to the underlying unknown dependence structure. The method is based on a simulation or resampling procedure, conditional on the data, which provides a simulated estimate of the permutation distribution of any statistic. Applications to some unusual and quite complex testing problems are shown.

Hostility and irritable mood in panic disorder with agoraphobia

Twenty patients suffering from panic disorder with agoraphobia were administered the hostility subscale of Kellners Symptom Questionnaire and the irritability scales of Paykels Clinical Interview for Depression and of Kellners Anxiety Rating Scale before and after behavioral treatment of agoraphobia. A matched control group of normal subjects had the same assessments at two similar points in time. Hostility and irritable mood decreased and friendliness increased in patients with panic disorder after treatment; upon recovery, there were no significant differences in hostility between patients and controls, whereas such differences were striking during the illness. The results suggest that increased hostility and irritable mood may be symptoms of panic disorder and improve with the treatment of agoraphobia.

Testing marginal homogeneity against stochastic order in multivariate ordinal data.

SUMMARY Many assessment instruments used in the evaluation of toxicity, safety, pain, or disease progression consider multiple ordinal endpoints to fully capture the presence and severity of treatment effects. Contingency tables underlying these correlated responses are often sparse and imbalanced, rendering asymptotic results unreliable or model fitting prohibitively complex without overly simplistic assumptions on the marginal and joint distribution. Instead of a modeling approach, we look at stochastic order and marginal inhomogeneity as an expression or manifestation of a treatment effect under much weaker assumptions. Often, endpoints are grouped together into physiological domains or by the body function they describe. We derive tests based on these subgroups, which might supplement or replace the individual endpoint analysis because they are more powerful. The permutation or bootstrap distribution is used throughout to obtain global, subgroup, and individual significance levels as they naturally incorporate the correlation among endpoints. We provide a theorem that establishes a connection between marginal homogeneity and the stronger exchangeability assumption under the permutation approach. Multiplicity adjustments for the individual endpoints are obtained via stepdown procedures, while subgroup significance levels are adjusted via the full closed testing procedure. The proposed methodology is illustrated using a collection of 25 correlated ordinal endpoints, grouped into six domains, to evaluate toxicity of a chemical compound.

Extending permutation conditional inference to unconditional ones

In this presentation we discuss the extension of permutation conditional inferences to unconditional or population ones. Within the parametric approach this extension is possible when the data set is randomly selected by well-designed sampling procedures on well-defined population distributions, provided that their nuisance parameters have boundely complete statistics in the null hypothesis or are provided with invariant statistics. When these conditions fail, especially if selection-bias procedures are used for data collection processes, in general most of the parametric inferential extensions are wrong or misleading. We will see that, since they are provided with similarity and conditional unbiasedness properties and if correctly applicable, permutation tests may extend, at least in a weak sense, conditional to unconditional inferences.

On Multi-Sided Permutation Tests

In fields such as clinical trials, the environment, epidemiology, genetics, pharmacology, social experiments, etc., situations in which the effect of a treatment can be positive on some individuals and negative on the rest may sometimes occur. In a two-sample design, this situation can be expressed by a response model where a random effect Δ in the alternative is such that , and . Such a situation is essentially different from the traditional two-sided test, in which the alternative is assumed to be active only on one of two directions. We consider alternatives in which two sub-alternatives (Δ < 0) and (Δ > 0) can be jointly active. In order to deal with such an atypical situation, we suggest to firstly apply two goodness-of-fit tests, one for the positive deviations from the empirical distribution function of control sample and the other for the negative, and then proceed with their nonparametric combination within a permutation framework. A simulation study inspects on properties of such a solution.

Union---intersection permutation solution for two-sample equivalence testing

One of the well-known problems with testing for sharp null hypotheses against two-sided alternatives is that, when sample sizes diverge, every consistent test rejects the null with a probability converging to one, even when it is true. This kind of problem emerges in practically all applications of traditional two-sided tests. The main purpose of the present paper is to overcome this very intriguing impasse by considering a general solution to the problem of testing for an equivalence null interval against a two one-sided alternative. Our goal is to go beyond the limitations of likelihood-based methods by working in a nonparametric permutation framework. This solution requires the nonparameteric Combination of dependent permutation tests, which is the methodological tool that achieves Roy’s Union–intersection principle. To obtain practical solutions, the related algorithm is presented. To appreciate its effectiveness for practical purposes, a simple example and some simulation results are also presented. In addition, for every pair of consistent partial test statistics it is proved that, if sample sizes diverge, when the effect lies in the open equivalence interval, the Rejection probability (RP) converges to zero. Analogously, if the effect lies outside that interval, the RP converges to one.