A. Martín Andrés

Fisher's mid-p-value arrangement in 2×2 comparative trials

Abstract Martin and Silva (1994) studied nine existing unconditional methods for comparing two proportions (independent samples), selecting Z methods (based on the classic chi-square statistics) and F1 (based on Fishers exact test) as the optimals, because their power is a higher than that of their competitors in computation time, and not too much lower than Barnards optimal method (the original B or its approximation B′). However Z or F1 are optimals depending on the value of K (which depends on the imbalance of the sample sizes), which means that a program is need which deals with both methods. In this paper the authors study thirteen new methods and show that the new method based on Fishers mid-p value is a solution halfway between methods Z and F1 (for every values of K), which it frequently surpasses in power, and approaching the B and B′ methods, especially in large samples (where B and B′ can not be applied). The authors conclude that the arrangement based on Fishers exact test mid-p value (for one- and two-tailed versions) is the optimal, because it maintains n adequate balance between its power and the computation time required.

Comparing the asymptotic power of exact tests in 2×2 tables

Abstract A 2×2 table may arise from three types of sampling, depending on the number of previously fixed marginals, and may yield three possible, differing, probabilistic models. From the unconditional point of view each model requires a specific solution but, within each model, the calculation time increases as the test procedure chosen is more powerful and, between the models, the calculation time decreases in the number of marginals fixed. Moreover, each model yields a test which is generally more powerful than the test of any other model with a larger number of marginals fixed. The condition under which a less powerful test, of the same or a different model, can substitute a more powerful test with a loss of power lower than 2% is determined. It is concluded that the Fisher exact test can be used as an approximation to Barnards exact test for a table with 0 or 1 fixed marginals, when the sample size is ⩾100 or when the smaller sample size is ⩾80, respectively. Similarly, Barnards exact test for a table with 1 fixed marginal can be used as an approximation of the same test for a table with 0 fixed marginals, when the sample size is ⩾50.

Chance-corrected measures of reliability and validity in K K tables

When studying the degree of overall agreement between the nominal responses of two raters, it is customary to use the coefficient kappa. A more detailed analysis requires the evaluation of the degree of agreement category by category, and this is carried out in two different ways: using the value of kappa in the collapsed table for each category or using the agreement index for each category (proportion of agreements observed). Both indices have disadvantages: the former is sensitive to marginal totals; the latter is not chance corrected; and neither distinguishes the case where one of the two raters is a gold standard (an expert) from the case where neither rater is a gold standard. This article suggests five chance-corrected indices which are not sensitive to marginal totals and which differ depending on whether there is a standard rater. The article also justifies the reason for poor performance of kappa when the two marginal totals are unbalanced (especially if they are so in opposite directions) and the reason for its good performance when analysing the various 2 2 tables obtained by the collapse of a wider table.

Exact unconditional non-classical tests on the difference of two proportions

Abstract The paper analyzes the three possible unconditional exact tests on the difference (d=p2−p1) of two independent binomial proportions: H SG : d⩽δ vs. K SG : d>δ,H PEG : d∉(δ 1 ,δ 2 ) vs. K PEG : d∈[δ 1 ,δ 2 ] and H SDG : d∈[δ 1 ,δ 2 ] vs. KSDG:d∉[δ1,δ2], where H(K) refers to the null (alternative) hypothesis. For this purpose the methodological bases of such tests are established; the properties of convexity and symmetry of the classic statistic z-pooled are studied; and, finally, the article indicates how to reduce computing problems by decreasing the parametric space and sample space involved (which can reduce the computing time up to tenfold). These tests are a generalization of the “classic” case (δ=0 in SG; δ1=δ2=0 in SDG) and the “non-classic” case (δ 0 in SG for establishing substantial superiority; δ2=−δ1>0 in PEG for establishing equivalence). The paper also proves that the present test PEG is more powerful than the one based on the classic TOST (two one-sided tests) method. Finally, the above tests were used to obtain exact confidence intervals.

Inferences about a linear combination of proportions

Statistical methods for carrying out asymptotic inferences (tests or confidence intervals) relative to one or two independent binomial proportions are very frequent. However, inferences about a linear combination of K independent proportions L = Σβipi (in which the first two are special cases) have had very little attention paid to them (focused exclusively on the classic Wald method). In this article the authors approach the problem from the more efficient viewpoint of the score method, which can be solved using a free programme, which is available from the webpage quoted in the article. In addition the article offers approximate formulas that are easy to calculate, gives a general proof of Agresti’s heuristic method (consisting of adding a certain number of successes and failures to the original results before applying Wald’s method) and, finally, it proves that the score method (which verifies the desirable properties of spatial and parametric convexity) is the best option in comparison with other methods.Statistical methods for carrying out asymptotic inferences (tests or confidence intervals) relative to one or two independent binomial proportions are very frequent. However, inferences about a linear combination of K independent proportions L = Σβipi (in which the first two are special cases) have had very little attention paid to them (focused exclusively on the classic Wald method). In this article the authors approach the problem from the more efficient viewpoint of the score method, which can be solved using a free programme, which is available from the webpage quoted in the article. In addition the article offers approximate formulas that are easy to calculate, gives a general proof of Agresti’s heuristic method (consisting of adding a certain number of successes and failures to the original results before applying Wald’s method) and, finally, it proves that the score method (which verifies the desirable properties of spatial and parametric convexity) is the best option in comparison with other methods.

On determining the P-value in 2 x 2 multinomial trials

Abstract One of the oldest problems in statistics is how to analyze a 2 x 2 table and it is still very much with us today. The case of zero fixed marginals, analyzed via the unconditional non-asymptotic method, is perhaps the one that has received least attention, owing to its computational complexity. This paper presents a critical review of existing publications on the topic, defines the conditions of equivalence, of symmetry, and of convexity to be satisfied by the points in the critical region (and takes advantage of the opportunity to generate, in an abbreviated manner, the sample space), defines two new criteria for ordering the elements in the sample space, analyzes the properties of symmetry of the error α (dependent on the two nuisance parameters) and their impact on the reduction of the parameter space to 1 4 (one-tailed test) or to 1 8 (two-tailed test) of its size by determining the P-value of an experimental table, shows how to minimize computations by grouping common elements, and finally, it gives various properties which allow the size of the parameter space to be reduced even more, and the number of times by which the size of the test must be determined. For purposes of comparison, it refers to the cases of one or two fixed marginals and to the assertions made by other authors.

Two-tailed approximate confidence intervals for the ratio of proportions

Various approximate methods have been proposed for obtaining a two-tailed confidence interval for the ratio R of two proportions (independent samples). This paper evaluates 73 different methods (64 of which are new methods or modifications of older methods) and concludes that: (1) none of the classic methods (including the well-known score method) is acceptable since they are too liberal; (2), the best of the classic methods is the one based on logarithmic transformation (after increasing the data by 0.5), but it is only valid for large samples and moderate values of R; (3) the best methods among the 73 methods is based on an approximation to the score method (after adding 0.5 to all the data), with the added advantage of obtaining the interval by a simple method (i.e. solving a second degree equation); and (4) an option that is simpler than the previous one, and which is almost as effective for moderate values of R, consists of applying the classic Wald method (after adding a quantity to the data which is usually

Asymptotical tests in 2 × 2 comparative trials (unconditional approach)

z_{\alpha /2}^{2}/4

Optimal correction for continuity and conditions for validity in the unconditional chi-squared test

).

The optimal method to make inferences about a linear combination of proportions

The unconditional Barnards test for the comparison of two independent proportions is difficult to apply even with moderately large samples. The alternative is to use a χ2 type, arc sine or mid-p asymptotic test. In the paper, the authors evaluate some 60 of these tests, some new and others that are already familiar. For the ordinary significances, the optimal tests are the arc sine methods (with the improvement proposed by Anscombe), the χ2 ones given by Pearson (with a correction for continuity of 2 or of 1 depending on whether the sample sizes are equal or different) and the mid-p-value ones given by Fisher (using the criterion proposed by Armitage, when applied as a two-tailed test). For one-(two) tailed tests, the first method generally produces reliable results E > 10.5 (E > 9 and unbalanced samples), the second method does so for E > 9 (E > 6) and the third does so for all cases, although for E ≤ 6 (E ≤ 10.5) it usually gives too many conservative results. E refers to the minimum expected quantity.