Nitin R. Patel

A Network Algorithm for Performing Fisher's Exact Test in r × c Contingency Tables

Abstract An exact test of significance of the hypothesis that the row and column effects are independent in an r × c contingency table can be executed in principle by generalizing Fishers exact treatment of the 2 × 2 contingency table. Each table in a conditional reference set of r × c tables with fixed marginal sums is assigned a generalized hypergeometric probability. The significance level is then computed by summing the probabilities of all tables that are no larger (on the probability scale) than the observed table. However, the computational effort required to generate all r × c contingency tables with fixed marginal sums severely limits the use of Fishers exact test. A novel technique that considerably extends the bounds of computational feasibility of the exact test is proposed here. The problem is transformed into one of identifying all paths through a directed acyclic network that equal or exceed a fixed length. Some interesting new optimization theorems are developed in the process. The numer...

Computing an Exact Confidence Interval for the Common Odds Ratio in Several 2×2 Contingency Tables

Abstract A quadratic time network algorithm is provided for computing an exact confidence interval for the common odds ratio in several 2×2 independent contingency tables. The algorithm is shown to be a considerable improvement on an existing algorithm developed by Thomas (1975), which relies on exhaustive enumeration. Problems that would formerly have consumed several CPU hours can now be solved in a few CPU seconds. The algorithm can easily handle sparse data sets where asymptotic results are suspect. The network approach, on which the algorithm is based, is also a powerful tool for exact statistical inference in other settings.

Exact significance testing to establish treatment equivalence with ordered categorical data.

This communication concerns the problem of establishing the therapeutic equivalence of two treatments that are being compared on the basis of ordered categorical data. The problem is formulated as a significance test in which the null hypothesis specifies a treatment difference. An efficient numerical algorithm for computing the exact significance level is provided, along with a simple method for obtaining the asymptotic significance level. Both methods are applied to a clinical trial of a new agent versus an active control. Guidelines for when to use the exact procedure and when to rely on asymptotic theory are provided.

Computing Distributions for Exact Logistic Regression

Abstract Logistic regression is a commonly used technique for the analysis of retrospective and prospective epidemiological and clinical studies with binary response variables. Usually this analysis is performed using large sample approximations. When the sample size is small or the data structure sparse, the accuracy of the asymptotic approximations is in question. On other occasions, singularity of the covariance matrix of parameter estimates precludes asymptotic analysis. Under these circumstances, use of exact inferential procedures would seem to be a prudent alternative. Cox (1970) showed that exact inference on the parameters of a logistic model with binary response requires consideration of the distribution of sufficient statistics for these parameters. To date, however, resorting to the exact method has not been computationally feasible except in a few special situations. This article presents an efficient recursive algorithm that generates the joint and conditional distributions of the sufficient...

ALGORITHM 643: FEXACT: a FORTRAN subroutine for Fisher's exact test on unordered r×c contingency tables

The computer code for Mehta and Patels (1983) network algorithm for Fishers exact test on unordered r×c contingency tables is provided. The code is written in double precision FORTRAN 77. This code provides the fastest currently available method for executing Fishers exact test, and is shown to be orders of magnitude superior to any other available algorithm. Many important details of data structures and implementation that have contributed crucially to the success of the network algorithm are recorded here.

The future of drug development: advancing clinical trial design

Declining pharmaceutical industry productivity is well recognized by drug developers, regulatory authorities and patient groups. A key part of the problem is that clinical studies are increasingly expensive, driven by the rising costs of conducting Phase II and III trials. It is therefore crucial to ensure that these phases of drug development are conducted more efficiently and cost-effectively, and that attrition rates are reduced. In this article, we argue that moving from the traditional clinical development approach based on sequential, distinct phases towards a more integrated view that uses adaptive design tools to increase flexibility and maximize the use of accumulated knowledge could have an important role in achieving these goals. Applications and examples of the use of these tools — such as Bayesian methodologies — in early- and late-stage drug development are discussed, as well as the advantages, challenges and barriers to their more widespread implementation.

Exact Inference for Contingency Tables with Ordered Categories

Abstract This article proposes an efficient numerical algorithm for small-sample exact inferences in contingency tables having ordinal classifications. The inferences, which apply conditional on the observed marginal totals, also provide an exact analysis for the log-linear model of linear-by-linear association for cell probabilities. An exact test of independence has a one-sided P value equal to the null probability that model-based maximum likelihood estimates of odds ratios are at least as large as the observed estimates. The conditional nonnull distribution yields confidence intervals for odds ratios having a linear-by-linear structure. The computations utilize an extension of the network algorithm proposed by Mehta and Patel (1983).

Efficient Monte Carlo Methods for Conditional Logistic Regression

Abstract Exact inference for the logistic regression model is based on generating the permutation distribution of the sufficient statistics for the regression parameters of interest conditional on the sufficient statistics for the remaining (nuisance) parameters. Despite the availability of fast numerical algorithms for the exact computations, there are numerous instances where a data set is too large to be analyzed by the exact methods, yet too sparse or unbalanced for the maximum likelihood approach to be reliable. What is needed is a Monte Carlo alternative to the exact conditional approach which can bridge the gap between the exact and asymptotic methods of inference. The problem is technically hard because conventional Monte Carlo methods lead to massive rejection of samples that do not satisfy the linear integer constraints of the conditional distribution. We propose a network sampling approach to the Monte Carlo problem that eliminates rejection entirely. Its advantages over alternative saddlepoint and Markov Chain Monte Carlo approaches are also discussed.

Exact inference for matched case-control studies

In an epidemiological study with a small sample size or a sparse data structure, the use of an asymptotic method of analysis may not be appropriate. In this paper we present an alternative method of analyzing data for case-control studies with a matched design that does not rely on large-sample assumptions. A recursive algorithm to compute the exact distribution of the conditional sufficient statistics of the parameters of the logistic model for such a design is given. This distribution can be used to perform exact inference on model parameters, the methodology of which is outlined. To illustrate the exact method, and compare it with the conventional asymptotic method, analyses of data from two case-control studies are also presented.

Clustering seasonality patterns in the presence of errors

Clustering is a very well studied problem that attempts to group similar data points. Most traditional clustering algorithms assume that the data is provided without measurement error. Often, however, real world data sets have such errors and one can obtain estimates of these errors. We present a clustering method that incorporates information contained in these error estimates. We present a new distance function that is based on the distribution of errors in data. Using a Gaussian model for errors, the distance function follows a Chi-Square distribution and is easy to compute. This distance function is used in hierarchical clustering to discover meaningful clusters. The distance function is scale-invariant so that clustering results are independent of units of measuring data. In the special case when the error distribution is the same for each attribute of data points, the rank order of pair-wise distances is the same for our distance function and the Euclidean distance function. The clustering method is applied to the seasonality estimation problem and experimental results are presented for the retail industry data as well as for simulated data, where it outperforms classical clustering methods.