Mayer Alvo

Statistical Methods for Ranking Data

This book introduces advanced undergraduate, graduate students and practitioners to statistical methods for ranking data. An important aspect of nonparametric statistics is oriented towards the use of ranking data. Rank correlation is defined through the notion of distance functions and the notion of compatibility is introduced to deal with incomplete data. Ranking data are also modeled using a variety of modern tools such as CART, MCMC, EM algorithm and factor analysis. This book deals with statistical methods used for analyzing such data and provides a novel and unifying approach for hypotheses testing. The techniques described in the book are illustrated with examples and the statistical software is provided on the authors website.

Testing Ordered Alternatives in the Presence of Incomplete Data

Abstract In testing the null hypothesis of no treatment effects in a randomized block experiment, a researcher may restrict attention to an ordered alternative and thereby increase the power of his test. Jonckheere and later Page proposed such test statistics based on the Kendall and Spearman correlation coefficients. Motivated by notions of distance between permutations, we generalize Jonckheeres and Pages tests to the situation in which one or more observations are missing from one or more blocks. Conditional on the pattern of missing observations, the resulting statistics are shown to be asymptotically normal. For a particular pattern of missing observations, the asymptotic efficiency of the extended Page test is found, in many cases, to be not much lower than for the standard Page test.

Rank correlation methods for missing data

The subject of rank correlation has had a rich history. It has been used in numerous applications in tests for trend and for independence. However, little has been said about how to define rank correlation when the data are incomplete. The practice has often been to ignore missing observations and to define rank correlation for the smaller complete record. We propose a new class of measures of rank correlation which are based on a notion of distance between incomplete rankings. There is the potential for a significant increase in efficiency over the approach which ignores missing observations as demonstrated by a specific case.

Testing for mean and correlation changes in microarray experiments: an application for pathway analysis

BackgroundMicroarray experiments examine the change in transcript levels of tens of thousands of genes simultaneously. To derive meaningful data, biologists investigate the response of genes within specific pathways. Pathways are comprised of genes that interact to carry out a particular biological function. Existing methods for analyzing pathways focus on detecting changes in the mean or over-representation of the number of differentially expressed genes relative to the total of genes within the pathway. The issue of how to incorporate the influence of correlation among the genes is not generally addressed.ResultsIn this paper, we propose a non-parametric rank test for analyzing pathways that takes into account the correlation among the genes and compared two existing methods, Global and Gene Set Enrichment Analysis (GSEA), using two publicly available data sets. A simulation study was conducted to demonstrate the advantage of the rank test method.ConclusionsThe data indicate the advantages of the rank test. The method can distinguish significant changes in pathways due to either correlations or changes in the mean or both. From the simulation study the rank test out performed Global and GSEA. The greatest gain in performance was for the sample size case which makes the application of the rank test ideal for microarray experiments.

A Unified Nonparametric Approach for Unbalanced Factorial Designs

Motivated by questions arising from the field of statistical genetics, we consider the problem of testing main, nested, and interaction effects in unbalanced factorial designs. Based on the concept of composite linear rank statistics, a new notion of weighted rank is proposed. Asymptotic normality of weighted linear rank statistics is established under mild conditions, and consistent estimators are developed for the corresponding limiting covariance structure. A unified framework to use weighted rank to construct test statistics for main, nested, and interaction effects in unbalanced factorial designs is established. The proposed tests statistics are applicable to unbalanced designs with arbitrary cell replicates greater than one per cell. The limiting distributions under both the null hypotheses and Pitman alternatives are derived. Monte Carlo simulations are conducted to confirm the validity and power of the proposed tests. Genetic datasets from a simulated backcross study are analyzed to demonstrate the application of the proposed tests in quantitative trait loci mapping.

A general rank based approach to the analysis of block data

In the following we present a unified approach to rank based methods for the analysis of block designs and develop various tests for general unbalanced incomplete designs. The methodology relies on using the concept of compatibility to extend the definition of distances between complete rankings to those which may have missing observations or ties. Through this device we produce various tests based on the distances of Spearman and Kendall. The tests generated are applicable to general block designs with missing observations, ties, and multiple observations per cell, and include standard tests as special cases as well as others which are novel. Unlike other tests for general block designs, the test statistics derived here are easy to calculate, and their relation to tests for simpler designs is immediate. The approach gives a new interpretation to some previous ad hoc methods such as the correction for ties in the Friedman test and provides a natural extension of the Friedman-Durbin test to cyclic-designs.

The evaluation of long-term sulfur deposition models

Abstract The International Sulfur Deposition Model Evaluation (ISDME) assessed the performance of 11 regional, long-term deposition models in predicting amounts of sulfur wet deposition. With few exceptions, each model predicted air concentrations and dry/wet deposition amounts of SO2 and SO42− at up to 66 sites across eastern North America for each season of 1980. Unlike its predecessors, this evaluation focused on the ability of the models to replicate, within the uncertainty limits of the data, the magnitude and position of the seasonal spatial patterns of S wet deposition. Both the spatial patterns and the uncertainties arising from measurement and interpolation errors were determined by a simple Kriging technique. Model performance measures included the per cent of subrogions where interpolated predictions and observations were significantly different and differences between the magnitudes and locations of maxima, centroids and orientations of the major axes. When data uncertainty limits are considered, the seasonal predictions of one Lagrangian and two statistical models were insignificantly different from the seasonal observations (at approximately the 95% confidence level) across at least 85% of the evaluation region. Three other models performed relatively well for two or three seasons, while the performance of three others was somewhat erratic. The final two models significantly underpredicted for every season across at least 40% of the evaluation region. In spring, all of the models correctly predicted the location of the observed maximum amount of S wet deposition. For the other three seasons the models predicted the locations within 170–350 km. Except for summer, most models predicted the magnitude of the observed maximum within the data uncertainty limits.

Multivariate non-parametric tests of trend when the data are incomplete

In environmental and medical studies, multivariate data are often recorded over regular time intervals and examined for monotone increasing or decreasing trends in one or more of the variables. Dietz and Killeen (J. Amer. Statist. Assoc. 76 (1981) 169) proposed a non-parametric test based on the Kendall measure of correlation and applied it to medical data. In this paper, we are concerned with situations when the data are partially incomplete. New test statistics based on the Spearman and Kendall correlation coefficients are proposed which are shown to be asymptotically chi squared. Results from a limited simulation study reveal that in most situations, the proposed test statistic performs better than its counterpart which deletes the missing data.

Applications of Hamming Distance to the Analysis of Block Designs

Publisher Summary Hamming distance is used in coding theory for binary strings and can be used to define a rank correlation. Using Hamming distance between rankings, we generate rank-based tests for the analysis of general complete and incomplete block designs. The methodology used here relies on using the concept of compatibility to extend the definition of Hamming distance between complete rankings to those which may have missing observations or ties. In the incomplete case, the asymptotic distribution of the resulting statistic relies on the eigen values of two matrices; one the so-called information matrix of a block design, and the other a matrix whose eigen values are determined through the use of orthogonal polynomials. In one section, there is a recall on the notion of compatibility and use it to extend the Hamming distance between two complete rankings to the situations where either missing observations or ties may be present. The resulting tests have forms which are easily calculated with asymptotic distributions that are linear combinations of independent chi squares. In the case of incomplete blocks, the coefficients of these linear combinations are products of the eigen values of two (t x t) matrices. When the number of missing observation is the same for each block, one set of these eigen values is completely specified, while the eigen values of the other matrix are well known for many designs. Once the coefficients are determined, the critical values can be approximated.

A general theory of hypothesis testing based on rankings

Abstract In nonparametric statistics, a hypothesis testing problem based on the ranks of the data gives rise to two separate permutation sets corresponding to the null and to the alternative hypothesis, respectively. A modification of Critchlows unified approach to hypothesis testing is proposed. By defining the distance between permutation sets to be the average distance between pairs of permutations, one from each set, various test statistics are derived for the multi-sample location problem and the two-way layout. The asymptotic distributions of the test statistics are computed under both the null and alternative hypotheses. Some comparisons are made on the basis of the asymptotic relative efficiency.