Frank Critchley

Identifying Regression Outliers and Mixtures Graphically

Abstract Regressions in practice can include outliers and other unknown subpopulation structures. For example, mixtures of regressions occur if there is an omitted categorical predictor, like gender or location, and different regressions occur within each category. The theory of regression graphics based on central subspaces can be used to construct graphical solutions to long-standing problems of this type. It is argued that in practice the central subspace automatically expands to incorporate outliers and regression mixtures. Thus methods of estimating the central subspace can be used to identify these phenomena, without specifying a model. Examples illustrating the power of the theory are presented.

An elementary account of Amari's expected geometry

Differential geometry has found fruitful application in statistical inference. In particular, Amari’s (1990) expected geometry is used in higher order asymptotic analysis, and in the study of sufficiency and ancillarity. However, we can see three drawbacks to the use of a differential geometric approach in econometrics and statistics more generally. Firstly, the mathematics is unfamiliar and the terms involved can be difficult for the econometrician to fully appreciate. Secondly, their statistical meaning can be less than completely clear, and finally the fact that, at its core, geometry is a visual subject can be obscured by the mathematical formalism required for a rigorous analysis, thereby hindering intuition. All three drawbacks apply particularly to the differential geometric concept of a non metric affine connection. The primary objective of this paper is to attempt to mitigate these drawbacks in the case of Amari’s expected geometric structure on a full exponential family. We aim to do this by providing an elementary account of this structure which is clearly based statistically, accessible geometrically and visually presented.

On the Differential Geometry of the Wald Test with Nonlinear Restrictions

In this paper we exploit the tools of differential geometry to provide a clear explanation for the finite sample lack of invariance of the Wald statistic to algebraically equivalent reformulations of the null hypothesis

Influence analysis based on the case sensitivity function

The case sensitivity function approach to influence analysis is introduced as a natural smooth extension of influence curve methodology in which both the insights of geometry and the power of (convex) analysis are available. In it, perturbation is defined as movement between probability vectors defining weighted empirical distributions. A Euclidean geometry is proposed giving such perturbations both size and direction. The notion of the salience of a perturbation is emphasized. This approach has several benefits. A general probability case weight analysis results. Answers to a number of outstanding questions follow directly. Rescaled versions of the three usual finite sample influence curve measures-seen now to be required for comparability across different-sized subsets of cases-are readily available. These new diagnostics directly measure the salience of the (infinitesimal) perturbations involved. Their essential unity, both within and between subsets, is evident geometrically. Finally it is shown how a relaxation strategy, in which a high dimensional (0(nCm)) discrete problem is replaced by a low dimensional (O(n)) continuous problem, can combine with (convex) optimization results to deliver better performance in challenging multiple-case influence problems. Further developments are briefly indicated.

A relaxed approach to combinatorial problems in robustness and diagnostics

A range of procedures in both robustness and diagnostics require optimisation of a target functional over all subsamples of given size. Whereas such combinatorial problems are extremely difficult to solve exactly, something less than the global optimum can be ‘good enough’ for many practical purposes, as shown by example. Again, a relaxation strategy embeds these discrete, high-dimensional problems in continuous, low-dimensional ones. Overall, nonlinear optimisation methods can be exploited to provide a single, reasonably fast algorithm to handle a wide variety of problems of this kind, thereby providing a certain unity. Four running examples illustrate the approach. On the robustness side, algorithmic approximations to minimum covariance determinant (MCD) and least trimmed squares (LTS) estimation. And, on the diagnostic side, detection of multiple multivariate outliers and global diagnostic use of the likelihood displacement function. This last is developed here as a global complement to Cook’s (in J. R. Stat. Soc. 48:133–169, 1986) local analysis. Appropriate convergence of each branch of the algorithm is guaranteed for any target functional whose relaxed form is—in a natural generalisation of concavity, introduced here—‘gravitational’. Again, its descent strategy can downweight to zero contaminating cases in the starting position. A simulation study shows that, although not optimised for the LTS problem, our general algorithm holds its own with algorithms that are so optimised. An adapted algorithm relaxes the gravitational condition itself.

Multiple deletion measures and conditional influence in regression model

The joint effect of the deletion of the ith and jih cases is given by Gray and Ling (1984), they discussed the influence measures for influential subsets in linear regression analysis. The present paper is concerned with multiple sets of deletion measures in the linear regression model. In particular we are interested in the effects of the jointly and conditional influence analysis for the detection of two influential subsets.

Computing with Fisher geodesics and extended exponential families

Recent progress using geometry in the design of efficient Markov chain Monte Carlo (MCMC) algorithms have shown the effectiveness of the Fisher Riemannian structure. Furthermore, the theory of the underlying geometry of spaces of statistical models has made an important breakthrough by extending the classical theory on exponential families to their closures, the so-called extended exponential families. This paper looks at the underlying geometry of the Fisher information, in particular its limiting behaviour near boundaries, which illuminates the excellent behaviour of the corresponding geometric MCMC algorithms. Further, the paper shows how Fisher geodesics in extended exponential families smoothly attach the boundaries of extended exponential families to their relative interior. We conjecture that this behaviour could be exploited for trans-dimensional MCMC algorithms.

Computational information geometry in statistics: foundations

This paper lays the foundations for a new framework for numerically and computationally applying information geometric methods to statistical modelling.

On preferred point geometry in statistics

A brief synopsis of progress in differential geometry in statistics is followed by a note of some points of tension in the developing relationship between these disciplines. The preferred point nature of much of statistics is described and suggests the adoption of a corresponding geometry which reduces these tensions. Applications of preferred point geometry in statistics are then reviewed. These include extensions of statistical manifolds, a statistical interpretation of duality in Amaris expected geometry, and removal of the apparent incompatibility between (Kullback–Leibler) divergence and geodesic distance. Equivalences between a number of new expected preferred point geometries are established and a new characterisation of total flatness shown. A preferred point geometry of influence analysis is briefly indicated. Technical details are kept to a minimum throughout to improve accessibility.

Influence functions of two families of robust estimators under proportional scatter matrices

In this paper, under a proportional model, two families of robust estimates for the proportionality constants, the common principal axes and their size are discussed. The first approach is obtained by plugging robust scatter matrices on the maximum likelihood equations for normal data. A projection- pursuit and a modified projection-pursuit approach, adapted to the proportional setting, are also considered. For all families of estimates, partial influence functions are obtained and asymptotic variances are derived from them. The performance of the estimates is compared through a Monte Carlo study.