Victor J. Yohai

ROBUST REGRESSION BY MEANS OF S-ESTIMATORS

There are at least two reasons why robust regression techniques are useful tools in robust time series analysis. First of all, one often wants to estimate autoregressive parameters in a robust way, and secondly, one sometimes has to fit a linear or nonlinear trend to a time series. In this paper we shall develop a class of methods for robust regression, and briefly comment on their use in time series. These new estimators are introduced because of their invulnerability to large fractions of contaminated data. We propose to call them “S-estimators” because they are based on estimators of scale.

High Breakdown-Point Estimates of Regression by Means of the Minimization of an Efficient Scale.

Abstract A new class of robust estimates, τ estimates, is introduced. The estimates have simultaneously the following properties: (a) they are qualitatively robust, (b) their breakdown point is .5, and (c) they are highly efficient for regression models with normal errors. They are defined by minimizing a new scale estimate, τ, applied to the residuals. Asymptotically, a τ estimate is equivalent to an M estimate with a ψ function given by a weighted average of two ψ functions, one corresponding to a very robust estimate and the other to a highly efficient estimate. The weights are adaptive and depend on the underlying error distribution. We prove consistency and asymptotic normality and give a convergent iterative computing algorithm. Finally, we compare the biases produced by gross error contamination in the τ estimates and optimal bounded-influence estimates.

Robust Estimates for ARMA Models

Abstract Two new classes of robust estimates for ARMA models are introduced: estimates based on residual autocovariances (RA estimates), and estimates based on truncated residual autocovariances (TRA estimates). A heuristic derivation of the asymptotic normal distribution is given. We also perform a Monte Carlo study to compare the robustness properties of these estimates with the least squares, M, and GM estimates. In this study we consider observations that correspond to a Gaussian model with additive outliers. The Monte Carlo results show that RA and TRA estimates compare favorably with respect to least squares, M, and GM estimates.

A Fast Algorithm for S-Regression Estimates

Equivariant high-breakdown point regression estimates are computationally expensive, and the corresponding algorithms become unfeasible for moderately large number of regressors. One important advance to improve the computational speed of one such estimator is the fast-LTS algorithm. This article proposes an analogous algorithm for computing S-estimates. The new algorithm, that we call “fast-S”, is also based on a “local improvement” step of the resampling initial candidates. This allows for a substantial reduction of the number of candidates required to obtain a good approximation to the optimal solution. We performed a simulation study which shows that S-estimators computed with the fast-S algorithm compare favorably to the LTS-estimators computed with the fast-LTS algorithm.

A Procedure for Robust Estimation and Inference in Linear Regression

Even if robust regression estimators have been around for nearly 20 years, they have not found widespread application. One obstacle is the diversity of estimator types and the necessary choices of tuning constants, combined with a lack of guidance for these decisions. While some participants of the IMA summer program have argued that these choices should always be made in view of the specific problem at hand, we propose a procedure which should fit many purposes reasonably well. A second obstacle is the lack of simple procedures for inference, or the reluctance to use the straightforward inference based on asymptotics.

Asymptotic behavior of general M-estimates for regression and scale with random carriers

SummaryLet (xini, yibe a sequence of independent identically distributed random variables, where xi∃Rpand yi∃R, and let θ∃Rpbe an unknown vector such that yi=x′iθ+ui(*), where uiis independent of xiand has distribution function F(u/σ), where σ>0 is an unknown parameter. This paper deals with a general class of M-estimates of regression and scale, (θ*,σ*), defined as solutions of the system:

Robust regression with both continuous and categorical predictors

\sum\limits_i \phi ({\text{x}}_i ,r_i )x_i = 0,\sum\limits_i \chi (|r_i |) = 0,

Optimal locally robust M-estimates of regression

, where r= (yi−xi1θ*/σ)*, with Φ∶ Rp×R→R and χ∶ R→R. This class contains estimators of (θ, σ) proposed by Huber, Mallows and Krasker and Welsch. The consistency and asymptotic normality of the general M-estimators are proved assuming general regularity conditions on Φ and χ and assuming the joint distribution of (xi, yi) to fulfill the model (*) only approximately.