Jan Ámos Víšek

On the diversity of estimates

The presented examples of regression analysis of some real data sets below confirm that it is possible that various estimators may produce considerably different estimates. Such situation will be called diversity of estimates. Employing high breakdown point estimators, namely the least median of squares (LMS) and the least trimmed squares (LTS), the diversity of estimates is demonstrated. Two examples with the artificial data and a formal reflection of the diversity of estimates hint why the diversity of estimates takes place. The theoretical reflection also shows that the diversity of estimates may appear for any sample size. A proposal an how to select from the diverse estimates of model is given and illustrated in examples of real data sets.

Sensitivity analysis of M-estimates

Bahadur representation of the difference of estimators of regression coefficients for the full data set and for the set from which one observation was deleted is given for the M-estimators which are generated by a continuous ψ-function. The representation is invariant with respect to the scale of residuals and it indicates that the bound of the norm of the difference is proportional to the gross error sensitivity. Then for the ψ-function which corresponds to the median it is shown that the difference of the estimates for the full data and for data without one observation, although being bounded in probability, can be much larger than indicated by the gross error sensitivity.

Sensitivity analysis of M-estimates of nonlinear regression model: Influence of data subsets

Asymptotic representations of the difference of M-estimators of the parameters of nonlinear regression model for the full data and for the subsample of data are given for the following three situation: i) fix number of points excluded from data, ii) increasing number, however asymptotically negligible part of data excluded, and finally iii) asymptotically fix portion of data excluded. Asymptotic normality of the difference of estimators (for the two latter cases) is proved.

Least Trimmed Squares

Least trimmed squares (LTS) is a statistical technique for estimation of unknown parameters of a linear regression model and provides a “robust” alternative to the classical regression method based on minimizing the sum of squared residuals.

Empirical distribution function under heteroscedasticity

Neglecting heteroscedasticity of error terms may imply the wrong identification of a regression model (see appendix). Employment of (heteroscedasticity resistent) Whites estimator of covariance matrix of estimates of regression coefficients may lead to the correct decision about the significance of individual explanatory variables under heteroscedasticity. However, Whites estimator of covariance matrix was established for least squares (LS)-regression analysis (in the case when error terms are normally distributed, LS- and maximum likelihood (ML)-analysis coincide and hence then Whites estimate of covariance matrix is available for ML-regression analysis, tool). To establish Whites-type estimate for another estimator of regression coefficients requires Bahadur representation of the estimator in question, under heteroscedasticity of error terms. The derivation of Bahadur representation for other (robust) estimators requires some tools. As the key too proved to be a tight approximation of the empirical distribution function (d.f.) of residuals by the theoretical d.f. of the error terms of the regression model. We need the approximation to be uniform in the argument of d.f. as well as in regression coefficients. The present paper offers this approximation for the situation when the error terms are heteroscedastic.

Robust error-term-scale estimate

Abstract: A scale-equivariant and regression-invariant estimator of the variance of error terms in the linear regression model is proposed and its consistency proved. The estimator is based on (down)weighting the order statistics of the squared residuals which corresponds to the consistent and scaleand regression-equivariant estimator of the regression coefficients. A small numerical study demonstrating the behaviour of the estimator under the various types of contamination is included.

Robust Estimation in Linear Model and its Computational Aspects

Heuristics of robust estimation is briefly explained and a survey of the most typical robust methods of the point estimation both of the location and scale parameters and in the linear model is presented. Basic notions of robust statistics as well as of some particular topics of robust diagnostics are discussed, too. Testing of submodels and the possibilities of adaptive approach in the estimation of parameters in the linear model are also commented.

Sensitivity of the test risk with respect to contamination

In a framework of testing statistical hypotheses the sensitivity of the test error probabilities is studied. The Edgeworth approximations of proposed characteristics of this sensitivity are derived and specialized for the local alternative setting. Results, similar to Steins lemma and Chernoffs theorem, describing the convergence rate of test risk sensitivity characteristic is given, too. An admissible character of contamination is also discussed.

Diagnostics of Regression Model: Test of Goodness of Fit

A test statistic for goodness of fit of the M-estimates of the linear regression model is proposed and its asymptotic behaviour is described. Numerical study on real data is included.

WEIGHTED RISK OF TEST: SENSITIVITY TO CONTAMINATION

Asymptotic behaviour of the weighted risk (weighted mean of the prerisk function) in the setting with a fixed hypothesis and alternative and in the local alternative one is derived. A modified directional derivative of the weighted risk is then studied as a characteristic of the test risk sensitivity with respect to a level of contamination. Asymptotic results for the above mentioned settings are also established. A similarity of the properties of the weighted risk and its modified directional derivative resembling alike heredity in the case of the error probabilities and the influence curve of the size and the power of test is pointed out.