Teófilo Valdés

Regularization in statistics

This paper is a selective review of the regularization methods scattered in statistics literature. We introduce a general conceptual approach to regularization and fit most existing methods into it. We have tried to focus on the importance of regularization when dealing with todays high-dimensional objects: data and models. A wide range of examples are discussed, including nonparametric regression, boosting, covariance matrix estimation, principal component estimation, subsampling.

Modal iterative estimation in linear models with unimodal errors and non-grouped and grouped data collected from different sources

In this paper we introduce an iterative estimation procedure based on conditional modes suitable to fit linear models when errors are known to be unimodal and, moreover, the dependent data stem from different sources and, consequently, may be either non-grouped or grouped with different classification criteria. The procedure requires, at each step, the imputation of the exact values of the grouped data and runs by means of a process that is similar to the EM algorithm with normal errors. The expectation step has been substituted with a mode step that avoids awkward integration with general errors and, in addition, we have substituted the maximisation step with a natural one which only coincides with it when the error distribution is normal. Notwithstanding the former modifications, we have proved that, on the one hand, the iterative estimating algorithm converges to a point which is unique and non-dependent on the starting values and, on the other hand, our final estimate, being anM-estimator, may enjoy good stochastic asymptotic properties such as consistency, boundness inL2, and limit normality.

An algorithm for robust linear estimation with grouped data

An algorithm which is valid to estimate the parameters of linear models under several robust conditions is presented. With respect to the robust conditions, firstly, the dependent variables may be either non-grouped or grouped. Secondly, the distribution of the errors may vary within the wide class of the strongly unimodal distributions, either symmetrical or non-symmetrical. Finally, the variance of the errors is unknown. Under these circumstances the algorithm is not only capable of estimating the parameters (slopes and error variance) of the linear model, but also the asymptotic covariance matrix of the linear parameters. This opens the possibility of making inferences in terms of either multiple confidence regions or hypothesis testing.

An iterative estimating procedure for probit-type nonresponse models in surveys with call backs

This work attempts to treat the negatives to respond in sample plans when several tries or call backs in the capture of individual data are assumed. We also maintain the assumption that the respondents supply all the variables of interest when they are captured although the retries are kept on, even after previous captures, for a predetermined number of tries,r, fixed only for estimating purposes. Supposing that the different retries or call backs are exerted with different capture intensities, the response probabilities, which may vary from one individual to another, are searched by probit models whose parameters are estimated using conditional likelihoods evaluated on the respondents only (other models, derived from error distributions different from normal, could also be possible by approximating numerical techniques quite similar to the ones proposed here). We present a numerical estimating algorithm, quite easy to implement, which may be used when the recorded information about data captures includes at least marginal results. Finally, we include some encouraging empirical simulations whose purpose is centred in testing and evaluating the practical performance of the procedure.

An algorithm based on discrete response regression models suitable to correct the bias of non-response in surveys with several capture tries

The use of survey plans, which contemplate several tries or call-backs when endeavouring to capture individual data, may supply unarguable information in certain sampling situations with non-ignorable non-response. This paper presents an algorithm whose final aim is the estimation of the individual non-response probabilities from a general perspective of discrete response regression models, which includes the well known probit and logit models. It will be assumed that the respondents supply all the variables of interest when they are captured. Nevertheless, the call-backs continue, even after previous captures, for a small number of tries, r, which has been fixed beforehand only for estimating purposes. The different retries or call-backs are supposed to be carried out with different capture intensities. As mentioned above, the response probabilities, which may vary from one individual to another, are sought by discrete response regression models, whose parameters are estimated from conditioned likelihoods evaluated on the respondents only. The algorithm, quick and easy to implement, may be used even when the capture indicator matrix has been partially recorded. Finally, the practical performance of the proposed procedure is tested and evaluated from empirical simulations whose results are undoubtedly encouraging.

Robust analysis of variance with imprecise data: an ad hoc algorithm

We present an easy to implement algorithm, which is valid to analyse the variance of data under several robust conditions. Firstly, the observations may be precise or imprecise. Secondly, the error distributions may vary within the wide class of the strongly unimodal distributions, symmetrical or not. Thirdly, the variance of the errors is unknown. The algorithm starts by estimating the parameters of the ANOVA linear model. Then, the asymptotic covariance matrix of the effects is estimated. Finally, the algorithm uses this matrix estimate to test ANOVA hypotheses posed in terms of linear combinations of the effects.

Recursive estimation in linear models with general errors and grouped data: a median-based procedure and related asymptotics

We introduce in this paper an iterative estimation procedure based on conditional medians valid to fit linear models when, on the one hand, the distribution of errors, assumed to be known, may be general and, on the other, the dependent data stem from different sources and, consequently, may be either non-grouped or grouped with different classification criteria. The procedure requires us at each step to interpolate the grouped data and is similar to the EM algorithm with normal errors. The expectation step has been replaced by a median-based step which avoids doing awkward integration with general errors and, also, we have substituted for the maximisation step, a natural one which only coincides with it when the errors are normally distributed. With these modifications, we have proved that the iterative estimating algorithm converges to a point which is unique and non-dependent on the starting values. Finally, our final estimate, being a Huber type M-estimator, may enjoy good stochastic asymptotic properties which have also been investigated in detail.

On Spectral Radii of Matrices Governing some Numerical Estimating Algorithms with Incomplete Data

SummaryMany estimating procedures are carried out with incomplete data by means of different types of EM algorithms. They allow us to obtain maximum likelihood parameter estimates in classical inference and also estimates based on the posterior mode in Bayesian inference. This paper analyzes in detail the spectral radii of the Jacobian matrices algorithm as a possible way to evaluate convergence rates. The eigenvalues of such matrices are explicitly obtained in some cases and, in all of them, a geometric convergence rate is, at least, guaranteed near the optimum. Finally, a comparison between the leading eigenvalues of EM and direct and approximate EM-Bayes algorithms may suggest the efficiency of each case.