Carlos Rivero

Regularization in statistics

This paper is a selective review of the regularization methods scattered in statistics literature. We introduce a general conceptual approach to regularization and fit most existing methods into it. We have tried to focus on the importance of regularization when dealing with todays high-dimensional objects: data and models. A wide range of examples are discussed, including nonparametric regression, boosting, covariance matrix estimation, principal component estimation, subsampling.

Modal iterative estimation in linear models with unimodal errors and non-grouped and grouped data collected from different sources

In this paper we introduce an iterative estimation procedure based on conditional modes suitable to fit linear models when errors are known to be unimodal and, moreover, the dependent data stem from different sources and, consequently, may be either non-grouped or grouped with different classification criteria. The procedure requires, at each step, the imputation of the exact values of the grouped data and runs by means of a process that is similar to the EM algorithm with normal errors. The expectation step has been substituted with a mode step that avoids awkward integration with general errors and, in addition, we have substituted the maximisation step with a natural one which only coincides with it when the error distribution is normal. Notwithstanding the former modifications, we have proved that, on the one hand, the iterative estimating algorithm converges to a point which is unique and non-dependent on the starting values and, on the other hand, our final estimate, being anM-estimator, may enjoy good stochastic asymptotic properties such as consistency, boundness inL2, and limit normality.

An algorithm for robust linear estimation with grouped data

An algorithm which is valid to estimate the parameters of linear models under several robust conditions is presented. With respect to the robust conditions, firstly, the dependent variables may be either non-grouped or grouped. Secondly, the distribution of the errors may vary within the wide class of the strongly unimodal distributions, either symmetrical or non-symmetrical. Finally, the variance of the errors is unknown. Under these circumstances the algorithm is not only capable of estimating the parameters (slopes and error variance) of the linear model, but also the asymptotic covariance matrix of the linear parameters. This opens the possibility of making inferences in terms of either multiple confidence regions or hypothesis testing.

EM-Based Radial Basis Function Training with Partial Information

This work presents an EM approach for nonlinear regression with incomplete data. Radial Basis Function (RBF) Neural Networks are employed since their architecture is appropriate for an efficient parameter estimation. The training algorithm expectation (E) step takes into account the censorship over the data, and the maximization (M) step can be implemented in several ways. The results guarantee the convergence of the algorithm in the GEM (Generalized EM) framework.

An algorithm based on discrete response regression models suitable to correct the bias of non-response in surveys with several capture tries

The use of survey plans, which contemplate several tries or call-backs when endeavouring to capture individual data, may supply unarguable information in certain sampling situations with non-ignorable non-response. This paper presents an algorithm whose final aim is the estimation of the individual non-response probabilities from a general perspective of discrete response regression models, which includes the well known probit and logit models. It will be assumed that the respondents supply all the variables of interest when they are captured. Nevertheless, the call-backs continue, even after previous captures, for a small number of tries, r, which has been fixed beforehand only for estimating purposes. The different retries or call-backs are supposed to be carried out with different capture intensities. As mentioned above, the response probabilities, which may vary from one individual to another, are sought by discrete response regression models, whose parameters are estimated from conditioned likelihoods evaluated on the respondents only. The algorithm, quick and easy to implement, may be used even when the capture indicator matrix has been partially recorded. Finally, the practical performance of the proposed procedure is tested and evaluated from empirical simulations whose results are undoubtedly encouraging.

Robust analysis of variance with imprecise data: an ad hoc algorithm

We present an easy to implement algorithm, which is valid to analyse the variance of data under several robust conditions. Firstly, the observations may be precise or imprecise. Secondly, the error distributions may vary within the wide class of the strongly unimodal distributions, symmetrical or not. Thirdly, the variance of the errors is unknown. The algorithm starts by estimating the parameters of the ANOVA linear model. Then, the asymptotic covariance matrix of the effects is estimated. Finally, the algorithm uses this matrix estimate to test ANOVA hypotheses posed in terms of linear combinations of the effects.

Recursive estimation in linear models with general errors and grouped data: a median-based procedure and related asymptotics

We introduce in this paper an iterative estimation procedure based on conditional medians valid to fit linear models when, on the one hand, the distribution of errors, assumed to be known, may be general and, on the other, the dependent data stem from different sources and, consequently, may be either non-grouped or grouped with different classification criteria. The procedure requires us at each step to interpolate the grouped data and is similar to the EM algorithm with normal errors. The expectation step has been replaced by a median-based step which avoids doing awkward integration with general errors and, also, we have substituted for the maximisation step, a natural one which only coincides with it when the errors are normally distributed. With these modifications, we have proved that the iterative estimating algorithm converges to a point which is unique and non-dependent on the starting values. Finally, our final estimate, being a Huber type M-estimator, may enjoy good stochastic asymptotic properties which have also been investigated in detail.