Eric Thierry

Minimum-entropy estimation in semi-parametric models

In regression problems where the density f of the errors is not known, maximum likelihood is unapplicable, and the use of alternative techniques like least squares or robust M-estimation generally implies inefficient estimation of the parameters. The search for adaptive estimators, that is, estimators that remain asymptotically efficient independently of the knowledge of f, has received a lot of attention, see in particular (Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, 1956, pp. 187; Ann. Stat. 3(2) (1975) 267; Ann. Stat. 10 (1982) 647) and the review paper (Econometric Rev. 3(2) (1984) 145). The paper considers a minimum-entropy parametric estimator that minimizes an estimate of the entropy of the distribution of the residuals. A first construction connects the method with the Stone-Bickel approach, where the estimation is decomposed into two steps. Then we consider a direct approach that does not involve any preliminary √n-consistent estimator. Some results are given that illustrate the good performance of minimum-entropy estimation for reasonable sample sizes when compared to standard methods, in particular concerning robustness in the presence of outliers.

Weighted averaging using adaptive estimation of the weights

Abstract Signal averaging is often used in biomedical signal processing. Unfortunately the classical hypothesis of perfect alignment of equal shape signals embedded in stationary and uncorrelated noise is rarely exactly verified. So the optimum improvement may be not achieved, and the estimated mean signal may be quite different from an individual realization. In this work we are interested in the noise power variation. In this case, it is more appropriate to use a weighted averaging instead of a uniform one. The main difficulty is the estimation of the signal to noise ratio for each realization in order to find the optimum weights. We show here that the optimum weights can be estimated adaptively using a constrained minimization algorithm. The constraint ensures that the estimated amplitude of the signal is not altered by the averaging procedure. Three different algorithms to estimate the optimal weights are presented and compared using simulations: (1) a direct method based on Lagrange multiplier; (2) an indirect method based on the penalty method; (3) a method based on the resolution of Kuhn and Tucker equations by means of Newton method. From the simulations, the Kuhn and Tucker method has the best performances and is used in the analysis of real electrocardiograms.

A minimum-entropy estimator for regression problems with unknown distribution of observation errors

We consider a nonlinear regression model, with independent observation errors identically distributed with an unknown probability density function f(⋅). Instead of minimizing the empirical version of the entropy of f(⋅) based on the residuals, which corresponds to maximum likelihood estimation and requires the knowledge of f(⋅), we minimize the entropy of a (symmetrized) kernel estimate fn,h of f(⋅), constructed from the residuals. Two examples are presented to illustrate the finite-sample behavior of this estimator (accuracy, robustness). Some (preliminary) consistency results are given.

A Minimum-Entropy Procedure for Robust Motion Estimation

We focus on motion estimation using a block matching approach and suggest using a minimum-entropy criterion. Many entropy-based estimation procedures exist, such as plug-in estimators based on Parzen windowing. We consider here an alternative that is applicable to data of any dimension and that circumvents the critical issues raised by kernel-based methods. To the best of our knowledge, this criterion has not yet been considered for image processing problems. The inherent robustness property of entropy is expected to provide a robust and efficient estimation of the motion vector of a block of a video sequence. In particular, the minimum-entropy estimator should be robust to occlusions and variations of luminance, for which standard approaches like SSD usually meet their limitations.

Asymptotic performance analysis of cyclic detectors

The paper deals with the analytical evaluation of the asymptotic detection and false-alarm probabilities of multicycle and single-cycle detectors operating in additive white Gaussian noise with unknown and nonrandom spectral level, which are based on the cyclostationarity properties of the signal to be intercepted. The receiver operating characteristics are derived by using the asymptotic complex normality and the covariance expression of the sample average estimator of the cyclic covariance in an observed discrete-time series. A numerical example for interception of a binary phase-shift keying signal is considered.

Nonlinear prediction by kriging, with application to noise cancellation

Abstract A semi-parametric approach based on kriging is suggested for nonlinear prediction. It does not rely on any specific model structure, which makes the approach much more flexible than those based on parametric behavioural models. At the same time, accurate predictions are obtained for extremely short-training sequences. Various examples are presented to illustrate the robustness of the method, with a discussion of the prior choices concerning the parametric part of the model. Application on real data is considered in the context of a noise-cancellation problem in underwater acoustics.

Design of experiments for response diversity

A design method is proposed that sequentially generates observation sites for the construction of a kriging predictive model. The objective of the construction is to allow a precise inversion of the system in the following sense: with any reachable target point T in the output space one wishes to be able to associate an input vector x_T such that the system response at x_T will be close to T (which requires that the model ensures a precise prediction of the response at x_T ). Intuitively, the observation sites should not be spread over the admissible input space, but should rather concentrate in areas such that, when mapped by the system, they cover the reachable output space. Two approaches are proposed that are shown on examples (one with five input and two outputs, derived from a problem in oil industry) to give satisfactory results. They are based on the maximization of a measure of dispersion of the observations in the output space and can cope with the presence of observation errors, a rather typical situation for experimentation with real physical systems.

Performance analysis of a statistical test for presence of cyclostationarity in a noisy observation

The purpose of this paper is to evaluate theoretically in terms of receiver operating characteristics (ROC) the performance of a statistical test (based on the second order statistic) for the presence of cyclostationarity in a noisy observation. In order to obtain false alarm and detection probabilities, we consider the following specific situation: we choose a cycle frequency contained in a binary-phase-shift-keyed (BPSK) signal of interest (SOI) (BPSK signal is considered for its increasing attention in digital communications) and we test the presence of cyclostationarity of the SOI in a noisy observation. Two examples are considered, the SOI is buried either in a stationary Gaussian noise or in an another BPSK signal.

Minimum Entropy Estimation in Semi-Parametric Models: a Candidate for Adaptive Estimation?

In regression problems with errors having an unknown density f, least squares or robust M-estimation is the usual alternative to maximum likelihood, with the loss of asymptotic efficiency as a consequence. The search for efficiency in the absence of knowledge of f (adaptive estimation) has motivated a large amount of work, see in particular (1956), (1975), (1982) and the review paper (1984). The present paper continues the work initiated in (2001a),(b). The estimator is obtained by minimizing (an estimate of) the entropy of the symmetrized residuals. Connections and differences with previous work are indicated. The focus is mainly on the location model but we show how the results can be extended to nonlinear regression problems, in particular when the design consists of replications of a fixed design. Numerical results illustrate that asymptotic efficiency is not necessarily in conflict with robustness.

Entropy minimization for parameter estimation problems with unknown distribution of the output noise

We consider the situation where the parameters /spl theta/ of a linear regression model have to be estimated from observations corrupted by an additive noise with unknown distribution f. Since maximum likelihood estimation cannot be used, we estimate /spl theta/ by minimizing the entropy of a kernel estimate of f, constructed from the residuals. An example of parameter estimation in the presence of interference with random binary signals is presented.