Lucien Birgé

From Model Selection to Adaptive Estimation

Many different model selection information criteria can be found in the literature in various contexts including regression and density estimation. There is a huge amount of literature concerning this subject and we shall, in this paper, content ourselves to cite only a few typical references in order to illustrate our presentation. Let us just mention AIC, C p , or C L , BIC and MDL criteria proposed by Akaike (1973), Mallows (1973), Schwarz (1978), and Rissanen (1978) respectively. These methods propose to select among a given collection of parametric models that model which minimizes an empirical loss (typically squared error or minus log-likelihood) plus some penalty term which is proportional to the dimension of the model. From one criterion to another the penalty functions differ by factors of log n, where n represents the number of observations.

Approximation dans les espaces métriques et théorie de l'estimation

SummaryWe investigate the relations between the speed of estimation and the metric structure of the parameter space Θ, especially in the case when its metric dimension is infinite. Given some distance d on Θ (generally Hellinger distance in the case of n i.i.d. variables), we consider the minimax risk for n observations: % MathType!MTEF!2!1!+-% feaafiart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr% 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9qq-Jar% pepeea0xd9q8as0-LqLs-Jirpepeea0-as0Fb9pgea0db9fr-xfr-x% frpeWZqaaeaabiGaciaacaqabeaadaqaaqaaaOqaaiaadkfadaWgaa% WcbaGaamOBaaqabaGccaGGOaGaamyCaiaacMcacqGH9aqpdaWfqaqa% aiGacMgacaGGUbGaaiOzaGqaciaa-bcaciGGZbGaaiyDaiaacchaaS% qaaiaadsfadaWgaaadbaGaamOBaiaa-bcaaeqaaSGaeqiUdeNaeyic% I4SaeuiMdefabeaakiaabMeacaqGfbWaaSbaaSqaaiabeI7aXbqaba% GccaGGBbGaamizamaaCaaaleqabaGaamyCaaaakiaacIcacqaH4oqC% caGGSaGaa8hiaiaadsfadaWgaaWcbaGaamOBaaqabaGccaGGPaGaai% yxaiaacYcaaaa!5A70!

Thresholding algorithms, maxisets and well-concentrated bases

R_n (q) = \mathop {\inf \sup }\limits_{T_{n } \theta \in \Theta } {\text{IE}}_\theta [d^q (\theta , T_n )],

Robust Testing for Independent Non Identically Distributed Variables and Markov Chains

, Tn being any estimate of θ. We shall look for functions r such that for positive constants C1(q) and C2(q) C1rq(n)≦Rn(q)≦C2rq(n). r(n) is the speed of estimation and we shall show under fairly general conditions (including i.i.d. variables and regular cases of Markov chains and stationnary gaussian processes) that r(n) is determined, up to multiplicative constants, by the metric structure of Θ. We shall also give a construction for some sort of “universal” estimates the risk of which is bounded by C2rq(n) in all cases where the preceding theory applies.

A new method for estimation and model selection:\rho -estimation

SummaryWe shall present here a general study of minimum contrast estimators in a nonparametric setting (although our results are also valid in the classical parametric case) for independent observations. These estimators include many of the most popular estimators in various situations such as maximum likelihood estimators, least squares and other estimators of the regression function, estimators for mixture models or deconvolution... The main theorem relates the rate of convergence of those estimators to the entropy structure of the space of parameters. Optimal rates depending on entropy conditions are already known, at least for some of the models involved, and they agree with what we get for minimum contrast estimators as long as the entropy counts are not too large. But, under some circumstances (“large” entropies or changes in the entropy structure due to local perturbations), the resulting the rates are only suboptimal. Counterexamples are constructed which show that the phenomenon is real for non-parametric maximum likelihood or regression. This proves that, under purely metric assumptions, our theorem is optimal and that minimum contrast estimators happen to be suboptimal.

Estimating composite functions by model selection

The aim of this paper is to synthetically analyse the performances of thresholding and wavelet estimation methods. In this connection, it is useful to describe the maximal sets where these methods attain a special rate of convergence. We relate these “maxisets” to other problems naturally arising in the context of non parametric estimation, as approximation theory or information reduction. A second part of the paper is devoted to isolate two very special properties especially shared by wavelet bases, which allow them to behave almost as in an Hilbertian context even for Lp risks.

Statistical estimation with model selection

Our purpose in this paper is to apply the general methodology for model selection based on T-estimators developed in Birge (Model selection via testing : an alternative to (penalized) maximum likelihood estimators, Ann. Inst. Henri Poincare Probab. et Statist., vol. 42, 273-325, 2006) to the particular situation of the estimation of the unknown mean measure of a Poisson process. We introduce a Hellinger type distance between finite positive measures to serve as our loss function and we build suitable tests between balls (with respect to this distance) in the set of mean measures. As a consequence of the existence of such tests, given a suitable family of approximating models, we can build T-estimators for the mean measure based on this family of models and analyze their performances. We provide a number of applications to adaptive intensity estimation when the square root of the intensity belongs to various smoothness classes. We also give a method for aggregation of preliminary estimators.

Risk bounds for model selection via penalization

In previous papers, it was shown that there exists good tests between two Hellinger balls P = B(P0, r) and Q = B(Q0, r), given n independent observations. We investigate here the performance of those tests when the true laws of the observations do not belong to those balls; this leads to some robustness result for those tests. The natural extension to non i.i.d. random variables is to define a distance H between product measures by: