Jeffrey S. Racine

Nonparametric estimation of regression functions with both categorical and continuous data

Abstract In this paper we propose a method for nonparametric regression which admits continuous and categorical data in a natural manner using the method of kernels. A data-driven method of bandwidth selection is proposed, and we establish the asymptotic normality of the estimator. We also establish the rate of convergence of the cross-validated smoothing parameters to their benchmark optimal smoothing parameters. Simulations suggest that the new estimator performs much better than the conventional nonparametric estimator in the presence of mixed data. An empirical application to a widely used and publicly available dynamic panel of patent data demonstrates that the out-of-sample squared prediction error of our proposed estimator is only 14–20% of that obtained by some popular parametric approaches which have been used to model this data set.

Cross-Validation and the Estimation of Conditional Probability Densities

Many practical problems, especially some connected with forecasting, require nonparametric estimation of conditional densities from mixed data. For example, given an explanatory data vector X for a prospective customer, with components that could include the customers salary, occupation, age, sex, marital status, and address, a company might wish to estimate the density of the expenditure, Y, that could be made by that person, basing the inference on observations of (X, Y) for previous clients. Choosing appropriate smoothing parameters for this problem can be tricky, not in the least because plug-in rules take a particularly complex form in the case of mixed data. An obvious difficulty is that there exists no general formula for the optimal smoothing parameters. More insidiously, and more seriously, it can be difficult to determine which components of X are relevant to the problem of conditional inference. For example, if the jth component of X is independent of Y, then that component is irrelevant to estimating the density of Y given X, and ideally should be dropped before conducting inference. In this article we show that cross-validation overcomes these difficulties. It automatically determines which components are relevant and which are not, through assigning large smoothing parameters to the latter and consequently shrinking them toward the uniform distribution on the respective marginals. This effectively removes irrelevant components from contention, by suppressing their contribution to estimator variance; they already have very small bias, a consequence of their independence of Y. Cross-validation also yields important information about which components are relevant; the relevant components are precisely those that cross-validation has chosen to smooth in a traditional way, by assigning them smoothing parameters of conventional size. Indeed, cross-validation produces asymptotically optimal smoothing for relevant components, while eliminating irrelevant components by oversmoothing. In the problem of nonparametric estimation of a conditional density, cross-validation comes into its own as a method with no obvious peers.

Entropy and predictability of stock market returns

Abstract We examine the predictability of stock market returns by employing a new metric entropy measure of dependence with several desirable properties. We compare our results with a number of traditional measures. The metric entropy is capable of detecting nonlinear dependence within the returns series, and is also capable of detecting nonlinear “affinity” between the returns and their predictions obtained from various models thereby serving as a measure of out-of-sample goodness-of-fit or model adequacy. Several models are investigated, including the linear and neural-network models as well as nonparametric and recursive unconditional mean models. We find significant evidence of small nonlinear unconditional serial dependence within the returns series, but fragile evidence of superior conditional predictability (profit opportunity) when using market-switching versus buy-and-hold strategies.

A Consistent Model Specification Test with Mixed Discrete and Continuous Data

In this paper we propose a nonparametric kernel-based model specification test that can be used when the regression model contains both discrete and continuous regressors. We employ discrete variable kernel functions and we smooth both the discrete and continuous regressors using least squares cross-validation (CV) methods. The test statistic is shown to have an asymptotic normal null distribution. We also prove the validity of using the wild bootstrap method to approximate the null distribution of the test statistic, the bootstrap being our preferred method for obtaining the null distribution in practice. Simulations show that the proposed test has significant power advantages over conventional kernel tests which rely upon frequency-based nonparametric estimators that require sample splitting to handle the presence of discrete regressors.

Nonparametric Estimation of Conditional CDF and Quantile Functions With Mixed Categorical and Continuous Data

We propose a new nonparametric conditional cumulative distribution function kernel estimator that admits a mix of discrete and categorical data along with an associated nonparametric conditional quantile estimator. Bandwidth selection for kernel quantile regression remains an open topic of research. We employ a conditional probability density function-based bandwidth selector proposed by Hall, Racine, and Li that can automatically remove irrelevant variables and has impressive performance in this setting. We provide theoretical underpinnings including rates of convergence and limiting distributions. Simulations demonstrate that this approach performs quite well relative to its peers; two illustrative examples serve to underscore its value in applied settings.

Nonparametric Estimation of Regression Functions in the Presence of Irrelevant Regressors

In this paper we consider a nonparametric regression model that admits a mix of continuous and discrete regressors, some of which may in fact be redundant (that is, irrelevant). We show that, asymptotically, a data-driven least squares cross-validation method can remove irrelevant regressors. Simulations reveal that this automatic dimensionality reduction feature is very effective in finite-sample settings.

Nonparametric Econometrics: A Primer

This review is a primer for those who wish to familiarize themselves with nonparametric econometrics. Though the underlying theory for many of these methods can be daunting for some practitioners, this article will demonstrate how a range of nonparametric methods can in fact be deployed in a fairly straightforward manner. Rather than aiming for encyclopedic coverage of the field, we shall restrict attention to a set of touchstone topics while making liberal use of examples for illustrative purposes. We will emphasize settings in which the user may wish to model a dataset comprised of continuous, discrete, or categorical data (nominal or ordinal), or any combination thereof. We shall also consider recent developments in which some of the variables involved may in fact be irrelevant, which alters the behavior of the estimators and optimal bandwidths in a manner that deviates substantially from conventional approaches.

Consistent Significance Testing for Nonparametric Regression

This article presents a framework for individual and joint tests of significance employing nonparametric estimation procedures. The proposed test is based on nonparametric estimates of partial derivatives, is robust to functional misspecification for general classes of models, and employs nested pivotal bootstrapping procedures. Two simulations and one application are considered to examine size and power relative to misspecified parametric models, and to test for the linear unpredictability of exchange-rate movements for G7 currencies.

Consistent cross-validatory model-selection for dependent data: hv-block cross-validation

This paper considers the impact of Shaos (1993) recent results regarding the asymptotic inconsistency of model selection via leave-one-out cross-validation on h-block cross-validation, a cross-validatory method for dependent data proposed by Burman, Chow and Nolan (1994, Journal of Time Series Analysis 13, 189–207). It is shown that h-block cross-validation is inconsistent in the sense of Shao (1993, Journal of American Statistical Association 88(422), 486–495) and therefore is not asymptotically optimal. A modification of the h-block method, dubbed ‘hv-block’ cross-validation, is proposed which is asymptotically optimal. The proposed approach is consistent for general stationary observations in the sense that the probability of selecting the model with the best predictive ability converges to 1 as the total number of observations approaches infinity. This extends existing results and yields a new approach which contains leave-one-out cross-validation, leave-nv-out cross-validation, and h-block cross-validation as special cases. Applications are considered.

Semiparametric ARX neural-network models with an application to forecasting inflation

We examine semiparametric nonlinear autoregressive models with exogenous variables (NLARX) via three classes of artificial neural networks: the first one uses smooth sigmoid activation functions; the second one uses radial basis activation functions; and the third one uses ridgelet activation functions. We provide root mean squared error convergence rates for these ANN estimators of the conditional mean and median functions with stationary beta-mixing data. As an empirical application, we compare the forecasting performance of linear and semiparametric NLARX models of US inflation. We find that all of our semiparametric models outperform a benchmark linear model based on various forecast performance measures. In addition, a semiparametric ridgelet NLARX model which includes various lags of historical inflation and the GDP gap is best in terms of both forecast mean squared error and forecast mean absolute deviation error.