S. Sundararajan

Predictive App roaches for Choosing Hyperparameters in Gaussian Processes

Gaussian processes are powerful regression models specified by parameterized mean and covariance functions. Standard approaches to choose these parameters (known by the name hyperparameters) are maximum likelihood and maximum a posteriori. In this article, we propose and investigate predictive approaches based on Geissers predictive sample reuse (PSR) methodology and the related Stones cross-validation (CV) methodology. More specifically, we derive results for Geissers surrogate predictive probability (GPP), Geissers predictive mean square error (GPE), and the standard CV error and make a comparative study. Within an approximation we arrive at the generalized cross-validation (GCV) and establish its relationship with the GPP and GPE approaches. These approaches are tested on a number of problems. Experimental results show that these approaches are strongly competitive with the existing approaches.

Fast generalized cross-validation algorithm for sparse model learning

We propose a fast, incremental algorithm for designing linear regression models. The proposed algorithm generates a sparse model by optimizing multiple smoothing parameters using the generalized cross-validation approach. The performances on synthetic and real-world data sets are compared with other incremental algorithms such as Tipping and Fauls fast relevance vector machine, Chen et al.s orthogonal least squares, and Orrs regularized forward selection. The results demonstrate that the proposed algorithm is competitive.

Predictive approaches for sparse model learning

In this paper we investigate cross validation and Geisser’s sample reuse approaches for designing linear regression models. These approaches generate sparse models by optimizing multiple smoothing parameters. Within certain approximation, we establish equivalence relationships that exist among these approaches. The computational complexity, sparseness and performance on some benchmark data sets are compared with those obtained using relevance vector machine.