Taeryon Choi

Gaussian process regression analysis for functional data

Introduction Functional Regression Models Gaussian Process Regression Some Data Sets and Associated Statistical Problems Bayesian Nonlinear Regression with Gaussian Process Priors Gaussian Process Prior and Posterior Posterior Consistency Asymptotic Properties of the Gaussian Process Regression Models Inference and Computation for Gaussian Process Regression Model Empirical Bayes Estimates Bayesian Inference and MCMC Numerical Computation Covariance Function and Model Selection Examples of Covariance Functions Selection of Covariance Functions Variable Selection Functional Regression Analysis Linear Functional Regression Model Gaussian Process Functional Regression Model GPFR Model with a Linear Functional Mean Model Mixed-Effects GPFR Models GPFR ANOVA Model Mixture Models and Curve Clustering Mixture GPR Models Mixtures of GPFR Models Curve Clustering Generalized Gaussian Process Regression for Non-Gaussian Functional Data Gaussian Process Binary Regression Model Generalized Gaussian Process Regression Generalized GPFR Model for Batch Data Mixture Models for Multinomial Batch Data Some Other Related Models Multivariate Gaussian Process Regression Model Gaussian Process Latent Variable Models Optimal Dynamic Control Using GPR Model RKHS and Gaussian Process Regression Appendices Bibliography Index Further Reading and Notes appear at the end of each chapter.

Remarks on consistency of posterior distributions

In recent years, the literature in the area of Bayesian asymptotics has been rapidly growing. It is increasingly important to understand the con- cept of posterior consistency and validate specific Bayesian methods, in terms of consistency of posterior distributions. In this paper, we build up some con- ceptual issues in consistency of posterior distributions, and discuss panoramic views of them by comparing various approaches to posterior consistency that have been investigated in the literature. In addition, we provide interesting results on posterior consistency that deal with non-exponential consistency, improper priors and non i.i.d. (independent but not identically distributed) observations. We describe a few examples for illustrative purposes.

A Gaussian process regression approach to a single-index model

We consider a Gaussian process regression (GPR) approach to analysing a single-index model (SIM) from the Bayesian perspective. Specifically, the unknown link function is assumed to be a Gaussian process a priori and a prior on the index vector is considered based on a simple uniform distribution on the unit sphere. The posterior distributions for the unknown parameters are derived, and the posterior inference of the proposed approach is performed via Markov chain Monte Carlo methods based on them. Particularly, in estimating the hyperparameters, different numerical schemes are implemented: fully Bayesian methods and empirical Bayes methods. Numerical illustration of the proposed approach is also made using simulation data as well as well-known real data. The proposed approach broadens the scope of the applicability of the SIM as well as the GPR. In addition, we discuss the theoretical aspect of the proposed method in terms of posterior consistency.

Partial correlation with copula modeling

We propose a new partial correlation approach using Gaussian copula. Our empirical study found that the Gaussian copula partial correlation has the same value as that which is obtained by performing a Pearsons partial correlation. With the proposed method, based on canonical vine and d-vine, we captured direct interactions among eight histone genes.

Penalized Gaussian Process Regression and Classification for High-Dimensional Nonlinear Data

The model based on Gaussian process (GP) prior and a kernel covariance function can be used to fit nonlinear data with multidimensional covariates. It has been used as a flexible nonparametric approach for curve fitting, classification, clustering, and other statistical problems, and has been widely applied to deal with complex nonlinear systems in many different areas particularly in machine learning. However, it is a challenging problem when the model is used for the large-scale data sets and high-dimensional data, for example, for the meat data discussed in this article that have 100 highly correlated covariates. For such data, it suffers from large variance of parameter estimation and high predictive errors, and numerically, it suffers from unstable computation. In this article, penalized likelihood framework will be applied to the model based on GPs. Different penalties will be investigated, and their ability in application given to suit the characteristics of GP models will be discussed. The asymptotic properties will also be discussed with the relevant proofs. Several applications to real biomechanical and bioinformatics data sets will be reported.

Generalized bivariate copulas and their properties

Copulas are useful devices to explain the dependence structure among variables by eliminating the influence of marginals. In this paper, we propose a new class of bivariate copulas to quantify dependency and incorporate it into various iterated copula families. We investigate properties of the new class of bivariate copulas and derive the measure of association, such as Spearmans �, Kendalls � , and the regression function for the new class. We also provide the concept of directional dependence in bivariate regression setting by using copulas.

A note on the Bayes factor in a semiparametric regression model

In this paper, we consider a semiparametric regression model where the unknown regression function is the sum of parametric and nonparametric parts. The parametric part is a finite-dimensional multiple regression function whereas the nonparametric part is represented by an infinite series of orthogonal basis. In this model, we investigate the large sample property of the Bayes factor for testing the parametric null model against the semiparametric alternative model. Under some conditions on the prior and design matrix, we identify the analytic form of the Bayes factor and show that the Bayes factor is consistent, i.e. converges to infinity in probability under the parametric null model, while converges to zero under the semiparametric alternative, as the sample size increases.

Smoothing and Mean–Covariance Estimation of Functional Data with a Bayesian Hierarchical Model

Functional data, with basic observational units being functions (e.g., curves, surfaces) varying over a continuum, are frequently encountered in various applications. While many statistical tools have been developed for functional data analysis, the issue of smoothing all functional observations simultaneously is less studied. Existing methods often focus on smoothing each individual function separately, at the risk of removing important systematic patterns common across functions. We propose a nonparametric Bayesian approach to smooth all functional observations simultaneously and nonparametrically. In the proposed approach, we assume that the functional observations are independent Gaussian processes subject to a common level of measurement errors, enabling the borrowing of strength across all observations. Unlike most Gaussian process regression models that rely on pre-specified structures for the covariance kernel, we adopt a hierarchical framework by assuming a Gaussian process prior for the mean function and an Inverse-Wishart process prior for the covariance function. These prior assumptions induce an automatic mean-covariance estimation in the posterior inference in addition to the simultaneous smoothing of all observations. Such a hierarchical framework is flexible enough to incorporate functional data with different characteristics, including data measured on either common or uncommon grids, and data with either stationary or nonstationary covariance structures. Simulations and real data analysis demonstrate that, in comparison with alternative methods, the proposed Bayesian approach achieves better smoothing accuracy and comparable mean-covariance estimation results. Furthermore, it can successfully retain the systematic patterns in the functional observations that are usually neglected by the existing functional data analyses based on individual-curve smoothing.

A hierarchical Bayesian regression model for the uncertain functional constraint using screened scale mixtures of Gaussian distributions

This paper considers a hierarchical Bayesian analysis of regression models using a class of Gaussian scale mixtures. This class provides a robust alternative to the common use of the Gaussian distribution as a prior distribution in particular for estimating the regression function subject to uncertainty about the constraint. For this purpose, we use a family of rectangular screened multivariate scale mixtures of Gaussian distribution as a prior for the regression function, which is flexible enough to reflect the degrees of uncertainty about the functional constraint. Specifically, we propose a hierarchical Bayesian regression model for the constrained regression function with uncertainty on the basis of three stages of a prior hierarchy with Gaussian scale mixtures, referred to as a hierarchical screened scale mixture of Gaussian regression models (HSMGRM). We describe distributional properties of HSMGRM and an efficient Markov chain Monte Carlo algorithm for posterior inference, and apply the proposed model to real applications with constrained regression models subject to uncertainty.

Convergence of Posterior Distribution in the Mixture of Regressions

Mixture models provide a method of modelling a complex probability distribution in terms of simpler structures. In particular, the method of mixture of regressions has received considerable attention due to its modelling flexibility and availability of convenient computational algorithms. This paper aims to contribute to theoretical justification for the mixtures of regression model from the Bayesian perspective. In particular, we establish consistency of posterior distribution and determine how fast posterior distribution converges to the true value of the parameter in the context of mixture of binary, Poisson, and Gaussian regressions.