Jouni Hartikainen

Kalman filtering and smoothing solutions to temporal Gaussian process regression models

In this paper, we show how temporal (i.e., time-series) Gaussian process regression models in machine learning can be reformulated as linear-Gaussian state space models, which can be solved exactly with classical Kalman filtering theory. The result is an efficient non-parametric learning algorithm, whose computational complexity grows linearly with respect to number of observations. We show how the reformulation can be done for Matérn family of covariance functions analytically and for squared exponential covariance function by applying spectral Taylor series approximation. Advantages of the proposed approach are illustrated with two numerical experiments.

On Gaussian Optimal Smoothing of Non-Linear State Space Models

In this note we shall present a new Gaussian approximation based framework for approximate optimal smoothing of non-linear stochastic state space models. The approximation framework can be used for efficiently solving non-linear fixed-interval, fixed-point and fixed-lag optimal smoothing problems. We shall also numerically compare accuracies of approximations, which are based on Taylor series expansion, unscented transformation, central differences and Gauss-Hermite quadrature.

Spatiotemporal Learning via Infinite-Dimensional Bayesian Filtering and Smoothing: A Look at Gaussian Process Regression Through Kalman Filtering

Gaussian process-based machine learning is a powerful Bayesian paradigm for nonparametric nonlinear regression and classification. In this article, we discuss connections of Gaussian process regression with Kalman filtering and present methods for converting spatiotemporal Gaussian process regression problems into infinite-dimensional state-space models. This formulation allows for use of computationally efficient infinite-dimensional Kalman filtering and smoothing methods, or more general Bayesian filtering and smoothing methods, which reduces the problematic cubic complexity of Gaussian process regression in the number of time steps into linear time complexity. The implication of this is that the use of machine-learning models in signal processing becomes computationally feasible, and it opens the possibility to combine machine-learning techniques with signal processing methods.

Recursive outlier-robust filtering and smoothing for nonlinear systems using the multivariate student-t distribution

Nonlinear Kalman filter and Rauch-Tung-Striebel smoother type recursive estimators for nonlinear discrete-time state space models with multivariate Students t-distributed measurement noise are presented. The methods approximate the posterior state at each time step using the variational Bayes method. The nonlinearities in the dynamic and measurement models are handled using the nonlinear Gaussian filtering and smoothing approach, which encompasses many known nonlinear Kalman-type filters. The method is compared to alternative methods in a computer simulation.

Non-linear noise adaptive Kalman filtering via variational Bayes

We consider joint estimation of state and time-varying noise covariance matrices in non-linear stochastic state space models. We propose a variational Bayes and Gaussian non-linear filtering based algorithm for efficient computation of the approximate filtering posterior distributions. The formulation allows the use of efficient Gaussian integration methods such as unscented transform, cubature integration and Gauss-Hermite integration along with the classical Taylor series approximations. The performance of the algorithm is illustrated in a simulated application.

Sparse Spatio-temporal Gaussian processes with general likelihoods

In this paper, we consider learning of spatio-temporal processes by formulating a Gaussian process model as a solution to an evolution type stochastic partial differential equation. Our approach is based on converting the stochastic infinite-dimensional differential equation into a finite dimensional linear time invariant (LTI) stochastic differential equation (SDE) by discretizing the process spatially. The LTI SDE is time-discretized analytically, resulting in a state space model with linear-Gaussian dynamics. We use expectation propagation to perform approximate inference on non-Gaussian data, and show how to incorporate sparse approximations to further reduce the computational complexity. We briefly illustrate the proposed methodology with a simulation study and with a real world modelling problem.

Posterior inference on parameters of stochastic differential equations via non-linear Gaussian filtering and adaptive MCMC

This article is concerned with Bayesian estimation of parameters in non-linear multivariate stochastic differential equation (SDE) models occurring, for example, in physics, engineering, and financial applications. In particular, we study the use of adaptive Markov chain Monte Carlo (AMCMC) based numerical integration methods with non-linear Kalman-type approximate Gaussian filters for parameter estimation in non-linear SDEs. We study the accuracy and computational efficiency of gradient-free sigma-point approximations (Gaussian quadratures) in the context of parameter estimation, and compare them with Taylor series and particle MCMC approximations. The results indicate that the sigma-point based Gaussian approximations lead to better approximations of the parameter posterior distribution than the Taylor series, and the accuracy of the approximations is comparable to that of the computationally significantly heavier particle MCMC approximations.

Sigma point methods in optimal smoothing of non-linear stochastic state space models

In this article, we shall show how the sigma-point based approximations that have previously been used in optimal filtering can also be used in optimal smoothing. In particular, we shall consider unscented transformation, Gauss-Hermite quadrature and central differences based optimal smoothers. We briefly present the smoother equations and compare performance of different methods in simulated scenarios.

Correction to “On Gaussian Optimal Smoothing of Nonlinear State Space Models” [Aug 10 1938-1941]

In the above titled paper (ibid., vol. 55, no. 8, pp. 1938-1941, Aug. 10), some of the error values reported in Table I were incorrect. The corrected errors are reported in the revised Table I presented here.