Arno Solin

Spatiotemporal Learning via Infinite-Dimensional Bayesian Filtering and Smoothing: A Look at Gaussian Process Regression Through Kalman Filtering

Gaussian process-based machine learning is a powerful Bayesian paradigm for nonparametric nonlinear regression and classification. In this article, we discuss connections of Gaussian process regression with Kalman filtering and present methods for converting spatiotemporal Gaussian process regression problems into infinite-dimensional state-space models. This formulation allows for use of computationally efficient infinite-dimensional Kalman filtering and smoothing methods, or more general Bayesian filtering and smoothing methods, which reduces the problematic cubic complexity of Gaussian process regression in the number of time steps into linear time complexity. The implication of this is that the use of machine-learning models in signal processing becomes computationally feasible, and it opens the possibility to combine machine-learning techniques with signal processing methods.

Dynamic retrospective filtering of physiological noise in BOLD fMRI: DRIFTER

In this article we introduce the DRIFTER algorithm, which is a new model based Bayesian method for retrospective elimination of physiological noise from functional magnetic resonance imaging (fMRI) data. In the method, we first estimate the frequency trajectories of the physiological signals with the interacting multiple models (IMM) filter algorithm. The frequency trajectories can be estimated from external reference signals, or if the temporal resolution is high enough, from the fMRI data. The estimated frequency trajectories are then used in a state space model in combination of a Kalman filter (KF) and Rauch-Tung-Striebel (RTS) smoother, which separates the signal into an activation related cleaned signal, physiological noise, and white measurement noise components. Using experimental data, we show that the method outperforms the RETROICOR algorithm if the shape and amplitude of the physiological signals change over time.

On Continuous-Discrete Cubature Kalman Filtering

Abstract This paper is concerned with application of cubature integration methods to Kalman filtering of discretely observed non-linear stochastic continuous-time systems. We compare two recently proposed variants of the continuous-discrete cubature Kalman filter (CD-CKF), which differ in the order how the discretization and the Gaussian approximation are done. Aside with theoretical analysis we test the performance of the different variants in a simulated application. The results indicate that the relative advantages of the approaches are application specific and neither one is unconditionally better than the other.

Infinite-dimensional Bayesian filtering for detection of quasiperiodic phenomena in spatiotemporal data

This paper introduces a spatiotemporal resonator model and an inference method for detection and estimation of nearly periodic temporal phenomena in spatiotemporal data. The model is derived as a spatial extension of a stochastic harmonic resonator model, which can be formulated in terms of a stochastic differential equation. The spatial structure is included by introducing linear operators, which affect both the oscillations and damping, and by choosing the appropriate spatial covariance structure of the driving time-white noise process. With the choice of the linear operators as partial differential operators, the resonator model becomes a stochastic partial differential equation, which is compatible with infinite-dimensional Kalman filtering. The resulting infinite-dimensional Kalman filtering problem allows for a computationally efficient solution as the computational cost scales linearly with measurements in the temporal dimension. This framework is applied to weather prediction and to physiological noise elimination in functional magnetic resonance imaging brain data.

Modeling and Interpolation of the Ambient Magnetic Field by Gaussian Processes

Anomalies in the ambient magnetic field can be used as features in indoor positioning and navigation. By using Maxwells equations, we derive and present a Bayesian nonparametric probabilistic modeling approach for interpolation and extrapolation of the magnetic field. We model the magnetic field components jointly by imposing a Gaussian process (GP) prior to the latent scalar potential of the magnetic field. By rewriting the GP model in terms of a Hilbert space representation, we circumvent the computational pitfalls associated with GP modeling and provide a computationally efficient and physically justified modeling tool for the ambient magnetic field. The model allows for sequential updating of the estimate and time-dependent changes in the magnetic field. The model is shown to work well in practice in different applications. We demonstrate mapping of the magnetic field both with an inexpensive Raspberry Pi powered robot and on foot using a standard smartphone.

Gaussian quadratures for state space approximation of scale mixtures of squared exponential covariance functions

Stationary one-dimensional Gaussian process models in machine learning can be reformulated as state space equations. This reduces the cubic computational complexity of the naive full GP solution to linear with respect to the number of training data points. For infinitely differentiable covariance functions the representation is an approximation. In this paper, we study a class of covariance functions that can be represented as a scale mixture of squared exponentials. We show how the generalized Gauss-Laguerre quadrature rule can be employed in a state space approximation in this class. The explicit form of the rational quadratic covariance function approximation is written out, and we demonstrate the results in a regression and log-Gaussian Cox process study.

The 10th annual MLSP competition: First place

The goal of the MLSP 2014 Schizophrenia Classification Challenge was to automatically diagnose subjects with schizophrenia based on multimodal features derived from their magnetic resonance imaging (MRI) brain scans. This challenge took place between June 5 and July 20, 2014, and was organized on Kaggle. We present how this classification problem can be solved in terms of a Bayesian machine learning paradigm known as Gaussian process (GP) classification. The proposed solution achieved an AUC score of 0.928, and it ranked first on the Kaggle private leaderboard.

Terrain navigation in the magnetic landscape: Particle filtering for indoor positioning

Variations in the ambient magnetic field can be used as features in indoor positioning and navigation. We describe a technique for map matching where the pedestrian movement is matched to a map of the magnetic landscape. The map matching algorithm is based on a particle filter, a recursive Monte Carlo method, and follows the classical terrain matching framework used in aircraft positioning and navigation. A recent probabilistic Gaussian process regression based method for modeling the ambient magnetic field is employed in the framework. The feasibility of this terrain matching approach is demonstrated in a simple real-life indoor positioning example, where both the mapping and positioning is done using a smartphone device.

The Blind Leading the Blind: Network-Based Location Estimation Under Uncertainty

We propose a probabilistic method for inferring the geographical locations of linked objects, such as users in a social network. Unlike existing methods, our model does not assume that the exact locations of any subset of the linked objects, like neighbors in a social network, are known. The method efficiently leverages prior knowledge on the locations, resulting in high geolocation accuracies even if none of the locations are initially known. Experiments are conducted for three scenarios: geolocating users of a location-based social network, geotagging historical church records, and geotagging Flickr photos. In each experiment, the proposed method outperforms two state-of-the-art network-based methods. Furthermore, the last experiment shows that the method can be employed not only to network-based but also to content-based location estimation.

Nonlinear state space model identification using a regularized basis function expansion

This paper is concerned with black-box identification of nonlinear state space models. By using a basis function expansion within the state space model, we obtain a flexible structure. The model is identified using an expectation maximization approach, where the states and the parameters are updated iteratively in such a way that a maximum likelihood estimate is obtained. We use recent particle methods with sound theoretical properties to infer the states, whereas the model parameters can be updated using closed-form expressions by exploiting the fact that our model is linear in the parameters. Not to over-fit the flexible model to the data, we also propose a regularization scheme without increasing the computational burden. Importantly, this opens up for systematic use of regularization in nonlinear state space models. We conclude by evaluating our proposed approach on one simulation example and two real-data problems.