Felipe A. Tobar

Multikernel Least Mean Square Algorithm

The multikernel least-mean-square algorithm is introduced for adaptive estimation of vector-valued nonlinear and nonstationary signals. This is achieved by mapping the multivariate input data to a Hilbert space of time-varying vector-valued functions, whose inner products (kernels) are combined in an online fashion. The proposed algorithm is equipped with novel adaptive sparsification criteria ensuring a finite dictionary, and is computationally efficient and suitable for nonstationary environments. We also show the ability of the proposed vector-valued reproducing kernel Hilbert space to serve as a feature space for the class of multikernel least-squares algorithms. The benefits of adaptive multikernel (MK) estimation algorithms are illuminated in the nonlinear multivariate adaptive prediction setting. Simulations on nonlinear inertial body sensor signals and nonstationary real-world wind signals of low, medium, and high dynamic regimes support the approach.

Quaternion Reproducing Kernel Hilbert Spaces: Existence and Uniqueness Conditions

The existence and uniqueness conditions of quaternion reproducing kernel Hilbert spaces (QRKHS) are established in order to provide a mathematical foundation for the development of quaternion-valued kernel learning algorithms. This is achieved through a rigorous account of left quaternion Hilbert spaces, which makes it possible to generalise standard RKHS to quaternion RKHS. Quaternion versions of the Riesz representation and Moore-Aronszajn theorems are next introduced, thus underpinning kernel estimation algorithms operating on quaternion-valued feature spaces. The difference between the proposed quaternion kernel concept and the existing real and vector approaches is also established in terms of both theoretical advantages and computational complexity. The enhanced estimation ability of the so-introduced quaternion-valued kernels over their real- and vector-valued counterparts is validated through kernel ridge regression applications. Simulations on real world 3D inertial body sensor data and nonlinear channel equalisation using novel quaternion cubic and Gaussian kernels support the approach.

A novel augmented complex valued kernel LMS

A novel class of complex valued kernel least mean square (CKLMS) algorithms is introduced with the aim to provide physical meaning to the mapping between the primal and dual space termed the independent CKLMS (iCKLMS). The general class of CKLMS algorithms is also extended in the widely linear sense to develop online kernel algorithms suitable for the processing of general complex valued signals, both circular and noncircular. The so-introduced augmented complex kernel least mean square (ACKLMS) algorithms are verified on adaptive prediction of nonlinear and nonstationary complex wind signals.

Learning stationary time series using Gaussian processes with nonparametric kernels

We introduce the Gaussian Process Convolution Model (GPCM), a two-stage non-parametric generative procedure to model stationary signals as the convolution between a continuous-time white-noise process and a continuous-time linear filter drawn from Gaussian process. The GPCM is a continuous-time nonparametric-window moving average process and, conditionally, is itself a Gaussian process with a nonparametric kernel defined in a probabilistic fashion. The generative model can be equivalently considered in the frequency domain, where the power spectral density of the signal is specified using a Gaussian process. One of the main contributions of the paper is to develop a novel variational free-energy approach based on inter-domain inducing variables that efficiently learns the continuous-time linear filter and infers the driving white-noise process. In turn, this scheme provides closed-form probabilistic estimates of the covariance kernel and the noise-free signal both in denoising and prediction scenarios. Additionally, the variational inference procedure provides closed-form expressions for the approximate posterior of the spectral density given the observed data, leading to new Bayesian nonparametric approaches to spectrum estimation. The proposed GPCM is validated using synthetic and real-world signals.

The quaternion kernel least squares

The quaternion kernel least squares algorithm (QKLS) is introduced as a generic kernel framework for the estimation of multivariate quaternion valued signals. This is achieved based on the concepts of quaternion inner product and quaternion positive definiteness, allowing us to define quaternion kernel regression. Next, the least squares solution is derived using the recently introduced Hℝ calculus. We also show that QKLS is a generic extension of standard kernel least squares, and their equivalence is established for real valued kernels. The superiority of the quaternion-valued linear kernel with respect to its real-valued counterpart is illustrated for both synthetic and real-world prediction applications, in terms of accuracy and robustness to overfitting.standard kernel least squares,quaternion-valued linear kernelreal-world prediction applications,real-world 3D inertial body sensor signals.synthetic autoregressive processes

Unsupervised State-Space Modeling Using Reproducing Kernels

A novel framework for the design of state-space models (SSMs) is proposed whereby the state-transition function of the model is parametrized using reproducing kernels. The nature of SSMs requires learning a latent function that resides in the state space and for which input-output sample pairs are not available, thus prohibiting the use of gradient-based supervised kernel learning. To this end, we then propose to learn the mixing weights of the kernel estimate by sampling from their posterior density using Monte Carlo methods. We first introduce an offline version of the proposed algorithm, followed by an online version which performs inference on both the parameters and the hidden state through particle filtering. The accuracy of the estimation of the state-transition function is first validated on synthetic data. Next, we show that the proposed algorithm outperforms kernel adaptive filters in the prediction of real-world time series, while also providing probabilistic estimates, a key advantage over standard methods.

Modelling of complex signals using gaussian processes

In complex-valued signal processing, estimation algorithms require complete knowledge (or accurate estimation) of the second order statistics, this makes Gaussian processes (GP) well suited for modelling complex signals, as they are designed in terms of covariance functions. Dealing with bivariate signals using GPs require four covariance matrices, or equivalently, two complex matrices. We propose a GP-based approach for modelling complex signals, whereby the second-order statistics are learnt through maximum likelihood; in particular, the complex GP approach allows for circularity coefficient estimation in a robust manner when the observed signal is corrupted by (circular) white noise. The proposed model is validated using climate signals, for both circular and noncircular cases. The results obtained open new possibilities for collaboration between the complex signal processing and Gaussian processes communities towards an appealing representation and statistical description of bivariate signals.

A particle filtering based kernel HMM predictor

A novel kernel algorithm is proposed for nonlinear prediction whereby the signal is modelled as a state of a hidden Markov model (HMM). The transition function of the HMM is approximated using kernels, whose weights are also part of the state of the system and are learnt in an unsupervised fashion by a sample importance resampling (SIR) particle filter. The SIR proposal density is designed so as to maintain a diverse population of particles, thus avoiding particle degeneracy arising from inaccuracies of early model estimates. The kernel HMM algorithm is further equipped with a sparsification criterion based on approximate linear dependence and its performance is evaluated against the KNLMS and KRLS algorithms for the prediction of synthetic signals and real world point-of-gaze data.

Initialising kernel adaptive filters via probabilistic inference

We present a probabilistic framework for both (i) determining the initial settings of kernel adaptive filters (KAFs) and (ii) constructing fully-adaptive KAFs whereby in addition to weights and dictionaries, kernel parameters are learnt sequentially. This is achieved by formulating the estimator as a probabilistic model and defining dedicated prior distributions over the kernel parameters, weights and dictionary, enforcing desired properties such as sparsity. The model can then be trained using a subset of data to initialise standard KAFs or updated sequentially each time a new observation becomes available. Due to the nonlinear/non-Gaussian properties of the model, learning and inference is achieved using gradient-based maximum-aposteriori optimisation and Markov chain Monte Carlo methods, and can be confidently used to compute predictions. The proposed framework was validated on nonlinear time series of both synthetic and real-world nature, where it outperformed standard KAFs in terms of mean square error and the sparsity of the learnt dictionaries.

Improving battery voltage prediction in an electric bicycle using altitude measurements and kernel adaptive filters

Abstract The time-varying nature of consumption patterns is critical in the development of reliable electric vehicles and real-time schemes for assessing energy autonomy. Most of these schemes use battery voltage observations as a primary source of information and neglect variables external to the vehicle that affect its autonomy and help to characterise the behaviour of the battery as main energy storage device. Using an electric bicycle as case study, we show that the incorporation of external variables (e.g., altitude measurements) improves predictions associated with evolution of the battery voltage in time. We achieve this by proposing a novel kernel adaptive filter for multiple inputs and with a data-dependent dictionary construction. This allows us to model the dependency between battery voltage and altitude variations in a sequential manner. The proposed methodology combines automatic discovery of the relationship between voltage and altitude from data, and a kernel-based voltage predictor to address an important issue in reliability of electric vehicles. The proposed method is validated against a standard kernel adaptive filter, fixed linear filters and adaptive linear filters as baselines on the short- and long-term prediction of real-world battery voltage data.