Marco Signoretto

Tensor Versus Matrix Completion: A Comparison With Application to Spectral Data

Tensor completion recently emerged as a generalization of matrix completion for higher order arrays. This problem formulation allows one to exploit the structure of data that intrinsically have multiple dimensions. In this work, we recall a convex formulation for minimum (multilinear) ranks completion of arrays of arbitrary order. Successively we focus on completion of partially observed spectral images; the latter can be naturally represented as third order tensors and typically exhibit intraband correlations. We compare different convex formulations and assess them through case studies.

2011 Special Issue: A kernel-based framework to tensorial data analysis

Tensor-based techniques for learning allow one to exploit the structure of carefully chosen representations of data. This is a desirable feature in particular when the number of training patterns is small which is often the case in areas such as biosignal processing and chemometrics. However, the class of tensor-based models is somewhat restricted and might suffer from limited discriminative power. On a different track, kernel methods lead to flexible nonlinear models that have been proven successful in many different contexts. Nonetheless, a naïve application of kernel methods does not exploit structural properties possessed by the given tensorial representations. The goal of this work is to go beyond this limitation by introducing non-parametric tensor-based models. The proposed framework aims at improving the discriminative power of supervised tensor-based models while still exploiting the structural information embodied in the data. We begin by introducing a feature space formed by multilinear functionals. The latter can be considered as the infinite dimensional analogue of tensors. Successively we show how to implicitly map input patterns in such a feature space by means of kernels that exploit the algebraic structure of data tensors. The proposed tensorial kernel links to the MLSVD and features an interesting invariance property; the approach leads to convex optimization and fits into the same primal-dual framework underlying SVM-like algorithms.

Incorporating structural information from the multichannel EEG improves patient-specific seizure detection.

OBJECTIVE A novel patient-specific seizure detection algorithm is presented. As the spatial distribution of the ictal pattern is characteristic for a patients seizures, this work incorporates such information into the data representation and provides a learning algorithm exploiting it. METHODS The proposed training algorithm uses nuclear norm regularization to convey structural information of the channel-feature matrices extracted from the EEG. This method is compared to two existing approaches utilizing the same feature set, but integrating the multichannel information in a different manner. The performances of the detectors are demonstrated on a publicly available dataset containing 131 seizures recorded in 892 h of scalp EEG from 22 pediatric patients. RESULTS The proposed algorithm performed significantly better compared to the reference approaches (p=0.0170 and p=0.0002). It reaches a median performance of 100% sensitivity, 0.11h(-1) false detection rate and 7.8s alarm delay, outperforming a method in the literature using the same dataset. CONCLUSION The strength of our method lies within conveying structural information from the multichannel EEG. Such formulation automatically includes crucial spatial information and improves detection performance. SIGNIFICANCE Our solution facilitates accurate classification performance for small training sets, therefore, it potentially reduces the time needed to train the detector before starting monitoring.

Nonlinear Acoustic Echo Cancellation Based on a Sliding-Window Leaky Kernel Affine Projection Algorithm

Acoustic echo cancellation (AEC) is used in speech communication systems where the existence of echoes degrades the speech intelligibility. Standard approaches to AEC rely on the assumption that the echo path to be identified can be modeled by a linear filter. However, some elements introduce nonlinear distortion and must be modeled as nonlinear systems. Several nonlinear models have been used with more or less success. The kernel affine projection algorithm (KAPA) has been successfully applied to many areas in signal processing but not yet to nonlinear AEC (NLAEC). The contribution of this paper is three-fold: (1) to apply KAPA to the NLAEC problem, (2) to develop a sliding-window leaky KAPA (SWL-KAPA) that is well suited for NLAEC applications, and (3) to propose a kernel function, consisting of a weighted sum of a linear and a Gaussian kernel. In our experiment set-up, the proposed SWL-KAPA for NLAEC consistently outperforms the linear APA, resulting in up to 12 dB of improvement in ERLE at a computational cost that is only 4.6 times higher. Moreover, it is shown that the SWL-KAPA outperforms, by 4-6 dB, a Volterra-based NLAEC, which itself has a much higher 413 times computational cost than the linear APA.

High level high performance computing for multitask learning of time-varying models

We propose an approach suitable to learn multiple time-varying models jointly and discuss an application in data-driven weather forecasting. The methodology relies on spectral regularization and encodes the typical multi-task learning assumption that models lie near a common low dimensional subspace. The arising optimization problem amounts to estimating a matrix from noisy linear measurements within a trace norm ball. Depending on the problem, the matrix dimensions as well as the number of measurements can be large. We discuss an algorithm that can handle large-scale problems and is amenable to parallelization. We then compare high level high performance implementation strategies that rely on Just-in-Time (JIT) decorators. The approach enables, in particular, to offload computations to a GPU without hard-coding computationally intensive operations via a low-level language. As such, it allows for fast prototyping and therefore it is of general interest for developing and testing novel computational models.

Classification of Multichannel Signals With Cumulant-Based Kernels

We consider the problem of training a discriminative classifier given a set of labelled multivariate time series (a.k.a. multichannel signals or vector processes). We propose a novel kernel function that exploits the spectral information of tensors of fourth-order cross-cumulants associated to each multichannel signal. Contrary to existing approaches the arising procedure does not require an (often nontrivial) blind identification step. Nonetheless, insightful connections with the dynamics of the generating systems can be drawn under specific modeling assumptions. The method is illustrated on both synthetic examples as well as on a brain decoding task where the direction, either left of right, towards where the subject modulates attention is predicted from magnetoencephalography (MEG) signals. Kernel functions for unstructured data do not leverage the underlying dynamics of multichannel signals. A comparison with these kernels as well as with state-of-the-art approaches, including generative methods, shows the merits of the proposed technique.

Kernel-based learning from infinite dimensional 2-way tensors

In this paper we elaborate on a kernel extension to tensor-based data analysis. The proposed ideas find applications in supervised learning problems where input data have a natural 2-way representation, such as images or multivariate time series. Our approach aims at relaxing linearity of standard tensor-based analysis while still exploiting the structural information embodied in the input data.

Quadratically Constrained Quadratic Programming for Subspace Selection in Kernel Regression Estimation

In this contribution we consider the problem of regression estimation. We elaborate on a framework based on functional analysis giving rise to structured models in the context of reproducing kernel Hilbert spaces. In this setting the task of input selection is converted into the task of selecting functional components depending on one (or more) inputs. In turn the process of learning with embedded selection of such components can be formalized as a convex-concave problem. This results in a practical algorithm that can be implemented as a quadratically constrained quadratic programming (QCQP) optimization problem. We further investigate the mechanism of selection for the class of linear functions, establishing a relationship with LASSO.

Convex estimation of cointegrated VAR models by a nuclear norm penalty

Abstract Cointegrated Vector AutoRegressive (VAR) processes arise in the study of long run equilibrium relations of stochastic dynamical systems. In this paper we introduce a novel convex approach for the analysis of these type of processes. The idea relies on an error correction representation and amounts at solving a penalized empirical risk minimization problem. The latter finds a model from data by minimizing a trade-off between a quadratic error function and a nuclear norm penalty used as a proxy for the cointegrating rank. We elaborate on properties of the proposed convex program; we then propose an easily implementable and provably convergent algorithm based on FISTA. This algorithm can be conveniently used for computing the regularization path, i.e., the entire set of solutions associated to the trade-off parameter. We show how such path can be used to build an estimator for the cointegrating rank and illustrate the proposed ideas with experiments.

Automatic seizure detection incorporating structural information

Traditional seizure detection algorithms act on single channels ignoring the synchronously recorded, inherently interdependent multichannel nature of EEG. However, the spatial distribution and evolution of the ictal pattern is a crucial characteristic of the seizure. Two different approaches aiming at including such structural information into the data representation are presented in this paper. Their performance is compared to the traditional approach both in a simulation study and a real-life example, showing that spatial and structural information facilitates precise classification.