Otto Debals

Breaking the Curse of Dimensionality Using Decompositions of Incomplete Tensors: Tensor-based scientific computing in big data analysis

Higher-order tensors and their decompositions are abundantly present in domains such as signal processing (e.g., higher-order statistics [1] and sensor array processing [2]), scientific computing (e.g., discretized multivariate functions [3]?[6]), and quantum information theory (e.g., representation of quantum many-body states [7]). In many applications, the possibly huge tensors can be approximated well by compact multilinear models or decompositions. Tensor decompositions are more versatile tools than the linear models resulting from traditional matrix approaches. Compared to matrices, tensors have at least one extra dimension. The number of elements in a tensor increases exponentially with the number of dimensions, and so do the computational and memory requirements. The exponential dependency (and the problems that are caused by it) is called the curse of dimensionality. The curse limits the order of the tensors that can be handled. Even for a modest order, tensor problems are often large scale. Large tensors can be handled, and the curse can be alleviated or even removed by using a decomposition that represents the tensor instead of using the tensor itself. However, most decomposition algorithms require full tensors, which renders these algorithms infeasible for many data sets. If a tensor can be represented by a decomposition, this hypothesized structure can be exploited by using compressed sensing (CS) methods working on incomplete tensors, i.e., tensors with only a few known elements.

Löwner-Based Blind Signal Separation of Rational Functions With Applications

A new blind signal separation (BSS) technique is proposed, enabling a deterministic separation of signals into rational functions. Rational functions can take on a wide range of forms, such as the well-known pole-like shape. The approach is a possible alternative for the well-known independent component analysis when the theoretical sources are not independent, such as for frequency spectra, or when only a small number of samples is available. The technique uses a low-rank decomposition on the tensorized version of the observed data matrix. The deterministic tensorization with Löwner matrices is comprehensively analyzed in this paper. Uniqueness properties are investigated, and a connection with the separation into exponential polynomials is made. Finally, the technique is illustrated for fetal electrocardiogram extraction and with an application in the domain of fluorescence spectroscopy, enabling the identification of chemical analytes using only a single excitation-emission matrix.

A Tensor-Based Method for Large-Scale Blind Source Separation Using Segmentation

Many real-life signals are compressible, meaning that they depend on much fewer parameters than their sample size. In this paper, we use low-rank matrix or tensor representations for signal compression. We propose a new deterministic method for blind source separation that exploits the low-rank structure, enabling a unique separation of the source signals and providing a way to cope with large-scale data. We explain that our method reformulates the blind source separation problem as the computation of a tensor decomposition, after reshaping the observed data matrix into a tensor. This deterministic tensorization technique is called segmentation and is closely related to Hankel-based tensorization. We apply the same strategy to the mixing coefficients of the blind source separation problem, as in many large-scale applications, the mixture is also compressible because of many closely located sensors. Moreover, we combine both strategies, resulting in a general technique that allows us to exploit the underlying compactness of the sources and the mixture simultaneously. We illustrate the techniques for fetal electrocardiogram extraction and direction-of-arrival estimation in large-scale antenna arrays.

Tensorlab 3.0 — Numerical optimization strategies for large-scale constrained and coupled matrix/tensor factorization

We give an overview of recent developments in numerical optimization-based computation of tensor decompositions that have led to the release of Tensorlab 3.0 in March 2016 (www.tensorlab.net). By careful exploitation of tensor product structure in methods such as quasi-Newton and nonlinear least squares, good convergence is combined with fast computation. A modular approach extends the computation to coupled factorizations and structured factors. Given large datasets, different compact representations (polyadic, Tucker,…) may be obtained by stochastic optimization, randomization, compressed sensing, etc. Exploiting the representation structure allows us to scale the algorithms for constrained/coupled factorizations to large problem sizes.

A novel deterministic method for large-scale blind source separation

A novel deterministic method for blind source separation is presented. In contrast to common methods such as independent component analysis, only mild assumptions are imposed on the sources. On the contrary, the method exploits a hypothesized (approximate) intrinsic low-rank structure of the mixing vectors. This is a very natural assumption for problems with many sensors. As such, the blind source separation problem can be reformulated as the computation of a tensor decomposition by applying a low-rank approximation to the tensorized mixing vectors. This allows the introduction of blind source separation in certain big data applications, where other methods fall short.

Blind signal separation of rational functions using Löwner-based tensorization

A novel deterministic blind signal separation technique for separating signals into rational functions is proposed, applicable in various situations. This new technique is based on a tensorization of the observed data matrix into a set of Löwner matrices. The obtained tensor can then be decomposed with a block tensor decomposition, resulting in a unique separation into rational functions under mild conditions. This approach provides a viable alternative to independent component analysis (ICA) in cases where the independence assumption is not valid or where the sources can be modeled well by rational functions, such as frequency spectra. In contrast to ICA, this technique is deterministic and not based on statistics, and therefore works well even with a small number of samples.

A tensor-based method for large-scale blind system identification using segmentation

A new method for the blind identification of large-scale finite impulse response (FIR) systems is presented. It exploits the fact that the system coefficients in large-scale problems often depend on much fewer parameters than the total number of entries in the coefficient vectors. We use low-rank models to compactly represent matricized versions of these compressible system coefficients. We show that blind system identification (BSI) then reduces to the computation of a structured tensor decomposition by using a deterministic tensorization technique called segmentation on the observed outputs. This careful exploitation of the low-rank structure enables the unique identification of both the system coefficients and the inputs. The approach does not require the input signals to be statistically independent.

Tensor-Based Large-Scale Blind System Identification Using Segmentation

Many real-life signals can be described in terms of much fewer parameters than the actual number of samples. Such compressible signals can often be represented very compactly with low-rank matrix and tensor models. The authors have adopted this strategy to enable large-scale instantaneous blind source separation. In this paper, we generalize the approach to the blind identification of large-scale convolutive systems. In particular, we apply the same idea to the system coefficients of finite impulse response systems. This allows us to reformulate blind system identification as a structured tensor decomposition. The tensor is obtained by applying a deterministic tensorization technique called segmentation on the observed output data. Exploiting the low-rank structure of the system coefficients enables a unique identification of the system and estimation of the inputs. We obtain a new type of deterministic uniqueness conditions. Moreover, the compactness of the low-rank models allows one to solve large-scale problems. We illustrate our method for direction-of-arrival estimation in large-scale antenna arrays and neural spike sorting in high-density microelectrode arrays.

Nonnegative Matrix Factorization Using Nonnegative Polynomial Approximations

Nonnegative matrix factorization is a key tool in many data analysis applications such as feature extraction, compression, and noise filtering. Many existing algorithms impose additional constraints to take into account prior knowledge and to improve the physical interpretation. This letter proposes a novel algorithm for nonnegative matrix factorization, in which the factors are modeled by nonnegative polynomials. Using a parametric representation of finite-interval nonnegative polynomials, we obtain an optimization problem without external nonnegativity constraints, which can be solved using conventional quasi-Newton or nonlinear least-squares methods. The polynomial model guarantees smooth solutions and may realize a noise reduction. A dedicated orthogonal compression enables a significant reduction of the matrix dimensions, without sacrificing accuracy. The overall approach scales well to large matrices. The approach is illustrated with applications in hyperspectral imaging and chemical shift brain imaging.

Coupled rank-(Lm, Ln, •) block term decomposition by coupled block simultaneous generalized Schur decomposition

Coupled decompositions of multiple tensors are fundamental tools for multi-set data fusion. In this paper, we introduce a coupled version of the rank-(Lm, Ln, *) block term decomposition (BTD), applicable to joint independent subspace analysis. We propose two algorithms for its computation based on a coupled block simultaneous generalized Schur decomposition scheme. Numerical results are given to show the performance of the proposed algorithms.