José Henrique De Morais Goulart

Tensor CP Decomposition With Structured Factor Matrices: Algorithms and Performance

The canonical polyadic decomposition (CPD) of high-order tensors, also known as Candecomp/Parafac, is very useful for representing and analyzing multidimensional data. This paper considers a CPD model having structured matrix factors, as e.g. Toeplitz, Hankel or circulant matrices, and studies its associated estimation problem. This model arises in signal processing applications such as Wiener-Hammerstein system identification and cumulant-based wireless communication channel estimation. After introducing a general formulation of the considered structured CPD (SCPD), we derive closed-form expressions for the Cramér-Rao bound (CRB) of its parameters under the presence of additive white Gaussian noise. Formulas for special cases of interest, as when the CPD contains identical factors, are also provided. Aiming at a more relevant statistical evaluation from a practical standpoint, we discuss the application of our formulas in a Bayesian context, where prior distributions are assigned to the model parameters. Three existing algorithms for computing SCPDs are then described: a constrained alternating least squares (CALS) algorithm, a subspace-based solution and an algebraic solution for SCPDs with circulant factors. Subsequently, we present three numerical simulation scenarios, in which several specialized estimators based on these algorithms are proposed for concrete examples of SCPD involving circulant factors. In particular, the third scenario concerns the identification of a Wiener-Hammerstein system via the SCPD of an associated Volterra kernel. The statistical performance of the proposed estimators is assessed via Monte Carlo simulations, by comparing their Bayesian mean-square error with the expected CRB.

An algebraic solution for the Candecomp/PARAFAC decomposition with circulant factors

The Candecomp/PARAFAC decomposition (CPD) is an important mathematical tool used in several fields of application. Yet, its computation is usually performed with iterative methods which are subject to reaching local minima and to exhibiting slow convergence. In some practical contexts, the data tensors of interest admit decompositions constituted by matrix factors with particular structure. Often, such structure can be exploited for devising specialized algorithms with superior properties in comparison with general iterative methods. In this paper, we propose a novel approach for computing a circulant-constrained CPD, i.e., a CPD of a hypercubic tensor whose factors are all circulant (and possibly tall). To this end, we exploit the algebraic structure of such tensor, showing that the elements of its frequency-domain counterpart satisfy homogeneous monomial equations in the eigenvalues of square circulant matrices associated with its factors, which we can therefore estimate by solving these equations. Then...

An iterative hard thresholding algorithm with improved convergence for low-rank tensor recovery

Recovering low-rank tensors from undercomplete linear measurements is a computationally challenging problem of great practical importance. Most existing approaches circumvent the intractability of the tensor rank by considering instead the multilinear rank. Among them, the recently proposed tensor iterative hard thresholding (TIHT) algorithm is simple and has low cost per iteration, but converges quite slowly. In this work, we propose a new step size selection heuristic for accelerating its convergence, relying on a condition which (ideally) ensures monotonic decrease of its target cost function. This condition is obtained by studying TIHT from the standpoint of the majorization-minimization strategy which underlies the normalized IHT algorithm used for sparse vector recovery. Simulation results are presented for synthetic data tensor recovery and brain MRI data tensor completion, showing that the performance of TIHT is notably improved by our heuristic, with a small to moderate increase of the cost per iteration.

Low-Rank Tensor Recovery using Sequentially Optimal Modal Projections in Iterative Hard Thresholding (SeMPIHT)

Iterative hard thresholding (IHT) is a simple and effective approach to parsimonious data recovery. Its multilinear rank (mrank)-based application to low-rank tensor recovery (LRTR) is especially valuable given the difficulties involved in this problem. In this paper, we propose a novel IHT algorithm for LRTR, choosing sequential per-mode SVD truncation as its thresholding operator. This operator is less costly than those used in existing IHT algorithms for LRTR, and often leads to superior performance. Furthermore, by exploiting the sequential optimality of the employed modal projections, we derive recovery guarantees relying on restricted isometry constants. Though these guarantees are suboptimal, our numerical studies indicate that a quasi-optimal number of Gaussian measurements suffices for perfect data reconstruction. We also investigate a continuation technique which yields a sequence of progressively more complex estimated models until attaining a target mrank. When recovering real-world data, this...

Evaluating the potential of Volterra-PARAFAC IIR models

The Volterra-PARAFAC (VP) nonlinear system model, which consists of a FIR filterbank followed by a memoryless nonlinearity, aims at offering a good compromise between accuracy and parametric complexity. Here, for an even better compromise, we propose a generalization with IIR filters (VPI model) and evaluate both models. For the evaluation, we consider the concrete case of two audio loudspeakers and initially compute reference Volterra kernels from their known physical state-space models, using an efficient procedure. Then, VP and VPI models are derived and their accuracy is tested. As shown, the VPI models have in this case only 15 to 26 % of the parametric complexity of VP models with the same accuracy, which points to a great potential for accurate and efficient nonlinear system modeling.

A novel non-iterative algorithm for low-multilinear-rank tensor approximation

Low-rank tensor approximation algorithms are building blocks in tensor methods for signal processing. In particular, approximations of low multilinear rank (mrank) are of central importance in tensor subspace analysis. This paper proposes a novel non-iterative algorithm for computing a low-mrank approximation, termed sequential low-rank approximation and projection (SeLRAP). Our algorithm generalizes sequential rank-one approximation and projection (SeROAP), which aims at the rank-one case. For third-order mrank-(1,R,R) approximations, SeLRAPs outputs are always at least as accurate as those of previously proposed methods. Our simulation results suggest that this is actually the case for the overwhelmingly majority of random third- and fourth-order tensors and several different mranks. Though the accuracy improvement is often small, we show it can make a large difference when repeatedly computing approximations, as happens, e.g., in an iterative hard thresholding algorithm for tensor completion.