Rafal Zdunek

Nonnegative Matrix and Tensor Factorization [Lecture Notes]

In these lecture notes, the authors have outlined several approaches to solve a NMF/NTF problem. The following main conclusions can be drawn: 1) Multiplicative algorithms are not necessary the best approaches for NMF, especially if data representations are not very redundant or sparse. 2) Much better performance can be achieved using the FP-ALS (especially for large-scale problems), IPC, and QN methods. 3) To achieve high performance it is quite important to use the multilayer structure with multistart initialization conditions. 4) To estimate physically meaningful nonnegative components it is often necessary to use some a priori knowledge and impose additional constraints or regularization terms (to control sparsity, boundness, continuity or smoothness of the estimated nonnegative components).

Improved FOCUSS Method With Conjugate Gradient Iterations

Focal Underdetermined System Solver (FOCUSS) is a powerful tool for sparse representation and underdetermined inverse problems. In this correspondence, we strengthen the FOCUSS method with the following main contributions: 1) we give a more rigorous derivation of the FOCUSS for the sparsity parameter 0 < p < 1 by a nonlinear transform and 2) we develop the CG-FOCUSS by incorporating the conjugate gradient (CG) method to the FOCUSS, which significantly reduces a computational cost with respect to the standard FOCUSS and extends its availability for large scale problems. We justify the CG-FOCUSS based on a probability theory. Furthermore, the high performance of the CG-FOCUSS is demonstrated with experiments.

Novel Multi-layer Non-negative Tensor Factorization with Sparsity Constraints

In this paper we present a new method of 3D non-negative tensor factorization (NTF) that is robust in the presence of noise and has many potential applications, including multi-way blind source separation (BSS), multi-sensory or multi-dimensional data analysis, and sparse image coding. We consider alpha- and beta-divergences as error (cost) functions and derive three different algorithms: (1) multiplicative updating; (2) fixed point alternating least squares (FPALS); (3) alternating interior-point gradient (AIPG) algorithm. We also incorporate these algorithms into multilayer networks. Experimental results confirm the very useful behavior of our multilayer 3D NTF algorithms with multi-start initializations.

Flexible Component Analysis for Sparse, Smooth, Nonnegative Coding or Representation

In the paper, we present a new approach to multi-way Blind Source Separation (BSS) and corresponding 3D tensor factorization that has many potential applications in neuroscience and multi-sensory or multidimensional data analysis, and neural sparse coding. We propose to use a set of local cost functions with flexible penalty and regularization terms whose simultaneous or sequential (one by one) minimization via a projected gradient technique leads to simple Hebbian-like local algorithms that work well not only for an over-determined case but also (under some weak conditions) for an under-determined case (i.e., a system which has less sensors than sources). The experimental results confirm the validity and high performance of the developed algorithms, especially with usage of the multi-layer hierarchical approach.

Blind Image Separation Using Nonnegative Matrix Factorization with Gibbs Smoothing

Nonnegative Matrix Factorization (NMF) has already found many applications in image processing and data analysis, including classification, clustering, feature extraction, pattern recognition, and blind image separation. In the paper, we extend the selected NMF algorithms by taking into account local smoothness properties of source images. Our modifications are related with incorporation of the Gibbs prior, which is well-known in many tomographic image reconstruction applications, to a underlying blind image separation model. The numerical results demonstrate the improved performance of the proposed methods in comparison to the standard NMF algorithms.

Sparse Super Symmetric Tensor Factorization

In the paper we derive and discuss a wide class of algorithms for 3D Super-symmetric Nonnegative Tensor Factorization (SNTF) or nonnegative symmetric PARAFAC, and as a special case: Symmetric Nonnegative Matrix Factorization (SNMF) that have many potential applications, including multi-way clustering, feature extraction, multi- sensory or multi-dimensional data analysis, and nonnegative neural sparse coding. The main advantage of the derived algorithms is relatively low complexity, and in the case of multiplicative algorithms possibility for straightforward extension of the algorithms to L-order tensors factorization due to some nice symmetric property. We also propose to use a wide class of cost functions such as Squared Euclidean, Kullback Leibler I-divergence, Alpha divergence and Beta divergence. Preliminary experimental results confirm the validity and good performance of some of these algorithms, especially when the data have sparse representations.