Morten Mørup

Parallel Factor Analysis as an exploratory tool for wavelet transformed event-related EEG

In the decomposition of multi-channel EEG signals, principal component analysis (PCA) and independent component analysis (ICA) have widely been used. However, as both methods are based on handling two-way data, i.e. two-dimensional matrices, multi-way methods might improve the interpretation of frequency transformed multi-channel EEG of channel x frequency x time data. The multi-way decomposition method Parallel Factor (PARAFAC), also named Canonical Decomposition (CANDECOMP), was recently used to decompose the wavelet transformed ongoing EEG of channel x frequency x time (Miwakeichi, F., Martinez-Montes, E., Valdes-Sosa, P.A., Nishiyama, N., Mizuhara, H., Yamaguchi, Y., 2004. Decomposing EEG data into space-time-frequency components using parallel factor analysis. Neuroimage 22, 1035-1045). In this article, PARAFAC is used for the first time to decompose wavelet transformed event-related EEG given by the inter-trial phase coherence (ITPC) encompassing ANOVA analysis of differences between conditions and 5-way analysis of channel x frequency x time x subject x condition. A flow chart is presented on how to perform data exploration using the PARAFAC decomposition on multi-way arrays. This includes (A) channel x frequency x time 3-way arrays of F test values from a repeated measures analysis of variance (ANOVA) between two stimulus conditions; (B) subject-specific 3-way analyses; and (C) an overall 5-way analysis of channel x frequency x time x subject x condition. The PARAFAC decompositions were able to extract the expected features of a previously reported ERP paradigm: namely, a quantitative difference of coherent occipital gamma activity between conditions of a visual paradigm. Furthermore, the method revealed a qualitative difference which has not previously been reported. The PARAFAC decomposition of the 3-way array of ANOVA F test values clearly showed the difference of regions of interest across modalities, while the 5-way analysis enabled visualization of both quantitative and qualitative differences. Consequently, PARAFAC is a promising data exploratory tool in the analysis of the wavelets transformed event-related EEG.

Defining a Local Arterial Input Function for Perfusion MRI Using Independent Component Analysis

Quantification of cerebral blood flow (CBF) using dynamic‐susceptibility contrast MRI relies on the deconvolution of the arterial input function (AIF), which is commonly estimated from the signal changes in a major artery. However, it has been shown that the presence of bolus delay/dispersion between the artery and the tissue of interest can be a significant source of error. These effects could be minimized if a local AIF were used, although the measurement of a local AIF can be problematic. This work describes a new methodology to define a local AIF using independent component analysis (ICA). The methodology was tested on data from patients with various cerebrovascular abnormalities and compared to the conventional approach of using a global AIF. The new methodology produced higher CBF and shorter mean transit time values (compared to the global AIF case) in areas with distorted AIFs, suggesting that the effects of delay/dispersion are minimized. The minimization of these effects using the calculated local AIF should lead to a more accurate quantification of CBF, which can have important implications for diagnosis and management of patients with cerebral ischemia. Magn Reson Med 52:789–797, 2004.

Nonnegative matrix factor 2-d deconvolution for blind single channel source separation

We present a novel method for blind separation of instruments in single channel polyphonic music based on a non-negative matrix factor 2-D deconvolution algorithm. The method is an extention of NMFD recently introduced by Smaragdis [1]. Using a model which is convolutive in both time and frequency we factorize a spectrogram representation of music into components corresponding to individual instruments. Based on this factorization we separate the instruments using spectrogram masking. The proposed algorithm has applications in computational auditory scene analysis, music information retrieval, and automatic music transcription.

Applications of tensor (multiway array) factorizations and decompositions in data mining

Tensor (multiway array) factorization and decomposition has become an important tool for data mining. Fueled by the computational power of modern computer researchers can now analyze large‐scale tensorial structured data that only a few years ago would have been impossible. Tensor factorizations have several advantages over two‐way matrix factorizations including uniqueness of the optimal solution and component identification even when most of the data is missing. Furthermore, multiway decomposition techniques explicitly exploit the multiway structure that is lost when collapsing some of the modes of the tensor in order to analyze the data by regular matrix factorization approaches. Multiway decomposition is being applied to new fields every year and there is no doubt that the future will bring many exciting new applications. The aim of this overview is to introduce the basic concepts of tensor decompositions and demonstrate some of the many benefits and challenges of modeling data multiway for a wide variety of data and problem domains.

Algorithms for sparse nonnegative tucker decompositions

There is a increasing interest in analysis of large-scale multiway data. The concept of multiway data refers to arrays of data with more than two dimensions, that is, taking the form of tensors. To analyze such data, decomposition techniques are widely used. The two most common decompositions for tensors are the Tucker model and the more restricted PARAFAC model. Both models can be viewed as generalizations of the regular factor analysis to data of more than two modalities. Nonnegative matrix factorization (NMF), in conjunction with sparse coding, has recently been given much attention due to its part-based and easy interpretable representation. While NMF has been extended to the PARAFAC model, no such attempt has been done to extend NMF to the Tucker model. However, if the tensor data analyzed are nonnegative, it may well be relevant to consider purely additive (i.e., nonnegative) Tucker decompositions). To reduce ambiguities of this type of decomposition, we develop updates that can impose sparseness in any combination of modalities, hence, proposed algorithms for sparse nonnegative Tucker decompositions (SN-TUCKER). We demonstrate how the proposed algorithms are superior to existing algorithms for Tucker decompositions when the data and interactions can be considered nonnegative. We further illustrate how sparse coding can help identify what model (PARAFAC or Tucker) is more appropriate for the data as well as to select the number of components by turning off excess components. The algorithms for SN-TUCKER can be downloaded from Mrup (2007).

Shift-invariant multilinear decomposition of neuroimaging data

We present an algorithm for multilinear decomposition that allows for arbitrary shifts along one modality. The method is applied to neural activity arranged in the three modalities space, time, and trial. Thus, the algorithm models neural activity as a linear superposition of components with a fixed time course that may vary across either trials or space in its overall intensity and latency. Its utility is demonstrated on simulated data as well as actual EEG, and fMRI data. We show how shift-invariant multilinear decompositions of multiway data can successfully cope with variable latencies in data derived from neural activity--a problem that has caused degenerate solutions especially in modeling neuroimaging data with instantaneous multilinear decompositions. Our algorithm is available for download at www.erpwavelab.org.

Non-linear calibration models for near infrared spectroscopy.

Different calibration techniques are available for spectroscopic applications that show nonlinear behavior. This comprehensive comparative study presents a comparison of different nonlinear calibration techniques: kernel PLS (KPLS), support vector machines (SVM), least-squares SVM (LS-SVM), relevance vector machines (RVM), Gaussian process regression (GPR), artificial neural network (ANN), and Bayesian ANN (BANN). In this comparison, partial least squares (PLS) regression is used as a linear benchmark, while the relationship of the methods is considered in terms of traditional calibration by ridge regression (RR). The performance of the different methods is demonstrated by their practical applications using three real-life near infrared (NIR) data sets. Different aspects of the various approaches including computational time, model interpretability, potential over-fitting using the non-linear models on linear problems, robustness to small or medium sample sets, and robustness to pre-processing, are discussed. The results suggest that GPR and BANN are powerful and promising methods for handling linear as well as nonlinear systems, even when the data sets are moderately small. The LS-SVM is also attractive due to its good predictive performance for both linear and nonlinear calibrations.

Infinite multiple membership relational modeling for complex networks

Learning latent structure in complex networks has become an important problem fueled by many types of networked data originating from practically all fields of science. In this paper, we propose a new non-parametric Bayesian multiple-membership latent feature model for networks. Contrary to existing multiple-membership models that scale quadratically in the number of vertices the proposed model scales linearly in the number of links admitting multiple-membership analysis in large scale networks. We demonstrate a connection between the single membership relational model and multiple membership models and show on “real” size benchmark network data that accounting for multiple memberships improves the learning of latent structure as measured by link prediction while explicitly accounting for multiple membership result in a more compact representation of the latent structure of networks.

Approximate L 0 constrained non-negative matrix and tensor factorization

Non-negative matrix factorization (NMF), i.e. V ap WH where both V, W and H are non-negative has become a widely used blind source separation technique due to its part based representation. The NMF decomposition is not in general unique and a part based representation not guaranteed. However, imposing sparseness both improves the uniqueness of the decomposition and favors part based representation. Sparseness in the form of attaining as many zero elements in the solution as possible is appealing from a conceptional point of view and corresponds to minimizing reconstruction error with an L0 norm constraint. In general, solving for a given L0 norm is an NP hard problem thus convex relaxation to regularization by the L1 norm is often considered, i.e., minimizing (1/2||V - WH||F 2 + lambda||H||1).An open problem is to control the degree of sparsity lambda imposed. We here demonstrate that a full regularization path for the L1 norm regularized least squares NMF for fixed W can be calculated at the cost of an ordinary least squares solution based on a modification of the least angle regression and selection (LARS) algorithm forming a non-negativity constrained LARS (NLARS). With the full regularization path, the L1 regularization strength lambda that best approximates a given L0 can be directly accessed and in effect used to control the sparsity of H. The MATLAB code for the NLARS algorithm is available for download.

Nonparametric Bayesian modeling of complex networks: an introduction

Modeling structure in complex networks using Bayesian nonparametrics makes it possible to specify flexible model structures and infer the adequate model complexity from the observed data. This article provides a gentle introduction to nonparametric Bayesian modeling of complex networks: Using an infinite mixture model as running example, we go through the steps of deriving the model as an infinite limit of a finite parametric model, inferring the model parameters by Markov chain Monte Carlo, and checking the model?s fit and predictive performance. We explain how advanced nonparametric models for complex networks can be derived and point out relevant literature.