Effrosini Kokiopoulou

Trace optimization and eigenproblems in dimension reduction methods

This paper gives an overview of the eigenvalue problems encountered in areas of data mining that are related to dimension reduction. Given some input high-dimensional data, the goal of dimension reduction is to map them to a low-dimensional space such that certain properties of the original data are preserved. Optimizing these properties among the reduced data can be typically posed as a trace optimization problem that leads to an eigenvalue problem. There is a rich variety of such problems and the goal of this paper is to unravel relations between them as well as to discuss effective solution techniques. First, we make a distinction between projective methods that determine an explicit linear mapping from the high-dimensional space to the low-dimensional space, and nonlinear methods where the mapping between the two is nonlinear and implicit. Then, we show that all of the eigenvalue problems solved in the context of explicit linear projections can be viewed as the projected analogues of the nonlinear or implicit projections. We also discuss kernels as a means of unifying linear and nonlinear methods and revisit some of the equivalences between methods established in this way. Finally, we provide some illustrative examples to showcase the behavior and the particular characteristics of the various dimension reduction techniques on real world data sets. Copyright c

Orthogonal neighborhood preserving projections

Orthogonal neighborhood preserving projections (ONPP) is a linear dimensionality reduction technique which attempts to preserve both the intrinsic neighborhood geometry of the data samples and the global geometry. The proposed technique constructs a weighted data graph where the weights are constructed in a data-driven fashion, similarly to locally linear embedding (LLE). A major difference with the standard LLE where the mapping between the input and the reduced spaces is implicit, is that ONPP employs an explicit linear mapping between the two. As a result, and in contrast with LLE, handling new data samples becomes straightforward, as this amounts to a simple linear transformation. ONPP shares some of the properties of locality preserving projections (LPP). Both ONPP and LPP rely on a k-nearest neighbor graph in order to capture the data topology. However, our algorithm inherits the characteristics of LLE in preserving the structure of local neighborhoods, while LPP aims at preserving only locality without specifically aiming at preserving the geometric structure. This feature makes ONPP an effective method for data visualization. We provide ample experimental evidence to demonstrate the advantageous characteristics of ONPP, using well known synthetic test cases as well as real life data from computational biology and computer vision.

Polynomial filtering in latent semantic indexing for information retrieval

Latent Semantic Indexing (LSI) is a well established and effective framework for conceptual information retrieval. In traditional implementations of LSI the semantic structure of the collection is projected into the k-dimensional space derived from a rank-k approximation of the original term-by-document matrix. This paper discusses a new way to implement the LSI methodology, based on polynomial filtering. The new framework does not rely on any matrix decomposition and therefore its computational cost and storage requirements are low relative to traditional implementations of LSI. Additionally, it can be used as an effective information filtering technique when updating LSI models based on user feedback.

Computation of Large Invariant Subspaces Using Polynomial Filtered Lanczos Iterations with Applications in Density Functional Theory

The most expensive part of all electronic structure calculations based on density functional theory lies in the computation of an invariant subspace associated with some of the smallest eigenvalues of a discretized Hamiltonian operator. The dimension of this subspace typically depends on the total number of valence electrons in the system, and can easily reach hundreds or even thousands when large systems with many atoms are considered. At the same time, the discretization of Hamiltonians associated with large systems yields very large matrices, whether with planewave or real-space discretizations. The combination of these two factors results in one of the most significant bottlenecks in computational materials science. In this paper we show how to efficiently compute a large invariant subspace associated with the smallest eigenvalues of a symmetric/Hermitian matrix using polynomially filtered Lanczos iterations. The proposed method does not try to extract individual eigenvalues and eigenvectors. Instead, it constructs an orthogonal basis of the invariant subspace by combining two main ingredients. The first is a filtering technique to dampen the undesirable contribution of the largest eigenvalues at each matrix-vector product in the Lanczos algorithm. This technique employs a well-selected low pass filter polynomial, obtained via a conjugate residual-type algorithm in polynomial space. The second ingredient is the Lanczos algorithm with partial reorthogonalization. Experiments are reported to illustrate the efficiency of the proposed scheme compared to state-of-the-art implicitly restarted techniques.

The design of a distributed MATLAB-based environment for computing pseudospectra

It has been documented in the literature that the pseudospectrum of a matrix is a powerful concept that broadens our understanding of phenomena based on matrix computations. When the matrix A is non-normal, however, the computation of the pseudospectrum becomes a very expensive computational task. Thus, the use of high performance computing resources becomes key to obtaining useful answers in acceptable amounts of time. In this work we describe the design and implementation of an environment that integrates a suite of state-of-the-art algorithms running on a cluster of workstations to enable the matrix pseudospectrum become a practical tool for scientists and engineers. The user interacts with the environment via the graphical user interface PPsGUI. The environment is constructed on top of CMTM, an existing environment that enables distributed computation via an MPI API for MATLAB.

Towards the effective parallel computation of matrix pseudospectra

Given a matrix A, the computation of its pseudospectrum A∈ (A) is a far more expensive task than the computation of characteristics such as the condition number and the matrix spectrum. As research of the last 15 years has shown, however, the matrix pseudospectrum provides valuable information that is not included in other indicators. So, we ask how to compute it efficiently and build a tool that would facilitate engineers and scientists to make such analyses? In this paper we focus on parallel algorithms for computing pseudospectra. The most widely used algorithm for computing pseudospectra is embarassingly parallel; nevertheless, it is extremely costly and one cannot hope achieve absolute high performance with it. We describe algorithms that have drastically improved performance while maintaining a high degree of large grain parallelism. We evaluate the effectiveness of these methods in the context of a MATLAB-based environment for parallel programming using MPI on small, off-the-shelf parallel systems.

Parallel Computation of Pseudospectra Using Transfer Functions on a MATLAB-MPI Cluster Platform

One of the most computationally expensive problems in numerical linear algebra is the computation of the ?-pseudospectrum of matrices, that is, the locus of eigenvalues of all matrices of the form A + E, where ||E|| ? ?. Several research efforts have been attempting to make the problem tractable by means of better algorithms and utilization of all possible computational resources. One common goal is to bring to users the power to extract pseudospectrum information from their applications, on the computational environments they generally use, at a cost that is sufficiently low to render these computations routine. To this end, we investigate a scheme based on i) iterative methods for computing pseudospectra via approximations of the resolvent norm, with ii) a computational platform based on a cluster of PCs and iii) a programming environment based on MATLAB enhanced with MPI functionality and show that it can achieve high performance for problems of significant size.