Mohammed A. Hasan

Natural gradient for minor component extraction

The paper proposes constrained optimization criteria for extracting in parallel multiple minor components using a weighted inverse Rayleigh quotient (WIRQ). This WIRQ is formulated into many constrained optimization problems which results in deriving many variations of MCA flows using the natural gradient concept. The derived minor component flows are analyzed and some of their convergence properties are presented. Procedures for converting MCA flows into a principal component flow are also discussed.

Diagonally weighted and shifted criteria for minor and principal component extraction

A framework for a class of minor and principal component learning rules is presented. These rules compute multiple eigenvectors and not only a basis for a multi-dimensional eigenspace. Several MCA/PCA cost functions which are weighted or shifted by a diagonal matrix are optimized subject to orthogonal or symmetric constraints. A number of minor and principal component learning rules for symmetric matrices and matrix pencils, many of which are new, are obtained by exploiting symmetry of constrained criteria. These algorithms may be seen as the counterparts or generalization of Ojas and Xus systems for computing multiple principal component analyzers. Procedures for converting minor component flows into principal component flows are also discussed.

Block eigenvalue decomposition using nth roots of the identity matrix

The matrix sign function has been utilized in recent years for block diagonalization of complex matrices. In this paper, nth roots of the identity matrix including the matrix sector function are utilized for block diagonalization of general matrices. Specifically, we derive classes of rational fixed point functions for nth roots of any nonsingular matrix which are then used for block eigen-decomposition. Based on these functions, algorithms may have any desired order of convergence are developed. Efficient implementation of these algorithms using the QR factorization is also presented. Several examples are presented to illustrate the performance of these methods.

On multi-set canonical correlation analysis

Two- and multi-set canonical correlation analysis (CCA) and (MCCA) techniques are used to find linear combinations that give maximal multivariate differences. This paper describes methods for deriving MCCA dynamical systems which converge to the desired canonical variates and canonical correlations. Unconstrained and constrained optimization methods over quadratic constraints are applied to derive several dynamical systems that converge to a solution of a generalized eigenvalue problem. These include merit functions that are based on generalized Rayleigh quotient, and logarithmic generalized Rayleigh quotient.

Fixed point iterations for computing square roots and the matrix sign function of complex matrices

The purpose of this work has been the development of new set of rational iterations for computing square roots and the matrix sign function of complex matrices. Given any positive integer r/spl ges/2, we presented a systematic way of deriving rth order convergent algorithms for matrix square roots, the matrix sign function, invariant subspaces in different half-planes, and the polar decomposition. We have shown, that these iterations are applicable for computing square roots of more general type of matrices than previously reported, such as matrices in which some of its eigenvalues are negative. Also, algorithms for computing square roots and the invariant subspace of a given matrix in any given half-plane are derived.

Self-Normalizing Dual Systems for Minor and Principal Component Extraction

In this paper classes of globally stable dynamical systems for dual-purpose extraction of principal and minor components are analyzed. The proposed systems may apply to both the standard and the generalized eigenvalue problems. Lyapunov stability theory and LaSalle invariance principle are used to derive invariant sets for these systems. Some of these systems may be viewed as generalizations of known learning rules such as Ojas and Xus systems and are shown to be applied, with some modifications, to symmetric and nonsymmetric matrices. Numerical examples are provided to examine the convergence behavior of the dual-purpose minor and principal component analyzers.

Constrained gradient descent and line search for solving optimization problem with elliptic constraints

Finding global minima and maxima of constrained optimization problems is an important task in engineering applications and scientific computation. In this paper, the necessary conditions of optimality will be solved sequentially using a combination of gradient descent and exact or approximate line search. The optimality conditions are enforced at each step while optimizing along the direction of the gradient of the Lagrangian of the problem. Among many applications, this paper proposes learning algorithms which extract adaptively reduced rank canonical variates and correlations, reduced rank Wiener filter, and principal and minor components within similar framework.

Rational invariant subspace approximations with applications

Subspace methods such as MUSIC, minimum norm, and ESPRIT have gained considerable attention due to their superior performance in sinusoidal and direction-of-arrival (DOA) estimation, but they are also known to be of high computational cost. In this paper, new fast algorithms for approximating signal and noise subspaces and that do not require exact eigendecomposition are presented. These algorithms approximate the required subspace using rational and power-like methods applied to the direct data or the sample covariance matrix. Several ESPRIT- as well as MUSIC-type methods are developed based on these approximations. A substantial computational saving can be gained comparing with those associated with the eigendecomposition-based methods. These methods are demonstrated to have performance comparable to that of MUSIC yet will require fewer computations to obtain the signal subspace matrix.

DOA and frequency estimation using fast subspace algorithms

Abstract Eigendecomposition-based methods such as MUSIC, minimum norm, and ESPRIT estimators are popular for their high resolution property in sinusoidal and direction of arrival (DOA) estimation but they are also known to be of high computational demand. In this paper, new fast and robust algorithms for DOA frequency estimation are presented. These algorithms approximate the required signal and noise subspace using rational and power-like methods applied to the sample covariance matrix and have been shown to be applicable to ESPRIT-type as well as MUSIC-type methods. It is also shown that a substantial computational saving would be gained compared to those associated with the eigendecomposition-based methods. Simulations results show that these approximated estimators have comparable performance at low signal-to-noise ratio (SNR) to their standard counterparts and are robust against overestimating the number of impinging signals.

Low-rank approximations with applications to principal singular component learning systems

In this paper, we present several dynamical systems for efficient and accurate computation of optimal low rank approximation of a real matrix. The proposed dynamical systems are gradient flows or weighted gradient flows derived from unconstrained optimization of certain objective functions. These systems are then modified to obtain power-like methods for computing a few dominant singular triplets of very large matrices simultaneously rather than just one at a time, by incorporating upper-triangular and diagonal matrices. The validity of the proposed algorithms was demonstrated through numerical experiments.