Jennifer B. Erway

Randomized SVD methods in hyperspectral imaging

We present a randomized singular value decomposition (rSVD) method for the purposes of lossless compression, reconstruction, classification, and target detection with hyperspectral (HSI) data. Recent work in low-rank matrix approximations obtained from random projections suggests that these approximations are well suited for randomized dimensionality reduction. Approximation errors for the rSVD are evaluated on HSI, and comparisons aremade to deterministic techniques and as well as to other randomized low-rank matrix approximation methods involving compressive principal component analysis. Numerical tests on real HSI data suggest that the method is promising and is particularly effective for HSI data interrogation.

On Efficiently Computing the Eigenvalues of Limited-Memory Quasi-Newton Matrices

In this paper, we consider the problem of efficiently computing the eigenvalues of limited-memory quasi-Newton matrices that exhibit a compact formulation. In addition, we produce a compact formula for quasi-Newton matrices generated by any member of the Broyden convex class of updates. Our proposed method makes use of efficient updates to the QR factorization that substantially reduces the cost of computing the eigenvalues after the quasi-Newton matrix is updated. Numerical experiments suggest that the proposed method is able to compute eigenvalues to high accuracy. Applications for this work include modified quasi-Newton methods and trust-region methods for large-scale optimization, the efficient computation of condition numbers and singular values, and sensitivity analysis.

Algorithm 943: MSS: MATLAB Software for L-BFGS Trust-Region Subproblems for Large-Scale Optimization

A MATLAB implementation of the Moré-Sorensen sequential (MSS) method is presented. The MSS method computes the minimizer of a quadratic function defined by a limited-memory BFGS matrix subject to a two-norm trust-region constraint. This solver is an adaptation of the Moré-Sorensen direct method into an L-BFGS setting for large-scale optimization. The MSS method makes use of a recently proposed stable fast direct method for solving large shifted BFGS systems of equations [Erway and Marcia 2012; Erway et al. 2012] and is able to compute solutions to any user-defined accuracy. This MATLAB implementation is a matrix-free iterative method for large-scale optimization. Numerical experiments on the CUTEr [Bongartz et al. 1995; Gould et al. 2003]) suggest that using the MSS method as a trust-region subproblem solver can require significantly fewer function and gradient evaluations needed by a trust-region method as compared with the Steihaug-Toint method.

On solving L-SR1 trust-region subproblems

In this article, we consider solvers for large-scale trust-region subproblems when the quadratic model is defined by a limited-memory symmetric rank-one (L-SR1) quasi-Newton matrix. We propose a solver that exploits the compact representation of L-SR1 matrices. Our approach makes use of both an orthonormal basis for the eigenspace of the L-SR1 matrix and the Sherman–Morrison–Woodbury formula to compute global solutions to trust-region subproblems. To compute the optimal Lagrange multiplier for the trust-region constraint, we use Newton’s method with a judicious initial guess that does not require safeguarding. A crucial property of this solver is that it is able to compute high-accuracy solutions even in the so-called hard case. Additionally, the optimal solution is determined directly by formula, not iteratively. Numerical experiments demonstrate the effectiveness of this solver.

Shifted L-BFGS systems

We investigate fast direct methods for solving systems of the form (B+G)x=y, where B is a limited-memory Broyden-Fletcher-Goldfarb-Shanno matrix and G is a symmetric positive-definite matrix. These systems, which we refer to as shifted L-BFGS systems, arise in several settings, including trust-region methods and preconditioning techniques for interior-point methods. We show that under mild assumptions, the system (B+G)x=y can be solved in an efficient manner that mitigates instability via a recursion that requires only vector inner products. We consider various shift matrices G and demonstrate the effectiveness and accuracy of the recursion method in numerical experiments.

Shifted limited-memory DFP systems

Quasi-Newton methods are optimization techniques suitable for large data-generated problems that are subject to errors and uncertainty since these methods only use first-order information, do not require storing large matrices, and are easily scalable to large problems. Generally speaking, because they make use of multiple updates to Hessian approximations, these methods enjoy faster convergence rates on nonconvex problems than other first-order methods. In this paper, we consider Davidon-Fletcher-Powell (DFP) quasi-Newton matrices. We derive a compact formulation of DFP updates and propose a method to solve shifted DFP quasi-Newton systems of the form (B ; σI)x = y where MDFP = B-1 is the DFP matrix approximation to the inverse Hessian and σ > 0 is a positive scalar. This paper extends work done in [1] and [2] on general quasi-Newton matrices (and, in particular, L-BFGS matrices) to DFP matrices. Finally, we generalize this method to solve systems of the form (B;G)x = y, where G is a symmetric positive-definite matrix.

Limited-memory trust-region methods for sparse relaxation

In this paper, we solve the ℓ2-ℓ1 sparse recovery problem by transforming the objective function of this problem into an unconstrained differentiable function and applying a limited-memory trust-region method. Unlike gradient projection-type methods, which uses only the current gradient, our approach uses gradients from previous iterations to obtain a more accurate Hessian approximation. Numerical experiments show that our proposed approach eliminates spurious solutions more effectively while improving computational time.

Compact representation of the full Broyden class of quasi-Newton updates: Compact Representation of the Full Broyden Class of Quasi-Newton Updates

In this paper, we present the compact representation for matrices belonging to the the Broyden class of quasi-Newton updates, where each update may be either rank-one or rank-two. This work extends previous results solely for the restricted Broyden class of rank-two updates. In this article, it is not assumed the same Broyden update is used each iteration; rather, different members of the Broyden class may be used each iteration. Numerical experiments suggest that a practical implementation of the compact representation is able to accurately represent matrices belonging to the Broyden class of updates. Furthermore, we demonstrate how to compute the compact representation for the inverse of these matrices, as well as a practical algorithm for solving linear systems with members of the Broyden class of updates. We demonstrate through numerical experiments that the proposed linear solver is able to efficiently solve linear systems with members of the Broyden class of matrices to high accuracy. As an immediate consequence of this work, it is now possible to efficiently compute the eigenvalues of any limited-memory member of the Broyden class of matrices, allowing for the computation of condition numbers and the ability perform sensitivity analysis.