Ricardo D. Fierro

Regularization by Truncated Total Least Squares

The total least squares (TLS) method is a successful method for noise reduction in linear least squares problems in a number of applications. The TLS method is suited to problems in which both the coefficient matrix and the right-hand side are not precisely known. This paper focuses on the use of TLS for solving problems with very ill-conditioned coefficient matrices whose singular values decay gradually (so-called discrete ill-posed problems), where some regularization is necessary to stabilize the computed solution. We filter the solution by truncating the small singular values of the TLS matrix. We express our results in terms of the singular value decomposition (SVD) of the coefficient matrix rather than the augmented matrix. This leads to insight into the filtering properties of the truncated TLS method as compared to regularized least squares solutions. In addition, we propose and test an iterative algorithm based on Lanczos bidiagonalization for computing truncated TLS solutions.

Low-Rank Orthogonal Decompositions for Information Retrieval Applications

Current methods to index and retrieve documents from databases usually depend on a lexical match between query terms and keywords extracted from documents in a database. These methods can produce incomplete or irrelevant results due to the use of synonyms and polysemus words. The association of terms with documents (or implicit semantic structure) can be derived using large sparse {\it term-by-document} matrices. In fact, both terms and documents can be matched with user queries using representations in k-space (where 100 ≤ k ≤ 200) derived from k of the largest approximate singular vectors of these term-by-document matrices. This completely automated approach called latent semantic indexing or LSI, uses subspaces spanned by the approximate singular vectors to encode important associative relationships between terms and documents in k-space. Using LSI, two or more documents may be closeto each other in k-space (and hence meaning) yet share no common terms. The focus of this work is to demonstrate the computational advantages of exploiting low-rank orthogonal decompositions such as the ULV (or URV) as opposed to the truncated singular value decomposition (SVD) for the construction of initial and updated rank-k subspaces arising from LSI applications.

Collinearity and Total Least Squares

The least squares (LS) and total least squares (TLS) methods are commonly used to solve the overdetermined system of equations

UTV Tools: Matlab templates for rank-revealing UTV decompositions

Ax \approx b

Low-rank revealing UTV decompositions

. The main objective of this paper is to examine TLS when

Perturbation theory for orthogonal projection methods with applications to least squares and total least squares

A

Lanczos and the Riemannian SVD in information retrieval applications

is nearly rank deficient by outlining its differences and similarities to the well-known truncated LS method. It is shown that TLS may be viewed as a regularization technique much like truncated LS, even though the rank reduction depends on

Perturbation Analysis For Two-Sided (or Complete) Orthogonal Decompositions

b

UTV Expansion Pack: Special-Purpose Rank-Revealing Algorithms

. The sensitivity of LS and TLS approximate nullspaces to perturbations in the data is also examined. Some numerical simulations are included.

Truncated VSV solutions to symmetric rank-deficient problems

We describe a Matlab 5.2 package for computing and modifying certain rank-revealing decompositions that have found widespread use in signal processing and other applications. The package focuses on algorithms for URV and ULV decompositions, collectively known as UTV decompositions. We include algorithms for the ULLV decomposition, which generalizes the ULV decomposition to a pair of matrices. For completeness a few algorithms for computation of the RRQR decomposition are also included. The software in this package can be used as is, or can be considered as templates for specialized implementations on signal processors and similar dedicated hardware platforms.