David Titley-Peloquin

Stopping Criteria for the Iterative Solution of Linear Least Squares Problems

We explain an interesting property of minimum residual iterative methods for the solution of the linear least squares (LS) problem. Our analysis demonstrates that the stopping criteria commonly used with these methods can in some situations be too conservative, causing any chosen method to perform too many iterations or even fail to detect that an acceptable iterate has been obtained. We propose a less conservative criterion to determine whether a given iterate is an acceptable LS solution. This is merely a sufficient condition, but it approaches a necessary condition in the limit as the given iterate approaches the exact LS solution. We also propose a necessary and sufficient condition to determine whether a given approximate LS solution is an acceptable LS solution, based on recent results on backward perturbation analysis of the LS problem. Although both of the above new conditions use quantities that are too expensive to compute in practical situations, we suggest potential approaches for estimating some of these quantities efficiently. We illustrate our results with several numerical examples.

Sensitivity and Conditioning of the Truncated Total Least Squares Solution

We present an explicit expression for the condition number of the truncated total least squares (TLS) solution of

Estimating the Backward Error in LSQR

Ax \approx b

Backward perturbation analysis for scaled total least-squares problems

. This expression is obtained using the notion of the Frechet derivative. We also give upper bounds on the condition number, which are simple to compute and interpret. These results generalize those in the literature for the untruncated TLS problem. Numerical experiments demonstrate that our bounds are often a very good estimate of the condition number, and provide a significant improvement to known bounds.

A conjugate gradient like method for p-norm minimization in functional spaces

We propose practical stopping criteria for the iterative solution of sparse linear least squares (LS) problems. Although we focus our discussion on the algorithm LSQR of Paige and Saunders, the ideas discussed here may also be applicable to other algorithms. We review why the 2-norm of the projection of the residual vector onto the range of

Differentiating the Method of Conjugate Gradients

A

On the Accuracy of the Karlson--Waldén Estimate of the Backward Error for Linear Least Squares Problems

is a useful measure of convergence, and we show how this projection can be estimated efficiently at every iteration of LSQR. We also give practical and cheaply computable estimates of the backward error for the LS problem.

Characterizing Matrices That Are Consistent with Given Solutions

The scaled total least-squares (STLS) method unifies the ordinary least-squares (OLS), the total least-squares (TLS), and the data least-squares (DLS) methods. In this paper we perform a backward perturbation analysis of the STLS problem. This also unifies the backward perturbation analyses of the OLS, TLS and DLS problems. We derive an expression for an extended minimal backward error of the STLS problem. This is an asymptotically tight lower bound on the true minimal backward error. If the given approximate solution is close enough to the true STLS solution (as is the goal in practice), then the extended minimal backward error is in fact the minimal backward error. Since the extended minimal backward error is expensive to compute directly, we present a lower bound on it as well as an asymptotic estimate for it, both of which can be computed or estimated more efficiently. Our numerical examples suggest that the lower bound gives good order of magnitude approximations, while the asymptotic estimate is an excellent estimate. We show how to use our results to easily obtain the corresponding results for the OLS and DLS problems in the literature. Copyright

Visualizing and Constructing Cycles in the Simplex Method

We develop an iterative algorithm to recover the minimum p-norm solution of the functional linear equation