Gene H. Golub

Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter

Consider the ridge estimate (λ) for β in the model unknown, (λ) = (X T X + nλI)−1 X T y. We study the method of generalized cross-validation (GCV) for choosing a good value for λ from the data. The estimate is the minimizer of V(λ) given by where A(λ) = X(X T X + nλI)−1 X T . This estimate is a rotation-invariant version of Allens PRESS, or ordinary cross-validation. This estimate behaves like a risk improvement estimator, but does not require an estimate of σ2, so can be used when n − p is small, or even if p ≥ 2 n in certain cases. The GCV method can also be used in subset selection and singular value truncation methods for regression, and even to choose from among mixtures of these methods.

Singular value decomposition and least squares solutions

Let A be a real m×n matrix with m≧n. It is well known (cf. [4]) that

Calculating the Singular Values and Pseudo-Inverse of a Matrix

A = U\sum {V^T}

A Nonlinear Primal-Dual Method for Total Variation-Based Image Restoration

(1) where

On Direct Methods for Solving Poisson’s Equations

{U^T}U = {V^T}V = V{V^T} = {I_n}{\text{ and }}\sum {\text{ = diag(}}{\sigma _{\text{1}}}{\text{,}} \ldots {\text{,}}{\sigma _n}{\text{)}}{\text{.}}

A Hessenberg-Schur method for the problem AX + XB= C

The matrix U consists of n orthonormalized eigenvectors associated with the n largest eigenvalues of AA T , and the matrix V consists of the orthonormalized eigenvectors of A T A. The diagonal elements of ∑ are the non-negative square roots of the eigenvalues of A T A; they are called singular values. We shall assume that