Clifford H. Thurber

Nonlinear Inverse Problems

Synopsis The nonlinear regression approaches of Chapter 9 are generalized to problems requiring regularization. The Tikhonov regularization and Occams inversion approaches are introduced. Seismic tomography and electrical conductivity inversion examples are used to illustrate the application of these methods. Resolution analysis for nonlinear problems is addressed. We introduce the nonlinear conjugate gradient method for solving large systems of nonlinear equations. The discrete adjoint method is described and illustrated with a regularized heat flow example that utilizes the nonlinear conjugate gradient method in its solution.

Rank Deficiency and Ill-Conditioning

The characteristics of rank-deficient and ill-conditioned linear systems of equations are explored using the singular value decomposition. The connection between model and data null spaces and solution uniqueness and ability to fit data is examined. Model and data resolution matrices are defined. The relationship between singular value size and singular vector roughness and its connection to the effect of noise on solutions are discussed in the context of the fundamental trade-off between model resolution and instability. Specific manifestations of these issues in rank-deficient and ill-conditioned discrete problems are shown in several examples.

Discretizing Problems Using Basis Functions

Techniques for discretizing continuous inverse problems characterized by Fredholm integral equations of the first kind using continuous basis functions are discussed, both for general basis functions and for representers. The Gram matrix is defined and used in analyzing these problems. The method of Backus and Gilbert is also introduced.

Additional Regularization Techniques

Alternatives or adjuncts to Tikhonov regularization are introduced. Bounds constraints allow the use of prior knowledge on the permissible range of parameter values. Sparsity regularization selects solutions with the minimum number of nonzero model parameters. A modification of the iteratively reweighted least squares algorithm presented in Chapter 2 for 1-norm parameter estimation is introduced to solve these problems. In compressive sensing, sparsity regularization is applied in association with a change of basis, where the basis is chosen so that the desired model will be sparse (i.e., have only a few nonzero coefficients in the model expansion). Total variation regularization uses a regularization term based on the 1-norm of the model gradient that allows for discontinuous jumps in the model to bias the solution towards piecewise-constant functions.