Eldad Haber

On optimization techniques for solving nonlinear inverse problems

This paper considers optimization techniques for the solution of nonlinear inverse problems where the forward problems, like those encountered in electromagnetics, are modelled by differential equations. Such problems are often solved by utilizing a Gauss-Newton method in which the forward model constraints are implicitly incorporated. Variants of Newtons method which use second-derivative information are rarely employed because their perceived disadvantage in computational cost per step offsets their potential benefits of faster convergence. In this paper we show that, by formulating the inversion as a constrained or unconstrained optimization problem, and by employing sparse matrix techniques, we can carry out variants of sequential quadratic programming and the full Newton iteration with only a modest additional cost. By working with the differential equation explicitly we are able to relate the constrained and the unconstrained formulations and discuss the advantages of each. To make the comparisons meaningful we adopt the same global optimization strategy for all inversions. As an illustration, we focus upon a 1D electromagnetic (EM) example simulating a magnetotelluric survey. This problem is sufficiently rich that it illuminates most of the computational complexities that are prevalent in multi-source inverse problems and we therefore describe its solution process in detail. The numerical results illustrate that variants of Newtons method which utilize second-derivative information can produce a solution in fewer iterations and, in some cases where the data contain significant noise, requiring fewer floating point operations than Gauss-Newton techniques. Although further research is required, we believe that the variants proposed here will have a significant impact on developing practical solutions to large-scale 3D EM inverse problems.

JOINT INVERSION : A STRUCTURAL APPROACH

We develop a methodology to invert two different data sets with the assumption that the underlying models have a common structure. Structure is defined in terms of absolute value of curvature of the model and two models are said to have common structure if the changes occur at the same physical locations. The joint inversion is solved by defining an objective function which quantifies the difference in structure between two models, and then minimizing this objective function subject to satisfying the data constraints. The problem is nonlinear and is solved iteratively using Krylov space techniques. Testing the algorithm on synthetic data sets shows that the joint inversion is superior to individual inversions. In an application to field data we show that the data sets are consistent with models that are quite similar.

Intensity Gradient Based Registration and Fusion of Multi-modal Images

OBJECTIVES A particular problem in image registration arises for multi-modal images taken from different imaging devices and/or modalities. Starting in 1995, mutual information has shown to be a very successful distance measure for multi-modal image registration. Therefore, mutual information is considered to be the state-of-the-art approach to multi-modal image registration. However, mutual information has also a number of well-known drawbacks. Its main disadvantage is that it is known to be highly non-convex and has typically many local maxima. METHODS This observation motivates us to seek a different image similarity measure which is better suited for optimization but as well capable to handle multi-modal images. RESULTS In this work, we investigate an alternative distance measure which is based on normalized gradients. CONCLUSIONS As we show, the alternative approach is deterministic, much simpler, easier to interpret, fast and straightforward to implement, faster to compute, and also much more suitable to numerical optimization.

Inversion of 3D electromagnetic data in frequency and time domain using an inexact all-at-once approach

We present a general formulation for inverting frequencyor time-domain electromagnetic data using an all-at-once approach. In this methodology, the forward modeling equations are incorporated as constraints and, thus, we need to solve a constrained optimization problem where the parameters are the electromagnetic fields, the conductivity model, and a set of Lagrange multipliers. This leads to a much larger problem than the traditional unconstrained formulation where only the conductivities are sought. Nevertheless, experience shows that the constrained problem can be solved faster than the unconstrained one. The primary reasons are that the forward problem does not have to be solved exactly until the very end of the optimization process, and that permitting the fields to be away from their constrained values in the initial stages introduces flexibility so that a stationary point of the objective function is found more quickly. In this paper, we outline the all-atonce approach and apply it to electromagnetic problems in both frequency and time domains. This is facilitated by a unified representation for forward modeling for these two types of data. The optimization problem is solved by finding a stationary point of the Lagrangian. Numerically, this leads to a nonlinear system that is solved iteratively using a Gauss-Newton strategy. At each iteration, a large, indefinite matrix is inverted, and we discuss how this can be accomplished. As a test, we invert frequency-domain synthetic data from a grounded electrode system that emulates a field CSAMT survey. For the time domain, we invert borehole data obtained from a current loop on the surface.

Numerical methods for volume preserving image registration

Image registration is one of todays challenging image processing problems, particularly in medical imaging. Since the problem is ill posed, one may like to add additional information about distortions. This applies, for example, to the registration of time series of contrast-enhanced images, where variations of substructures are not related to patient motion but to contrast uptake. Here, one may only be interested in registrations which do not alter the volume of any substructure. In this paper, we discuss image registration techniques with a focus on volume preserving constraints. These constraints can reduce the non-uniqueness of the registration problem significantly. Our implementation is based on a constrained optimization formulation. Upon discretization, we obtain a large, discrete, highly nonlinear optimization problem and the necessary conditions for the solution form a discretized nonlinear partial differential equation. To solve the problem we use a variant of the sequential quadratic programming method. Finally, we present results on synthetic as well as on real-life data.

RESINVM3D: A 3D resistivity inversion package

We have developed an open source 3D, MATLAB based, resistivity inversion package. The forward solution to the governing partial differential equation is efficiently computed using a second-order finite volume discretization coupled with a preconditioned, biconjugate, stabilized gradient algorithm. Using the analytical solution to a potential field in a homogeneous half space, we evaluate the accuracy of our numerical forward solution and, subsequently, develop a source correction factor that reduces forward modeling errors associated with boundary effects and source electrode singularities. For the inversion algorithm we have implemented an inexact Gauss-Newton solver, with the model update being calculated using a preconditioned conjugate gradient algorithm. The inversion uses a combination of zero and first order Tikhonov regularization. Two synthetic examples demonstrate the usefulness of this code. The first example considers a surface resistivity survey with 3813 measurements. The discretized model s...

Fast Finite Volume Simulation of 3D Electromagnetic Problems with Highly Discontinuous Coefficients

We consider solving three-dimensional electromagnetic problems in parameter regimes where the quasi-static approximation applies and the permeability, permittivity, and conductivity may vary significantly. The difficulties encountered include handling solution discontinuities across interfaces and accelerating convergence of traditional iterative methods for the solution of the linear systems of algebraic equations that arise when discretizing Maxwells equations in the frequency domain. The present article extends methods we proposed earlier for constant permeability [E. Haber, U. Ascher, D. Aruliah, and D. Oldenburg, J. Comput. Phys., 163 (2000), pp. 150--171; D. Aruliah, U. Ascher, E. Haber, and D. Oldenburg, Math. Models Methods Appl. Sci., to appear.] to handle also problems in which the permeability is variable and may contain significant jump discontinuities. In order to address the problem of slow convergence we reformulate Maxwells equations in terms of potentials, applying a Helmholtz decomposition to either the electric field or the magnetic field. The null space of the curl operators can then be annihilated by adding a stabilizing term, using a gauge condition, and thus obtaining a strongly elliptic differential operator. A staggered grid finite volume discretization is subsequently applied to the reformulated PDE system. This scheme works well for sources of various types, even in the presence of strong material discontinuities in both conductivity and permeability. The resulting discrete system is amenable to fast convergence of ILU-preconditioned Krylov methods. We test our method using several numerical examples and demonstrate its robust efficiency. We also compare it to the classical Yee method using similar iterative techniques for the resulting algebraic system, and we show that our method is significantly faster, especially for electric sources.

A Multilevel Method for Image Registration

In this paper we introduce a new framework for image registration. Our formulation is based on consistent discretization of the optimization problem coupled with a multigrid solution of the linear system which evolves in a Gauss--Newton iteration. We show that our discretization is

Intensity gradient based registration and fusion of multi-modal images

h

A GCV based method for nonlinear ill-posed problems

-elliptic independent of parameter choice, and therefore a simple multigrid implementation can be used. To overcome potential large nonlinearities and to further speed up computation, we use a multilevel continuation technique. We demonstrate the efficiency of our method on a realistic highly nonlinear registration problem.