Nils Papenberg

Highly Accurate Optic Flow Computation with Theoretically Justified Warping

In this paper, we suggest a variational model for optic flow computation based on non-linearised and higher order constancy assumptions. Besides the common grey value constancy assumption, also gradient constancy, as well as the constancy of the Hessian and the Laplacian are proposed. Since the model strictly refrains from a linearisation of these assumptions, it is also capable to deal with large displacements. For the minimisation of the rather complex energy functional, we present an efficient numerical scheme employing two nested fixed point iterations. Following a coarse-to-fine strategy it turns out that there is a theoretical foundation of so-called warping techniques hitherto justified only on an experimental basis. Since our algorithm consists of the integration of various concepts, ranging from different constancy assumptions to numerical implementation issues, a detailed account of the effect of each of these concepts is included in the experimental section. The superior performance of the proposed method shows up by significantly smaller estimation errors when compared to previous techniques. Further experiments also confirm excellent robustness under noise and insensitivity to parameter variations.

A Survey on Variational Optic Flow Methods for Small Displacements

Optic fow describes the displacement field in an image sequence. Its reliable computation constitutes one of the main challenges in computer vision, and variational methods belong to the most successful techniques for achieving this goal. Variational methods recover the optic flow field as a minimiser of a suitable energy functional that involves data and smoothness terms. In this paper we present a survey on different model assumptions for each of these terms and illustrate their impact by experiments. We restrict ourselves to rotationally invariant convex functionals with a linearised data term. Such models are appropriate for small displacements. Regarding the data term, constancy assumptions on the brightness, the gradient, the Hessian, the gradient magnitude, the Laplacian, and the Hessian determinant are investigated. Local integration and nonquadratic penalisation are considered in order to improve robustness under noise. With respect to the smoothness term, we review a recent taxonomy that links regularisers to diffusion processes. It allows to distinguish five types of regularisation strategies: homogeneous, isotropic image-driven, anisotropic image-driven, isotropic flow-driven, and anisotropic flow-driven. All these regularisations can be performed either in the spatial or the spatiotemporal domain. After discussing well-posedness results for convex optic flow functionals, we sketch some numerical ideas in order to achieve realtime performance on a standard PC by means of multigrid methods, and we survey a simple and intuitive confidence measure.

Image Registration for CT and intra-operative ultrasound data of the liver

The paper is concerned with image registration algorithms for the alignment of computer tomography (CT) and 3D-ultrasound (US) images of the liver. The necessity of registration arises from the surgeons request to benefit from the planning data during surgery. The goal is to align the planning data, derived from pre-operative CT-images, with the current US-images of the liver acquired during the surgery. The registration task is complicated by the fact, that the images are of a different modality, that the US-images are severely corrupted by noise, and that the surgeon is looking for a fast and robust scheme. To guide and support the registration, additional pairs of corresponding landmarks are prepared. We will present two different approaches for registration. The first one is based on the pure alignment of the landmarks using thin plate splines. It has been successfully applied in various applications and is now transmitted to liver surgery. In the second approach, we mix a volumetric distance measure with the landmark interpolation constraints. In particular, we investigate the promising normalized gradient field distance measure. We use data from actual liver surgery to illustrate the applicability and the characteristics of both approaches. It turns out that both approaches are suitable for the registration of multi-modal images of the liver.

Matching CT and ultrasound data of the liver by landmark constrained image registration

In navigated liver surgery the key challenge is the registration of pre-operative planing and intra-operative navigation data. Due to the patients individual anatomy the planning is based on segmented, pre-operative CT scans whereas ultrasound captures the actual intra-operative situation. In this paper we derive a novel method based on variational image registration methods and additional given anatomic landmarks. For the first time we embed the landmark information as inequality hard constraints and thereby allowing for inaccurately placed landmarks. The yielding optimization problem allows to ensure the accuracy of the landmark fit by simultaneous intensity based image registration. Following the discretize-then-optimize approach the overall problem is solved by a generalized Gauss-Newton-method. The upcoming linear system is attacked by the MinRes solver. We demonstrate the applicability of the new approach for clinical data which lead to convincing results.

Robust and staining-invariant elastic registration of a series of images from histologic slices

In image registration of medical data a common and challenging problem is handling intensity-inhomogeneities. These inhomogeneities appear for instance in images of serially sectioned brains caused by the histological staining process or in medical imaging with contrast agents. Beneath this, natural outliers (for instance cells or vessels) produced by the underlying material itself may be mistaken as noise. Both image registration applications have in common that the well known sum of squared differences (SSD) measure would detect false differences. To deal with these kinds of problems, we supplement the common SSD-measure with image derivatives of higher order. Additionally we introduce a non-quadratic penalizer function to the distance measure leading to robust energy. The concepts are well known in optical flow. Overall, we present a variational model which combines all of these properties. This formulation leads to a fast and efficient algorithm. We demonstrate its applicability at the problems described above.

A fully parallel algorithm for multimodal image registration using normalized gradient fields

We present a super fast variational algorithm for the challenging problem of multimodal image registration. It is capable of registering full-body CT and PET images in about a second on a standard CPU with virtually no memory requirements. The algorithm is founded on a Gauss-Newton optimization scheme with specifically tailored, mathematically optimized computations for objective function and derivatives. It is fully parallelized and perfectly scalable, thus directly suitable for usage in many-core environments. The accuracy of our method was tested on 21 PET-CT scan pairs from clinical routine. The method was able to correct random distortions in the range from -10 cm to 10 cm translation and from -15° to 15° degree rotation to subvoxel accuracy. In addition, it exhibits excellent robustness to noise.

A Variational Approach for Volume-to-Slice Registration

In this work we present a new variational approach for image registration where part of the data is only known on a low-dimensional manifold. Our work is motivated by navigated liver surgery. Therefore, we need to register 3D volumetric CT data and tracked 2D ultrasound (US) slices. The particular problem is that the set of all US slices does not assemble a full 3D domain. Other approaches use so-called compounding techniques to interpolate a 3D volume from the scattered slices. Instead of inventing new data by interpolation here we only use the given data. Our variational formulation of the problem is based on a standard approach. We minimize a joint functional made up from a distance term and a regularizer with respect to a 3D spatial deformation field. In contrast to existing methods we evaluate the distance of the images only on the two-dimensional manifold where the data is known. A crucial point here is regularization. To avoid kinks and to achieve a smooth deformation it turns out that at least second order regularization is needed. Our numerical method is based on Newton-type optimization. We present a detailed discretization and give some examples demonstrating the influence of regularization. Finally we show results for clinical data.

Landmark constrained non-parametric image registration with isotropic tolerances

The incorporation of additional user knowledge into a nonrigid registration process is a promising topic in modern registration schemes. The combination of intensity based registration and some interactively chosen landmark pairs is a major approach in this direction. There exist different possibilities to incorporate landmark pairs into a variational non-parametric registration framework. As the interactive localization of point landmarks is always prone to errors, a demand for precise landmark matching is bound to fail. Here, the treatment of the distances of corresponding landmarks as penalties within a constrained optimization problem offers the possibility to control the quality of the matching of each landmark pair individually. More precisely, we introduce inequality constraints, which allow for a sphere-like tolerance around each landmark. We illustrate the performance of this new approach for artificial 2D images as well as for the challenging registration of preoperative CT data to intra-operative 3D ultrasound data of the liver.

Deformable image registration for adaptive radiotherapy with guaranteed local rigidity constraints

BackgroundDeformable image registration (DIR) is a key component in many radiotherapy applications. However, often resulting deformations are not satisfying, since varying deformation properties of different anatomical regions are not considered. To improve the plausibility of DIR in adaptive radiotherapy in the male pelvic area, this work integrates a local rigidity deformation model into a DIR algorithm.MethodsA DIR framework is extended by constraints, enforcing locally rigid deformation behavior for arbitrary delineated structures. The approach restricts those structures to rigid deformations, while surrounding tissue is still allowed to deform elastically. The algorithm is tested on ten CT/CBCT male pelvis datasets with active rigidity constraints on bones and prostate and compared to the Varian SmartAdapt deformable registration (VSA) on delineations of bladder, prostate and bones.ResultsThe approach with no rigid structures (REG0) obtains an average dice similarity coefficient (DSC) of 0.87 ± 0.06 and a Hausdorff-Distance (HD) of 8.74 ± 5.95 mm. The new approach with rigid bones (REG1) yields a DSC of 0.87 ± 0.07, HD 8.91 ± 5.89 mm. Rigid deformation of bones and prostate (REG2) obtains 0.87 ± 0.06, HD 8.73 ± 6.01 mm, while VSA yields a DSC of 0.86 ± 0.07, HD 10.22 ± 6.62 mm. No deformation grid foldings are observed for REG0 and REG1 in 7 of 10 cases; for REG2 in 8 of 10 cases, with no grid foldings in prostate, an average of 0.08 % in bladder (REG2: no foldings) and 0.01 % inside the body contour. VSA exhibits grid foldings in each case, with an average percentage of 1.81 % for prostate, 1.74 % for bladder and 0.12 % for the body contour. While REG1 and REG2 keep bones rigid, elastic bone deformations are observed with REG0 and VSA. An average runtime of 26.2 s was achieved with REG1; 31.1 s with REG2, compared to 10.5 s with REG0 and 10.7 s with VMS.ConclusionsWith accuracy in the range of VSA, the new approach with constraints delivers physically more plausible deformations in the pelvic area with guaranteed rigidity of arbitrary structures. Although the algorithm uses an advanced deformation model, clinically feasible runtimes are achieved.

Registrierung im Fokus

In vielen praktischen Problemstellungen ist der Anwender nur in wenigen ausgezeichneten Bildbereichen an einer hochgenauen Registrierung interessiert. Dieser Umstand wird in der vorliegenden Arbeit konsequent umgesetzt. Es wird eine Multiresolutionsstrategie vorgestellt, die es dem Anwender erstmalig erlaubt, auf ausgewahlte Bildbereiche zu fokussieren. Das Verfahren ist in einen variationellen Kontext eingebettet und bietet einen deutlichen Geschwindigkeitsvorteil gegenuber herkommlichen Methoden. Neben der Herleitung wird die Wirkungsweise des Verfahrens beispielhaft illustriert und die Qualitat der Ergebnisse diskutiert. Es zeigt sich, dass dieser neue Ansatz den problemangepassten Einsatz variationeller Methoden in zeitkritischen Anwendungen erlaubt.