Fabrice Heitz

Multimodal estimation of discontinuous optical flow using Markov random fields

The estimation of dense velocity fields from image sequences is basically an ill-posed problem, primarily because the data only partially constrain the solution. It is rendered especially difficult by the presence of motion boundaries and occlusion regions which are not taken into account by standard regularization approaches. In this paper, the authors present a multimodal approach to the problem of motion estimation in which the computation of visual motion is based on several complementary constraints. It is shown that multiple constraints can provide more accurate flow estimation in a wide range of circumstances. The theoretical framework relies on Bayesian estimation associated with global statistical models, namely, Markov random fields. The constraints introduced here aim to address the following issues: optical flow estimation while preserving motion boundaries, processing of occlusion regions, fusion between gradient and feature-based motion constraint equations. Deterministic relaxation algorithms are used to merge information and to provide a solution to the maximum a posteriori estimation of the unknown dense motion field. The algorithm is well suited to a multiresolution implementation which brings an appreciable speed-up as well as a significant improvement of estimation when large displacements are present in the scene. Experiments on synthetic and real world image sequences are reported. >

Discrete Markov image modeling and inference on the quadtree

Noncasual Markov (or energy-based) models are widely used in early vision applications for the representation of images in high-dimensional inverse problems. Due to their noncausal nature, these models generally lead to iterative inference algorithms that are computationally demanding. In this paper, we consider a special class of nonlinear Markov models which allow one to circumvent this drawback. These models are defined as discrete Markov random fields (MRF) attached to the nodes of a quadtree. The quadtree induces causality properties which enable the design of exact, noniterative inference algorithms, similar to those used in the context of Markov chain models. We first introduce an extension of the Viterbi algorithm which enables exact maximum a posteriori (MAP) estimation on the quadtree. Two other algorithms, related to the MPM criterion and to Bouman and Shapiros (1994) sequential-MAP (SMAP) estimator are derived on the same hierarchical structure. The estimation of the model hyper parameters is also addressed. Two expectation-maximization (EM)-type algorithms, allowing unsupervised inference with these models are defined. The practical relevance of the different models and inference algorithms is investigated in the context of image classification problem, on both synthetic and natural images.

Automatic change detection in multimodal serial MRI: application to multiple sclerosis lesion evolution

The automatic analysis of subtle changes between MRI scans is an important tool for assessing disease evolution over time. Manual labeling of evolutions in 3D data sets is tedious and error prone. Automatic change detection, however, remains a challenging image processing problem. A variety of MRI artifacts introduce a wide range of unrepresentative changes between images, making standard change detection methods unreliable. In this study we describe an automatic image processing system that addresses these issues. Registration errors and undesired anatomical deformations are compensated using a versatile multiresolution deformable image matching method that preserves significant changes at a given scale. A nonlinear intensity normalization method is associated with statistical hypothesis test methods to provide reliable change detection. Multimodal data is optionally exploited to reduce the false detection rate. The performance of the system was evaluated on a large database of 3D multimodal, MR images of patients suffering from relapsing remitting multiple sclerosis (MS). The method was assessed using receiver operating characteristics (ROC) analysis, and validated in a protocol involving two neurologists. The automatic system outperforms the human expert, detecting many lesion evolutions that are missed by the expert, including small, subtle changes.

3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization

This paper deals with topology preservation in three-dimensional (3-D) deformable image registration. This work is a nontrivial extension of , which addresses the case of two-dimensional (2-D) topology preserving mappings. In both cases, the deformation map is modeled as a hierarchical displacement field, decomposed on a multiresolution B-spline basis. Topology preservation is enforced by controlling the Jacobian of the transformation. Finding the optimal displacement parameters amounts to solving a constrained optimization problem: The residual energy between the target image and the deformed source image is minimized under constraints on the Jacobian. Unlike the 2-D case, in which simple linear constraints are derived, the 3-D B-spline-based deformable mapping yields a difficult (until now, unsolved) optimization problem. In this paper, we tackle the problem by resorting to interval analysis optimization techniques. Care is taken to keep the computational burden as low as possible. Results on multipatient 3-D MRI registration illustrate the ability of the method to preserve topology on the continuous image domain.

Affine-invariant geometric shape priors for region-based active contours

We present a new way of constraining the evolution of a region-based active contour with respect to a reference shape. Minimizing a shape prior, defined as a distance between shape descriptors based on the Legendre moments of the characteristic function, leads to a geometric flow that can be used with benefits in a two-class segmentation application. The shape model includes intrinsic invariance with regard to pose and affine deformations

A hierarchical Markov modeling approach for the segmentation and tracking of deformable shapes

In many applications of dynamic scene analysis, the objects or structures to be analyzed undergo deformations that have to be modeled. In this paper, we develop a hierarchical statistical modeling framework for the representation, segmentation, and tracking of 2D deformable structures in image sequences. The model relies on the specification of a template, on which global as well as local deformations are defined. Global deformations are modeled using a statistical modal analysis of the deformations observed on a representative population. Local deformations are represented by a (first-order) Markov random process. A model-based segmentation of the scene is obtained by a joint bayesian estimation of global deformation parameters and local deformation variables. Spatial or spatio-temporal observations are considered in this estimation procedure, yielding an edge-based or a motion-based segmentation of the scene. The segmentation procedure is combined with a temporal tracking of the deformable structure over long image sequences, using a Kalman filtering approach. This combined segmentation-tracking procedure has produced reliable extraction of deformable parts from long image sequences in adverse situations such as low signal-to-noise ratio, nongaussian noise, partial occlusions, or random initialization. The approach is demonstrated on a variety of synthetic as well as real-world image sequences featuring different classes of deformable objects.

Registration of MR/MR and MR/SPECT Brain Images by Fast Stochastic Optimization of Robust Voxel Similarity Measures

This paper describes a robust, fully automated algorithm to register intrasubject 3D single and multimodal images of the human brain. The proposed technique accounts for the major limitations of the existing voxel similarity-based methods: sensitivity of the registration to local minima of the similarity function and inability to cope with gross dissimilarities in the two images to be registered. Local minima are avoided by the implementation of a stochastic iterative optimization technique (fast simulated annealing). In addition, robust estimation is applied to reject outliers in case the images show significant differences (due to lesion evolution, incomplete acquisition, non-Gaussian noise, etc.). In order to evaluate the performance of this technique, 2D and 3D MR and SPECT human brain images were artificially rotated, translated, and corrupted by noise. A test object was acquired under different angles and positions for evaluating the accuracy of the registration. The approach has also been validated on real multiple sclerosis MR images of the same patient taken at different times. Furthermore, robust MR/SPECT image registration has permitted the representation of functional features for patients with partially complex seizures. The fast simulated annealing algorithm combined with robust estimation yields registration errors that are less than 1 degree in rotation and less than 1 voxel in translation (image dimensions of 128(3)). It compares favorably with other standard voxel similarity-based approaches.

Retrospective evaluation of a topology preserving non-rigid registration method

This paper proposes a comprehensive evaluation of a monomodal B-spline-based non-rigid registration algorithm allowing topology preservation in 3-D. This article is to be considered as the companion of [Noblet, V., Heinrich, C., Heitz, F., Armspach, J.-P., 2005. 3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization. IEEE Transactions on Image Processing, 14 (5), 553-566] where this algorithm, based on the minimization of an objective function, was introduced and detailed. Overall assessment is based on the estimation of synthetic deformation fields, on average brain construction, on atlas-based segmentation and on landmark mapping. The influence of the model parameters is characterized. Comparison between several objective functions is carried out and impact of their symmetrization is pointed out. An original intensity normalization scheme is also introduced, leading to significant improvements of the registration quality. The comparison benchmark is the popular demons algorithm [Thirion, J.-P., 1998. Image matching as a diffusion process: an analogy with Maxwells demons. Medical Image Analysis, 2 (3), 243-260], that exhibited best results in a recent comparison between several non-rigid 3-D registration methods [Hellier, P., Barillot, C., Corouge, I., Gibaud, B., Le Goualher, G., Collins, D.L., Evans, A., Malandain, G., Ayache, N., Christensen, G.E., Johnson, H.J., 2003. Retrospective evaluation of intersubject brain registration. IEEE Transactions on Medical Imaging, 22 (9), 1120-1130]. The topology preserving B-spline-based method proved to outperform the commonly available ITK implementation of the demons algorithms on many points. Some limits of intensity-based registration methods are also highlighted through this work.

Statistical deformable model-based segmentation of image motion

We present a statistical method for the motion-based segmentation of deformable structures undergoing nonrigid movements. The proposed approach relies on two models describing the shape of interest, its variability, and its movement. The first model corresponds to a statistical deformable template that constrains the shape and its deformations. The second model is introduced to represent the optical flow field inside the deformable template. These two models are combined within a single probability distribution, which enables to derive shape and motion estimates using a maximum likelihood approach. The method requires no manual initialization and is demonstrated on synthetic data and on a medical X-ray image sequence.

Multi-Reference Shape Priors for Active Contours

In this paper, we present a new way of constraining the evolution of an active contour with respect to a set of fixed reference shapes. This approach is based on a description of shapes by the Legendre moments computed from their characteristic function. This provides a region-based representation that can handle arbitrary shape topologies. Moreover, exploiting the properties of moments, it is possible to include intrinsic affine invariance in the descriptor, which solves the issue of shape alignment without increasing the number of d.o.f. of the initial problem and allows introducing geometric shape variabilities. Our new shape prior is based on a distance, in terms of descriptors, between the evolving curve and the reference shapes. Minimizing the corresponding shape energy leads to a geometric flow that does not rely on any particular representation of the contour and can be implemented with any contour evolution algorithm. We introduce our prior into a two-class segmentation functional, showing its benefits on segmentation results in presence of severe occlusions and clutter. Examples illustrate the ability of the model to deal with large affine deformation and to take into account a set of reference shapes of different topologies.