Christian Daul

Mosaicing of Bladder Endoscopic Image Sequences: Distortion Calibration and Registration Algorithm

Cancers located on the internal wall of bladders can be detected in image sequences acquired with endoscopes. The clinical diagnosis and follow-up can be facilitated by building a unique panoramic image of the bladder with the images acquired from different viewpoints. This process, called image mosaicing, consists of two steps. In the first step, consecutive images are pairwise registered to find the local transformation matrices linking geometrically consecutive images. In the second step, all images are placed in a common and global coordinate system. In this contribution, a mutual information-based similarity measure and a stochastic gradient optimization method were implemented in the registration process. However, the images have to be preprocessed in order to register the data in a robust way. Thus, a simple correction method of the distortions affecting endoscopic images is presented. After the placement of all images in the global coordinate system, the parameters of the local transformation matrices are all adjusted to improve the visual aspect of the panoramic images. Phantoms are used to evaluate the global mosaicing accuracy and the limits of the registration algorithm. The mean distances between ground truth positions in the mosaiced image range typically in 1-3 pixels. Results given for in vivo patient data illustrate the ability of the algorithm to give coherent panoramic images in the case of bladders.

Fast construction of panoramic images for cystoscopic exploration.

Cystoscopy is used as a reference clinical examination in the detection and visualization of pathological bladder lesions. Evolution observation and analysis of these lesions is easier when panoramic images from internal bladder walls are used instead of video sequences. This work describes a fast and automatic mosaicing algorithm applied to cystoscopic video sequences, where perspective geometric transformations link successive image pairs. This mosaicing algorithm begins with a fast initialization of translation parameters computed by a cross-correlation of images, followed by an iterative optimization of transformation parameters. Finally, registered images are projected onto a global common coordinate system. A quantifying test protocol applied over a phantom yielded a mosaicing mean error lower than 4 pixels for a 1947 x 1187 pixels panoramic image. Qualitative evaluation of 10 panoramic images resulting from videos of clinical cystoscopies was performed. An analysis performed over translation values from these clinical sequences (in vivo) is used to modify the mosaicing algorithm to be able to do a dynamic selection of image pairs. Construction time of panoramic images takes some minutes. At last, algorithm limits are discussed.

Flexible calibration of structured-light systems projecting point patterns

Structured-light systems (SLSs) are widely used in active stereo vision to perform 3D modelling of a surface of interest. We propose a flexible method to calibrate SLSs projecting point patterns. The method is flexible in two respects. First, the calibration is independent of the number of points and their spatial distribution inside the pattern. Second, no positioning device is required since the projector geometry is determined in the camera coordinate system based on unknown positions of the calibration board. The projector optical center is estimated together with the 3D rays originating from the projector using a numerical optimization procedure. We study the 3D point reconstruction accuracy for two SLSs involving a laser based projector and a pico-projector, respectively, and for three point patterns. We finally illustrate the potential of our active vision system for a medical endoscopy application where a 3D cartography of the inspected organ (a large field of view surface also including image textures) can be reconstructed from a video acquisition using the laser based SLS.

From the Hough Transform to a New Approach for the Detection and Approximation of Elliptical Arcs

The edges of simple geometrical (e.g., manufactured) parts can generally be approximated sufficiently accurately by straight-line segments and elliptical arcs in order, for example, to carry out a dimensional analysis of these parts, such as required by inspection tasks. Hough transforms are robust methods for the detection of straight-line segments but are not directly suitable for the detection of elliptical arcs, for which the processing time and memory space necessary are too important. We present in this paper a new method which allows the determination of the parameters of elliptical arcs (or of ellipses) by use of a two-dimensional discretized parameter space defined similarly to the usual Hough space. This new method allows both the detection and characterization of ellipses whose major and minor axis lengths can be as small as four pixels long or of elliptical arcs with a small angular aperture.

Endoscopic bladder image registration using sparse graph cuts

Video endoscopy is one of the standard clinical procedures for visually detecting lesions on the internal wall of human bladders. In order to facilitate the diagnosis, it is helpful to build panoramic maps by registering consecutive images from the video sequence. We show how to efficiently reduce the computation time of graph cut based image registration by an order of magnitude. The number of nodes in a graph is greatly reduced using spatial image properties in order to minimize the loss of information. The set of edges in this sparse graph is obtained by applying a watershed transform on the set of nodes. This graph reduction has negligible negative effects on image registration quality compared to a dense graph cut, so that visually coherent panoramic maps of bladder walls can be built. Results demonstrate that the method improves the registration accuracy and reduces the computation time of other endoscopic bladder image registration methods. This work is an important step towards real time map construction.

Building a color classification system for textured and hue homogeneous surfaces: system calibration and algorithm

Abstract. We present a system for classifying the color aspect of textured surfaces having a nearly constant hue (such as wooden boards, textiles, wallpaper, etc.). The system is designed to compensate for small fluctuations (over time) of the light source and for inhomogeneous illumination conditions (shading correction). This is an important feature because even in industrial environments where the lighting conditions are controlled, a constant and homogeneous illumination cannot be guaranteed. Together with an appropriate camera calibration (which includes a periodic update), our approach offers a robust system which is able to “distinguish” (i.e., classify correctly) between surface classes which exhibit visually barely perceptible color variations. In particular, our approach is based on relative (not absolute) color measurements. In this paper, we outline the classification algorithm while focusing in detail on the camera calibration and a method for compensating for fluctuations of the light source.

A novel 3D surface construction approach: Application to three-dimensional endoscopic data

Video-endoscopy is the standard clinical procedure for visual exploration of internal walls of hollow organs. For the bladder, the lesion diagnosis is complex because the endoscopic images are bi-dimensional and cover only small bladder areas. 3D endoscopes, based on stereoscopic active vision principles, were recently proposed and validated. This paper presents a 3D reconstruction algorithm using 2D texture images and a few 3D points located on the internal wall surfaces provided by such endoscopes. The algorithm constructs a 3D panoramic surface using the 3D reconstruction method guided by 2D image registration. We show on realistic bladder phantoms that the algorithm is able to reconstruct 3D points and surfaces with a sub-millimetre accuracy.

A simplified method of endoscopic image distortion correction based on grey level registration

We present a new method of endoscopic camera calibration for non-linear radial distortion correction. The algorithm implemented computes both projective (camera) and polynomial (distortion) transformations. The optimization process registrates the corrected distorted pattern image with the non-distorted one. Mutual information was used as measure of similarity and stochastic gradient descent method for optimization. The algorithm was tested with two b/w (chessboard, concentric circles) and one grey level patterns, for 3 angular positions of the endoscope (0/spl deg/, 5/spl deg/ and 10/spl deg/ to perpendicular). Convergence time increased with the angle. Maximal mean correction error was less than 0.45 % with optimized distortion parameters calculated for the grey level pattern. Tested inclinations did not have significant effects on errors. Results obtained show the interest of the method proposed that requires only approximative perpendicular positioning of the endoscope and simple grey level calibration patterns without precise geometrical characteristics.

3D sculptured surface analysis using a structured-light approach

The contribution aims at describing a computer-based structured light imaging system to be applied to automated recovery of quantitative 3D information on sculptured surfaces, in order to take in charge (industrial) inspection/3D reconstruction tasks. Recovery is based on evaluation of images of the light pattern induced by projection into the scene of a specifically deviced parallel grid. The system has been designed for direct use in industrial environments, e.g. for integration into on-line quality control systems. Consequently, particular emphasis has been put on efforts for fulfilling requirements usually implied by this type of application, such as simplicity of set-up, application real-time, high accuracy, and low cost. This paper gives a discription of the system realized, including the algorithms specifically designed and implemented for calibration, nonambiguous labeling of the imaged fringes, and subpixel evaluation of their locations. The integration of the system into an on-line inspection system for 100% control of manufactured parts illustrates its application. Inspection is based on comparison of extracted features gained from a CAD model of the part and including tolerance information. Currently, a measurement accuracy of the order of 25 micrometers can be routinely achieved.

Illumination invariant optical flow using neighborhood descriptors

Optical flow approach robust towards illumination changes and texture variability.Accurate multiscale TV-l1 approach for both small and large displacements.Accurate results for very different scenes with constant algorithm parameters.High performance was obtained on the Middlebury, KITTI and MPI Sintel databases.The algorithm enabled mosaicing of endoscopic images under different modalities. Total variational (TV) methods using l1-norm are efficient approaches for optical flow determination. This contribution presents a multi-resolution TV-l1 approach using a data-term based on neighborhood descriptors and a weighted non-local regularizer. The proposed algorithm is robust to illumination changes. The benchmarking of the proposed algorithm is done with three reference databases (Middlebury, KITTI and MPI Sintel). On these databases, the proposed approach exhibits an optimal compromise between robustness, accuracy and computation speed. Numerous tests performed both on complicated data of the reference databases and on challenging endoscopic images acquired under three different modalities demonstrate the robustness and accuracy of the method against the presence of large or small displacements, weak texture information, varying illumination conditions and modality changes.