Jean Triboulet

Three-dimensional heart motion estimation using endoscopic monocular vision system: From artificial landmarks to texture analysis

In robot-assisted beating heart surgery, motion of the heart surface might be virtually stabilized to let the surgeon work as in on-pump cardiac surgery. Virtual stabilization means to compensate physically the relative motion between the instrument tool tip and the region of interest on the heart surface, and to offer surgeon a stable visual display of the scene. To this end, motion of the heart must be estimated. This article focusses on motion estimation of the heart surface. Two approaches are considered in the paper. The first one is based on landmark tracking allowing 3D pose estimation. The second is based on texture tracking. Classical computer vision methods, as well as a new texture-based tracking scheme has been applied to track the heart motion, and, when possible, reconstruct 3D distance to the heart surface. Experimental results obtained on in vivo images show the estimated motion of heart surface points.

Efficient Tracking of the Heart Using Texture

Performing motion tracking in real-time is an old and recurrent problem in computer vision. It has been addressed through a large set of approaches but achieving a high level of robustness is still a challenge, especially with low definition input. In the considered application, tracking the heart motion in endoscopic beating heart sequences, the sensitivity of existing algorithms to visual artifacts and variations in illumination is an issue that calls for improvements. In the prospect of developing a motion compensation architecture for robotically assisted beating heart surgery, we address the problem of visual information retrieval by proposing a new composite tracking algorithm using both template matching and texture analysis. As we will show in this paper, the use of texture characterization of the heart surface improves the overall precision and robustness, in comparison with other prior approaches.

Matching segments in stereoscopic vision

We have shown that its possible to realize a stereoscopic sensor with poor cameras. We developed image processing that is robust and allows us to quickly obtain results for the matching algorithm. We computed an important number of features on each segment, and with these features, we built 16-component vector used in the classification step. After an exhaustive study, we decided to combine two methods, Bayesian and neural, to construct an efficient classifier. The tests for indoor images had better than 90% good matching. With segment couples, it is possible to compute the 3D coordinates of the objects. Therefore, the mobile robot is able to localize and move about in the environment.

3D HEART MOTION ESTIMATION USING ENDOSCOPIC MONOCULAR VISION SYSTEM

Abstract In robotic assisted beating heart surgery, motion of the heart surface might be virtually stabilized to let the surgeon work as in on-pump cardiac surgery. Virtual stabilization means to compensate physically the relative motion between instrument tool tip and point of interest on the heart surface, and to offer surgeon a stable visual display of the scene. In this way, motion of the heart must be estimated. This article focus on motion estimation of heart surface. Classical computer vision method has been applied to reconstruct 3D pose of interest point. Experimental results obtained on in vivo images show the estimated motion of heart surface points.

Comparative study of two 3D reconstruction methods for underwater archaeology

The underwater 3D reconstruction cartography has made great progress in the last decade. The work presented in this paper is about the analysis and 3D reconstruction of archeological objects. Using a calibrated single camera and an uncalibrated system, we propose to describe a method to perform the Euclidian 3D reconstruction of unknown objects. A comparison of two methods is presented and tested on synthetic and real underwater pictures. Filters are proposed to simulate underwater environment and inherent problems. Finally, robust and stable features have been extracted from underwater pictures and used to perform the 3D model.

A camcorder for 3D underwater reconstruction of archeological objects

The underwater cartography has made great progress in the last decade. In this paper, we discuss of the 3D underwater cartography problem and propose a multimodal fusion approach. The work presented in this paper is about the analyze and 3D reconstruction of archeological objects. Using an uncalibrated single camera, we propose to describe a method with enables first, to calibrate this camera integrating variation of the water optical refractive index to the distortion model. And secondly, robust and stable features have been extracted. A comparison of two methods of detecting and matching points is presented. Finally, we estimate the movement between different images and compare two methods.

Highly specific pose estimation with a catadioptric omnidirectional camera

This article presents a new method for estimating the pose of para-catadioptric vision systems. It is based on the estimation of vanishing points associated with vertical edges of the environment. However, unlike classical approaches no feature (line, circle) extraction and/or identification is needed. A sampled domain of possible vanishing points is tested and histograms are build to characterize the soundness of these points. A specificity index allows to find the more relevant histogram and the pose of the sensor. This method has been tested on simulated and real images giving very promising results (maximum angular error of 0.18 degree).

Texture Features Selection for Visual Servoing of the Beating Heart

Texture features are widely used for image classification and retrieval. They offer an efficient way to add prior knowledge in image processing, using an appropriate prior parametrization. In this paper, we present the methods we used to select among numerous existing features the most adapted to deal with beating heart tracking, using an experimental database of images and data mining techniques. Then we introduce the way we plan to use this information to reinforce region tracking algorithms in order to track the motion of characteristic landmarks on the heart surface

Evaluation of the Influence of Probe Pressure on the B-mode Ultrasound Measurement of Arterial Diameter

This paper deals with the influence of the echographic probe pressure on the measurement of arterial diameter. It is common sense that arterial diameter measurements are affected by the pressure exerted on the B-mode ultrasound probe but we found no report in the current literature analyzing and quantifying this effect. We demonstrated this influence by the analysis of arterial images obtained with a conventional B-mode ultrasound system while monitoring the pressure exerted on the probe with a force sensor. Our preliminary results demonstrated that continuous measurement was feasible and could be used to assess the relationship between the blood pressure and the arterial diameter. This may prove useful for the quantitative evaluation and the follow-up of patients with cardio-vascular diseases

Modelling of a camera-3D range finder system

A laser range finder mounted on a site and azimuth turret is used as a 3D range camera. It forms, associated with a video camera, an original stereovision system. The internal structure of both images are the same but the resolution of 3D image stays low. By ignoring the acquiring speed of measures, spatial resolution is limited by the accuracy of deviation device and the laser footprint. The fact that the impact of the beam is not a point introduces spatial integration.To correct the average at depth discontinuities due to the beam footprint, a neural-network-based solution is reported.The use of such a multisensor system requires its calibration. As camera calibration is a well-known problem, the paper focuses on models and calibration methods of the range finder. Experimental results illustrate the quality of the calibration step in terms of accuracy and stability.The footprint correction is evaluated for both 1D and 2D range finder scannings.