Sa Thompson

Toward Detection and Localization of Instruments in Minimally Invasive Surgery

Methods for detecting and localizing surgical instruments in laparoscopic images are an important element of advanced robotic and computer-assisted interventions. Robotic joint encoders and sensors integrated or mounted on the instrument can provide information about the tools position, but this often has inaccuracy when transferred to the surgeons point of view. Vision sensors are currently a promising approach for determining the position of instruments in the coordinate frame of the surgical camera. In this study, we propose a vision algorithm for localizing the instruments pose in 3-D leaving only rotation in the axis of the tools shaft as an ambiguity. We propose a probabilistic supervised classification method to detect pixels in laparoscopic images that belong to surgical tools. We then use the classifier output to initialize an energy minimization algorithm for estimating the pose of a prior 3-D model of the instrument within a level set framework. We show that the proposed method is robust against noise using simulated data and we perform quantitative validation of the algorithm compared to ground truth obtained using an optical tracker. Finally, we demonstrate the practical application of the technique on in vivo data from minimally invasive surgery with traditional laparoscopic and robotic instruments.

2D-3D Pose Tracking of Rigid Instruments in Minimally Invasive Surgery

Instrument localization and tracking is an important challenge for advanced computer assisted techniques in minimally invasive surgery and image-based solutions to instrument localization can provide a non-invasive, low cost solution. In this study, we present a novel algorithm capable of recovering the 3D pose of laparoscopic surgical instruments combining constraints from a classification algorithm, multiple point features, stereo views (when available) and a linear motion model to robustly track the tool in surgical videos. We demonstrate the improved robustness and performance of our algorithm with optically tracked ground truth and additionally qualitatively demonstrate its performance on in vivo images.