Anton Deguet

An open-source research kit for the da Vinci® Surgical System

We present a telerobotics research platform that provides complete access to all levels of control via open-source electronics and software. The electronics employs an FPGA to enable a centralized computation and distributed I/O architecture in which all control computations are implemented in a familiar development environment (Linux PC) and low-latency I/O is performed over an IEEE-1394a (FireWire) bus at speeds up to 400 Mbits/sec. The mechanical components are obtained from retired first-generation da Vinci ® Surgical Systems. This system is currently installed at 11 research institutions, with additional installations underway, thereby creating a research community around a common open-source hardware and software platform.

Software components and frameworks for medical robot control

Robots are increasingly being used in computer integrated surgery (CIS) systems, yet to our knowledge, there is no open source software that is specifically targeted at this application domain. In this paper, we derive unique requirements for medical robot controllers that are based on our experiences in the field. We present an overview of the second-generation open source software package (the cisst package), currently under development, that would include a family of application frameworks with dynamically loaded software components. We then focus on some key design details, which include extensive application of the command pattern as well as a dynamic interface query mechanism, and present a simple example to illustrate the concepts

MRI image overlay: Application to arthrography needle insertion

Magnetic Resonance Imaging (MRI) offers great potential for planning, guiding, monitoring and controlling interventions. MR arthrography (MRAr) is the imaging gold standard for assessing small ligament and fibrocartilage injury in joints. In contemporary practice, MRAr consists of two consecutive sessions: (1) an interventional session where a needle is driven to the joint space and MR contrast is injected under fluoroscopy or CT guidance; and (2) a diagnostic MRI imaging session to visualize the distribution of contrast inside the joint space and evaluate the condition of the joint. Our approach to MRAr is to eliminate the separate radiologically guided needle insertion and contrast injection procedure by performing those tasks on conventional high-field closed MRI scanners. We propose a 2D augmented reality image overlay device to guide needle insertion procedures. This approach makes diagnostic high-field magnets available for interventions without a complex and expensive engineering entourage. In preclinical trials, needle insertions have been performed in the joints of porcine and human cadavers using MR image overlay guidance; in all cases, insertions successfully reached the joint space on the first attempt.

Image overlay for CT-guided needle insertions

Objective: We present a 2D image overlay device to assist needle placement on computed tomography (CT) scanners. Materials and Methods: The system consists of a flat display and a semitransparent mirror mounted on the gantry. When the physician looks at the patient through the mirror, the CT image appears to be floating inside the body with correct size and position as if the physician had 2D ‘X-ray vision’. The physician draws the optimal path on the CT image. The composite image is rendered on the display and thus reflected in the mirror. The reflected image is used to guide the physician in the procedure. In this article, we describe the design and various embodiments of the 2D image overlay system, followed by the results of phantom and cadaver experiments in multiple clinical applications. Results: Multiple skeletal targets were successfully accessed with one insertion attempt. Generally, successful access was recorded on liver targets when a clear path opened, but the number of attempts and accuracy showed variability because of occasional lack of access. Soft tissue deformation further reduced the accuracy and consistency in comparison to skeletal targets. Conclusion: The system demonstrated strong potential for reducing faulty needle insertion attempts, thereby reducing X-ray dose and patient discomfort.

Robotic needle guide for prostate brachytherapy: Clinical testing of feasibility and performance

PURPOSE Optimization of prostate brachytherapy is constrained by tissue deflection of needles and fixed spacing of template holes. We developed and clinically tested a robotic guide toward the goal of allowing greater freedom of needle placement. METHODS AND MATERIALS The robot consists of a small tubular needle guide attached to a robotically controlled arm. The apparatus is mounted and calibrated to operate in the same coordinate frame as a standard template. Translation in x and y directions over the perineum ±40 mm are possible. Needle insertion is performed manually. RESULTS Five patients were treated in an institutional review board-approved study. Confirmatory measurements of robotic movements for initial 3 patients using infrared tracking showed mean error of 0.489 mm (standard deviation, 0.328 mm). Fine adjustments in needle positioning were possible when tissue deflection was encountered; adjustments were performed in 54 (30.2%) of 179 needles placed, with 36 (20.1%) of 179 adjustments of >2mm. Twenty-seven insertions were intentionally altered to positions between the standard template grid to improve the dosimetric plan or avoid structures such as pubic bone and blood vessels. CONCLUSIONS Robotic needle positioning provided a means of compensating for needle deflections and the ability to intentionally place needles into areas between the standard template holes. To our knowledge, these results represent the first clinical testing of such a system. Future work will be incorporation of direct control of the robot by the physician, adding software algorithms to help avoid robot collisions with the ultrasound, and testing the angulation capability in the clinical setting.

An Image Overlay System with Enhanced Reality for Percutaneous Therapy Performed Inside CT Scanner

We describe a simple, safe, and inexpensive image overlay system to assist surgical interventions inside a conventional CT scanner. The overlay system is mounted non-invasively on the gantry of the CT scanner and it consists of a seven degrees-of-freedom passive mounting arm, a flat LCD display, and a light brown acrylic plate as a half mirror. In a preoperative calibration process, the display, half-mirror, and imaging plane of the scanner are spatially registered by imaging a triangular calibration object. Following the calibration, the patient is brought into the scanner, an image is acquired and sent to the overlay display via DICOM transfer. Looking at the patient through the half-mirror, the CT image appears to be floating inside the patient in correct size and position. This vision enables the physician to see both the surface and the inside of the patient at the same time, which can be utilized in guiding a surgical intervention. The complete system fits into a carry-on suitcase (except the mounting adapter), it is easy to calibrate, mounts non-invasively on the scanner, without utilizing vendor-specific features of he scanner.

A component-based architecture for flexible integration of robotic systems

While a robot control framework generally focuses on real-time performance and efficient data exchange between cooperating tasks or processes, an application such as robot-assisted surgery often demands information from, and integration with, a number of other devices. Thus, the software framework for the integrated system may have different requirements and priorities than a framework for real-time robot control. This paper reports on a component-based architecture that seamlessly bridges the gap between real-time robot control and a distributed, integrated system. The starting point is the cisst library, which provides a component-based framework for lock-free and efficient data exchange between multiple threads within a single process, which is suitable for real-time robot control. This paper describes the extension of the cisst library to support distributed systems, while keeping the same programming model as the single-process, multi-threaded scenario. Thus, application software does not need to know whether the component providing services is within the same process, in a different process, or on a different computer. In comparison, most standard middleware packages support components that fall within the last two categories (different processes on the same computer or different computers). This does not allow them to take advantage of the higher performance that can be achieved using standard lock-free data structures that do not rely on the operating system or on middleware services. Thus, the novelty of this approach is that the same component-based architecture and associated programming model extends from a multi-threaded scenario (which provides the best real-time performance) to a standard multi-process distributed system.

Intra-operative 3D guidance in prostate brachytherapy using a non-isocentric C-arm

Intra-operative guidance in Transrectal Ultrasound (TRUS) guided prostate brachytherapy requires localization of inserted radioactive seeds relative to the prostate. Seeds were reconstructed using a typical C-arm, and exported to a commercial brachytherapy system for dosimetry analysis. Technical obstacles for 3D reconstruction on a non-isocentric C-arm included pose-dependent C-arm calibration; distortion correction; pose estimation of C-arm images; seed reconstruction; and C-arm to TRUS registration. In precision-machined hard phantoms with 40-100 seeds, we correctly reconstructed 99.8% seeds with a mean 3D accuracy of 0.68 mm. In soft tissue phantoms with 45-87 seeds and clinically realistic 15 degrees C-arm motion, we correctly reconstructed 100% seeds with an accuracy of 1.3 mm. The reconstructed 3D seed positions were then registered to the prostate segmented from TRUS. In a Phase-1 clinical trial, so far on 4 patients with 66-84 seeds, we achieved intra-operative monitoring of seed distribution and dosimetry. We optimized the 100% prescribed iso-dose contour by inserting an average of 3.75 additional seeds, making intra-operative dosimetry possible on a typical C-arm, at negligible additional cost to the existing clinical installation.

Ultrasound-Fluoroscopy Registration for Prostate Brachytherapy Dosimetry

Prostate brachytherapy is a treatment for prostate cancer using radioactive seeds that are permanently implanted in the prostate. The treatment success depends on adequate coverage of the target gland with a therapeutic dose, while sparing the surrounding tissue. Since seed implantation is performed under transrectal ultrasound (TRUS) imaging, intraoperative localization of the seeds in ultrasound can provide physicians with dynamic dose assessment and plan modification. However, since all the seeds cannot be seen in the ultrasound images, registration between ultrasound and fluoroscopy is a practical solution for intraoperative dosimetry. In this manuscript, we introduce a new image-based nonrigid registration method that obviates the need for manual seed segmentation in TRUS images and compensates for the prostate displacement and deformation due to TRUS probe pressure. First, we filter the ultrasound images for subsequent registration using thresholding and Gaussian blurring. Second, a computationally efficient point-to-volume similarity metric, an affine transformation and an evolutionary optimizer are used in the registration loop. A phantom study showed final registration errors of 0.84 ± 0.45 mm compared to ground truth. In a study on data from 10 patients, the registration algorithm showed overall seed-to-seed errors of 1.7 ± 1.0 mm and 1.5 ± 0.9 mm for rigid and nonrigid registration methods, respectively, performed in approximately 30s per patient.

Automatic segmentation of radiographic fiducial and seeds from X-ray images in prostate brachytherapy

Prostate brachytherapy guided by transrectal ultrasound is a common treatment option for early stage prostate cancer. Prostate cancer accounts for 28% of cancer cases and 11% of cancer deaths in men with 217,730 estimated new cases and 32,050 estimated deaths in 2010 in the United States alone. The major current limitation is the inability to reliably localize implanted radiation seeds spatially in relation to the prostate. Multimodality approaches that incorporate X-ray for seed localization have been proposed, but they require both accurate tracking of the imaging device and segmentation of the seeds. Some use image-based radiographic fiducials to track the X-ray device, but manual intervention is needed to select proper regions of interest for segmenting both the tracking fiducial and the seeds, to evaluate the segmentation results, and to correct the segmentations in the case of segmentation failure, thus requiring a significant amount of extra time in the operating room. In this paper, we present an automatic segmentation algorithm that simultaneously segments the tracking fiducial and brachytherapy seeds, thereby minimizing the need for manual intervention. In addition, through the innovative use of image processing techniques such as mathematical morphology, Hough transforms, and RANSAC, our method can detect and separate overlapping seeds that are common in brachytherapy implant images. Our algorithm was validated on 55 phantom and 206 patient images, successfully segmenting both the fiducial and seeds with a mean seed segmentation rate of 96% and sub-millimeter accuracy.