Stephen A. Thompson

Use of a CT statistical deformation model for multi-modal pelvic bone segmentation

We present a segmentation algorithm using a statistical deformation model constructed from CT data of adult male pelves coupled to MRI appearance data. The algorithm allows the semi-automatic segmentation of bone for a limited population of MRI data sets. Our application is pelvic bone delineation from pre-operative MRI for image guided pelvic surgery. Specifically, we are developing image guidance for prostatectomies using the daVinci telemanipulator. Hence the use of male pelves only. The algorithm takes advantage of the high contrast of bone in CT data, allowing a robust shape model to be constructed relatively easily. This shape model can then be applied to a population of MRI data sets using a single data set that contains both CT and MRI data. The model is constructed automatically using fluid based non-rigid registration between a set of CT training images, followed by principal component analysis. MRI appearance data is imported using CT and MRI data from the same patient. Registration optimisation is performed using differential evolution. Based on our limited validation to date, the algorithm may outperform segmentation using non-rigid registration between MRI images without the use of shape data. The mean surface registration error achieved was 1.74 mm. The algorithm shows promise for use in segmentation of pelvic bone from MRI, though further refinement and validation is required. We envisage that the algorithm presented could be extended to allow the rapid creation of application specific models in various imaging modalities using a shape model based on CT data.

Improved Modelling of Tool Tracking Errors by Modelling Dependent Marker Errors

Accurate understanding of equipment tracking error is essential for decision making in image guided surgery. For tools tracked using markers attached to a rigid body, existing error estimation methods use the assumption that the individual marker errors are independent random variables. This assumption is not valid for all tracking systems. This paper presents a method to estimate a more accurate tracking error function, consisting of a systematic and random component. The proposed method does not require detailed knowledge of the tracking system physics. Results from a pointer calibration are used to demonstrate that the proposed method provides a better match to observed results than the existing state of the art. A simulation of the pointer calibration process is then used to show that existing methods can underestimate the pointer calibration error by a factor of two. A further simulation of laparoscopic camera tracking is used to show that existing methods cannot model important variations in system performance due to the angular arrangement of the tracking markers. By arranging the markers such that the systematic errors are nearly identical for all markers, the rotational component of the tracking error can be reduced, resulting in a significant reduction in target tracking errors.

Fast Semi-dense Surface Reconstruction from Stereoscopic Video in Laparoscopic Surgery

Liver resection is the main curative option for liver metastases. While this offers a 5-year survival rate of 50%, only about 20% of all patients are suitable for laparoscopic resection and thus being able to take advantage of minimally invasive surgery. One underlying difficulty is the establishment of a safe resection margin while avoiding critical structures. Intra-operative registration of patient scan data may provide a solution. However, this relies on fast and accurate reconstruction methods to obtain the current shape of the liver. Therefore, this paper presents a method for high-resolution stereoscopic surface reconstruction at interactive rates. To this end, a feature-matching propagation method is adapted to multi-resolution processing to enable parallelisation, remove global synchronisation issues and hence become amenable to a GPU-based implementation. Experiments are conducted on a planar target for reconstruction noise estimation and a visually realistic silicone liver phantom. Results highlight an average reconstruction error of 0.6 mm on the planar target, 2.4–5.7 mm on the phantom and processing times averaging around 370 milliseconds for input images of size 1920 x 540.

Accuracy validation of an image guided laparoscopy system for liver resection

We present an analysis of the registration component of a proposed image guidance system for image guided liver surgery, using contrast enhanced CT. The analysis is performed on a visually realistic liver phantom and in-vivo porcine data. A robust registration process that can be deployed clinically is a key component of any image guided surgery system. It is also essential that the accuracy of the registration can be quantified and communicated to the surgeon. We summarise the proposed guidance system and discuss its clinical feasibility. The registration combines an intuitive manual alignment stage, surface reconstruction from a tracked stereo laparoscope and a rigid iterative closest point registration to register the intra-operative liver surface to the liver surface derived from CT. Testing of the system on a liver phantom shows that subsurface landmarks can be localised to an accuracy of 2.9 mm RMS. Testing during five porcine liver surgeries demonstrated that registration can be performed during surgery, with an error of less than 10 mm RMS for multiple surface landmarks.

Development and Phantom Validation of a 3-D-Ultrasound-Guided System for Targeting MRI-Visible Lesions During Transrectal Prostate Biopsy

Objective: Three- and four-dimensional transrectal ultrasound transducers are now available from most major ultrasound equipment manufacturers, but currently are incorporated into only one commercial prostate biopsy guidance system. Such transducers offer the benefits of rapid volumetric imaging, but can cause substantial measurement distortion in electromagnetic tracking sensors, which are commonly used to enable 3-D navigation. In this paper, we describe the design, development, and validation of a 3-D-ultrasound-guided transrectal prostate biopsy system that employs high-accuracy optical tracking to localize the ultrasound probe and prostate targets in 3-D physical space. Methods: The accuracy of the system was validated by evaluating the targeted needle placement error after inserting a biopsy needle to sample planned targets in a phantom using standard 2-D ultrasound guidance versus real-time 3-D guidance provided by the new system. Results: The overall mean needle-segment-to-target distance error was 3.6 ± 4.0 mm and mean needle-to-target distance was 3.2 ± 2.4 mm. Conclusion: A significant increase in needle placement accuracy was observed when using the 3-D guidance system compared with visual targeting of invisible (virtual) lesions using a standard B-mode ultrasound-guided biopsy technique.

Database-Based Estimation of Liver Deformation under Pneumoperitoneum for Surgical Image-Guidance and Simulation

The insufflation of the abdomen in laparoscopic liver surgery leads to significant deformation of the liver. The estimation of the shape and position of the liver after insufflation has many important applications, such as providing surface-based registration algorithms used in image guidance with an initial guess and realistic patient-specific surgical simulation.

Deep residual networks for automatic segmentation of laparoscopic videos of the liver

Motivation: For primary and metastatic liver cancer patients undergoing liver resection, a laparoscopic approach can reduce recovery times and morbidity while offering equivalent curative results; however, only about 10% of tumours reside in anatomical locations that are currently accessible for laparoscopic resection. Augmenting laparoscopic video with registered vascular anatomical models from pre-procedure imaging could support using laparoscopy in a wider population. Segmentation of liver tissue on laparoscopic video supports the robust registration of anatomical liver models by filtering out false anatomical correspondences between pre-procedure and intra-procedure images. In this paper, we present a convolutional neural network (CNN) approach to liver segmentation in laparoscopic liver procedure videos. Method: We defined a CNN architecture comprising fully-convolutional deep residual networks with multi-resolution loss functions. The CNN was trained in a leave-one-patient-out cross-validation on 2050 video frames from 6 liver resections and 7 laparoscopic staging procedures, and evaluated using the Dice score. Results: The CNN yielded segmentations with Dice scores ≥0.95 for the majority of images; however, the inter-patient variability in median Dice score was substantial. Four failure modes were identified from low scoring segmentations: minimal visible liver tissue, inter-patient variability in liver appearance, automatic exposure correction, and pathological liver tissue that mimics non-liver tissue appearance. Conclusion: CNNs offer a feasible approach for accurately segmenting liver from other anatomy on laparoscopic video, but additional data or computational advances are necessary to address challenges due to the high inter-patient variability in liver appearance.

Intelligent viewpoint selection for efficient CT to video registration in laparoscopic liver surgery

PurposeMinimally invasive surgery offers advantages over open surgery due to a shorter recovery time, less pain and trauma for the patient. However, inherent challenges such as lack of tactile feedback and difficulty in controlling bleeding lower the percentage of suitable cases. Augmented reality can show a better visualisation of sub-surface structures and tumour locations by fusing pre-operative CT data with real-time laparoscopic video. Such augmented reality visualisation requires a fast and robust video to CT registration that minimises interruption to the surgical procedure.MethodsWe propose to use view planning for efficient rigid registration. Given the trocar position, a set of camera positions are sampled and scored based on the corresponding liver surface properties. We implement a simulation framework to validate the proof of concept using a segmented CT model from a human patient. Furthermore, we apply the proposed method on clinical data acquired during a human liver resection.ResultsThe first experiment motivates the viewpoint scoring strategy and investigates reliable liver regions for accurate registrations in an intuitive visualisation. The second experiment shows wider basins of convergence for higher scoring viewpoints. The third experiment shows that a comparable registration performance can be achieved by at least two merged high scoring views and four low scoring views. Hence, the focus could change from the acquisition of a large liver surface to a small number of distinctive patches, thereby giving a more explicit protocol for surface reconstruction. We discuss the application of the proposed method on clinical data and show initial results.ConclusionThe proposed simulation framework shows promising results to motivate more research into a comprehensive view planning method for efficient registration in laparoscopic liver surgery.

Breathing motion compensated registration of laparoscopic liver ultrasound to CT.

Laparoscopic Ultrasound (LUS) is regularly used during laparoscopic liver resection to locate critical vascular structures. Many tumours are iso-echoic, and registration to pre-operative CT or MR has been proposed as a method of image guidance. However, factors such as abdominal insufflation, LUS probe compression and breathing motion cause deformation of the liver, making this task far from trivial. Fortunately, within a smaller local region of interest a rigid solution can suffice. Also, the respiratory cycle can be expected to be consistent. Therefore, in this paper we propose a feature-based local rigid registration method to align tracked LUS data with CT while compensating for breathing motion. The method employs the Levenberg-Marquardt Iterative Closest Point (LMICP) algorithm, registers both on liver surface and vessels and requires two LUS datasets, one for registration and another for breathing estimation. Breathing compensation is achieved by fitting a 1D breathing model to the vessel points. We evaluate the algorithm by measuring the Target Registration Error (TRE) of three manually selected landmarks of a single porcine subject. Breathing compensation improves accuracy in 77% of the measurements. In the best case, TRE values below 3mm are obtained. We conclude that our method can potentially correct for breathing motion without gated acquisition of LUS and be integrated in the surgical workflow with an appropriate segmentation.

Explicit contact modeling for surgical computer guidance and simulation

Realistic modelling of mechanical interactions between tissues is an important part of surgical simulation, and may become a valuable asset in surgical computer guidance. Unfortunately, it is also computationally very demanding. Explicit matrix-free FEM solvers have been shown to be a good choice for fast tissue simulation, however little work has been done on contact algorithms for such FEM solvers. This work introduces such an algorithm that is capable of handling both deformable-deformable (soft-tissue interacting with soft-tissue) and deformable-rigid (e.g. soft-tissue interacting with surgical instruments) contacts. The proposed algorithm employs responses computed with a fully matrix-free, virtual node-based version of the model first used by Taylor and Flanagan in PRONTO3D. For contact detection, a bounding-volume hierarchy (BVH) capable of identifying self collisions is introduced. The proposed BVH generation and update strategies comprise novel heuristics to minimise the number of bounding volumes visited in hierarchy update and collision detection. Aside from speed, stability was a major objective in the development of the algorithm, hence a novel method for computation of response forces from C0-continuous normals, and a gradual application of response forces from rate constraints has been devised and incorporated in the scheme. The continuity of the surface normals has advantages particularly in applications such as sliding over irregular surfaces, which occurs, e.g., in simulated breathing. The effectiveness of the scheme is demonstrated on a number of meshes derived from medical image data and artificial test cases.