Andrew P. King

Design and evaluation of a system for microscope-assisted guided interventions (MAGI)

The problem of providing surgical navigation using image overlays on the operative scene can be split into four main tasks-calibration of the optical system; registration of preoperative images to the patient; system and patient tracking, and display using a suitable visualization scheme. To achieve a convincing result in the magnified microscope view a very high alignment accuracy is required. The authors have simulated an entire image overlay system to establish the most significant sources of error and improved each of the stages involved. The microscope calibration process has been automated. The authors have introduced bone-implanted markers for registration and incorporated a locking acrylic dental stent (LADS) for patient tracking. The LADS can also provide a less-invasive registration device with mean target error of 0.7 mm in volunteer experiments. These improvements have significantly increased the alignment accuracy of the authors overlays. Phantom accuracy is 0.3-0.5 mm and clinical overlay errors were 0.5-1.0 mm on the bone fiducials and 0.5-4 mm on target structures. The authors have improved the graphical representation of the stereo overlays. The resulting system provides three-dimensional surgical navigation for microscope-issisted guided interventions (MAGI).

Stereo Augmented Reality in the Surgical Microscope

This paper describes the MAGI (microscope-assisted guided interventions) augmented-reality system, which allows surgeons to view virtual features segmented from preoperative radiological images accurately overlaid in stereo in the optical path of a surgical microscope. The aim of the system is to enable the surgeon to see in the correct 3-D position the structures that are beneath the physical surface. The technical challenges involved are calibration, segmentation, registration, tracking, and visualization. This paper details our solutions to these problems. As it is difficult to make reliable quantitative assessments of the accuracy of augmented-reality systems, results are presented from a numerical simulation, and these show that the system has a theoretical overlay accuracy of better than 1 mm at the focal plane of the microscope. Implementations of the system have been tested on volunteers, phantoms, and seven patients in the operating room. Observations are consistent with this accuracy prediction.

A System for Microscope-Assisted Guided Interventions

We present a system for surgical navigation using stereo overlays in the operating microscope aligned to the operative scene. This augmented reality system provides 3D information about nearby structures and offers a significant advancement over pointer-based guidance, which provides only the location of one point and requires the surgeon to look away from the operative scene. With a previous version of this system, we demonstrated feasibility, but it became clear that to achieve convincing guidance through the magnified microscope view, a very high alignment accuracy was required. We have made progress with several aspects of the system, including automated calibration, error simulation, bone-implanted fiducials and a dental attachment for tracking. We have performed experiments to establish the visual display parameters required to perceive overlaid structures beneath the operative surface. Easy perception of real and virtual structures with the correct transparency has been demonstrated in a laboratory and through the microscope. The result is a system with a predicted accuracy of 0.9 mm and phantom errors of 0.5 mm. In clinical practice errors are 0.5–1.5 mm, rising to 2–4 mm when brain deformation occurs.

A Stochastic Iterative Closest Point Algorithm (stochastICP)

We present a modification to the iterative closest point algorithm which improves the algorithms robustness and precision. At the start of each iteration, before point correspondence is calculated between the two feature sets, the algorithm randomly perturbs the point positions in one feature set. These perturbations allow the algorithm to move out of some local minima to find a minimum with a lower residual error. The size of this perturbation is reduced during the registration process. The algorithm has been tested using multiple starting positions to register three sets of data: a surface of a femur, a skull surface and a registration to hepatic vessels and a liver surface. Our results show that, if local minima are present, the stochastic ICP algorithm is more robust and is more precise than the standard ICP algorithm.

Bayesian Estimation of Intra-operative Deformation for Image-Guided Surgery Using 3-D Ultrasound

This paper describes the application of Bayesian theory to the problem of compensating for soft tissue deformation to improve the accuracy of image-guided surgery. A triangular surface mesh segmented from a pre-operative image is used as the input to the algorithm, and intra-operatively acquired ultrasound data compounded into a 3-D volume is used to guide the deformation process. Prior probabilities are defined for the boundary points of the segmented structure based on knowledge of the direction of gravity, the position of the surface of the surgical scene, and knowledge of the tissue properties. The posterior probabilities of the locations of each of the boundary points are then maximised according to Bayes’ theorem. A regularisation term is included to constrain deformation to the global structure of the object.

Tracking liver motion using 3-D ultrasound and a surface based statistical shape model

We present a technique for registering information from preoperative CT or MR images to physical space using intraoperatively acquired 3-D ultrasound data and a surface-based statistical shape model. The model is subject-specific and captures the statistical modes of variation of the liver surface through the breathing cycle. The registration uses a Bayesian formulation, which enables information about the likely position in the breathing cycle to be incorporated in the form of prior knowledge. It is computed using the model and the ultrasound image intensities, and is constrained by the model to produce realistic surfaces. Once an initial registration is computed, the liver motion and deformation can be tracked using a single ultrasound image combined with the statistical model. The technique is demonstrated by registering models constructed for 3 different volunteers to ultrasound data acquired at different points in the breathing cycle. This method has potential application in treatment of any abdominal organ which is affected by breathing motion.

Registration of Video Images to Tomographic Images by Optimising Mutual Information Using Texture Mapping

In this paper we propose a novel tracking method to update the pose of stereo video cameras with respect to a surface model derived from a 3D tomographic image. This has a number of applications in image guided interventions and therapy. Registration of 2D video images to the pre-operative 3D image provides a mapping between image and physical space and enables a perspective projection of the pre-operative data to be overlaid onto the video image. Assuming an initial registration can be achieved, we propose a method for updating the registration, which is based on image intensity and texture mapping. We performed five experiments on simulated, phantom and volunteer data and validated the algorithm against an accurate gold standard in all three cases. We measured the mean 3D error of our tracking algorithm to be 1.05 mm for the simulation and 1.89 mm for the volunteer data. Visually this corresponds to a good registration.

Design and Evaluation of a System for Microscope-Assisted Guided Interventions (MAGI)

The problem of providing surgical navigation using image overlays on the operative scene can be split into four main tasks-calibration of the optical system; registration of preoperative images to the patient; system and patient tracking, and display using a suitable visualization scheme. To achieve a convincing result in the magnified microscope view a very high alignment accuracy is required. The authors have simulated an entire image overlay system to establish the most significant sources of error and improved each of the stages involved. The microscope calibration process has been automated. The authors have introduced bone-implanted markers for registration and incorporated a locking acrylic dental stent (LADS) for patient tracking. The LADS can also provide a less-invasive registration device with mean target error of 0.7 mm in volunteer experiments. These improvements have significantly increased the alignment accuracy of the authors overlays. Phantom accuracy is 0.3-0.5 mm and clinical overlay errors were 0.5-1.0 mm on the bone fiducials and 0.5-4 mm on target structures. The authors have improved the graphical representation of the stereo overlays. The resulting system provides three-dimensional surgical navigation for microscope-issisted guided interventions (MAGI).

Estimating Sparse Deformation Fields Using Multiscale Bayesian Priors and 3-D Ultrasound

This paper presents an extension to the standard Bayesian image analysis paradigm to explicitly incorporate a multiscale approach. This new technique is demonstrated by applying it to the problem of compensating for soft tissue deformation of pre-segmented surfaces for image-guided surgery using 3-D ultrasound. The solution is regularised using knowledge of the mean and Gaussian curvatures of the surface estimate. Results are presented from testing the method on ultrasound data acquired from a volunteers liver. Two structures were segmented from an MR scan of the volunteer: the liver surface and the portal vein. Accurate estimates of the deformed surfaces were successfully computed using the algorithm, based on prior probabilities defined using a minimal amount of human intervention. With a more accurate prior model, this technique has the possibility to completely automate the process of compensating for intraoperative deformation in image-guided surgery.