Neil T. Clancy

Comparative Validation of Single-Shot Optical Techniques for Laparoscopic 3-D Surface Reconstruction

Intra-operative imaging techniques for obtaining the shape and morphology of soft-tissue surfaces in vivo are a key enabling technology for advanced surgical systems. Different optical techniques for 3-D surface reconstruction in laparoscopy have been proposed, however, so far no quantitative and comparative validation has been performed. Furthermore, robustness of the methods to clinically important factors like smoke or bleeding has not yet been assessed. To address these issues, we have formed a joint international initiative with the aim of validating different state-of-the-art passive and active reconstruction methods in a comparative manner. In this comprehensive in vitro study, we investigated reconstruction accuracy using different organs with various shape and texture and also tested reconstruction robustness with respect to a number of factors like the pose of the endoscope as well as the amount of blood or smoke present in the scene. The study suggests complementary advantages of the different techniques with respect to accuracy, robustness, point density, hardware complexity and computation time. While reconstruction accuracy under ideal conditions was generally high, robustness is a remaining issue to be addressed. Future work should include sensor fusion and in vivo validation studies in a specific clinical context. To trigger further research in surface reconstruction, stereoscopic data of the study will be made publically available at www.open-CAS.com upon publication of the paper.

Spectrally encoded fiber-based structured lighting probe for intraoperative 3D imaging.

Three dimensional quantification of organ shape and structure during minimally invasive surgery (MIS) could enhance precision by allowing the registration of multi-modal or pre-operative image data (US/MRI/CT) with the live optical image. Structured illumination is one technique to obtain 3D information through the projection of a known pattern onto the tissue, although currently these systems tend to be used only for macroscopic imaging or open procedures rather than in endoscopy. To account for occlusions, where a projected feature may be hidden from view and/or confused with a neighboring point, a flexible multispectral structured illumination probe has been developed that labels each projected point with a specific wavelength using a supercontinuum laser. When imaged by a standard endoscope camera they can then be segmented using their RGB values, and their 3D coordinates calculated after camera calibration. The probe itself is sufficiently small (1.7 mm diameter) to allow it to be used in the biopsy channel of commonly used medical endoscopes. Surgical robots could therefore also employ this technology to solve navigation and visualization problems in MIS, and help to develop advanced surgical procedures such as natural orifice translumenal endoscopic surgery.

Narrow band 3 × 3 Mueller polarimetric endoscopy

Mueller matrix polarimetric imaging has shown potential in tissue diagnosis but is challenging to implement endoscopically. In this work, a narrow band 3 × 3 Mueller matrix polarimetric endoscope was designed by rotating the endoscope to generate 0°, 45° and 90° linearly polarized illumination and positioning a rotating filter wheel in front of the camera containing three polarisers to permit polarization state analysis for backscattered light. The system was validated with a rotating linear polarizer and a diffuse reflection target. Initial measurements of 3 × 3 Mueller matrices on a rat are demonstrated, followed by matrix decomposition into the depolarization and retardance matrices for further analysis. Our work shows the feasibility of implementing polarimetric imaging in a rigid endoscope conveniently and economically in order to reveal diagnostic information.

Intraoperative measurement of bowel oxygen saturation using a multispectral imaging laparoscope

Intraoperative monitoring of tissue oxygen saturation (StO2 ) has potentially important applications in procedures such as organ transplantation or colorectal surgery, where successful reperfusion affects the viability and integrity of repaired tissues. In this paper a liquid crystal tuneable filter-based multispectral imaging (MSI) laparoscope is described. Motion-induced image misalignments are reduced, using feature-based registration, before regression of the tissue reflectance spectra to calculate relative quantities of oxy- and deoxyhaemoglobin. The laparoscope was validated in vivo, during porcine abdominal surgery, by making parallel MSI and blood gas measurements of the small bowel vasculature. Ischaemic conditions were induced by local occlusion of the mesenteric arcade and monitored using the system. The MSI laparoscope was capable of measuring StO2 over a wide range (30-100%) with a temporal error of ± 7.5%. The imager showed sensitivity to spatial changes in StO2 during dynamic local occlusions, as well as tracking the recovery of tissues post-occlusion.

Multispectral image alignment using a three channel endoscope in vivo during minimally invasive surgery.

Sequential multispectral imaging is an acquisition technique that involves collecting images of a target at different wavelengths, to compile a spectrum for each pixel. In surgical applications it suffers from low illumination levels and motion artefacts. A three-channel rigid endoscope system has been developed that allows simultaneous recording of stereoscopic and multispectral images. Salient features on the tissue surface may be tracked during the acquisition in the stereo cameras and, using multiple camera triangulation techniques, this information used to align the multispectral images automatically even though the tissue or camera is moving. This paper describes a detailed validation of the set-up in a controlled experiment before presenting the first in vivo use of the device in a porcine minimally invasive surgical procedure. Multispectral images of the large bowel were acquired and used to extract the relative concentration of haemoglobin in the tissue despite motion due to breathing during the acquisition. Using the stereoscopic information it was also possible to overlay the multispectral information on the reconstructed 3D surface. This experiment demonstrates the ability of this system for measuring blood perfusion changes in the tissue during surgery and its potential use as a platform for other sequential imaging modalities.

Polarised stereo endoscope and narrowband detection for minimal access surgery

Polarisation imaging has the potential to provide enhanced contrast based on variations in the optical properties, such as scattering or birefringence, of the tissue of interest. Examining the signal at different wavebands in the visible spectrum also allows interrogation of different depths and structures. A stereo endoscope has been adapted to allow snapshot acquisition of orthogonal linear polarisation images to generate difference of linear polarisation images. These images are acquired in three narrow bands using a triple-bandpass filter and pair of colour cameras. The first in vivo results, acquired during a surgical procedure on a porcine subject, are presented that show wavelength dependent variations in vessel visibility and an increase in contrast under polarised detection.

A new device for assessing changes in skin viscoelasticity using indentation and optical measurement.

Background/aims: Skin is a viscoelastic material, comprised of fluidic and fibrous components. Changes in viscoelasticity can arise due to a number of conditions including dehydration, swelling (associated with injury or disease), impaired heart function, rehydration therapy, ageing, scarring, sun exposure and genetic conditions affecting connective tissue. Quantification of changes in skin viscoelasticity due to these processes is of great clinical interest in the fields of therapy monitoring, wound healing and disease screening. However, devices currently available to measure aspects of the mechanical properties of skin have limitations in ease‐of‐use, accessibility, and depth of measurement. This paper describes a new technique to follow changes in the viscoelasticity of the skin, using a novel approach to an indentation manoeuvre. The device is portable, low‐cost and easy to use while at the same time providing rich information on the mechanical response of the skin.

Robust surface tracking combining features, intensity and illumination compensation.

PurposeRecovering tissue deformation during robotic-assisted minimally invasive surgery procedures is important for providing intra-operative guidance, enabling in vivo imaging modalities and enhanced robotic control. The tissue motion can also be used to apply motion stabilization and to prescribe dynamic constraints for avoiding critical anatomical structures.MethodsImage-based methods based independently on salient features or on image intensity have limitations when dealing with homogeneous soft tissues or complex reflectance. In this paper, we use a triangular geometric mesh model in order to combine the advantages of both feature and intensity information and track the tissue surface reliably and robustly.ResultsSynthetic and in vivo experiments are performed to provide quantitative analysis of the tracking accuracy of our method, and we also show exemplar results for registering multispectral images where there is only a weak image signal.ConclusionCompared to traditional methods, our hybrid tracking method is more robust and has improved convergence in the presence of larger displacements, tissue dynamics and illumination changes.

An endoscopic structured lighting probe using spectral encoding

Recovering the three dimensional (3D) surface shape of tissues in minimally invasive surgery (MIS) is important for developing advanced image-guidance and navigation systems. Passive techniques for 3D reconstruction based on computational stereo are limited by the saliency of tissue texture and the view-dependent reflectance characteristics of the scene. Structured lighting provides a viable alternative by projecting known features onto the tissue surface. However, the correspondence problem (distinguishing individual projected features computationally) becomes difficult in tissue due to the presence of occlusions. Furthermore, miniaturisation of a light projection system for use in MIS, while maintaining the required light intensity, is a significant challenge. In this paper, a fibre-based probe is described that projects a spectrally-encoded pattern onto the target surface from its distal end. A dispersed broadband light source is used to project features of varying spectral content. The dominant wavelengths of imaged spots may be deduced from the RGB values of a standard colour camera using an algorithm that locates each colour on a chromaticity diagram. The results show that individual spots of a specified wavelength may be segmented and their centres of mass calculated, despite varying background colour. The probe has also been demonstrated on ex vivo tissue.

Robust near real-time estimation of physiological parameters from megapixel multispectral images with inverse Monte Carlo and random forest regression

PurposeMultispectral imaging can provide reflectance measurements at multiple spectral bands for each image pixel. These measurements can be used for estimation of important physiological parameters, such as oxygenation, which can provide indicators for the success of surgical treatment or the presence of abnormal tissue. The goal of this work was to develop a method to estimate physiological parameters in an accurate and rapid manner suited for modern high-resolution laparoscopic images.MethodsWhile previous methods for oxygenation estimation are based on either simple linear methods or complex model-based approaches exclusively suited for off-line processing, we propose a new approach that combines the high accuracy of model-based approaches with the speed and robustness of modern machine learning methods. Our concept is based on training random forest regressors using reflectance spectra generated with Monte Carlo simulations.ResultsAccording to extensive in silico and in vivo experiments, the method features higher accuracy and robustness than state-of-the-art online methods and is orders of magnitude faster than other nonlinear regression based methods.ConclusionOur current implementation allows for near real-time oxygenation estimation from megapixel multispectral images and is thus well suited for online tissue analysis.