Daniel S. Elson

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-operative morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeons navigation capabilities by observing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted instruments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D optical imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions.

Studying biological tissue with fluorescence lifetime imaging: microscopy, endoscopy, and complex decay profiles

We have applied fluorescence lifetime imaging (FLIM) to the autofluorescence of different kinds of biological tissue in vitro, including animal tissue sections and knee joints as well as human teeth, obtaining two-dimensional maps with functional contrast. We find that fluorescence decay profiles of biological tissue are well described by the stretched exponential function (StrEF), which can represent the complex nature of tissue. The StrEF yields a continuous distribution of fluorescence lifetimes, which can be extracted with an inverse Laplace transformation, and additional information is provided by the width of the distribution. Our experimental results from FLIM microscopy in combination with the StrEF analysis indicate that this technique is ready for clinical deployment, including portability that is through the use of a compact picosecond diode laser as the excitation source. The results obtained with our FLIM endoscope successfully demonstrated the viability of this modality, though they need further optimization. We expect a custom-designed endoscope with optimized illumination and detection efficiencies to provide significantly improved performance.

Fluorescence lifetime imaging microscopy for brain tumor image-guided surgery

We demonstrate for the first time the application of an endoscopic fluorescence lifetime imaging microscopy (FLIM) system to the intraoperative diagnosis of glioblastoma multiforme (GBM). The clinically compatible FLIM prototype integrates a gated (down to 0.2 ns) intensifier imaging system with a fiber-bundle (fiber image guide of 0.5 mm diameter, 10,000 fibers with a gradient index lens objective 0.5 NA, and 4 mm field of view) to provide intraoperative access to the surgical field. Experiments conducted in three patients undergoing craniotomy for tumor resection demonstrate that FLIM-derived parameters allow for delineation of tumor from normal cortex. For example, at 460±25-nm wavelength band emission corresponding to NADH/NADPH fluorescence, GBM exhibited a weaker fluorescence intensity (35% less, p-value<0.05) and a longer lifetime τGBM-Amean=1.59±0.24 ns than normal cortex τNC-Amean=1.28±0.04 ns (p-value<0.005). Current results demonstrate the potential use of FLIM as a tool for image-guided surgery of brain tumors.

Wide-field fluorescence lifetime imaging of cancer

Optical imaging of tissue autofluorescence has the potential to provide rapid label-free screening and detection of surface tumors for clinical applications, including when combined with endoscopy. Quantitative imaging of intensity-based contrast is notoriously difficult and spectrally resolved imaging does not always provide sufficient contrast. We demonstrate that fluorescence lifetime imaging (FLIM) applied to intrinsic tissue autofluorescence can directly contrast a range of surface tissue tumors, including in gastrointestinal tissues, using compact, clinically deployable instrumentation achieving wide-field fluorescence lifetime images of unprecedented clarity. Statistically significant contrast is observed between cancerous and healthy colon tissue for FLIM with excitation at 355 nm. To illustrate the clinical potential, wide-field fluorescence lifetime images of unstained ex vivo tissue have been acquired at near video rate, which is an important step towards real-time FLIM for diagnostic and interoperative imaging, including for screening and image-guided biopsy applications.

Comparative Validation of Single-Shot Optical Techniques for Laparoscopic 3-D Surface Reconstruction

Intra-operative imaging techniques for obtaining the shape and morphology of soft-tissue surfaces in vivo are a key enabling technology for advanced surgical systems. Different optical techniques for 3-D surface reconstruction in laparoscopy have been proposed, however, so far no quantitative and comparative validation has been performed. Furthermore, robustness of the methods to clinically important factors like smoke or bleeding has not yet been assessed. To address these issues, we have formed a joint international initiative with the aim of validating different state-of-the-art passive and active reconstruction methods in a comparative manner. In this comprehensive in vitro study, we investigated reconstruction accuracy using different organs with various shape and texture and also tested reconstruction robustness with respect to a number of factors like the pose of the endoscope as well as the amount of blood or smoke present in the scene. The study suggests complementary advantages of the different techniques with respect to accuracy, robustness, point density, hardware complexity and computation time. While reconstruction accuracy under ideal conditions was generally high, robustness is a remaining issue to be addressed. Future work should include sensor fusion and in vivo validation studies in a specific clinical context. To trigger further research in surface reconstruction, stereoscopic data of the study will be made publically available at www.open-CAS.com upon publication of the paper.

Whole-field five-dimensional fluorescence microscopy combining lifetime and spectral resolution with optical sectioning

We report a novel whole-field three-dimensional fluorescence lifetime imaging microscope that incoporates multispectral imaging to provide five-dimensional (5-D) fluorescence microscopy. This instrument, which can acquire a 5-D data set in less than a minute, is based on potentially compact and inexpensive diode-pumped solid-state laser technology. We demonstrate that spectral discrimination as well as optical sectioning minimize artifacts in lifetime determination and illustrate how spectral discrimination improves the lifetime contrast of biological tissue.

FLIM FRET Technology for Drug Discovery: Automated Multiwell-Plate High-Content Analysis, Multiplexed Readouts and Application in Situ

A fluorescence lifetime imaging (FLIM) technology platform intended to read out changes in Förster resonance energy transfer (FRET) efficiency is presented for the study of protein interactions across the drug-discovery pipeline. FLIM provides a robust, inherently ratiometric imaging modality for drug discovery that could allow the same sensor constructs to be translated from automated cell-based assays through small transparent organisms such as zebrafish to mammals. To this end, an automated FLIM multiwell-plate reader is described for high content analysis of fixed and live cells, tomographic FLIM in zebrafish and FLIM FRET of live cells via confocal endomicroscopy. For cell-based assays, an exemplar application reading out protein aggregation using FLIM FRET is presented, and the potential for multiple simultaneous FLIM (FRET) readouts in microscopy is illustrated.

Simultaneous time- and wavelength-resolved fluorescence spectroscopy for near real-time tissue diagnosis

A novel fiber-optic-based method for simultaneous time- and wavelength-resolved fluorescence spectroscopy for the rapid diagnosis of diseased tissue is demonstrated. By combining multiple bandpass and dichroic filters (405/40, 460/50, and 550/50) with different lengths of optical fiber (1, 10, and 19 m) acting as an optical delay this system enables the near real-time acquisition and characterization of time-resolved fluorescence spectra using a single detector and excitation input. The recording of multiple fluorescence response pulses at selected wavelengths can be completed in hundreds of nanoseconds, which provides the capability of a real-time characterization of biological systems.

Spectrally encoded fiber-based structured lighting probe for intraoperative 3D imaging.

Three dimensional quantification of organ shape and structure during minimally invasive surgery (MIS) could enhance precision by allowing the registration of multi-modal or pre-operative image data (US/MRI/CT) with the live optical image. Structured illumination is one technique to obtain 3D information through the projection of a known pattern onto the tissue, although currently these systems tend to be used only for macroscopic imaging or open procedures rather than in endoscopy. To account for occlusions, where a projected feature may be hidden from view and/or confused with a neighboring point, a flexible multispectral structured illumination probe has been developed that labels each projected point with a specific wavelength using a supercontinuum laser. When imaged by a standard endoscope camera they can then be segmented using their RGB values, and their 3D coordinates calculated after camera calibration. The probe itself is sufficiently small (1.7 mm diameter) to allow it to be used in the biopsy channel of commonly used medical endoscopes. Surgical robots could therefore also employ this technology to solve navigation and visualization problems in MIS, and help to develop advanced surgical procedures such as natural orifice translumenal endoscopic surgery.

Miniaturized side-viewing imaging probe for fluorescence lifetime imaging (FLIM): validation with fluorescence dyes, tissue structural proteins and tissue specimens

We report a side viewing fibre-based endoscope that is compatible with intravascular imaging and fluorescence lifetime imaging microscopy (FLIM). The instrument has been validated through testing with fluorescent dyes and collagen and elastin powders using the Laguerre expansion deconvolution technique to calculate the fluorescence lifetimes. The instrument has also been tested on freshly excised unstained animal vascular tissues.