Liangbo Shen

Live volumetric (4D) visualization and guidance of in vivo human ophthalmic surgery with intraoperative optical coherence tomography

Minimally-invasive microsurgery has resulted in improved outcomes for patients. However, operating through a microscope limits depth perception and fixes the visual perspective, which result in a steep learning curve to achieve microsurgical proficiency. We introduce a surgical imaging system employing four-dimensional (live volumetric imaging through time) microscope-integrated optical coherence tomography (4D MIOCT) capable of imaging at up to 10 volumes per second to visualize human microsurgery. A custom stereoscopic heads-up display provides real-time interactive volumetric feedback to the surgeon. We report that 4D MIOCT enhanced suturing accuracy and control of instrument positioning in mock surgical trials involving 17 ophthalmic surgeons. Additionally, 4D MIOCT imaging was performed in 48 human eye surgeries and was demonstrated to successfully visualize the pathology of interest in concordance with preoperative diagnosis in 93% of retinal surgeries and the surgical site of interest in 100% of anterior segment surgeries. In vivo 4D MIOCT imaging revealed sub-surface pathologic structures and instrument-induced lesions that were invisible through the operating microscope during standard surgical maneuvers. In select cases, 4D MIOCT guidance was necessary to resolve such lesions and prevent post-operative complications. Our novel surgical visualization platform achieves surgeon-interactive 4D visualization of live surgery which could expand the surgeon’s capabilities.

Enhanced volumetric visualization for real time 4D intraoperative ophthalmic swept-source OCT.

Current-generation software for rendering volumetric OCT data sets based on ray casting results in volume visualizations with indistinct tissue features and sub-optimal depth perception. Recent developments in hand-held and microscope-integrated intrasurgical OCT designed for real-time volumetric imaging motivate development of rendering algorithms which are both visually appealing and fast enough to support real time rendering, potentially from multiple viewpoints for stereoscopic visualization. We report on an enhanced, real time, integrated volumetric rendering pipeline which incorporates high performance volumetric median and Gaussian filtering, boundary and feature enhancement, depth encoding, and lighting into a ray casting volume rendering model. We demonstrate this improved model implemented on graphics processing unit (GPU) hardware for real-time volumetric rendering of OCT data during tissue phantom and live human surgical imaging. We show that this rendering produces enhanced 3D visualizations of pathology and intraoperative maneuvers compared to standard ray casting.

Novel microscope-integrated stereoscopic heads-up display for intrasurgical optical coherence tomography.

Intra-operative optical coherence tomography (OCT) requires a display technology which allows surgeons to visualize OCT data without disrupting surgery. Previous research and commercial intrasurgical OCT systems have integrated heads-up display (HUD) systems into surgical microscopes to provide monoscopic viewing of OCT data through one microscope ocular. To take full advantage of our previously reported real-time volumetric microscope-integrated OCT (4D MIOCT) system, we describe a stereoscopic HUD which projects a stereo pair of OCT volume renderings into both oculars simultaneously. The stereoscopic HUD uses a novel optical design employing spatial multiplexing to project dual OCT volume renderings utilizing a single micro-display. The optical performance of the surgical microscope with the HUD was quantitatively characterized and the addition of the HUD was found not to substantially effect the resolution, field of view, or pincushion distortion of the operating microscope. In a pilot depth perception subject study, five ophthalmic surgeons completed a pre-set dexterity task with 50.0% (SD = 37.3%) higher success rate and in 35.0% (SD = 24.8%) less time on average with stereoscopic OCT vision compared to monoscopic OCT vision. Preliminary experience using the HUD in 40 vitreo-retinal human surgeries by five ophthalmic surgeons is reported, in which all surgeons reported that the HUD did not alter their normal view of surgery and that live surgical maneuvers were readily visible in displayed stereoscopic OCT volumes.

Optical Coherence Tomography for Retinal Surgery: Perioperative Analysis to Real-Time Four-Dimensional Image-Guided Surgery

Magnification of the surgical field using the operating microscope facilitated profound innovations in retinal surgery in the 1970s, such as pars plana vitrectomy. Although surgical instrumentation and illumination techniques are continually developing, the operating microscope for vitreoretinal procedures has remained essentially unchanged and currently limits the surgeons depth perception and assessment of subtle microanatomy. Optical coherence tomography (OCT) has revolutionized clinical management of retinal pathology, and its introduction into the operating suite may have a similar impact on surgical visualization and treatment. In this article, we review the evolution of OCT for retinal surgery, from perioperative analysis to live volumetric (four-dimensional, 4D) image-guided surgery. We begin by briefly addressing the benefits and limitations of the operating microscope, the progression of OCT technology, and OCT applications in clinical/perioperative retinal imaging. Next, we review intraoperative OCT (iOCT) applications using handheld probes during surgical pauses, two-dimensional (2D) microscope-integrated OCT (MIOCT) of live surgery, and volumetric MIOCT of live surgery. The iOCT discussion focuses on technological advancements, applications during human retinal surgery, translational difficulties and limitations, and future directions.

4D microscope-integrated OCT improves accuracy of ophthalmic surgical maneuvers

Ophthalmic surgeons manipulate micron-scale tissues using stereopsis through an operating microscope and instrument shadowing for depth perception. While ophthalmic microsurgery has benefitted from rapid advances in instrumentation and techniques, the basic principles of the stereo operating microscope have not changed since the 1930’s. Optical Coherence Tomography (OCT) has revolutionized ophthalmic imaging and is now the gold standard for preoperative and postoperative evaluation of most retinal and many corneal procedures. We and others have developed initial microscope-integrated OCT (MIOCT) systems for concurrent OCT and operating microscope imaging, but these are limited to 2D real-time imaging and require offline post-processing for 3D rendering and visualization. Our previously presented 4D MIOCT system can record and display the 3D surgical field stereoscopically through the microscope oculars using a dual-channel heads-up display (HUD) at up to 10 micron-scale volumes per second. In this work, we show that 4D MIOCT guidance improves the accuracy of depth-based microsurgical maneuvers (with statistical significance) in mock surgery trials in a wet lab environment. Additionally, 4D MIOCT was successfully performed in 38/45 (84%) posterior and 14/14 (100%) anterior eye human surgeries, and revealed previously unrecognized lesions that were invisible through the operating microscope. These lesions, such as residual and potentially damaging retinal deformation during pathologic membrane peeling, were visualized in real-time by the surgeon. Our integrated system provides an enhanced 4D surgical visualization platform that can improve current ophthalmic surgical practice and may help develop and refine future microsurgical techniques.

Novel microscope-integrated stereoscopic display for intrasurgical optical coherence tomography

The first generation of intrasurgical optical coherence tomography (OCT) systems displayed OCT data onto a separate computer monitor, requiring surgeons to look away from the surgical microscope. In order to provide real-time OCT feedback without requiring surgeons to look away during surgeries, recent prototype research and commercial intrasurgical OCT systems have integrated heads-up display (HUD) systems into the surgical microscopes to allow the surgeons to access the OCT data and the surgical field through the oculars concurrently. However, all current intrasurgical OCT systems with a HUD are only capable of imaging through one ocular limiting the surgeon’s depth perception of OCT volumes. Stereoscopy is an effective technology to dramatically increase depth perception by presenting an image from slightly different angles to each eye. Conventional stereoscopic HUD use a pair of micro displays which require bulky optics. Several new approaches for HUDs are reported to use only one micro display at the expense of image brightness or increased footprint. Therefore, these techniques for HUD are not suitable to be integrated into microscopes. We have developed a novel stereoscopic HUD which uses spatial multiplexing to project stereo views into both oculars simultaneously with only one micro-display and three optical elements for our microscope-integrated OCT system. Simultaneous stereoscopic views of OCT volumes are computed in real time by GPU-enabled OCT system software. We present, to our knowledge, the first microscope integrated stereoscopic HUD used for intrasurgical OCT with a novel optical design for stereoscopic viewing devices and report on its preliminary use in human vitreoretinal surgeries.

Novel real-time volumetric tool segmentation algorithm for intraoperative microscope integrated OCT(Conference Presentation)

Optical coherence tomography (OCT) allows for micron scale imaging of the human retina and cornea. Current generation research and commercial intrasurgical OCT prototypes are limited to live B-scan imaging. Our group has developed an intraoperative microscope integrated OCT system capable of live 4D imaging. With a heads up display (HUD) 4D imaging allows for dynamic intrasurgical visualization of tool tissue interaction and surgical maneuvers. Currently our system relies on operator based manual tracking to correct for patient motion and motion caused by the surgeon, to track the surgical tool, and to display the correct B-scan to display on the HUD. Even when tracking only bulk motion, the operator sometimes lags behind and the surgical region of interest can drift out of the OCT field of view. To facilitate imaging we report on the development of a fast volume based tool segmentation algorithm. The algorithm is based on a previously reported volume rendering algorithm and can identify both the tool and retinal surface. The algorithm requires 45 ms per volume for segmentation and can be used to actively place the B-scan across the tool tissue interface. Alternatively, real-time tool segmentation can be used to allow the surgeon to use the surgical tool as an interactive B-scan pointer.