Noah Bedard

Real-time snapshot hyperspectral imaging endoscope

Hyperspectral imaging has tremendous potential to detect important molecular biomarkers of early cancer based on their unique spectral signatures. Several drawbacks have limited its use for in vivo screening applications: most notably the poor temporal and spatial resolution, high expense, and low optical throughput of existing hyperspectral imagers. We present the development of a new real-time hyperspectral endoscope (called the image mapping spectroscopy endoscope) based on an image mapping technique capable of addressing these challenges. The parallel high throughput nature of this technique enables the device to operate at frame rates of 5.2 frames per second while collecting a (x, y, λ) datacube of 350 × 350 × 48. We have successfully imaged tissue in vivo, resolving a vasculature pattern of the lower lip while simultaneously detecting oxy-hemoglobin.

Depth-resolved image mapping spectrometer (IMS) with structured illumination

We present a depth-resolved Image Mapping Spectrometer (IMS) which is capable of acquiring 4D (x, y, z, λ) datacubes. Optical sectioning is implemented by structured illumination. The device’s spectral imaging performance is demonstrated in a multispectral microsphere and mouse kidney tissue fluorescence imaging experiment. We also compare quantitatively the depth-resolved IMS with a hyperspectral confocal microscope (HCM) in a standard fluorescent bead imaging experiment. The comparison results show that despite the use of a light source with four orders of magnitude lower intensity in the IMS than that in the HCM, the image signal-to-noise ratio acquired by the IMS is 2.6 times higher than that achieved by the equivalent confocal approach.

Multimodal snapshot spectral imaging for oral cancer diagnostics: a pilot study

Optical imaging and spectroscopy have emerged as effective tools for detecting malignant changes associated with oral cancer. While clinical studies have demonstrated high sensitivity and specificity for detection, current devices either interrogate a small region or can have reduced performance for some benign lesions. We describe a snapshot imaging spectrometer that combines the large field-of-view of widefield imaging with the diagnostic strength of spectroscopy. The portable device can stream RGB images at 7.2 frames per second and record both autofluorescence and reflectance spectral datacubes in < 1 second. We report initial data from normal volunteers and oral cancer patients.

Emerging roles for multimodal optical imaging in early cancer detection: a global challenge.

Medical imaging technologies have become increasingly important in the clinical management of cancer, and now play key roles in cancer screening, diagnosis, staging, and monitoring response to treatment. Standard imaging modalities such as MRI, PET, and CT require significant financial resources and infrastructure, which limits access to these modalities to those patients in high-resource settings. In contrast, optical imaging strategies, with the potential for reduced cost and enhanced portability, are emerging as additional tools to facilitate the early detection and diagnosis of cancer. This article presents a vision for an expanding role for optical imaging in global cancer management, including screening, early detection at the point-of-care, biopsy guidance, and real-time histology. Multi-modal optical imaging - the combination of widefield and high resolution imaging - has the potential to aid in the detection and management of precancer and early cancer for traditionally underserved populations. Several recent widefield and high-resolution optical imaging technologies are described, along with requirements for implementing such devices into lower-resource settings.

Real-time video mosaicing with a high-resolution microendoscope

Microendoscopes allow clinicians to view subcellular features in vivo and in real-time, but their field-of-view is inherently limited by the small size of the probe’s distal end. Video mosaicing has emerged as an effective technique to increase the acquired image size. Current implementations are performed post-procedure, which removes the benefits of live imaging. In this manuscript we present an algorithm for real-time video mosaicing using a low-cost high-resolution microendoscope. We present algorithm execution times and show image results obtained from in vivo tissue.

Real-time hyperspectral fluorescence imaging of pancreatic β-cell dynamics with the image mapping spectrometer.

Summary The development of multi-colored fluorescent proteins, nanocrystals and organic fluorophores, along with the resulting engineered biosensors, has revolutionized the study of protein localization and dynamics in living cells. Hyperspectral imaging has proven to be a useful approach for such studies, but this technique is often limited by low signal and insufficient temporal resolution. Here, we present an implementation of a snapshot hyperspectral imaging device, the image mapping spectrometer (IMS), which acquires full spectral information simultaneously from each pixel in the field without scanning. The IMS is capable of real-time signal capture from multiple fluorophores with high collection efficiency (∼65%) and image acquisition rate (up to 7.2 fps). To demonstrate the capabilities of the IMS in cellular applications, we have combined fluorescent protein (FP)-FRET and [Ca2+]i biosensors to measure simultaneously intracellular cAMP and [Ca2+]i signaling in pancreatic &bgr;-cells. Additionally, we have compared quantitatively the IMS detection efficiency with a laser-scanning confocal microscope.

Image mapping spectrometry: calibration and characterization

Abstract. Image mapping spectrometry (IMS) is a hyperspectral imaging technique that simultaneously captures spatial and spectral information about an object in real-time. We present a new calibration procedure for the IMS as well as the first detailed evaluation of system performance. We correlate optical components and device calibration to performance metrics such as light throughput, scattered light, distortion, spectral image coregistration, and spatial/spectral resolution. Spectral sensitivity and motion artifacts are also evaluated with a dynamic biological experiment. The presented methodology of evaluation is useful in assessment of a variety of hyperspectral and multi-spectral modalities. Results are important to any potential users/developers of an IMS instrument and to anyone who may wish to compare the IMS to other imaging spectrometers.

Light field otoscope design for 3D in vivo imaging of the middle ear

We present a light field digital otoscope designed to measure three-dimensional shape of the tympanic membrane. This paper describes the optical and anatomical considerations we used to develop the prototype, along with the simulation and experimental measurements of vignetting, field curvature, and lateral resolution. Using an experimental evaluation procedure, we have determined depth accuracy and depth precision of our system to be 0.05-0.07 mm and 0.21-0.44 mm, respectively. To demonstrate the application of our light field otoscope, we present the first three-dimensional reconstructions of tympanic membranes in normal and otitis media conditions, acquired from children who participated in a feasibility study at the Childrens Hospital of Pittsburgh of the University of Pittsburgh Medical Center.

Design and optimization of a near-eye multifocal display system for augmented reality

We present a binocular, near-eye multifocal display prototype capable of multiplexing 6-focal-plane, flicker-free virtual content into the eyes. The prototype’s hardware design, focal sampling strategy, and rendered multifocal images are presented.

Real-time hyperspectral endoscope for early cancer diagnostics

Hyperspectral imaging has tremendous potential to detect important molecular biomarkers of early cancer based on their unique spectral signatures. Several drawbacks have limited their use for in vivo screening applications: most notably their poor temporal and spatial resolution, high expense, and low optical throughput. We present the development of a new real-time hyperspectral endoscope (called the IMS Endoscope) based on an image mapping technique which makes it capable of addressing these challenges. The parallel, high throughput nature of this technique enables the device to operate at frame rates of 3-10 fps while collecting a 3D (x, y, λ) datacube of 350 x 350 x 48.