Radoslav Marinov

260 frames-per-second 648x488 resolution division-of-focal-plane polarimeter with structural dynamics and tracking applications

We have designed an image sensor that can capture the first three Stokes parameters at 648 by 488 spatial resolution at 260 frames per second. The sensor consists of a CCD image sensor monolithically integrated with pixel pitch-matched aluminum nanowire polarization filters. The sensor demonstrates a Malus law response over all pixels, and has a relatively uniform diattenuation over the visible spectrum. We demonstrate two potential applications for the sensor. The first uses circular polarization in transmission mode to observe high-speed stress failure in polycarbonate. The second uses polarization in reflected mode to track high speed automobile traffic.

A 110 × 64 150 mW 28 frames/s integrated visible/near-infrared CMOS image sensor with dual exposure times for image guided surgery

A 110 × 64 pixel CMOS image sensor integrated with pixelated spectral interference filters is used for image guided surgery. The sensor operates with short exposure times for capturing visible light using blue, green, and red pixelated interference filters, and long exposure time for capturing near-infrared fluorescence with a co-located, pixelated NIR filter. The sensor was used to successfully detect the fluorescence of ICG accumulation in sentinel lymph node on patients with breast cancer.

Bio-inspired imager improves sensitivity in near-infrared fluorescence image-guided surgery

Image-guided surgery can enhance cancer treatment by decreasing, and ideally eliminating, positive tumor margins and iatrogenic damage to healthy tissue. Current state-of-the-art near-infrared fluorescence imaging systems are bulky and costly, lack sensitivity under surgical illumination, and lack co-registration accuracy between multimodal images. As a result, an overwhelming majority of physicians still rely on their unaided eyes and palpation as the primary sensing modalities for distinguishing cancerous from healthy tissue. Here we introduce an innovative design, comprising an artificial multispectral sensor inspired by the Morpho butterflys compound eye, which can significantly improve image-guided surgery. By monolithically integrating spectral tapetal filters with photodetectors, we have realized a single-chip multispectral imager with 1000 × higher sensitivity and 7 × better spatial co-registration accuracy compared to clinical imaging systems in current use. Preclinical and clinical data demonstrate that this technology seamlessly integrates into the surgical workflow while providing surgeons with real-time information on the location of cancerous tissue and sentinel lymph nodes. Due to its low manufacturing cost, our bio-inspired sensor will provide resource-limited hospitals with much-needed technology to enable more accurate value-based health care.

A 4-Megapixel Cooled CCD Division of Focal Plane Polarimeter for Celestial Imaging

The field of astronomy relies on spectral and polarization imagery recorded across a wide range of spectra to make inferences about imaged objects from nearby and distant galaxies. One of the challenges in recording celestial polarization information is recording multiple images filtered with various polarization optics, such as linear polarization filters or retarders, and with low-noise, low-dark-current sensors. In this paper, we present a division of focal plane polarimeter that can operate at room temperature down to −20 °C. When the imaging sensor operates at −20 °C, the dark currents is reduced by two orders of magnitude, which improves the polarization extinction ratio by ~5-fold. Comprehensive optoelectronic tests are presented with data recorded with the polarimeter.

Biologically inspired imaging sensors for multi-spectral and polarization imagery

Multi-spectral and polarization imaging have enabled and exploited a wide range of applications, from remote sensing to biomedical applications such as early cancer detection for image-guided surgery. However, state-of-the-art multispectral and polarization cameras are still based on conventional advances in optics and integrated circuits, yielding bulky form factors and poor signal reconstruction. Thus, these technologies have failed to be adopted as research or clinical imaging tools. Nature is full of examples of animals that take advantage of multi-spectral and polarization phenomena to gain an evolutionary advantage. For example, elegant low-power and compact biological visual systems, capable of multispectral and polarization imaging surpassing any man-made imaging system, can be found in the compound eyes of many arthropods. Here, we demonstrate radically novel, multi-spectral and polarization imaging sensors that function on the same fundamental principles as do the ommatidia of the mantis shrimp. Our bio-inspired imaging systems combine vertically stacked photodiodes, for single-pixel trichromatic vision, with an array of pixelated polarization filters, resulting in compact and low-power architectures. Our single-chip imager comprises of 1280-by-720 pixels, yielding a 62 dB and 48 dB dynamic range and signal-to-noise ratio, respectively, and operates at a maximum frame rate of 24 fps. This topology inherently co-registers in time and space the different spectral and polarization channels. This novel and ergonomic technology is enabling real-time in situ underwater polarization imaging as well as applications in biomedical fields.

Hexachromatic imager for near-infrared fluorescence image-guided surgery (Conference Presentation)

Image-guided surgery (IGS) can improve the patient’s outcome by providing meaningful real-time information about the location of cancerous tumors and surrounding tissue, aiding in the elimination of positive tumor margins and reducing iatrogenic damage. However, the clinical need for imaging systems that can provide real-time feedback under real operating room settings remains unmet. State-of-the-art imaging systems for near-infrared fluorescence IGS rely on a series of complex optics and several imaging sensors. As a result, these systems are bulky and expensive, and their architecture lacks the versatility to simultaneously image multiple fluorophores, effectively making them cumbersome when merged into the current surgical workflow. To address these shortcomings, we have designed a multi-spectral imager capable of spatially co-registered hexachromatic vision: three spectral channels in the visible spectrum for the identification of anatomical features in color and three spectral channels in the near-infrared spectrum for the simultaneous identification of multiple near-infrared fluorescence dyes used in IGS. Our single-chip imaging sensor combines the vertically stacked photodetectors technology with pixelated interference filters to create a multi-spectral imager that can help surgeons make clinically relevant decisions in real time, with an effective resolution of 1280x720x3 photodiodes and a frame rate of 24 FPS. Our imager has the ability to identify different shades of near-infrared fluorescent light, allowing the surgeon to use and differentiate multiple fluorophores as molecular probes with high sensitivity. Pre-clinical data is shown where simultaneous imaging of anatomical features in color, and identification of nerves and cancerous tumors, are achieved using multiple near-infrared fluorescent agents.

A compact bio-inspired visible/NIR imager for image-guided surgery(Conference Presentation)

Inspired by the visual system of the morpho butterfly, we have designed, fabricated, tested and clinically translated an ultra-sensitive, light weight and compact imaging sensor capable of simultaneously capturing near infrared (NIR) and visible spectrum information. The visual system of the morpho butterfly combines photosensitive cells with spectral filters at the receptor level. The spectral filters are realized by alternating layers of high and low dielectric constant, such as air and cytoplasm. We have successfully mimicked this concept by integrating pixelated spectral filters, realized by alternating silicon dioxide and silicon nitrate layers, with an array of CCD detectors. There are four different types of pixelated spectral filters in the imaging plane: red, green, blue and NIR. The high optical density (OD) of all spectral filters (OD>4) allow for efficient rejections of photons from unwanted bands. The single imaging chip weighs 20 grams with form factor of 5mm by 5mm. The imaging camera is integrated with a goggle display system. A tumor targeted agent, LS301, is used to identify all spontaneous tumors in a transgenic PyMT murine model of breast cancer. The imaging system achieved sensitivity of 98% and selectivity of 95%. We also used our imaging sensor to locate sentinel lymph nodes (SLNs) in patients with breast cancer using indocyanine green tracer. The surgeon was able to identify 100% of SLNs when using our bio-inspired imaging system, compared to 93% when using information from the lymphotropic dye and 96% when using information from the radioactive tracer.

A 250 frames-per-second 640 by 480 pixel division-of-focal-plane polarimeter for the visible spectrum

The most common method of polarimetery involves imaging a scene through a polarization analyzer at multiple configurations. Switching among these configurations requires capturing multiple images of the scene, limiting the ability to capture real-time polarization data due to multiple scene sampling and motion artifacts. Advances in nanofabrication technology have allowed direct integration of polarization analyzers onto the sensor, enabling the capture of multiple analyzer intensities from a single frame. Using this technique, we have fabricated a high frame rate, VGA resolution, division of focal plane polarization imager for the visible spectrum. The imaging sensor is realized by monolithic integration of aluminum nanowires with an array of CCD imaging elements. The pixelated nanowire polarization filters are at four different orientations offset by 45° This allows for recording of the first three Stokes parameters at every super pixel, and subsequently the degree of linear polarization and angle of polarization are computed at 250 frames per second at full VGA resolution and over 1000 when limited to a subsection of the array. The imaging sensor also employs a per pixel calibration scheme which mitigates the variations in the aluminum nanowire sizes. We present an optical characterization of the sensor, and then utilize the increased frame rate to capture high speed polarization images of pieces of polycarbonate plastic placed under stress. The high frame rate allows us to recover strain information that regular rate sensors cannot.