Hideharu Mikami

Ultrafast optical imaging technology: principles and applications of emerging methods

Abstract High-speed optical imaging is an indispensable technology for blur-free observation of fast transient dynamics in virtually all areas including science, industry, defense, energy, and medicine. High temporal resolution is particularly important for microscopy as even a slow event appears to occur “fast” in a small field of view. Unfortunately, the shutter speed and frame rate of conventional cameras based on electronic image sensors are significantly constrained by their electrical operation and limited storage. Over the recent years, several unique and unconventional approaches to high-speed optical imaging have been reported to circumvent these technical challenges and achieve a frame rate and shutter speed far beyond what can be reached with the conventional image sensors. In this article, we review the concepts and principles of such ultrafast optical imaging methods, compare their advantages and disadvantages, and discuss an entirely new class of applications that are possible using them.

Rapid mesoscale multiphoton microscopy of human skin

We present a multiphoton microscope designed for mesoscale imaging of human skin. The system is based on two-photon excited fluorescence and second-harmonic generation, and images areas of ~0.8x0.8 mm2 at speeds of 0.8 fps (800x800 pixels; 12 frame averages) for high signal-to-noise ratio, with lateral and axial resolutions of 0.5µm and 3.3µm, respectively. The main novelty of this instrument is the design of the scan head, which includes a fast galvanometric scanner, optimized relay optics, a beam expander and high NA objective lens. Computed aberrations in focus are below the Marechal criterion of 0.07λ rms for diffraction-limited performance. We demonstrate the practical utility of this microscope by ex-vivo imaging of wide areas in normal human skin.

Ultrafast confocal fluorescence microscopy beyond the fluorescence lifetime limit

Laser-scanning confocal fluorescence microscopy is an indispensable tool for biomedical research by virtue of its high spatial resolution. Its temporal resolution is equally important, but is still inadequate for many applications. Here we present a confocal fluorescence microscope that, for the first time to our knowledge, surpasses the highest possible frame rate constrained only by the fluorescence lifetime of fluorophores (typically a few to several nanoseconds). This microscope is enabled by integrating a broadband, spatially distributed, dual-frequency comb or spatial dual-comb and quadrature amplitude modulation for optimizing spectral efficiency into frequency-division multiplexing with single-pixel photodetection for signal integration. Specifically, we demonstrate confocal fluorescence microscopy at a record high frame rate of 16,000xa0frames/s. To show its broad biomedical utility, we use the microscope to demonstrate 3D volumetric confocal fluorescence microscopy of cellular dynamics at 104xa0volumes/s and confocal fluorescence imaging flow cytometry of hematological and microalgal cells at up to 2xa0m/s.

Enhanced speed in fluorescence imaging using beat frequency multiplexing

Fluorescence imaging using radiofrequency-tagged emission (FIRE) is an emerging technique that enables higher imaging speed (namely, temporal resolution) in fluorescence microscopy compared to conventional fluorescence imaging techniques such as confocal microscopy and wide-field microscopy. It works based on the principle that it uses multiple intensity-modulated fields in an interferometric setup as excitation fields and applies frequency-division multiplexing to fluorescence signals. Unfortunately, despite its high potential, FIRE has limited imaging speed due to two practical limitations: signal bandwidth and signal detection efficiency. The signal bandwidth is limited by that of an acousto-optic deflector (AOD) employed in the setup, which is typically 100-200 MHz for the spectral range of fluorescence excitation (400-600 nm). The signal detection efficiency is limited by poor spatial mode-matching between two interfering fields to produce a modulated excitation field. Here we present a method to overcome these limitations and thus to achieve higher imaging speed than the prior version of FIRE. Our method achieves an increase in signal bandwidth by a factor of two and nearly optimal mode matching, which enables the imaging speed limited by the lifetime of the target fluorophore rather than the imaging system itself. The higher bandwidth and better signal detection efficiency work synergistically because higher bandwidth requires higher signal levels to avoid the contribution of shot noise and amplifier noise to the fluorescence signal. Due to its unprecedentedly high-speed performance, our method has a wide variety of applications in cancer detection, drug discovery, and regenerative medicine.

Large field of view multiphoton microscopy of human skin

Clinical examination crucially relies on the ability to quickly examine large tissue areas and rapidly zoom in to regions of interest. Skin lesions often show irregularity in color and appearance in general, especially when they start to progress towards malignancy. Large field of view (FOV) and automatic translation of the imaging area are critical in the assessment of the entire lesion. Imaging of limited FOVs of the lesion can easily result in false negative diagnosis. We present a multiphoton microscope based on two-photon excited fluorescence and second-harmonic generation that images FOVs of about 0.8 mm2 (without stitching adjacent FOVs) at speeds of 10 frames/second (800 x 800 pixels) with lateral and axial resolutions of 0.5 μm and 2.5 μm, respectively. The main novelty of this instrument is the design of the scan head, which includes a fast galvanometric scanner, relay optics, a beam expander and a high NA objective lens. We optimized the system based on the Olympus 25x, 1.05NA water immersion lens, that features a long working distance of 1 mm. Proper tailoring of the beam expander, which consists of the scan and tube lens elements, enables scaling of the FOV. The design criteria include a flat wavefront of the beam, minimum field curvature, and suppressed spherical aberrations. All aberrations in focus are below the Marechal criterion of 0.07λ rms for diffraction-limited performance. We demonstrate the practical utility of this microscope by ex-vivo imaging of wide FOVs in normal human skin.

High-throughput fluorescence imaging flow cytometry with light-sheet excitation and machine learning (Conference Presentation)

Fluorescence imaging flow cytometry offers highly accurate analysis of a large number of cells compared with conventional flow cytometry by virtue of its imaging capability. Unfortunately, the throughput of conventional fluorescence imaging flow cytometers is limited to ~1,000 cells/sec, which is one order of magnitude lower than that of conventional non-imaging flow cytometers. This is due to the low data transfer rate of a CCD image sensor with a time-delay integration technique employed to achieve sufficient sensitivity for fluorescence imaging of fast flowing cells. Replacing the CCD image sensor with a CMOS image sensor can potentially overcome the throughput limitation by virtue of its high data transfer rate, but critically sacrifice the imaging sensitivity because the time-delay integration cannot be employed to current CMOS image sensors. Here we present a fluorescence imaging flow cytometer that achieves comparable throughput and sensitivity with non-imaging flow cytometers. It is enabled by high-energy-density light-sheet excitation of flowing cells on a mirror-embedded PDMS-based microfluidic chip and by fluorescence image acquisition with a CMOS image sensor. The light-sheet excitation allows us obtain fluorescence images of flowing cells at a speed of >1 m/s, corresponding to a high throughput of >10,000 cells/sec. To show its biomedical utility, we use it combined with machine learning to demonstrate accurate screening of white blood cells and real-time identification of cancer cells in blood.

Accurate classification of microalgal cells by frequency-division-multiplexed confocal imaging flow cytometry (Conference Presentation)

Fluorescence imaging flow cytometry is an emerging technique for analyzing a large number of cells with high accuracy over conventional flow cytometry by virtue of its imaging capability. Unfortunately, the cell throughput of conventional fluorescence imaging flow cytometers (~1,000 cells/sec) is much lower than that of standard non-imaging flow cytometers due to the use of a CCD image sensor having a limited data transfer rate, making it difficult to analyze a large population of cells. Here we report our experimental demonstration of highly accurate classification of microalgae with a frequency-division-multiplexed confocal imaging flow cytometer (IFC) that enables imaging of every single microalgal cell with an unprecedentedly high throughput of 20,000 cells/sec. The high-speed imaging performance of the IFC is enabled by employing frequency-division-multiplexed confocal microscopy, which uses a sensitive single-pixel photodetector such as an avalanche photodetector or a photomultiplier tube to obtain images of flowing cells. We stained three species of microalgae (Chlamydomonas reinhardtii, Haematococcus lacustris, and Euglena gracilis) with SYTO16 and obtained three-color images of the cells (bright-field, fluorescence staining of nuclei, and autofluorescence of chlorophyll). We extracted 243-dimensional features from each three-color image and employed a support vector machine to classify the cells with the obtained multi-dimensional data. As a result, the cells were successfully classified with an accuracy of 99.7%. Due to the IFC’s multi-color imaging capability with an unprecedentedly high throughput, our technique has a wide variety of potential applications other than microalga classification, such as accurate blood screening and liquid biopsy.

Intelligent Image-Activated Cell Sorting

A fundamental challenge of biology is to understand the vast heterogeneity of cells, particularly how cellular composition, structure, and morphology are linked to cellular physiology. Unfortunately, conventional technologies are limited in uncovering these relations. We present a machine-intelligence technology based on a radically different architecture that realizes real-time image-based intelligent cell sorting at an unprecedented rate. This technology, which we refer to as intelligent image-activated cell sorting, integrates high-throughput cell microscopy, focusing, and sorting on a hybrid software-hardware data-management infrastructure, enabling real-time automated operation for data acquisition, data processing, decision-making, and actuation. We use it to demonstrate real-time sorting of microalgal and blood cells based on intracellular protein localization and cell-cell interaction from large heterogeneous populations for studying photosynthesis and atherothrombosis, respectively. The technology is highly versatile and expected to enable machine-based scientific discovery in biological, pharmaceutical, and medical sciences.

High-speed bioimaging with frequency-division-multiplexed fluorescence confocal microscopy

We present methods of fluorescence confocal microscopy that enable unprecedentedly high frame rate of > 10,000 fps. The methods are based on a frequency-division multiplexing technique, which was originally developed in the field of communication engineering. Specifically, we achieved a broad bandwidth (~400 MHz) of detection signals using a dual- AOD method and overcame limitations in frame rate, due to a scanning device, by using a multi-line focusing method, resulting in a significant increase in frame rate. The methods have potential biomedical applications such as observation of sub-millisecond dynamics in biological tissues, in-vivo three-dimensional imaging, and fluorescence imaging flow cytometry.

Efficient generation of a three-photon W state

A three-photon W state was generated with 30 times higher efficiency and 10 times lower pump power than a previous work by utilizing weak coherent light. The scheme can be naturally extended to higher order.