Zoltán Göröcs

Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy

We discuss unique features of lens-free computational imaging tools and report some of their emerging results for wide-field on-chip microscopy, such as the achievement of a numerical aperture (NA) of ∼0.8–0.9 across a field of view (FOV) of more than 20 mm2 or an NA of ∼0.1 across a FOV of ∼18 cm2, which corresponds to an image with more than 1.5 gigapixels. We also discuss the current challenges that these computational on-chip microscopes face, shedding light on their future directions and applications.

Fluorescent Imaging of Single Nanoparticles and Viruses on a Smart Phone

Optical imaging of nanoscale objects, whether it is based on scattering or fluorescence, is a challenging task due to reduced detection signal-to-noise ratio and contrast at subwavelength dimensions. Here, we report a field-portable fluorescence microscopy platform installed on a smart phone for imaging of individual nanoparticles as well as viruses using a lightweight and compact opto-mechanical attachment to the existing camera module of the cell phone. This hand-held fluorescent imaging device utilizes (i) a compact 450 nm laser diode that creates oblique excitation on the sample plane with an incidence angle of ~75°, (ii) a long-pass thin-film interference filter to reject the scattered excitation light, (iii) an external lens creating 2× optical magnification, and (iv) a translation stage for focus adjustment. We tested the imaging performance of this smart-phone-enabled microscopy platform by detecting isolated 100 nm fluorescent particles as well as individual human cytomegaloviruses that are fluorescently labeled. The size of each detected nano-object on the cell phone platform was validated using scanning electron microscopy images of the same samples. This field-portable fluorescence microscopy attachment to the cell phone, weighing only ~186 g, could be used for specific and sensitive imaging of subwavelength objects including various bacteria and viruses and, therefore, could provide a valuable platform for the practice of nanotechnology in field settings and for conducting viral load measurements and other biomedical tests even in remote and resource-limited environments.

On-Chip Biomedical Imaging

Lab-on-a-chip systems have been rapidly emerging to pave the way toward ultra-compact, efficient, mass producible and cost-effective biomedical research and diagnostic tools. Although such microfluidic and microelectromechanical systems have achieved high levels of integration, and are capable of performing various important tasks on the same chip, such as cell culturing, sorting and staining, they still rely on conventional microscopes for their imaging needs. Recently, several alternative on-chip optical imaging techniques have been introduced, which have the potential to substitute conventional microscopes for various lab-on-a-chip applications. Here we present a critical review of these recently emerging on-chip biomedical imaging modalities, including contact shadow imaging, lens-free holographic microscopy, fluorescent on-chip microscopy and lens-free optical tomography.

Pixel super-resolution using wavelength scanning

Undersampling and pixelation affect a number of imaging systems, limiting the resolution of the acquired images, which becomes particularly significant for wide-field microscopy applications. Various super-resolution techniques have been implemented to mitigate this resolution loss by utilizing sub-pixel displacements in the imaging system, achieved, for example, by shifting the illumination source, the sensor array and/or the sample, followed by digital synthesis of a smaller effective pixel by merging these sub-pixel-shifted low-resolution images. Herein, we introduce a new pixel super-resolution method that is based on wavelength scanning and demonstrate that as an alternative to physical shifting/displacements, wavelength diversity can be used to boost the resolution of a wide-field imaging system and significantly increase its space-bandwidth product. We confirmed the effectiveness of this new technique by improving the resolution of lens-free as well as lens-based microscopy systems and developed an iterative algorithm to generate high-resolution reconstructions of a specimen using undersampled diffraction patterns recorded at a few wavelengths covering a narrow spectrum (10–30 nm). When combined with a synthetic-aperture-based diffraction imaging technique, this wavelength-scanning super-resolution approach can achieve a half-pitch resolution of 250 nm, corresponding to a numerical aperture of ~1.0, across a large field of view (>20 mm2). We also demonstrated the effectiveness of this approach by imaging various biological samples, including blood and Papanicolaou smears. Compared with displacement-based super-resolution techniques, wavelength scanning brings uniform resolution improvement in all directions across a sensor array and requires significantly fewer measurements. This technique would broadly benefit wide-field imaging applications that demand larger space-bandwidth products.

Deep learning microscopy

We demonstrate that a deep neural network can significantly improve optical microscopy, enhancing its spatial resolution over a large field-of-view and depth-of-field. After its training, the only input to this network is an image acquired using a regular optical microscope, without any changes to its design. We blindly tested this deep learning approach using various tissue samples that are imaged with low-resolution and wide-field systems, where the network rapidly outputs an image with remarkably better resolution, matching the performance of higher numerical aperture lenses, also significantly surpassing their limited field-of-view and depth-of-field. These results are transformative for various fields that use microscopy tools, including e.g., life sciences, where optical microscopy is considered as one of the most widely used and deployed techniques. Beyond such applications, our presented approach is broadly applicable to other imaging modalities, also spanning different parts of the electromagnetic spectrum, and can be used to design computational imagers that get better and better as they continue to image specimen and establish new transformations among different modes of imaging.

Giga-pixel fluorescent imaging over an ultra-large field-of-view using a flatbed scanner

We demonstrate a new fluorescent imaging technique that can screen for fluorescent micro-objects over an ultra-wide field-of-view (FOV) of ~532 cm(2), i.e., 19 cm × 28 cm, reaching a space-bandwidth product of more than 2 billion. For achieving such a large FOV, we modified the hardware and software of a commercially available flatbed scanner, and added a custom-designed absorbing fluorescent filter, a two-dimensional array of external light sources for computer-controlled and high-angle fluorescent excitation. We also re-programmed the driver of the scanner to take full control of the scanner hardware and achieve the highest possible exposure time, gain and sensitivity for detection of fluorescent micro-objects through the gradient index self-focusing lens array that is positioned in front of the scanner sensor chip. For example, this large FOV of our imaging platform allows us to screen more than 2.2 mL of undiluted whole blood for detection of fluorescent micro-objects within <5 minutes. This high-throughput fluorescent imaging platform could be useful for rare cell research and cytometry applications by enabling rapid screening of large volumes of optically dense media. Our results constitute the first time that a flatbed scanner has been converted to a fluorescent imaging system, achieving a record large FOV.

Propagation phasor approach for holographic image reconstruction

To achieve high-resolution and wide field-of-view, digital holographic imaging techniques need to tackle two major challenges: phase recovery and spatial undersampling. Previously, these challenges were separately addressed using phase retrieval and pixel super-resolution algorithms, which utilize the diversity of different imaging parameters. Although existing holographic imaging methods can achieve large space-bandwidth-products by performing pixel super-resolution and phase retrieval sequentially, they require large amounts of data, which might be a limitation in high-speed or cost-effective imaging applications. Here we report a propagation phasor approach, which for the first time combines phase retrieval and pixel super-resolution into a unified mathematical framework and enables the synthesis of new holographic image reconstruction methods with significantly improved data efficiency. In this approach, twin image and spatial aliasing signals, along with other digital artifacts, are interpreted as noise terms that are modulated by phasors that analytically depend on the lateral displacement between hologram and sensor planes, sample-to-sensor distance, wavelength, and the illumination angle. Compared to previous holographic reconstruction techniques, this new framework results in five- to seven-fold reduced number of raw measurements, while still achieving a competitive resolution and space-bandwidth-product. We also demonstrated the success of this approach by imaging biological specimens including Papanicolaou and blood smears.

Computational out-of-focus imaging increases the space–bandwidth product in lens-based coherent microscopy

The space–bandwidth product (SBP) of modern objective lenses is often significantly larger than the pixel count of opto-electronic image sensor chips, and, therefore, much of the information transmitted by the optical system cannot be adequately sampled or digitized. To resolve this mismatch, microscopes are in general designed to maintain the resolution of the optical system while significantly wasting the field of view (FOV) and SBP of the objective lens. We introduce a wide-field and high-resolution coherent imaging method that uses a stack of out-of-focus images to provide much better utilization of the SBP of an objective lens. We demonstrate our approach on a benchtop microscope by using a demagnification camera adapter to match the active area of the image sensor chip to the FOV of an objective lens. We show that the resulting spatial undersampling caused by capturing a large FOV can be mitigated through an iterative pixel super-resolution algorithm that uses e.g., ∼three to five slightly out-of-focus images, yielding an ∼8-fold increase in the SBP of the microscope. Furthermore, the same pixel super-resolution algorithm also achieves phase retrieval, revealing the optical phase information of the specimen. We compared our method against traditional off-axis and phase-shifting digital holographic microscopy modalities and demonstrated at least 3-fold reduction in the number of images required to achieve the same SBP. This technique could be used to maximize the throughput and SBP of lens-based coherent imaging and holography systems and inspire new microscopy designs that benefit from the inherent autofocusing steps of a scanning microscope to increase its SBP.

Enhanced light collection in fluorescence microscopy using self-assembled micro-reflectors

In fluorescence microscopy, the signal-to-noise ratio (SNR) of the optical system is directly linked to the numerical aperture (NA) of the microscope objective, which creates detection challenges for low-NA, wide-field and high-throughput imaging systems. Here we demonstrate a method to increase the light collection efficiency from micron-scale fluorescent objects using self-assembled vapor-condensed polyethylene glycol droplets, which act as micro-reflectors for fluorescent light. Around each fluorescent particle, a liquid meniscus is formed that increases the excitation efficiency and redirects part of the laterally-emitted fluorescent light towards the detector due to internal reflections at the liquid-air interface of the meniscus. The three-dimensional shape of this micro-reflector can be tuned as a function of time, vapor temperature, and substrate contact angle, providing us optimized SNR performance for fluorescent detection. Based on these self-assembled micro-reflectors, we experimentally demonstrate ~2.5-3 fold enhancement of the fluorescent signal from 2-10 μm sized particles. A theoretical explanation of the formation rate and shapes of these micro-reflectors is presented, along with a ray tracing model of their optical performance. This method can be used as a sample preparation technique for consumer electronics-based microscopy and sensing tools, thus increasing the sensitivity of low-NA systems that image fluorescent micro-objects.

Deep Learning Enhanced Mobile-Phone Microscopy

Mobile-phones have facilitated the creation of field-portable, cost-effective imaging and sensing technologies that approach laboratory-grade instrument performance. However, the optical imaging interfaces of mobile-phones are not designed for microscopy and produce spatial and spectral distortions in imaging microscopic specimens. Here, we report on the use of deep learning to correct such distortions introduced by mobile-phone-based microscopes, facilitating the production of high-resolution, denoised and colour-corrected images, matching the performance of benchtop microscopes with high-end objective lenses, also extending their limited depth-of-field. After training a convolutional neural network, we successfully imaged various samples, including blood smears, histopathology tissue sections, and parasites, where the recorded images were highly compressed to ease storage and transmission for telemedicine applications. This method is applicable to other low-cost, aberrated imaging systems, and could offer alternatives for costly and bulky microscopes, while also providing a framework for standardization of optical images for clinical and biomedical applications.