Sreya Ghosh

FEPS: a sensory substitution system for the blind to perceive facial expressions

This work demonstrates a visual-to-auditory Sensory Substitution System (SSD) called Facial Expression Perception through Sound (FEPS). It is designed to enable the visually impaired people to participate in a more effective social communication by perceiving their interlocutors facial expressions. The earlier SSDs provided feedback on inferred emotions, where as, this system responds to the facial movements. This is a better method than emotion inference due to complexities in expression-to-emotion mapping, the problem of capturing multitude of possible emotions derived from a limited facial movements and the difficulty to correctly predict emotions due to lack in ground truth data. In this work, the users ability to understand the facial expressions has been ensured by a usability study.

Characterization of a three-dimensional double-helix point-spread function for fluorescence microscopy in the presence of spherical aberration

Abstract. We characterize the three-dimensional (3-D) double-helix (DH) point-spread function (PSF) for depth-variant fluorescence microscopy imaging motivated by our interest to integrate the DH-PSF in computational optical sectioning microscopy (COSM) imaging. Physical parameters, such as refractive index and thickness variability of imaging layers encountered in 3-D microscopy give rise to depth-induced spherical aberration (SA) that change the shape of the PSF at different focusing depths and render computational approaches less practical. Theoretical and experimental studies performed to characterize the DH-PSF under varying imaging conditions are presented. Results show reasonable agreement between theoretical and experimental DH-PSFs suggesting that our model can predict the main features of the data. The depth-variability of the DH-PSF due to SA, quantified using a normalized mean square error, shows that the DH-PSF is more robust to SA than the conventional PSF. This result is also supported by the frequency analysis of the DH-PSF shown. Our studies suggest that further investigation of the DH-PSF’s use in COSM is warranted, and that particle localization accuracy using the DH-PSF calibration curve in the presence of SA can be improved by accounting for the axial shift due to SA.

Fluorescence microscopy point spread function model accounting for aberrations due to refractive index variability within a specimen

Abstract. A three-dimensional (3-D) point spread function (PSF) model for wide-field fluorescence microscopy, suitable for imaging samples with variable refractive index (RI) in multilayered media, is presented. This PSF model is a key component for accurate 3-D image restoration of thick biological samples, such as lung tissue. Microscope- and specimen-derived parameters are combined with a rigorous vectorial formulation to obtain a new PSF model that accounts for additional aberrations due to specimen RI variability. Experimental evaluation and verification of the PSF model was accomplished using images from 175-nm fluorescent beads in a controlled test sample. Fundamental experimental validation of the advantage of using improved PSFs in depth-variant restoration was accomplished by restoring experimental data from beads (6  μm in diameter) mounted in a sample with RI variation. In the investigated study, improvement in restoration accuracy in the range of 18 to 35% was observed when PSFs from the proposed model were used over restoration using PSFs from an existing model. The new PSF model was further validated by showing that its prediction compares to an experimental PSF (determined from 175-nm beads located below a thick rat lung slice) with a 42% improved accuracy over the current PSF model prediction.

Three-dimensional block-based restoration integrated with wide-field fluorescence microscopy for the investigation of thick specimens with spatially variant refractive index.

Abstract. Development of a block-based restoration (BBR) method that addresses spatially variant (SV) imaging in wide-field fluorescence microscopy of thick samples is presented. The BBR method is based on a block-based imaging model, which approximates SV imaging using an efficient orthonormal basis decomposition of multiple SV point-spread functions computed at block vertices. The effect of reducing the number of blocks needed to account for SV imaging on the restoration accuracy was investigated with simulations using a numerical lung tissue phantom relevant to biological studies. Results show that reducing the number of blocks by 82% and 98% resulted in a 19% and 27% reduction in restoration accuracy, respectively, thereby establishing a reasonable tradeoff between computational resources and accuracy. Comparison of the BBR method to existing methods (deconvolution) that do not account for SV imaging demonstrates a 90% improvement in restoration accuracy. BBR results from synthetic and experimental images of a controlled test sample with SV refractive index (RI) show consistency, providing a validation of the BBR approach. In this study, information from DIC and fluorescence images was combined to identify regions with changing RI within the imaging volume. The BBR method provides a first step toward computationally tractable reconstruction of images from thick samples.

Effect of double-helix point-spread functions on 3D imaging in the presence of spherical aberrations

Double Helix point-spread functions (DH-PSFs), the result of PSF engineering, are used for super resolution microscopy. The DH-PSF design features two dominant lobes in the image plane which rotate with the change in axial (z) position of the light point source. The center of the DH-PSF gives the precise XY location of the point source, while the orientation of the lobes gives the axial location. In this paper we investigate the effect of spherical aberrations on the DH-PSF. Physical parameters such as the lens used, the size of the particle, refractive index of medium, and depth i.e., location within the underlying object, contribute to the amount of spherical aberration. DH-PSFs with spherical aberrations are computed for different imaging conditions. Three-dimensional images were generated of computer-generated objects using both space-invariant and depth-variant approach. Different approaches to estimate intensity and location of points from these images were investigated. Our results show that the DH-PSFs are susceptible to spherical aberration leading to an apparent shift in the location of the point source with increasing spherical aberrations which is comparable to the conventional PSF. Estimation algorithms like the depth variant expectation maximization (DVEM) can be used to obtain estimates of the true underlying object from the image obtained with DH-PSFs.

A block-based forward imaging model for improved sample volume representation in computational optical sectioning microscopy

In typical fluorescence imaging systems the refractive index (RI) variability between the immersion medium of the objective lens, the coverslip, and the specimen, changes the spherical wave-front of the emitted light and introduces spherical aberrations (SA) in the acquired 3D image. In existing computational optical sectioning algorithms (COSM) to simplify the complexity of the problem, the specimen is either assumed to be thin or in the case of depth-variant algorithms to have a constant RI which is an invalid assumption for biological samples. Accurate modeling of biological samples requires a space variant (SV) imaging system i.e. a different point spread functions (PSF) for each pixel. To reduce the computational load an approximate block-based forward model is introduced in this study. The entire object space is divided into a collection of small 3D blocks where the PSFs at the faces of the blocks are known. An optimized combination of overlap-save and overlap-add methods of interpolation are used to obtain the final SV (axially and laterally variant) image. Simulated SV images using the new imaging model, of a numerical object comprising of similar structures dispersed in a medium with spatially variant RI are discussed. Images of fluorescent microspheres (6-μm in diameter) dispersed in a controlled sample with two distinct RIs are compared to simulated images of a numerical object subjected to the same imaging condition, to evaluate the new model. The accuracy of the block-based forward model to model the effect of space variance within a specimen was assessed using intensity profiles through the microspheres. The qualitative similarities in the appearance of the experimental and simulated image indicate the validity of the blockbased forward model to appropriately model samples with lateral variability in RI.

Further developments in addressing depth-variant 3D fluorescence microscopy imaging

Three-dimensional (3D) imaging with optical sectioning microscopy uses computational methods to obtain the true fluorescence distribution by ameliorating the effect of defocus, spherical aberration and noise. Inverse algorithms improve image quality at a fraction of the cost of implementing an optical system by accurate modeling of the imaging system. Good inverse imaging algorithms need to be accurate as well as fast. Better understanding of the image formation model is vital to obtain improved restoration through model-based algorithms. Forward imaging models based on a depth-varying point-spread function (DV-PSF) leads to a substantial improvement in the resulting images because it accounts for depth-induced aberrations present in the imaging system. PSFs at every layer can be represented using their principal components. Computation of the forward imaging model using a principal component analysis (PCA) representation of the DV-PSF requires fewer convolutions than a strata based approach investigated in the past. In this paper we present a new algorithm for maximum likelihood image restoration developed based on a PCA representation of the DV-PSF and an accelerated conjugate gradient (CG) iteration scheme. Results obtained with this PCA-CG algorithm from both simulated and experimental fluorescence microscope data are discussed and compared with results obtained from a CG iteration method based on the strata model and linear interpolation of the DV-PSF. The performance of the PCA-CG algorithms is shown to be promising for practical applications.

Double helix PSF engineering for computational fluorescence microscopy imaging

Point spread function engineering with a double helix (DH) phase mask has been recently used in a joint computationaloptical approach for the determination of depth and intensity information from fluorescence images. In this study, theoretically determined DH-PSFs computed from a model that incorporates different amounts of depth-induced spherical aberration (SA) due to refractive-index mismatch in the three-dimensional imaging layers, are evaluated through a comparison to empirically determined DH-PSFs measured from quantum dots. The theoretically-determined DH-PSFs show a trend that captures the main effects observed in the empirically-determined DH-PSFs. Calibration curves computed from these DH-PSFs show that SA slows down the rate of rotation observed in a DH-PSF which results in: 1) an extended range of rotation; and 2) asymmetric rotation ranges as the focus is moved in opposite directions. Thus, for accurate particle localization different calibration curves need to be known for different amounts of SA. Results also show that the DH-PSF is less sensitive to SA than the conventional PSF. Based on this result, it is expected that fewer depth-variant (DV) DH-PSFs will be required for 3D computational microscopy imaging in the presence of SA compared to the required number of conventional DV PSFs.

Space-variant image formation for 3D fluorescence microscopy using a computationally efficient block-based model

This study is an initial attempt to address space-variant (SV) image formation in 3D fluorescence microscopy using a computationally tractable block-based model. Spherical aberration (SA) introduces space-variance and can be attributed to refractive index (RI) and depth variation within the sample, hence affecting the imaging of most biological samples. Application of restoration algorithms to SV images is not practical, because it requires a different point-spread function (PSF) for each pixel. In this study, we use principal component analysis (PCA) to represent SV-PSFs, hence reducing the dimensionality of a block-based forward imaging problem. The PCA-based SV-imaging model is used to simulate the image of a test object with non-uniform RI. Images obtained from the PCA-based model and the existing block-based model show a 0.98 cross-correlation with an 85% reduction in computational resources when PCA is used. This computational efficiency can be exploited in the future by restoration algorithms to obtain improved biological images.

Block-Based Restoration Method for Wide-field Microscopy of Samples with Variable Refractive Index

Evaluation of a block based restoration (BBR) method, developed for imaging of samples with variable refractive index, is investigated using a lung tissue phantom. The BBR method results in a 98% accuracy improvement over depth-variant restoration.