Jason Deglint

Simultaneous Projector-Camera Self-Calibration for Three-Dimensional Reconstruction and Projection Mapping

Automatic calibration of structured-light systems, generally consisting of a projector and camera, is of great importance for a variety of practical applications. We propose a novel optimization approach for geometric calibration of a projector-camera system that estimates the intrinsic, extrinsic, and distortion parameters of both the camera and projector in an automatic fashion using structured light. Our approach benefits from a novel multifactor objective function that finds maximum-likelihood estimates from noisy point correspondences using constraints on focal lengths and resolves ambiguities estimating the fundamental matrix by enforcing epipolar geometry on the rectified noisy data. This new formulation allows estimation of all calibration parameters simultaneously and minimization is ensured by a greedy descent algorithm that decreases the cost function at each iteration. This provides more accurate parameter estimation, reconstruction accuracy, and robustness to noise and poor initialization compared to previous methods. Experimental results demonstrate the stability and robustness of our method, and show that the proposed solution outperforms a currently leading approach to an automatic geometric projector-camera calibration.

Virtual spectral multiplexing for applications in in-situ imaging microscopy of transient phenomena

Multispectral sensing is specifically designed to provide quantitative spectral information about various materials or scenes. Using spectral information, various properties of objects can be measured and analysed. Microscopy, the observing and imaging of objects at the micron- or nano-scale, is one application where multispectral sensing can be advantageous, as many fields of science and research that use microscopy would benefit from observing a specimen in multiple wavelengths. Multispectral microscopy is available, but often requires the operator of the device to switch filters which is a labor intensive process. Furthermore, the need for filter switching makes such systems particularly limiting in cases where the sample contains live species that are constantly moving or exhibit transient phenomena. Direct methods for capturing multispectral data of a live sample simultaneously can also be challenging for microscopy applications as it requires an elaborate optical systems design which uses beamsplitters and a number of detectors proportional to the number of bands sought after. Such devices can therefore be quite costly to build and difficult to maintain, particularly for microscopy. In this paper, we present the concept of virtual spectral demultiplexing imaging (VSDI) microscopy for low-cost in-situ multispectral microscopy of transient phenomena. In VSDI microscopy, the spectral response of a color detector in the microscope is characterized and virtual spectral demultiplexing is performed on the simultaneously-acquired broadband detector measurements based on the developed spectral characterization model to produce microscopic imagery at multiple wavelengths. The proposed VSDI microscope was used to observe colorful nanowire arrays at various wavelengths simultaneously to illustrate its efficacy.

Time-Frequency Domain Analysis via Pulselets for Non-contact Heart Rate Estimation from Remotely Acquired Photoplethysmograms

A novel method for remote heart rate estimation via analysis in the time-frequency domain is proposed. A photoplethysmogram (PPG) waveform is constructed via a Bayesian minimization approach with the required posterior probability obtained through an importance-weighted Monte Carlo sampling method. A pulselet (wavelet chosen for its similarities with a finger pulse oximiter PPG waveform), is used in the continuous wavelet transform to produce a map of the wavelet energy response in the time-frequency domain. This allows the heart rate frequency to be estimated at each time step, accounting for naturally occurring changes in heart rate over time which may cause error with frequency domain based methods. The frequency corresponding to the highest wavelet response at each time step is averaged across the entire time series to estimate the average heart rate. Experimental results against a data set of 30 videos show an improvement over current state-of-the-art methods.

Numerical Demultiplexing of Color Image Sensor Measurements via Non-linear Random Forest Modeling.

The simultaneous capture of imaging data at multiple wavelengths across the electromagnetic spectrum is highly challenging, requiring complex and costly multispectral image devices. In this study, we investigate the feasibility of simultaneous multispectral imaging using conventional image sensors with color filter arrays via a novel comprehensive framework for numerical demultiplexing of the color image sensor measurements. A numerical forward model characterizing the formation of sensor measurements from light spectra hitting the sensor is constructed based on a comprehensive spectral characterization of the sensor. A numerical demultiplexer is then learned via non-linear random forest modeling based on the forward model. Given the learned numerical demultiplexer, one can then demultiplex simultaneously-acquired measurements made by the color image sensor into reflectance intensities at discrete selectable wavelengths, resulting in a higher resolution reflectance spectrum. Experimental results demonstrate the feasibility of such a method for the purpose of simultaneous multispectral imaging.

Inference of dense spectral reflectance images from sparse reflectance measurement using non-linear regression modeling

One method to acquire multispectral images is to sequentially capture a series of images where each image contains information from a different bandwidth of light. Another method is to use a series of beamsplitters and dichroic filters to guide different bandwidths of light onto different cameras. However, these methods are very time consuming and expensive and perform poorly in dynamic scenes or when observing transient phenomena. An alternative strategy to capturing multispectral data is to infer this data using sparse spectral reflectance measurements captured using an imaging device with overlapping bandpass filters, such as a consumer digital camera using a Bayer filter pattern. Currently the only method of inferring dense reflectance spectra is the Wiener adaptive filter, which makes Gaussian assumptions about the data. However, these assumptions may not always hold true for all data. We propose a new technique to infer dense reflectance spectra from sparse spectral measurements through the use of a non-linear regression model. The non-linear regression model used in this technique is the random forest model, which is an ensemble of decision trees and trained via the spectral characterization of the optical imaging system and spectral data pair generation. This model is then evaluated by spectrally characterizing different patches on the Macbeth color chart, as well as by reconstructing inferred multispectral images. Results show that the proposed technique can produce inferred dense reflectance spectra that correlate well with the true dense reflectance spectra, which illustrates the merits of the technique.

Quantification of cyanobacterial cells via a novel imaging-driven technique with an integrated fluorescence signature

A novel imaging-driven technique with an integrated fluorescence signature to enable automated enumeration of two species of cyanobacteria and an alga of somewhat similar morphology to one of the cyanobacteria is presented to demonstrate proof-of-concept that high accuracy, imaging-based, rapid water quality analysis can be with conventional equipment available in typical water quality laboratories-this is not currently available. The results presented herein demonstrate that the developed method identifies and enumerates cyanobacterial cells at a level equivalent to or better than that achieved using standard manual microscopic enumeration techniques, but in less time, and requiring significantly fewer resources. When compared with indirect measurement methods, the proposed method provides better accuracy at both low and high cell concentrations. It extends the detection range for cell enumeration while maintaining accuracy and increasing enumeration speed. The developed method not only accurately estimates cell concentrations, but it also reliably distinguishes between cells of Anabaena flos-aquae, Microcystis aeruginosa, and Ankistrodesmus in mixed cultures by taking advantage of additional contrast between the target cell and complex background gained under fluorescent light. Thus, the proposed image-driven approach offers promise as a robust and cost-effective tool for identifying and enumerating microscopic cells based on their unique morphological features.

SAPPHIRE: Stochastically acquired photoplethysmogram for heart rate inference in realistic environments

A novel method, Stochastically Acquired Photoplethysmo-gram for Heart rate Inference in Realistic Environments (SAPPHIRE), is proposed for robust remote heart rate measurement through broadband video. A set of stochastically sampled points from the cheek region is tracked and used to construct corresponding time series observations via skin erythema transforms. From these observations, a photo-plethysmogram (PPG) waveform is estimated via Bayesian minimization, with the required posterior probability inferred using a Monte Carlo approach. To mitigate the effects of noise, the contribution of each observation is weighted based on the observations likelihood to contain relevant data. A bandpass filter is applied to the estimated PPG waveform to omit implausible heart rate frequencies, and the heart rate is estimated through frequency domain analysis. Experimental results acquired from a set of thirty videos indicate significantly improved performance in comparison to state-of-the-art methods.

Photoplethysmographic imaging via spectrally demultiplexed erythema fluctuation analysis for remote heart rate monitoring

Traditional photoplethysmographic imaging (PPGI) systems use the red, green, and blue (RGB) broadband measurements of a consumer digital camera to remotely estimate a patients heart rate; however, these broadband RGB signals are often corrupted by ambient noise, making the extraction of subtle fluctuations indicative of heart rate difficult. Therefore, the use of narrow-band spectral measurements can significantly improve the accuracy. We propose a novel digital spectral demultiplexing (DSD) method to infer narrow-band spectral information from acquired broadband RGB measurements in order to estimate heart rate via the computation of motion- compensated skin erythema fluctuation. Using high-resolution video recordings of human participants, multiple measurement locations are automatically identified on the cheeks of an individual, and motion-compensated broadband reflectance measurements are acquired at each measurement location over time via measurement location tracking. The motion-compensated broadband reflectance measurements are spectrally demultiplexed using a non-linear inverse model based on the spectral sensitivity of the cameras detector. A PPG signal is then computed from the demultiplexed narrow-band spectral information via skin erythema fluctuation analysis, with improved signal-to-noise ratio allowing for reliable remote heart rate measurements. To assess the effectiveness of the proposed system, a set of experiments involving human motion in a front-facing position were performed under ambient lighting conditions. Experimental results indicate that the proposed system achieves robust and accurate heart rate measurements and can provide additional information about the participant beyond the capabilities of traditional PPGI methods.

Bayesian Compensated Microscopy

We present a novel Bayesian compensated microscopy (BCM) technique designed for enhancing microscopy image quality. The proposed BCM technique provides a computational approach to jointly compensate for microscopy image degradations due to (1) optical aberrations, (2) illumination non-uniformities, and (3) imaging noise within a probabilistic framework. Experimental results based on a stained pathology sample of spleen tissue with leukemia demonstrate the effectiveness of the proposed BCM technique for the quality enhancement in microscopy imaging. The proposed BCM technique can lead to improved visualization of fine tissue structures as well as a more consistent visualization across the entire sample, which can be beneficial for accurate analysis and better interpretation of microscopy samples.