Ayush Bhandari

Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles

Time of flight cameras produce real-time range maps at a relatively low cost using continuous wave amplitude modulation and demodulation. However, they are geared to measure range (or phase) for a single reflected bounce of light and suffer from systematic errors due to multipath interference. We re-purpose the conventional time of flight device for a new goal: to recover per-pixel sparse time profiles expressed as a sequence of impulses. With this modification, we show that we can not only address multipath interference but also enable new applications such as recovering depth of near-transparent surfaces, looking through diffusers and creating time-profile movies of sweeping light. Our key idea is to formulate the forward amplitude modulated light propagation as a convolution with custom codes, record samples by introducing a simple sequence of electronic time delays, and perform sparse deconvolution to recover sequences of Diracs that correspond to multipath returns. Applications to computer vision include ranging of near-transparent objects and subsurface imaging through diffusers. Our low cost prototype may lead to new insights regarding forward and inverse problems in light transport.

Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization

Time-of-flight (ToF) cameras calculate depth maps by reconstructing phase shifts of amplitude-modulated signals. For broad illumination of transparent objects, reflections from multiple scene points can illuminate a given pixel, giving rise to an erroneous depth map. We report here a sparsity-regularized solution that separates K interfering components using multiple modulation frequency measurements. The method maps ToF imaging to the general framework of spectral estimation theory and has applications in improving depth profiles and exploiting multiple scattering.

Sampling and Reconstruction of Sparse Signals in Fractional Fourier Domain

Sampling theory for continuous time signals which have a bandlimited representation in fractional Fourier transform (FrFT) domain-a transformation which generalizes the conventional Fourier transform-has blossomed in the recent past. The mechanistic principles behind Shannons sampling theorem for fractional bandlimited (or fractional Fourier bandlimited) signals are the same as for the Fourier domain case i.e. sampling (and reconstruction) in FrFT domain can be seen as an orthogonal projection of a signal onto a subspace of fractional bandlimited signals. As neat as this extension of Shannons framework is, it inherits the same fundamental limitation that is prevalent in the Fourier regime-what happens if the signals have singularities in the time domain (or the signal has a nonbandlimited spectrum)? In this paper, we propose a uniform sampling and reconstruction scheme for a class of signals which are nonbandlimited in FrFT sense. Specifically, we assume that samples of a smoothed version of a periodic stream of Diracs (which is sparse in time-domain) are accessible. In its parametric form, this signal has a finite number of degrees of freedom per unit time. Based on the representation of this signal in FrFT domain, we derive conditions under which exact recovery of parameters of the signal is possible. Knowledge of these parameters leads to exact reconstruction of the original signal.

Shift-Invariant and Sampling Spaces Associated With the Fractional Fourier Transform Domain

Shift-invariant spaces play an important role in sampling theory, multiresolution analysis, and many other areas of signal and image processing. A special class of the shift-invariant spaces is the class of sampling spaces in which functions are determined by their values on a discrete set of points. One of the vital tools used in the study of sampling spaces is the Zak transform. The Zak transform is also related to the Poisson summation formula and a common thread between all these notions is the Fourier transform. In this paper, we extend some of these notions to the fractional Fourier transform (FrFT) domain. First, we introduce two definitions of the discrete fractional Fourier transform and two semi-discrete fractional convolutions associated with them. We employ these definitions to derive necessary and sufficient conditions pertaining to FrFT domain, under which integer shifts of a function form an orthogonal basis or a Riesz basis for a shift-invariant space. We also introduce the fractional Zak transform and derive two different versions of the Poisson summation formula for the FrFT. These extensions are used to obtain new results concerning sampling spaces, to derive the reproducing-kernel for the spaces of fractional band-limited signals, and to obtain a new simple proof of the sampling theorem for signals in that space. Finally, we present an application of our shift-invariant signal model which is linked with the problem of fractional delay filtering.

Automatic Electrooculogram Classification for Microcontroller Based Interface Design

In this paper, we present a simple and novel technique for classification of multiple channel Electrooculogram signals (EOG). In particular, a viable real time EOG signal classifier for microcontrollers is proposed. The classifier is based on Deterministic Finite Automata (DFA). The system is capable of classifying sixteen different EOG signals. The viability of the system was tested by performing online experiments with able bodied subjects.

Blind and reference-free fluorescence lifetime estimation via consumer time-of-flight sensors

Fluorescence lifetime imaging (FLI) is a popular method for extracting useful information that is otherwise unavailable from a conventional intensity image. Usually, however, it requires expensive equipment, is often limited to either distinctly frequency- or time-domain modalities, and demands calibration measurements and precise knowledge of the illumination signal. Here, we present a generalized time-based, cost-effective method for estimating lifetimes by repurposing a consumer-grade time-of-flight sensor. By developing mathematical theory that unifies time- and frequency-domain approaches, we can interpret a time-based signal as a combination of multiple frequency measurements. We show that we can estimate lifetimes without knowledge of the illumination signal and without any calibration. We experimentally demonstrate this blind, reference-free method using a quantum dot solution and discuss the method’s implementation in FLI applications.

Demultiplexing illumination via low cost sensing and nanosecond coding

Several computer vision algorithms require a sequence of photographs taken in different illumination conditions, which has spurred development in the area of illumination multiplexing. Various techniques for optimizing the multiplexing process already exist, but are geared toward regular or high speed cameras. Such cameras are fast, but code on the order of milliseconds. In this paper we propose a fusion of two popular contexts, time of flight range cameras and illumination multiplexing. Time of flight cameras are a low cost, consumer-oriented technology capable of acquiring range maps at 30 frames per second. Such cameras have a natural connection to conventional illumination multiplexing strategies as both paradigms rely on the capture of multiple shots and synchronized illumination. While previous work on illumination multiplexing has exploited coding at millisecond intervals, we repurpose sensors that are ordinarily used in time of flight imaging to demultiplex via nanosecond coding strategies.

Resolving Multipath Interference in Kinect: An Inverse Problem Approach

Multipath interference is an inherent unsolved problem in amplitude modulated continuous wave time-of-flight imaging that results in erroneous depth and amplitude measurements. This kind of interference is the result of light which travels along different optical paths and reaches the same pixel. Important scenarios with strong interference include scenes with highly reflective surfaces, imaging through translucent material, and subsurface scattering and imaging a corner. In this paper, we demonstrate that multiple frequency measurements made with a noncustomized Kinect for the XBox One (the new Kinect) camera can be used to resolve the interfering paths. By formulating our problem as a spectral estimation problem, we provide a closed-form, noniterative technique.

Sparse Linear Operator identification without sparse regularization? Applications to mixed pixel problem in Time-of-Flight/Range imaging

In this paper, we consider the problem of Sparse Linear Operator identification which is also linked with the topic of Sparse Deconvolution. In its abstract form, the problem can be stated as follows: Given a well behaved probing function, is it possible to identify a Sparse Linear Operator from its response to the function? We present a constructive solution to this problem. Furthermore, our approach is devoid of any sparsity inducing penalty term and explores the idea of parametric modeling. Consequently, our algorithm is non-iterative by design and circumvents tuning of any regularization parameter. Our approach is computationally efficient when compared the ℓ0/ℓ1-norm regularized counterparts. Our work addresses a problem of industrial significance: decomposition of mixed-pixels in Time-of-Flight/Range imaging. In this case, each pixel records range measurements from multiple contributing depths and the goal is to isolate each depth. Practical experiments corroborate our theoretical set-up and establish the efficiency of our approach, that is, speed-up in processing with lesser mean squared error. We also derive Cramér-Rao Bounds for performance characterization.

Wavelet based Novel Technique for Signal Conditioning of Electro-Oculogram Signals

In this paper, we present a novel and simple technique for signal conditioning of EOG signals which primarily involves denoising corrupted signals and post-processing for signal enhancement. Researches in the past have mainly focused on the EOG signals where the problem of removal of ocular artifacts from the electroencephalogram was dealt. We present, from a new perspective, a scheme which essentially deals with enhancement of EOG Signals. The non-stationary and time-varying EOG signals are processed using methodologies anchored on multiresolution analyses and the wavelet transform theory. Coiflet wavelets are used for subsequent removal of noise from the (awgn) corrupted EOG signals using the concept of coefficient thresholding. SURE is used for threshold selection. Its performance, in terms of SNR, is compared with strategies suggested by Birge-Massart and Donoho and Johnstone. Haar based wavelets of higher orders are used for post-processing of EOG signals. A pronounced advantage of post-processing of signals is that it facilitates the estimation of time instants and durations of intentional eye gestures which mainly find application in the development of human-computer interface based devices