Ehsan Miandji

Compressive Image Reconstruction in Reduced Union of Subspaces

We present a new compressed sensing framework for reconstruction of incomplete and possibly noisy images and their higher dimensional variants, e.g. animations and light‐fields. The algorithm relies on a learning‐based basis representation. We train an ensemble of intrinsically two‐dimensional (2D) dictionaries that operate locally on a set of 2D patches extracted from the input data. We show that one can convert the problem of 2D sparse signal recovery to an equivalent 1D form, enabling us to utilize a large family of sparse solvers. The proposed framework represents the input signals in a reduced union of subspaces model, while allowing sparsity in each subspace. Such a model leads to a much more sparse representation than widely used methods such as K‐SVD. To evaluate our method, we apply it to three different scenarios where the signal dimensionality varies from 2D (images) to 3D (animations) and 4D (light‐fields). We show that our method outperforms state‐of‐the‐art algorithms in computer graphics and image processing literature.

Photorealistic rendering of mixed reality scenes

Photo‐realistic rendering of virtual objects into real scenes is one of the most important research problems in computer graphics. Methods for capture and rendering of mixed reality scenes are driven by a large number of applications, ranging from augmented reality to visual effects and product visualization. Recent developments in computer graphics, computer vision, and imaging technology have enabled a wide range of new mixed reality techniques including methods for advanced image based lighting, capturing spatially varying lighting conditions, and algorithms for seamlessly rendering virtual objects directly into photographs without explicit measurements of the scene lighting. This report gives an overview of the state‐of‐the‐art in this field, and presents a categorization and comparison of current methods. Our in‐depth survey provides a tool for understanding the advantages and disadvantages of each method, and gives an overview of which technique is best suited to a specific problem.

Learning based compression of surface light fields for real-time rendering of global illumination scenes

We present an algorithm for compression and real-time rendering of surface light fields (SLF) encoding the visual appearance of objects in static scenes with high frequency variations. We apply a non-local clustering in order to exploit spatial coherence in the SLF data. To efficiently encode the data in each cluster, we introduce a learning based approach, Clustered Exemplar Orthogonal Bases (CEOB), which trains a compact dictionary of orthogonal basis pairs, enabling efficient sparse projection of the SLF data. In addition, we discuss the application of the traditional Clustered Principal Component Analysis (CPCA) on SLF data, and show that in most cases, CEOB outperforms CPCA, K-SVD and spherical harmonics in terms of memory footprint, rendering performance and reconstruction quality. Our method enables efficient reconstruction and real-time rendering of scenes with complex materials and light sources, not possible to render in real-time using previous methods.

Real-time multi-band synthesis of ocean water with new iterative up-sampling technique

Adapting natural phenomena rendering for real-time applications has become a common practice in computer graphics. We propose a GPU-based multi-band method for optimized synthesis of “far from coast” ocean waves using an empirical Fourier domain model. Instead of performing two independent syntheses for low- and high-band frequencies of ocean waves, we perform only low-band synthesis and employ results to reproduce high frequency details of ocean surface by an optimized iterative up-sampling stage. Our experimental results show that this approach greatly improves the performance of original multi-band synthesis while maintaining image quality.

On Probability of Support Recovery for Orthogonal Matching Pursuit Using Mutual Coherence

In this letter, we present a new coherence-based performance guarantee for the orthogonal matching pursuit (OMP) algorithm. A lower bound for the probability of correctly identifying the support of a sparse signal with additive white Gaussian noise is derived. Compared to previous research work, the new bound takes into account the signal parameters, such as dynamic range, noise variance, and sparsity. Numerical simulations show significant improvements over previous research work and a closer match to empirically obtained results of the OMP algorithm.

OMP-based DOA estimation performance analysis

In this paper, we present a new performance guarantee for Orthogonal Matching Pursuit (OMP) in the context of the Direction Of Arrival (DOA) estimation problem. For the first time, the effect of pa ...

A Performance Guarantee for Orthogonal Matching Pursuit Using Mutual Coherence

In this paper, we present a new performance guarantee for the orthogonal matching pursuit (OMP) algorithm. We use mutual coherence as a metric for determining the suitability of an arbitrary overcomplete dictionary for exact recovery. Specifically, a lower bound for the probability of correctly identifying the support of a sparse signal with additive white Gaussian noise and an upper bound for the mean square error is derived. Compared to the previous work, the new bound takes into account the signal parameters such as dynamic range, noise variance, and sparsity. Numerical simulations show significant improvements over previous work and a much closer correlation to empirical results of OMP.

On nonlocal image completion using an ensemble of dictionaries

In this paper we consider the problem of nonlocal image completion from random measurements and using an ensemble of dictionaries. Utilizing recent advances in the field of compressed sensing, we derive conditions under which one can uniquely recover an incomplete image with overwhelming probability. The theoretical results are complemented by numerical simulations using various ensembles of analytical and training-based dictionaries.

Facial Expression Recognition Using Facial Graph

Automatic analysis of human facial expression is one of the challenging problems in machine vision systems. It has many applications in human-computer interactions, social robots, deceit detection, interactive video and behavior monitoring. In this paper, we developed a new method for automatic facial expression recognition based on verifying movable facial elements and tracking nodes in sequential frames. The algorithm plots a face model graph in each frame and extracts features by measuring the ratio of the facial graph sides. Seven facial expressions, including neutral pose are being classified in this study using support vector machine and other classifiers on JAFFE databases. The approach does not rely on action units, and therefore eliminates errors which are otherwise propagated to the final result due to incorrect initial identification of action units. Experimental results show that analyzing facial movements gives accurate and efficient information in order to identify different facial expressions.

Learning based compression for real-time rendering of surface light fields

Photo-realistic rendering in real-time is a key challenge in computer graphics. A number of techniques where the light transport in a scene is pre-computed, compressed and used for real-time image synthesis have been proposed, e.g. [Ramamoorthi 2009]. We extend on this idea and present a technique where the radiance distribution in a scene, including arbitrarily complex materials and light sources, is pre-computed and stored as surface light fields (SLF) at each surface. An SLF describes the full appearance of each surface in a scene as a 4D function over the spatial and angular domains. An SLF is a complex data set with a large memory footprint often in the order of several GB per object in the scene. The key contribution in this work is a novel approach for compression of SLFs enabling real-time rendering of complex scenes. Our learning-based compression technique is based on exemplar orthogonal bases (EOB) [Gurumoorthy et al. 2010], and trains a compact dictionary of full-rank orthogonal basis pairs with sparse coefficients. Our results outperform the widely used CPCA method [Miandji et al. 2011] in terms of storage cost, visual quality and rendering speed. Compared to PRT techniques for real-time global illumination, our approach is limited to static scenes but can represent high frequency materials and any type of light source in a unified framework.