Abdolreza Abdolhosseini Moghadam

Compressive Framework for Demosaicing of Natural Images

Typical consumer digital cameras sense only one out of three color components per image pixel. The problem of demosaicing deals with interpolating those missing color components. In this paper, we present compressive demosaicing (CD), a framework for demosaicing natural images based on the theory of compressed sensing (CS). Given sensed samples of an image, CD employs a CS solver to find the sparse representation of that image under a fixed sparsifying dictionary Ψ. As opposed to state of the art CS-based demosaicing approaches, we consider a clear distinction between the interchannel (color) and interpixel correlations of natural images. Utilizing some well-known facts about the human visual system, those two types of correlations are utilized in a nonseparable format to construct the sparsifying transform Ψ. Our simulation results verify that CD performs better (both visually and in terms of PSNR) than leading demosaicing approaches when applied to the majority of standard test images.

Complex sparse projections for compressed sensing

Sparse projections for compressed sensing have been receiving some attention recently. In this paper, we consider the problem of recovering a k-sparse signal (x) in an n-dimensional space from a limited number (m) of linear, noiseless compressive samples (y) using complex sparse projections. Our approach is based on constructing complex sparse projections using strategies rooted in combinatorial design and expander graphs. We are able to recover the non-zero coefficients of the k-sparse signal (x) iteratively using a low-complexity algorithm that is reminiscent of well-known iterative channel decoding methods. We show that the proposed framework is optimal in terms of sample requirements for signal recovery (m = O (k log(n/k))) and has a decoding complexity of O (m log(n/m)), which represents a tangible improvement over recent solvers. Moreover we prove that using the proposed complex-sparse framework, on average 2k ≪ m ≤ 4k real measurements (where each complex sample is counted as two real measurements) suffice to recover a k-sparse signal perfectly.

Bayer and panchromatic color filter array demosaicing by sparse recovery

The utility of Compressed Sensing (CS) for demosaicing of digital images have been explored by few recent efforts. Most recently, a Compressive Demosaicing [3] framework, based on employing a random panchromatic Color Filter Array (CFA) at the sensing stage, has provided compelling CS-based demosaicing results by visually outperforming other leading techniques. Meanwhile, it is well known that the Bayer pattern is arguably the most popular CFA used in low-cost consumer digital cameras. In this paper, we explore and compare the Bayer and random panchromatic CFA structures using a generic approach for demosaicing of images based on recent advances in the field of CS. In particular, a key objective of this work is to provide a comparative analysis between these two CFA patterns (Bayer and random panchromatic) under the general umbrella of sparse recovery, which represents the cornerstone of CS-based decoding. We demonstrate the viability of the Bayer pattern under certain CS conditions. Meanwhile, we show that a random panchromatic CFA, which meets certain incoherence constraints, can visually outperform a Bayer based sparse recovery. As illustrated in our simulation results, a panchromatic CFA is more consistent in terms of providing better visual quality when tested on a wide range of color images.

Compressive demosaicing for periodic color filter arrays

The utility of Compressed Sensing (CS) for demosaicing of images captured using random panchromatic color filter arrays (CFA) has been investigated in [1]. Meanwhile, most camera manufacturers employ periodic CFAs such as the popular Bayer CFA. In this paper, we derive a CS-based solution to demosaicing images captured using the general class of periodic CFAs. It is well known that periodic CFAs can be designed to effectively separate luminance and chrominance frequency bands [2, 3]. We employ this ability to reduce artifacts associated with luminance-chrominance overlap at the solver side. We show that the modified compressive demo-saicing method coupled with the additional constraint that chrominance channels have smooth surfaces achieves further improved results for most periodic CFAs.

Incoherent color frames for compressive demosaicing

In this paper, we explore the notion of using frames to project sensed colors within their inherently 3D space onto a larger number of color basis vectors. In particular, we develop a new frame design, Incoherent Color Frames (ICF), which can include an arbitrary number of incoherent color vectors. An ICF frame possesses key desired properties including the ability to sparsify colors in 3D and to decorrelate color channels utilizing a spatial-frequency selective strategy. We present a low complexity algorithm for constructing ICF frames targeted for the problem of image demosaicing. Our simulation results show that when incorporating the proposed ICF within a Compressive Demosaicing (CD) framework [8], significant visual improvements can be achieved when compared with traditional and Compressed Sesnsing-based demosaicing solutions.

Common and Innovative Visuals: A Sparsity Modeling Framework for Video

Efficient video representation models are critical for many video analysis and processing tasks. In this paper, we present a framework based on the concept of finding the sparsest solution to model video frames. To model the spatio-temporal information, frames from one scene are decomposed into two components: (1) a common frame, which describes the visual information common to all the frames in the scene/segment and (2) a set of innovative frames, which depicts the dynamic behaviour of the scene. The proposed approach exploits and builds on recent results in the field of compressed sensing to jointly estimate the common frame and the innovative frames for each video segment. We refer to the proposed modeling framework by common and innovative visuals (CIV). We show how the proposed model can be utilized to find scene change boundaries and extend CIV to videos from multiple scenes. Furthermore, the proposed model is robust to noise and can be used for various video processing applications without relying on motion estimation and detection or image segmentation. Results for object tracking, video editing (object removal, inpainting), and scene change detection are presented to demonstrate the efficiency and performance of the proposed model.

Sparse Expander-like Real-valued Projection (SERP) matrices for compressed sensing

Sparse binary projection matrices are arguably the most commonly used sensing matrices in combinatorial approaches to Compressed Sensing (CS). In this paper, we are interested in properties of Sparse Expander-like Real-valued Projection (SERP) matrices that are constructed by replacing the non-zero entries of sparse binary projection matrices by Gaussian random variables. We prove that these sparse real-valued matrices have a “weak” form of Restricted Isometery Property (RIP). We show that such weak RIP enables this class of matrices to be utilized in all three approaches to the problem of Compressed Sensing, i.e. greedy, geometrical and combinatorial.

Hybrid Compressed Sensing of images

We consider the problem of recovering a signal/image (x) with a k-sparse representation, from hybrid (complex and real), noiseless linear samples (y) using a mixture of complex-valued sparse and real-valued dense projections within a single matrix. The proposed Hybrid Compressed Sensing (HCS) employs the complex-sparse part of the projection matrix to divide the n-dimensional signal (x) into subsets. In turn, each subset of the signal (coefficients) is mapped onto a complex sample of the measurement vector (y). Under a worst-case scenario of such sparsity-induced mapping, when the number of complex sparse measurements is sufficiently large then this mapping leads to the isolation of a significant fraction of the k non-zero coefficients into different complex measurement samples from y. Using a simple property of complex numbers (namely complex phases) one can identify the isolated non-zeros of x. After reducing the effect of the identified non-zero coefficients from the compressive samples, we utilize the real-valued dense submatrix to form a full rank system of equations to recover the signal values in the remaining indices (that are not recovered by the sparse complex projection part). We show that the proposed hybrid approach can recover a k-sparse signal (with high probability) while requiring only m ≈ 3√n/2k real measurements (where each complex sample is counted as two real measurements). We also derive expressions for the optimal mix of complex-sparse and real-dense rows within an HCS projection matrix. Further, in a practical range of sparsity ratio (k/n) suitable for images, the hybrid approach outperforms even the most complex compressed sensing frameworks (namely basis pursuit with dense Gaussian matrices). The theoretical complexity of HCS is less than the complexity of solving a full-rank system of m linear equations. In practice, the complexity can be lower than this bound.

Sensitivity analysis in RIPless compressed sensing

Sensitivity analysis in optimization theory explores how the solution to a particular optimization problem changes as the objective function or constraints of such optimization problem perturb. A recent and yet important class of optimization problems is the framework of compressed sensing where the objective is to find the sparsest solution to an under-determined and possibly noisy system of linear equations. In this paper, we show that by utilizing some tools in sensitivity analysis, namely Invariant Support Sets (ISS), one can improve certain developed results in the field of compressed sensing. More specifically, we show that in a noiseless and RIP-less setting [11], the recovery process of a compressed sensing framework is a binary event in the sense that either all vectors with the same support and sign pattern can be recovered from their compressive samples or none can be estimated correctly. However, in a noisy and RIP-less setting, recovering only one signal from its limited noisy samples guarantees that there exist signals (possibly even with different supports and sign patterns) and noise vectors that shall be recovered with good accuracies by using Lasso.

Randomness-in-Structured Ensembles for compressed sensing of images

Leading compressed sensing (CS) methods require m = O (k log(n)) compressive samples to perfectly reconstruct a k-sparse signal x of size n using random projection matrices (e.g., Gaussian or random Fourier matrices). For a given m, perfect reconstruction usually requires high complexity methods, such as Basis Pursuit (BP), which has complexity O(n3). Meanwhile, low-complexity greedy algorithms do not achieve the same level of performance (as BP) in terms of the quality of the reconstructed signal for the same m. In this paper, we introduce a new CS framework, which we refer to as Randomness-in-Structured Ensemble (RISE) projection. RISE projection matrices enable compressive sampling of image coefficients from random locations within the k-sparse image vector while imposing small structured overlaps. We prove that RISE-based compressed sensing requires only m = ck samples (where c is not a function of n) to perfectly recover a k-sparse image signal. For the case of n ≤ O(k2), the complexity of our solver is O(nk) which is less than the complexity of the popular greedy algorithm Orthogonal Matching Pursuit (OMP). Moreover, in practice we only need m = 2k samples to reconstruct the signal. We present simulation results that demonstrate the RISE frameworks ability to recover the original image with higher than 50 dB PSNR, whereas other leading approaches (such as BP) can achieve PSNR values around 30 dB only.