Soosan Beheshti

A new information-theoretic approach to signal denoising and best basis selection

The problem of signal denoising with an orthogonal basis is considered. The existing approaches convert the considered problem into one of finding a threshold for estimates of basis coefficients. In this paper, a new solution to the denoising problem is proposed. The method is based on the description length of the noiseless data in subspaces of the bases. For each subspace, we estimate the desired description length and suggest choosing the subspace for which this quantity is minimized. We provide a method of probabilistically estimating the reconstruction error. This estimate is used for probabilistic validation of the desired description length. In existing thresholding methods, the optimum threshold is obtained as a function of the additive noise variance. In practical problems, where the noise variance is unknown, the first step is to estimate the noise variance. The estimated noise variance is then used in calculating the optimum threshold. Unlike such approaches, in the proposed method, the noise variance estimation and the signal denoising are done simultaneously.

Joint intersymbol and multiple-access interference suppression algorithms for CDMA systems

Two promising classes of techniques are developed for efficient multiuser detection in code-division multiple-access (CDMA) communication systems subject to fading due to time-varying multipath propagation. Both are designed to jointly suppress both intersymbol and multiple-access interference inherent in such systems, and exploit all available time and frequency diversity. The first is a family of linear receivers for time-varying multiuser channels that generalize familiar linear equalizers designed for traditional single-user linear time-invariant channels. Minimum mean-square error, zero-forcing, and matched-filter versions of such multiuser detectors are all developed within a common state-space framework, and have convenient recursive implementations. Performance issues as well as natural decision-feedback variants of the detector structure are both discussed. The second is a family of nonlinear receivers that are specifically designed for use with spread-signature CDMA systems on time-varying multipath channels. These multiuser detectors employ a batch-iterative (multipass or “turbo” stayl) decoding algorithm based on a successive-cancellation strategy. Several aspects of the performance of this algorithm are developed, including its monotonic convergence property. Collectively, these two classes of algorithmic structures for joint equalization, interference suppression, demodulation, and detection are representative of several emerging and interrelated approaches to receiver design for next-generation CDMA systems.

Adaptive Noise Variance Estimation in BayesShrink

A method of noise variance estimation in BayesShrink image denoising is presented. The proposed approach competes with the well known MAD-based method and outperforms this method in more than 99% of our experimental results. The approach, called Residual Autocorrelation Power (RAP), provides a more accurate noise variance estimate and results in a smaller MSE.

Noise Invalidation Denoising

A denoising technique based on noise invalidation is proposed. The adaptive approach derives a noise signature from the noise order statistics and utilizes the signature to denoise the data. The novelty of this approach is in presenting a general-purpose denoising in the sense that it does not need to employ any particular assumption on the structure of the noise-free signal, such as data smoothness or sparsity of the coefficients. An advantage of the method is in denoising the corrupted data in any complete basis transformation (orthogonal or non-orthogonal). Experimental results show that the proposed method, called noise invalidation denoising (NIDe), outperforms existing denoising approaches in terms of mean square error (MSE).

Simultaneous Denoising and Intrinsic Order Selection in Hyperspectral Imaging

In this paper, we address the problem of order selection in noisy hyperspectral applications. In conventional unmixing methods, this problem has been divided into two separate processes of order selection and unmixing. Order selection methods generally use a denoising approach at the beginning stage. The data in this case pass through three stages: denoising, order selection, and unmixing. Each of these steps mainly aims to optimize a different criterion independently. In addition, any error created in the denoising process will be propagated not only to the order selection stage but also consequently to the unmixing results. Commonly used denoising methods such as eigenvalue-decomposition-based methods, e.g., singular-value-decomposition-based methods, provide a threshold value to separate the noise from the signal. These approaches are heavily sensitive to the threshold value and signal-to-noise ratio (SNR). Moreover, these methods tend to lose their efficiency rapidly for lower SNRs. Note that both the denoising step and the dimension estimation step aim to provide the optimum estimate of the same noiseless data. Consequently, adopting a simultaneous denoising and dimension estimation method with a goal to provide the optimum estimate of the desired noiseless data is rational. This process not only avoids possible error propagations from the denoising stage to the dimension estimation stage but also unifies the optimization criteria that were used in each of these steps. In this paper, a simultaneous denoising and dimension estimation method is introduced. The approach is based on minimizing the estimated mean square error. Minimization is done by comparing the estimated data in a range of subspaces dictated by a simultaneous process. Minimizing the error at once, the proposed method denoises the data and provides the optimum dimension simultaneously. Owing to the parallel processing of denoising and dimension estimation, the simulation results show the advantages of the proposed method over some of the state-of-the-art approaches and illustrate a substantial performance, particularly for cases with a lower SNR.

Calculation of abundance factors in hyperspectral imaging using genetic algorithm

Spatial resolution is a limiting factor in satellite imaging systems. It is usually very difficult to successfully interpret objects from a coarse resolution image. Images at such coarse resolutions result in mixed pixels. Mixed-pixel decomposition or spectral unmixing applies to derivation of constituent components, endmembers(EM), and their fractional proportions(abundances) at the subpixel scale. The mathematical intractability of the abundance non-negative constraint results in complex and extensive numerical approaches. Due to such mathematical intractability, many least square error(LSE) based methods are unconstrained and can only produce sub-optimal solutions. In this paper we propose a mixed genetic algorithm and LSE-based EM estimation method (LSEM) to extract the EM matrix and related abundances vectors. We apply the proposed GA-LSEM method to the subject of unmixing hyperspectral data. The experimental results obtained from simulated images show the effectiveness of the proposed method, specifically the robustness to noise.

Correlation Based Online Dictionary Learning Algorithm

The goal of dictionary learning algorithms is to learn a set of atoms called dictionary from a set of training data such that each training data can be represented sparsely by the dictionary. Most dictionary learning algorithms use two alternating steps, sparse coding and dictionary update, to solve this problem. In this paper, we propose a new online dictionary learning algorithm with a novel dictionary update step. In the new update method, only the atoms involved in the sparse representation of new training data are adaptively updated. The adaptive update also includes using the previous training data that have common atoms with the new training data in their sparse representation. The proposed algorithm reduces the computational complexity by reducing the number of atoms updating at each iteration and the number of training data contributing in the dictionary update. In the proposed algorithm, different weights are given to training data, so each has the proper level of influence in the dictionary update. The simulation results show this improves the performance of the algorithm, in convergence speed and representation error. We also show that our algorithm has the ability of full recovery of dictionary with only one trial over training data and does not need any dictionary pruning to remove the unused atoms of the dictionary. We compare our proposed algorithm with well-known online and batch based methods using synthetic and autoregressive data.

Adaptive Bayesian Denoising for General Gaussian Distributed Signals

We study behavior of the Bayesian estimator for noisy General Gaussian Distributed (GGD) data and show that this estimator can be well estimated with a simple shrinkage function. The four parameters of this shrinkage function are functions of GGDs shape parameter and data variance. The Shrinkage map, denoted by Rigorous BayesShrink (R-BayesShrink), models the Bayesian estimator for any value of shape parameter. In addition, when the shape parameter is between 0.5 and 1, this Shrinkage function transforms into a simple soft threshold. This result places the role of soft thresholding image denoising methods, such as BayesSkrink, in a new theoretical perspective. Moreover, BayesShrink is shown to be a special case of R-BayesShrink when the shape parameter is one (Laplacian distribution). Our simulation results confirm optimality of R-BayesShrink in GGD signal denoising in the sense of Peak Signal to Noise Ratio (PSNR) and Structural Similarity (SSIM) index for a range of shape parameters.

CANDECOMP/PARAFAC model order selection based on Reconstruction Error in the presence of Kronecker structured colored noise

Canonical Decomposition (CANDECOMP) also known as Parallel Factor Analysis (PARAFAC) is a well-known multiway model in high-dimensional data modeling. Approaches that use CANDECOMP/PARAFAC for parametric modeling of a noisy observation require an estimate of the number of signal components (rank) of the data as well. In real applications, the true model of data is unknown and model order selection is a challenging step of these algorithms. In addition, considering noise samples with correlation in different dimensions makes the model order selection even more challenging. Model order selection methods generally minimize a criterion to find the optimum model order. In this paper, we propose using the Reconstruction error, which is the error between the reconstructed data and the unavailable noiseless data, for a range of possible ranks, and use an estimate of this error as the desired criterion for order selection. Furthermore, we propose using the CORCONDIA measure for determining the range of possible model orders. In the presence of the colored noise with Kronecker structure, our proposed algorithm performs the multidimensional prewhitening prior to the model order selection. In addition, our method is able to estimate the noise covariance through an iterative algorithm when no prior information about the noise covariance is available. Simulation results show that the proposed method can be effectively exploited for robustly detecting the true rank of the observed tensor even in mid and low SNRs (i.e. 0-10 dB). It also has an advantage over the state-of-the-art methods, such as different variants of CORCONDIA, by having a better Probability of Detection (PoD) with almost no extra computational overhead after the CANDECOMP/PARAFAC decomposition.

MACE-means clustering

Abstract In this paper, we tackle the problem of estimating the correct number of clusters from a new perspective. The proposed method probabilistically estimates the Average Central Error (ACE), which is the difference between the true cluster centers and their estimations. The novelty of this work is partly in estimating the unavailable ACE by using the available cluster compactness that is the difference between estimated centers and their members. The application of this approach is explored with K-means clustering. The proposed method denoted by Minimum ACE K-means (MACE-means) is shown to have unique advantages both with synthetic and real data. MACE-means clustering is applied to benchmark real world data sets from UCI machine learning repository and other synthesized clusters that represent a wide class of clustering scenarios. Our analysis confirms superiority of MACE-means over the state of the art clustering methods in robustness to the initialization error, accuracy in detecting the correct number of clusters, having less time complexity, and robustness to cluster overlapping.