Shahar Bar

Bayesian Estimation in the Presence of Deterministic Nuisance Parameters—Part I: Performance Bounds

How accurately can one estimate a random parameter subject to unknown deterministic nuisance parameters? The hybrid Cramér-Rao bound (HCRB) provides an answer to this question for a restricted class of estimators. The HCRB is the most popular performance bound on the mean-square-error (MSE) for random parameter estimation problems which involve deterministic parameters. The HCRB is useful when one is interested in both the random and the deterministic parameters and in the coupling between their estimation errors. This bound refers to locally weak-sense unbiased estimators with respect to (w.r.t.) the deterministic parameters. However, if these parameters are nuisance, it is unnecessary to restrict their estimation as unbiased. This paper is the first of a two-part study of Bayesian parameter estimation in the presence of deterministic nuisance parameters. It begins with a study on order relations between existing Cramér-Rao (CR)-type bounds of mean-unbiased Bayesian estimators. Then, a new CR-type bound is developed with no assumption of unbiasedness on the nuisance parameters. Alternatively, Lehmanns concept of unbiasedness is employed rather than conventional mean-unbiasedness. It is imposed on a risk that measures the distance between the estimator and the minimum MSE (MMSE) estimator which assumes perfect knowledge of the nuisance parameters. In the succeeding paper, asymptotic performances of some Bayesian estimators with maximum likelihood based estimates for the nuisance parameters are investigated. The proposed risk-unbiased bound (RUB) is proved to be asymptotically achieved by the MMSE estimator with maximum likelihood estimates for the nuisance parameters, while the existing CR-type bounds are not necessarily achievable.

Bayesian Estimation in the Presence of Deterministic Nuisance Parameters—Part II: Estimation Methods

One of the fundamental issues of estimation theory is the presence of deterministic nuisance parameters. While in the Bayesian paradigm the model parameters are random, introduction of deterministic nuisance parameters into the model exceeds the Bayesian framework to the hybrid framework. In this type of scenarios, the conventional Bayesian estimators are not valid, as they assume knowledge of the deterministic nuisance parameters. This paper is the second of a two-part study of Bayesian parameter estimation in the presence of deterministic nuisance parameters. In part I, a new Cramér-Rao (CR)-type bound on the mean-square-error (MSE) for Bayesian estimation in the presence of deterministic nuisance parameters was established based on the concept of risk-unbiasedness. The proposed bound was named risk-unbiased bound (RUB). This paper presents properties of asymptotic uniform mean- and risk-unbiasedness of some Bayesian estimators: 1) the minimum MSE (MMSE) or maximum a posteriori probability (MAP) estimators with maximum likelihood (ML) estimates substituting the deterministic parameters, named MS-ML and MAP-ML, respectively, and 2) joint MAP and ML estimator, named JMAP-ML. Furthermore, an asymptotic performance analysis of the MS-ML and MAP-ML estimators is presented. These estimators are shown to asymptotically achieve the RUB, while the existing CR-type bounds can be achieved only in distinct cases. Simulations verify these results for the problem of blind separation of nonstationary sources. It is shown that unlike existing CR-type bounds, the RUB is asymptotically tight.

Performance analysis for pilot-based 1-bit channel estimation with unknown quantization threshold

Parameter estimation using quantized observations is of importance in many practical applications. Under a symmetric 1-bit setup, consisting of a zero-threshold hard-limiter, it is well known that the large sample performance loss for low signal-to-noise ratios (SNRs) is moderate (2/Π or -1.96dB). This makes low-complexity analog-to-digital converters (ADCs) with 1-bit resolution a promising solution for future wireless communications and signal processing devices. However, hardware imperfections and external effects introduce the quantizer with an unknown hard-limiting level different from zero. In this paper, the performance loss associated with pilot-based channel estimation, subject to an asymmetric hard limiter with unknown offset, is studied under two setups. The analysis is carried out via the Cramér-Rao lower bound (CRLB) and an expected CRLB for a setup with random parameter. Our findings show that the unknown threshold leads to an additional information loss, which vanishes for low SNR values or when the offset is close to zero.

Bayesian cramér-rao type bound for risk-unbiased estimation with deterministic nuisance parameters

In this paper, we derive a Bayesian Cramér-Rao type bound in the presence of unknown nuisance deterministic parameters. The most popular bound for parameter estimation problems which involves both deterministic and random parameters is the hybrid Cramér-Rao bound (HCRB). This bound is very useful especially, when one is interested in both the deterministic and random parameters and in the coupling between their estimation errors. The HCRB imposes locally unbiasedness for the deterministic parameters. However, in many signal processing applications, the unknown deterministic parameters are treated as nuisance, and it is unnecessary to impose unbiasedness on these parameters. In this work, we establish a new Cramér-Rao type bound on the mean square error (MSE) of Bayesian estimators with no unbiasedness condition on the nuisance parameters. Alternatively, we impose unbiasedness in the Lehmann sense for a risk that measures the distance between the estimator and the minimum MSE estimator which assumes perfect knowledge of the nuisance parameters. The proposed bound is compared to the HCRB and MSE of Bayesian estimators with maximum likelihood estimates for the nuisance parameters. Simulations show that the proposed bound provides tighter lower bound for these estimators.

New observations on efficiency of variance estimation of white Gaussian signal with unknown mean

The uniformly minimum variance unbiased estimator (UMVUE) for mean and variance of white Gaussian noise is known to be not efficient. This is due to the fact that according to the Cramér-Rao bound (CRB), no coupling exists between mean and variance of Gaussian observations, while it is clear that knowledge or lack of knowledge of the mean has impact on estimation of the variance. In this work, we consider the problem of variance estimation in the presence of unknown mean of white Gaussian signals, where the unknown mean is considered to be a nuisance parameter. For this purpose, a Cramér-Rao-type bound on the mean-squared-error (MSE) of non-Bayesian estimators, which has been recently introduced, is analyzed. This bound considers no unbiasedness condition on the nuisance parameters. Alternatively, Lehmanns concept of unbiasedness is imposed for a risk that measures the distance between the estimator and the locally best unbiased estimator, which assumes perfect knowledge of the model parameters. It is analytically shown that the MSE of the well-known UMVUE coincides with the proposed risk-unbiased CRB, and therefore it is called risk-efficient estimator.

A risk-unbiased approach to a new Cramér-Rao bound

How accurately can one estimate a deterministic parameter subject to other unknown deterministic model parameters? The most popular answer to this question is given by the Cramer-Rao bound (CRB). The main assumption behind the derivation of the CRB is local unbiased estimation of all model parameters. The foundations of this work rely on doubting this assumption. Each parameter in its turn is treated as a single parameter of interest, while the other model parameters are treated as nuisance, as their mis-knowledge interferes with the estimation of the parameter of interest. Correspondingly, a new Cramer-Rao-type bound on the mean squared error (MSE) of non-Bayesian estimators is established with no unbiasedness condition on the nuisance parameters. Alternatively, Lehmanns concept of unbiasedness is imposed for a risk that measures the distance between the estimator and the locally best unbiased (LBU) estimator which assumes perfect knowledge of the nuisance parameters. The proposed bound is compared to the CRB and MSE of the maximum likelihood estimator (MLE). Simulations show that the proposed bound provides a tight lower bound for this estimator, compared with the CRB.

Geometrical interpretation of the adaptive coherence estimator for hyperspectral target detection

A hyperspectral cube consists of a set of images taken at numerous wavelengths. Hyperspectral image data analysis uses each material’s distinctive patterns of reflection, absorption and emission of electromagnetic energy at specific wavelengths for classification or detection tasks. Because of the size of the hyperspectral cube, data reduction is definitely advantageous; when doing this, one wishes to maintain high performances with the least number of bands. Obviously in such a case, the choice of the bands will be critical. In this paper, we will consider one particular algorithm, the adaptive coherence estimator (ACE) for the detection of point targets. We give a quantitative interpretation of the dependence of the algorithm on the number and identity of the bands that have been chosen. Results on simulated data will be presented.

Adaptive waveform design for target detection with sequential composite hypothesis testing

This paper addresses the problem of adaptive waveform design for target detection with composite sequential hypothesis testing. We begin with an asymptotic analysis of the generalized sequential probability ratio test (GSPRT). The analysis is based on Bayesian considerations, similar to the ones used for the derivation of the Bayesian information criterion (BIC) for model order selection. Following the analysis, a novel test, named penalized GSPRT (PGSPRT), is proposed on the basis of restraining the exponential growth of the GSPRT with respect to the sequential probability ratio test (SPRT). The performance measures of the PGSPRT in terms of average sample number (ASN) and error probabilities are also investigated. In the proposed waveform design scheme, the transmit spatial waveform (beamforming) is adaptively determined at each step based on observations in the previous steps. The spatial waveform is determined to minimize the ASN of the PGSPRT. Simulations demonstrate the performance measures of the new algorithm for target detection in a multiple input, single output channel.

Cramér-Rao-type bound for state estimation in linear discrete-time system with unknown system parameters

Tracking problems are usually investigated using the Bayesian approach. Many practical tracking problems involve some unknown deterministic nuisance parameters such as the system parameters or noise statistical parameters. This paper addresses the problem of state estimation in linear discrete-time dynamic systems in the presence of unknown deterministic system parameters. A Cramér-Rao-type bound on the mean-sqaure-error (MSE) of the state estimation is introduced. The bound is based on the concept of risk-unbiasedness and can be computed recursively. It allows evaluating the optimality of the estimation procedure. Some sequential estimators for this problem are proposed such that the estimation procedure can be considered an on-line technique. Simulation results show that the proposed bound is asymptotically achieved by the considered estimators.

Risk-unbiased bound for random signal estimation in the presence of unknown deterministic channel

Estimation of a signal transmitted through a communication channel usually involves channel identification. This scenario can be modeled as random parameter estimation in the presence of unknown deterministic parameter. In this paper, we address the question of how accurately one can estimate a random signal intercepted by an array of sensors, subject to an unknown deterministic array response. The commonly used hybrid Cramér-Rao bound (HCRB) is restricted to mean-unbiased estimation of all model parameters with no distinction of their character and leads to optimistic and unachievable performance analysis. Instead, A Bayesian Cramér-Rao (CR)- type bound on the mean-square-error (MSE) is derived for the considered scenario. The bound is based on the risk-unbiased bound (RUB) which assumes risk-unbiased estimation of the signals of interest. Simulations show that the RUB provides a tight and achievable performance analysis for the MSE of conventional hybrid estimators.