José L. Sanz-González

Performance Analysis of Neural Network Detectors by Importance Sampling Techniques

Often, Neural Networks are involved in binary detectors of communication, radar or sonar systems. The design phase of a neural network detector usually requires the application of Monte Carlo trials in order to estimate some performance parameters.The classical Monte Carlo method is suitable to estimate high event probabilities (higher than 0.01), but not suitable to estimate very low event probabilities (say, 10−5 or less). For estimations of very low false alarm probabilities (or error probabilities), a modified Monte Carlo technique, the so-called Importance Sampling (IS) technique, is considered in this paper; some topics are developed, such as optimal and suboptimal IS probability density functions (biasing density functions), control parameters and new algorithms for the minimization of the estimator error.The main novelty of this paper is the application of an efficient IS technique on neural networks, drastically reducing the number of patterns required for testing events of low probability. As a practical application, the IS technique is applied to a neural detector on a radar (or sonar) system.

Comparison of a neural network detector vs Neyman-Pearson optimal detector

We optimize a neural network applied to binary detection such as those found in radar or sonar. Topics about designing the structure, training procedure and evaluating the performance, are discussed. The detector optimization is based on the use of a criterion function that yields a solution significantly superior to the typical sum-of-square-error. Using a modeled input, its performance is evaluated by Monte Carlo trials. As a result, detection curves are compared with the theoretical optimum ones (Neyman-Pearson detectors). For the model, and despite of the blind learning of the neural network, its performance is very close to optimal.

Nonparametric rank detectors under K-distributed clutter in radar applications

This correspondence deals with a comparative analysis of parametric detectors versus rank ones for radar applications, under K-distributed clutter and nonfluctuating and Swerling II target models. We show that the locally optimum detectors (LODs) (optimum for very low signal-to-clutter ratio (SCR)) under K-distributed clutter are not practical detectors; on the contrary, asymptotically optimum detectors (optimum for high SCR) are the practical ones. The performance analysis of the parametric log-detector and the nonparametric (linear rank) detector is carried out for independent and identically distributed (IID) clutter samples, correlated clutter samples, and nonhomogeneous clutter samples. Some results of Monte Carlo simulations for detection probability (P/sub d/) versus SCR are presented in curves for different detector parameter values.

Importance Sampling and Mean-Square Error in Neural Detector Training

This letter deals with the use of Importance Sampling (IS) techniques and the Mean-Square (MS) error in neural network training, for applications to detection in communication systems. Topics such as modifications of the MS objective function, optimal and suboptimal IS probability density functions, and adaptive importance sampling are presented. A genetic algorithm was used for the neural network training, having considered adaptive IS techniques for improving MS error estimations in each iteration of the training. Also, some experimental results of the training process are shown in this letter. Finally, we point out that the mean-square error (estimated by importance sampling) attains quasi-optimum training in the sense of minimum error probability (or minimum misclassification error).

Importance Sampling Techniques in Neural Detector Training

Importance Sampling is a modified Monte Carlo technique applied to the estimation of rare event probabilities (very low probabilities). In this paper, we propose and develop the use of Importance Sampling (IS) techniques in neural network training, for applications to detection in communication systems. Some key topics are introduced, such as modifications of the error probability objective function, optimal and suboptimal IS probability density functions (biasing density functions), and experimental results of training with a genetic algorithm. Also, it is shown that the genetic algorithm with the IS technique attains quasi-optimum training in the sense of minimum error probability (or minimum misclassification probability).

Signal processing algorithms by permutation test in radar application

A hypothesis H is parametric if every distribution from the process defined by H belongs to a family of distributions characterized by a finite number of parameters; on the other hand, if the distribution can not be defined by a finite number of parameters, the hypothesis is nonparametric. In this paper, we analyze a detector based on the optimum permutation test, applied to nonparametric radar detection which provide good performances without a large computational work, and we compare it with the parametric test and rank test in the Neyman-Pearson sense. The computational complexity of the detector is high and its implementation in real time is difficult, due to the number of operations increase with the factorial of the number of samples. Also, we present an algorithm that reduces the computational work required. We also present the detectability characteristic of the optimum permutation test against rank test and parametric test under Gaussian noise environments and different types of target models (nonfluctuating, Swerling I and Swerling II). The detection probability versus signal-to-noise ratio is estimated by Monte-Carlo simulations for different parameter values (N pulse, M reference samples and false alarm probability Pfa).

Importance Sampling for Objective Function Estimations in Neural Detector Training Driven by Genetic Algorithms

To train Neural Networks (NNs) in a supervised way, estimations of an objective function must be carried out. The value of this function decreases as the training progresses and so, the number of test observations necessary for an accurate estimation has to be increased. Consequently, the training computational cost is unaffordable for very low objective function value estimations, and the use of Importance Sampling (IS) techniques becomes convenient. The study of three different objective functions is considered, which implies the proposal of estimators of the objective function using IS techniques: the Mean-Square error, the Cross Entropy error and the Misclassification error criteria. The values of these functions are estimated by IS techniques, and the results are used to train NNs by the application of Genetic Algorithms. Results for a binary detection in Gaussian noise are provided. These results show the evolution of the parameters during the training and the performances of the proposed detectors in terms of error probability and Receiver Operating Characteristics curves. At the end of the study, the obtained results justify the convenience of using IS in the training.

Permutation tests for nonparametric detection

In this paper, the authors provide a methodology to design nonparametric permutation tests and, in particular, nonparametric rank tests for applications in detection. In the first part of the paper, the authors develop the optimization theory of both permutation and rank tests in the Neyman-Pearson sense; in the second part of the paper, they carry out a comparative performance analysis of the permutation and rank tests (detectors) against the parametric ones in radar applications. First, a brief review of some contributions on nonparametric tests is realized. Then, the optimum permutation and rank tests are derived. Finally, a performance analysis is realized by Monte-Carlo simulations for the corresponding detectors, and the results are shown in curves of detection probability versus signal-to-noise ratio.

Neyman-Pearson Neural Detectors

This paper is devoted to the design of a neural alternative to binary detectors optimized in the Neyman-Pearson sense. These detectors present a configurable low probability of classifying binary symbol 1 when symbol O is the correct decision. This kind of error, referred in the scientific literature as false-posetive or false a l a m probability has a high cost in many real applications as medical Computer Aided Diagnosis or Radar and Sonar Target Detection, and the possibility of controlling its maximum value is crucial. The novelty and interest of the detector is the application of a Multilayer Perceptron instead of a classical design. Under some conditions, the Neural Detector presents a performance competitive with classical designs adding the typical advantages of Neural Networks. So, the presented Neural Detectors may be considered as an alternative to classical ones.

Complexity analysis of permutation test versus rank test for nonparametric radar detection

In this paper, we analyze the complexity of an optimal algorithm for realizing the permutation test applied to nonparametric radar detection against the complexity of rank test realization. For a primitive permutation test algorithm, the computational work is very high and its implementation in real-time is difficult, due to the number of operations increases with the number of reference samples (M) to the power of the number of integrated pulses (N) (i.e. MN). We propose new permutation test and rank test algorithms, and analyze the complexity with respect to N and M for a given false-alarm probability (Pfa); also, the detection probability (Pd) will be evaluated for each case. Optimum values of N and M for a given Pfa will be given for the permutation test and the rank test, resulting in similar values of computational complexity of both of them, i.e. computational complexity proportional to N (DOT) M. We also show the detectability curves of the optimum permutation test versus optimum rank test under Gaussian noise environments for different values of N and M and different target models.