Dimitri Kazakos

Recursive estimation of prior probabilities using a mixture

The problem of estimating the prior probabilities q = (q_{1} \cdots q_{m-1}) of m statistical classes with known probability density functions F_{1}(X) \cdots F_{m}(x) on the basis of n statistically independent observations (X_{l} \cdots x_{n}) is considered. The mixture density g(x|q) = \sum^{m-1}_{j-1}q_{j}F_{j}(x) + (1 - \sum^{m-1}_{\tau = 1}q_{\tau})F_{m(x) is used to show that the maximum likelihood estimate of q is asymptotically efficient and weakly consistent under very mild constraints on the set of density functions. A recursive estimate is proposed for q . By using stochastic approximation theory and optimizing the gain sequence, it is shown that the recursive estimate is asymptotically efficient for the m = 2 class case. For m > 2 classes, the rate of convergence is computed and shown to be very close to asymptotic efficiency.

Distributed binary hypothesis testing with feedback

The problem of binary hypothesis testing is revisited in the context of distributed detection with feedback. Two basic distributed structures with decision feedback are considered. The first structure is the fusion center network, with decision feedback connections from the fusion center element to each one of the subordinate decisionmakers. The second structure consists of a set of detectors that are fully interconnected via decision feedback. Both structures are optimized in the Neyman-Pearson sense by optimizing each decision-maker individually. Then, the time evolution of the power of the tests is derived. Definite conclusions regarding the gain induced by the feedback process and direct comparisons between the two structures and the optimal centralized scheme are obtained through asymptotic studies (that is, assuming the presence of asymptotically many local detectors). The behavior of these structures is also examined in the presence of variations in the statistical description of the hypotheses. Specific robust designs are proposed and the benefits from robust operations are established. Numerical results provide additional support to the theoretical arguments. >

A Decision Theory Approach to the Approximation of Discrete Probability Densities

The problem of approximating a probability density function by a simpler one is considered from a decision theory viewpoint. Among the family of candidate approximating densities, we seek the one that is most difficult to discriminate from the original. This formulation leads naturaliy to the density at the smallest Bhattacharyya distance. The resulting optimization problem is analyzed in detail.

Fundamental structures and asymptotic performance criteria in decentralized binary hypothesis testing

Two fundamental distributed decision network structures are considered: the first system consists of finite number of sensors, each collecting asymptotically many data, while the second one employs asymptotically many sensors, each collecting a single datum. For binary hypothesis testing, the Neyman-Pearson criterion is utilized and justified via information theoretic arguments. An asymptotic relative efficiency performance measure is used to establish tradeoffs between the two structures, by comparing the performance characteristics of the decentralized detection systems to their centralized counterparts. >

The Bhattacharyya distance and detection between Markov chains

When the statistical structure under each of two hypotheses is time varying, the collection of infinitely many observations does not guarantee an error probability that approaches zero. A recursive formula for the Bhattacharyya distance between two Markov chains is derived, and it is used to derive necessary and sufficient conditions for asymptotically perfect detection (APD). It is shown that the use of incorrect prior probabilities in the Bayes detection rulee does not affect AID. The results are also extended to time-continuons finite-state Markov observations. An application is analyzed, in which the behavior of a message buffer is monitored for the purpose of detecting malfunctions in a computer communication network.

On the approximation of the output process of multiuser random-access communication networks

Bernoulli and first-order Markov processes are used to approximate the output process of a class of slotted multiuser random-access communication networks. The output process is defined as the process of the successfully transmitted packets within the network. The parameters of the approximating processes are analytically calculated for a network operating under a specific random access algorithm. The applied methods are general and can be used to calculate these parameters in the case of any random access algorithm within a class. To evaluate the accuracy of the approximations, a star topology of interconnected multiuser random-access communication networks is considered. The mean time that a packet spends in the central node of the star topology is calculated under the proposed approximations of the output processes of the interconnected networks. The results are compared to simulation results of the actual system. It turns out that the memoryless approximation gives satisfactory results up to a certain per network traffic load. Beyond that per network traffic load, the first-order Markov process performs better. >

Signal detection under mismatch (Corresp.)

A binary detection problem of the Neymann-Pearson type, in which the probability density functions used are inaccurate versions of the true ones, are considered. The performance of the above suboptimal detection scheme as the number of observations increases is investigated. A necessary and sufficient condition is given for the exponential convergence to zero of the two error probabilities as the number of observations increases. The condition is in terms of an inequality between differences of asymptotic per sample informational divergence expressions.

An improved decision-directed detector (Corresp.)

A decision-directed detection scheme for multiple hypotheses is developed and analyzed. It is assumed that the probability density functions \{f_{i}(x)\} under each of the m+1 hypotheses are known, and the prior probablities \{\pi_{i}\} are unknown and sequentially estimated on the basis of previous decisions. Using a set of nonlinear transformations of the data and applying results from the stochastic approximation theory, improved algorithms are given for achieving asymptotically unbiased estimates and accelerated convergence to the true priors.

Robust noiseless source coding through a game theoretical approach

Noiseless coding of a discrete source with partially known statistics is formulated as a two-person game. The payoff is the average codeword length, using Shannon codes. The code designer picks a source probability distribution for the design of the code, while an opponent picks the actual source probability distribution. It is shown that if the class of probability distributions allowed is convex, then there is a saddle point solution which is determined by the maximum entropy distribution of the convex class. The maximum entropy element is derived for three families of source probability mass functions (pmf): a) the class of c-contaminated pmfs; b) the class of pmfs for which each probability is known only through an upper and lower bound; c) the class of pmfs which is a convex hull of a finite number of known pmfs. An extension of the robust noiseless source coding problem for families of sources modeled as first-order Markov chains is discussed.

A Nonparametric Sequential Test for Data with Markov Dependence

A nonparametric sequential probability ratio test has been formed for dependent data. Operating characteristics are provided and their dependence upon the parameters is investigated. An example is given for second-order Markov dependence.