Dayu Huang

Universal and Composite Hypothesis Testing via Mismatched Divergence

For the universal hypothesis testing problem, where the goal is to decide between the known null hypothesis distribution and some other unknown distribution, Hoeffding proposed a universal test in the nineteen sixties. Hoeffdings universal test statistic can be written in terms of Kullback-Leibler (K-L) divergence between the empirical distribution of the observations and the null hypothesis distribution. In this paper a modification of Hoeffdings test is considered based on a relaxation of the K-L divergence, referred to as the mismatched divergence. The resulting mismatched test is shown to be a generalized likelihood-ratio test (GLRT) for the case where the alternate distribution lies in a parametric family of distributions characterized by a finite-dimensional parameter, i.e., it is a solution to the corresponding composite hypothesis testing problem. For certain choices of the alternate distribution, it is shown that both the Hoeffding test and the mismatched test have the same asymptotic performance in terms of error exponents. A consequence of this result is that the GLRT is optimal in differentiating a particular distribution from others in an exponential family. It is also shown that the mismatched test has a significant advantage over the Hoeffding test in terms of finite sample size performance for applications involving large alphabet distributions. This advantage is due to the difference in the asymptotic variances of the two test statistics under the null hypothesis.

Approximate dynamic programming using fluid and diffusion approximations with applications to power management

TD learning and its refinements are powerful tools for approximating the solution to dynamic programming problems. However, the techniques provide the approximate solution only within a prescribed finite-dimensional function class. Thus, the question that always arises is how should the function class be chosen? The goal of this paper is to propose an approach for TD learning based on choosing the function class using the solutions to associated fluid and diffusion approximations. In order to illustrate this new approach, the paper focuses on an application to dynamic speed scaling for power management.

Classification with high-dimensional sparse samples

The task of the binary classification problem is to determine which of two distributions has generated a length-n test sequence. The two distributions are unknown; two training sequences of length N, one from each distribution, are observed. The distributions share an alphabet of size m, which is significantly larger than n and N. How does N,n,m affect the probability of classification error? We characterize the achievable error rate in a high-dimensional setting in which N,n,m all tend to infinity, under the assumption that probability of any symbol is O(m-1). The results are: 1) There exists an asymptotically consistent classifier if and only if m = o(min{N2, Nn}). This extends the previous consistency result in [1] to the case N ≠ n. 2) For the sparse sample case where max{n, N} = o(m), finer results are obtained: The best achievable probability of error decays as - log(Pe) = J min {N2, Nn}(1 +o(1))/m with J >; 0. 3) A weighted coincidence-based classifier has non-zero generalized error exponent J. 4) The ℓ2-norm based classifier has J = 0.

Statistical SVMs for robust detection, supervised learning, and universal classification

The support vector machine (SVM) has emerged as one of the most popular approaches to classification and supervised learning. It is a flexible approach for solving the problems posed in these areas, but the approach is not easily adapted to noisy data in which absolute discrimination is not possible. We address this issue in this paper by returning to the statistical setting. The main contribution is the introduction of a statistical support vector machine (SSVM) that captures all of the desirable features of the SVM, along with desirable statistical features of the classical likelihood ratio test. In particular, we establish the following: (i) The SSVM can be designed so that it forms a continuous function of the data, yet also approximates the potentially discontinuous log likelihood ratio test. (ii) Extension to universal detection is developed, in which only one hypothesis is labeled (a semi-supervised learning problem). (iii) The SSVM generalizes the robust hypothesis testing problem based on a moment class. Motivation for the approach and analysis are each based on ideas from information theory. A detailed performance analysis is provided in the special case of i.i.d. observations. This research was partially supported by NSF under grant CCF 07-29031, by UTRC, Motorola, and by the DARPA ITMANET program. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF, UTRC, Motorola, or DARPA.

Feature extraction for universal hypothesis testing via rank-constrained optimization

This paper concerns the construction of tests for universal hypothesis testing problems, in which the alternate hypothesis is poorly modeled and the observation space is large. The mismatched universal test is a feature-based technique for this purpose. In prior work it is shown that its finite-observation performance can be much better than the (optimal) Hoeffding test, and good performance depends crucially on the choice of features. The contributions of this paper include: (i) We obtain bounds on the number of ε-distinguishable distributions in an exponential family. (ii) This motivates a new framework for feature extraction, cast as a rank-constrained optimization problem. (iii) We obtain a gradient-based algorithm to solve the rank-constrained optimization problem and prove its local convergence.

Weak Convergence Analysis of Asymptotically Optimal Hypothesis Tests

In recent years, solutions to various hypothesis testing problems in the asymptotic setting have been proposed using the results from large deviation theory. Such tests are optimal in terms of appropriately defined error exponents. For the practitioner, however, error probabilities in the finite sample size setting are more important. In this paper, we show how results on weak convergence of the test statistic can be used to obtain better approximations for the error probabilities in the finite sample size setting. While this technique is popular among statisticians for common tests, we demonstrate its applicability for several recently proposed asymptotically optimal tests, including tests for robust goodness of fit, homogeneity tests, outlier hypothesis testing, and graphical model estimation.

Generalized Error Exponents for Small Sample Universal Hypothesis Testing

The small sample universal hypothesis testing problem is investigated in this paper, in which the number of samples n is smaller than the number of possible outcomes m. The goal of this paper is to find an appropriate criterion to analyze statistical tests in this setting. A suitable model for analysis is the high-dimensional model in which both n and m increase to infinity, and n=o(m). A new performance criterion based on large deviations analysis is proposed and it generalizes the classical error exponent applicable for large sample problems (in which m=O(n)). This generalized error exponent criterion provides insights that are not available from asymptotic consistency or central limit theorem analysis. The following results are established for the uniform null distribution: 1) The best achievable probability of error Pe decays as Pe=exp{-(n2/m) J (1+o(1))} for some J > 0. 2) A class of tests based on separable statistics, including the coincidence-based test, attains the optimal generalized error exponents. 3) Pearsons chi-square test has a zero generalized error exponent and thus its probability of error is asymptotically larger than the optimal test.

Error exponents for composite hypothesis testing with small samples

We consider the small sample composite hypothesis testing problem, where the number of samples n is smaller than the size of the alphabet m. A suitable model for analysis is the high-dimensional model in which both n and m tend to infinity, and n = o(m). We propose a new performance criterion based on large deviation analysis, which generalizes the classical error exponent applicable for large sample problems (in which m = O(n)). The results are: (i) The best achievable probability of error Pe decays as -log(Pe) = (n2/m)(1 + o(1))J for some J >; 0, shown by upper and lower bounds. (ii) A coincidence-based test has non-zero generalized error exponent J, and is optimal in the generalized error exponent of missed detection. (iii) The widely-used Pearsons chi-square test has a zero generalized error exponent. (iv) The contributions (i)-(iii) are established under the assumption that the null hypothesis is uniform. For the non-uniform case, we propose a new test with nonzero generalized error exponent.

Feature selection for composite hypothesis testing with small samples: Fundamental limits and algorithms

This paper considers the problem of feature selection for composite hypothesis testing: The goal is to select, from m candidate features, r relevant ones for distinguishing the null hypothesis from the composite alternative hypothesis; the training data are given as L sequences of observations, of which each is an n-sample sequence coming from one distribution in the alternative hypothesis. What is the fundamental limit for successful feature selection? Are there any algorithms that achieve this limit? We investigate this problem in a small-sample high-dimensional setting, with n = o(m), and obtain a tight pair of achievability and converse results: (i) There exists a function f(L, n, r,m) such that if f(L, n, r,m) ↓ 0, then no asymptotically consistent feature selection algorithm exists; (ii) We propose a feature selection algorithm that is asymptotically consistent whenever f(L, n, r,m) ↑ ∞.

Angular domain processing for MIMO wireless systems with non-uniform antenna arrays

Many works have proposed spatial channel models for MIMO applications in the 1 to 10 GHz band. However, attention has shifted to the 60 GHz regime where the opening up of the spectrum has created new opportunities for multi-antenna communications. With a broad focus on the 60 GHz regime, we revisit MIMO channel modeling in the classical setting and propose an angular domain decomposition of the channel in this work. In the special case of non-uniform linear arrays, this proposal corresponds to non-uniformly partitioning the angular domain commensurate with the non-uniformity structure of the arrays and estimating the channel gains in each partition with a multi-window spectral estimator. We show that the optimal choice of basis functions that minimize the worst-case bias of the spectral estimator are solutions to certain generalized eigenvalue problems. Based on this structure, we develop a spatial model that generalizes existing MIMO channel modeling paradigms.