Shaofeng Zou

Estimation of KL divergence between large-alphabet distributions

The problem of estimating the KL divergence between two unknown distributions is studied. The alphabet size k of the distributions can scale to infinity. The estimation is based on m and n independent samples respectively drawn from the two distributions. It is first shown that there does not exist any consistent estimator to guarantee asymptotic small worst-case quadratic risk over the set of all pairs of distributions. A restricted set that contains pairs of distributions with bounded ratio f(k) is further considered. An augmented plug-in estimator is proposed, and is shown to be consistent if and only if m = ω(k ⋁ log2(f(k)) and n = ω(k f(k)). Furthermore, if f(k) ≥ log2k and log2(f(k)) = o(k), it is shown that any consistent estimator must satisfy the necessary conditions: m = ω( k/log k ⋁ log2(f(k)) and n = ω( k f(k)/log k).

Broadcast Networks With Layered Decoding and Layered Secrecy: Theory and Applications

Recent information-theoretic results on a class of broadcast channels with layered decoding and/or layered secrecy are reviewed. In this class of models, a transmitter sends multiple messages to a set of legitimate receivers in the presence of a set of eavesdroppers, whose channels can be ordered based on the quality of received signals. Receivers with better channel quality are required to decode more messages, and eavesdroppers with worse channel quality are required to be kept ignorant of more messages. The design of achievable schemes and the characterization of the corresponding secrecy capacity regions are presented. Comparison of the designs for different models is discussed. Applications of these information-theoretic models to the study of secure communication over fading wiretap channels and secret sharing are also presented to illustrate potential applications of these models.

Degraded broadcast channel: Secrecy outside of a bounded range

A four-receiver degraded broadcast channel with secrecy outside a bounded range is studied, over which a transmitter sends four messages to four receivers. In the model considered, the channel quality gradually degrades from receiver 4 to receiver 1, and receiver k is required to decode the first k messages for k = 1, ..., 4. Furthermore, message 3 is required to be secured from receiver 1, and message 4 is required to be secured from receivers 1 and 2. The secrecy capacity region is established. The achievable scheme includes not only superposition, binning and embedded coding used in previous studies, but also rate splitting and sharing particularly designed for this model, which is shown to be critical to further enlarge the achievable region and enable the development of the converse proof.A three-receiver degraded broadcast channel with secrecy outside of a bounded range is studied, in which the channel quality gradually degrades from receiver 3 to receiver 1. The transmitter has three messages intended for the receivers with receiver 3 decoding all messages, receiver 2 decoding the first two messages, and receiver 1 decoding only the first message. Furthermore, the third message should be kept secure from receiver 1. The discrete memoryless channel is studied and the secrecy capacity region is characterized. The achievable scheme is based on superposition coding and random binning, in which one superposition layer and random binning together provide secrecy. The converse proof is derived based on the insight obtained from the achievable scheme so that manipulations of terms yield tight rate bounds.

Layered decoding and secrecy over degraded broadcast channels

A K-receiver degraded broadcast channel with layered decoding and secrecy constraints is investigated, in which receivers are ordered by their channel quality. Each receiver is required to decode one more message compared to the receiver with one level worse channel quality, and this message should be kept secure from all receivers with worse channel quality. For both the discrete memoryless channel and the Gaussian channel, the secrecy capacity region is characterized. The achievability scheme is based on stochastic encoding and superposition coding schemes. Novel generalization of the analysis of leakage rates and of the proof of the converse is developed for the K-receiver scenario.

Nonparametric detection of an anomalous disk over a two-dimensional lattice network

Nonparametric detection of existence of an anomalous disk over a lattice network is investigated. If an anomalous disk exists, then all nodes belonging to the disk observe samples generated by a distribution q, whereas all other nodes observe samples generated by a distribution p that is distinct from q. If there does not exist an anomalous disk, then all nodes receive samples generated by p. The distributions p and q are arbitrary and unknown. The goal is to design statistically consistent test as the network size becomes asymptotically large. A kernel-based test is proposed based on maximum mean discrepancy (MMD) which measures the distance between mean embeddings of distributions into a reproducing kernel Hilbert space (RKHS). A sufficient condition on the minimum size of candidate anomalous disks is characterized in order to guarantee the consistency of the proposed test. A necessary condition that any universally consistent test must satisfy is further derived. Comparison of sufficient and necessary conditions yields that the proposed test is order-level optimal.

Rate splitting and sharing for degraded broadcast channel with secrecy outside a bounded range

A four-receiver degraded broadcast channel with secrecy outside a bounded range is studied, over which a transmitter sends four messages to four receivers. In the model considered, the channel quality gradually degrades from receiver 4 to receiver 1, and receiver k is required to decode the first k messages for k = 1, …, 4. Furthermore, message 3 is required to be secured from receiver 1, and message 4 is required to be secured from receivers 1 and 2. The secrecy capacity region is established. The achievable scheme includes not only superposition, binning and embedded coding used in previous studies, but also rate splitting and sharing particularly designed for this model, which is shown to be critical to further enlarge the achievable region and enable the development of the converse proof.

Layered secure broadcasting over MIMO channels and application in secret sharing

In this paper, the degraded Gaussian Multiple-Input-Multiple-Output (MIMO) broadcast channel with layered decoding and secrecy constraints is investigated. In this model, there are in total K messages and K receivers that are ordered by the channel quality. Each receiver is required to decode one more message than the receiver with one level worse channel quality. Furthermore, this message should be kept secure from the receivers with worse channel qualities. The secrecy capacity region for this model is fully characterized. The converse proof relies on a novel construction of a series of covariance matrices. An application of this model to the problem of sharing multiple secrets, which is difficult to solve using number theoretic tools, is investigated. The secret sharing capacity region is characterized by reformulating the secret sharing problem as the secure communication problem over the K-receiver degraded Gaussian MIMO broadcast channel.

Universal outlying sequence detection for continuous observations

The following detection problem is studied, in which there are M sequences of samples out of which one outlier sequence needs to be detected. Each typical sequence contains n independent and identically distributed (i.i.d.) continuous observations from a known distribution π, and the outlier sequence contains n i.i.d. observations from an outlier distribution μ, which is distinct from n, but otherwise unknown. A universal test based on Kullback-Leibler (KL) divergence is built to approximate the maximum likelihood test, with known π and unknown μ. A KL divergence estimator based on data-dependent partitions is employed, and is shown to converge to its true value exponentially fast when the density ratio satisfies 0 <; Kl ≤ dμ/dπ ≤ K2, where K1 and K2 are positive constants. The performance of such a KL divergence estimator further implies that the outlier detection test is exponentially consistent. The detection performance of the KL divergence based test is compared with that of a recently introduced test for this problem based on the machine learning approach of maximum mean discrepancy (MMD). Regimes in which the KL divergence based test is better than the MMD based test are identified.

A KERNEL-BASED NONPARAMETRIC TEST FOR ANOMALY DETECTION OVER LINE NETWORKS

The nonparametric problem of detecting existence of an anomalous interval over a one-dimensional line network is studied. Nodes corresponding to an anomalous interval (if one exists) receive samples generated by a distribution q, which is different from the distribution p that generates samples for other nodes. If an anomalous interval does not exist, then all nodes receive samples generated by p. It is assumed that the distributions p and q are arbitrary, and are unknown. In order to detect whether an anomalous interval exists, a test is built based on mean embeddings of distributions into a reproducing kernel Hilbert space (RKHS) and the metric of maximum mean discrepancy (MMD). It is shown that as the network size n goes to infinity, if the minimum length of candidate anomalous intervals is larger than a threshold which has the order O(log n), the proposed test is asymptotically successful. An efficient algorithm to perform the test with substantial computational complexity reduction is proposed, and is shown to be asymptotically successful if the condition on the minimum length of candidate anomalous interval is satisfied. Numerical results are provided, which are consistent with the theoretical results.

Kernel-based nonparametric anomaly detection

An anomaly detection problem is investigated, in which there are totally n sequences, with s anomalous sequences to be detected. Each normal sequence contains m independent and identically distributed (i.i.d.) samples drawn from a distribution p, whereas each anomalous sequence contains m i.i.d. samples drawn from a distribution q that is distinct from p. The distributions p and q are assumed to be unknown a priori. The scenario with a reference sequence generated by p is studied. Distribution-free tests are constructed using maximum mean discrepancy (MMD) as the metric, which is based on mean embeddings of distributions into a reproducing kernel Hilbert space (RKHS). It is shown that as the number n of sequences goes to infinity, if the value of s is known, then the number m of samples in each sequence should be of order O(log n) or larger in order for the developed tests to consistently detect s anomalous sequences. If the value of s is unknown, then m should be of order strictly larger than O(log n). The computational complexity of all developed tests is shown to be polynomial. Numerical results demonstrate that these new tests outperform (or perform as well as) tests based on other competitive traditional statistical approaches and kernel-based approaches under various cases.