Salman Avestimehr

Coded computation over heterogeneous clusters

In large-scale distributed computing clusters, such as Amazon EC2, there are several types of “system noise” that can result in major degradation of performance: system failures, bottlenecks due to limited communication bandwidth, latency due to straggler nodes, etc. On the other hand, these systems enjoy abundance of redundancy — a vast number of computing nodes and large storage capacity. There have been recent results that demonstrate the impact of coding for efficient utilization of computation and storage redundancy to alleviate the effect of stragglers and communication bottlenecks in homogeneous clusters. In this paper, we focus on general heterogeneous distributed computing clusters consisting of a variety of computing machines with different capabilities. We propose a coding framework for speeding up distributed computing in heterogeneous clusters with straggling servers by trading redundancy for reducing the latency of computation. In particular, we propose Heterogeneous Coded Matrix Multiplication (HCMM) algorithm for performing distributed matrix multiplication over heterogeneous clusters that is provably asymptotically optimal. Moreover, if the number of worker nodes in the cluster is n, we show that HCMM is Θ(log n) times faster than any uncoded scheme. We further provide numerical results demonstrating significant speedups of up to 49% and 34% for HCMM in comparison to the “uncoded” and “homogeneous coded” schemes, respectively.

MISO Broadcast Channel With Hybrid CSIT: Beyond Two Users

We study the impact of heterogeneity of channel-state-information available at the transmitters (CSIT) on the capacity of broadcast channels with a multiple-antenna transmitter and k single-antenna receivers (MISO BC). In particular, we consider the k-user MISO BC, where the CSIT with respect to each receiver can be either instantaneous/perfect, delayed, or not available; and we study the impact of this heterogeneity of CSIT on the degrees-of-freedom (DoFs) of such network. We first focus on the three-user MISO BC, and we completely characterize the DoF region for all possible heterogeneous CSIT configurations, assuming linear encoding strategies at the transmitters. The result shows that the state-of-the-art achievable schemes in the literature are indeed sum-DoF optimal, when restricted to linear encoding schemes. To prove the result, we develop a novel bound, called interference decomposition bound, which provides a lower bound on the interference dimension at a receiver which supplies delayed CSIT based on the average dimension of constituents of that interference, thereby decomposing the interference into its individual components. Furthermore, we extend our outer bound on the DoF region to the general k-user MISO BC, and demonstrate that it leads to an approximate characterization of linear sum-DoF to within an additive gap of 0.5 for a broad range of CSIT configurations. Moreover, for the special case where only one receiver supplies delayed CSIT, we completely characterize the linear sum-DoF.

Three-user MISO broadcast channel: How much can CSIT heterogeneity help?

We study the impact of heterogeneity of channel state information available to the transmitters (CSIT) on the performance of multi-antenna multi-user wireless networks. More specifically, we consider the 3-user multiple-input single-output (MISO) broadcast channel, where the available CSIT with respect to each receiver can be instantaneous (P), delayed (D), or none (N); and we characterize the extent to which such heterogeneity of CSIT impacts its linear degrees of freedom (LDoF). In particular, we completely characterize the DoF region for all possible CSIT configurations, assuming linear encoding strategies at the transmitters. The converse, which is the main contribution of the paper, is based on a novel lemma, called Interference Decomposition Bound, which provides a lower bound on the interference dimension at a receiver with delayed CSIT, based on the dimension of constituents of that interference, thereby decomposing the interference into its individual components.

Active learning for community detection in stochastic block models

The stochastic block model (SBM) is an important generative model for random graphs in network science and machine learning, useful for benchmarking community detection (or clustering) algorithms. The symmetric SBM generates a graph with 2n nodes which cluster into two equally sized communities. Nodes connect with probability p within a community and q across different communities. We consider the case of p = a ln(n)/n and q = b ln(n)/n. In this case, it was recently shown that recovering the community membership (or label) of every node with high probability (w.h.p.) using only the graph is possible if and only if the Chernoff-Hellinger (CH) divergence D(a; b) = (√a - √a)2 ≥ 1. In this work, we study if, and by how much, community detection below the clustering threshold (i.e. D(a; b) <; 1) is possible by querying the labels of a limited number of chosen nodes (i.e., active learning). Our main result is to show that, under certain conditions, sampling the labels of a vanishingly small fraction of nodes (a number sub-linear in n) is sufficient for exact community detection even when D(a; b) <; 1. Furthermore, we provide an efficient learning algorithm which recovers the community memberships of all nodes w.h.p. as long as the number of sampled points meets the sufficient condition. We also show that recovery is not possible if the number of observed labels is less than n1-D(a;b). The validity of our results is demonstrated through numerical experiments.