Ramtin Pedarsani

On the construction of polar codes

We consider the problem of efficiently constructing polar codes over binary memoryless symmetric (BMS) channels. The complexity of designing polar codes via an exact evaluation of the polarized channels to find which ones are “good” appears to be exponential in the block length. In [3], Tal and Vardy show that if instead the evaluation if performed approximately, the construction has only linear complexity. In this paper, we follow this approach and present a framework where the algorithms of [3] and new related algorithms can be analyzed for complexity and accuracy. We provide numerical and analytical results on the efficiency of such algorithms, in particular we show that one can find all the “good” channels (except a vanishing fraction) with almost linear complexity in block-length (except a polylogarithmic factor).

Online coded caching

We consider a basic content distribution scenario consisting of a single origin server connected through a shared bottleneck link to a number of users each equipped with a cache of finite memory. The users issue a sequence of content requests from a set of popular files, and the goal is to operate the caches as well as the server such that these requests are satisfied with the minimum number of bits sent over the shared link. Assuming a basic Markov model for renewing the set of popular files, we characterize approximately the optimal long-term average rate of the shared link. We further prove that the optimal online scheme has approximately the same performance as the optimal offline scheme, in which the cache contents can be updated based on the entire set of popular files before each new request. To support these theoretical results, we propose an online coded caching scheme termed coded least-recently sent (LRS) and simulate it for a demand time series derived from the dataset made available by Netflix for the Netflix Prize. For this time series, we show that the proposed coded LRS algorithm significantly outperforms the popular least-recently used caching algorithm.

Speeding up distributed machine learning using codes

Distributed machine learning algorithms that are widely run on modern large-scale computing platforms face several types of randomness, uncertainty and system “noise.” These include stragglers1, system failures, maintenance outages, and communication bottlenecks. In this work, we view distributed machine learning algorithms through a coding-theoretic lens, and show how codes can equip them with robustness against this system noise. Motivated by their importance and universality, we focus on two of the most basic building blocks of distributed learning algorithms: data shuffling and matrix multiplication. In data shuffling, we use codes to reduce communication bottlenecks: when a constant fraction of the data can be cached at each worker node, and n is the number of workers, coded shuffling reduces the communication cost by up to a factor Θ(n) over uncoded shuffling. For matrix multiplication, we use codes to alleviate the effects of stragglers, also known as the straggler problem. We show that if the number of workers is n, and the runtime of each subtask has an exponential tail, the optimal coded matrix multiplication is Θ(log n) times faster than the uncoded matrix multiplication or the optimal task replication scheme.

Coded computation over heterogeneous clusters

In large-scale distributed computing clusters, such as Amazon EC2, there are several types of “system noise” that can result in major degradation of performance: system failures, bottlenecks due to limited communication bandwidth, latency due to straggler nodes, etc. On the other hand, these systems enjoy abundance of redundancy — a vast number of computing nodes and large storage capacity. There have been recent results that demonstrate the impact of coding for efficient utilization of computation and storage redundancy to alleviate the effect of stragglers and communication bottlenecks in homogeneous clusters. In this paper, we focus on general heterogeneous distributed computing clusters consisting of a variety of computing machines with different capabilities. We propose a coding framework for speeding up distributed computing in heterogeneous clusters with straggling servers by trading redundancy for reducing the latency of computation. In particular, we propose Heterogeneous Coded Matrix Multiplication (HCMM) algorithm for performing distributed matrix multiplication over heterogeneous clusters that is provably asymptotically optimal. Moreover, if the number of worker nodes in the cluster is n, we show that HCMM is Θ(log n) times faster than any uncoded scheme. We further provide numerical results demonstrating significant speedups of up to 49% and 34% for HCMM in comparison to the “uncoded” and “homogeneous coded” schemes, respectively.

PhaseCode: Fast and efficient compressive phase retrieval based on sparse-graph codes

We consider the problem of recovering a complex signal x ϵ Cn from m intensity measurements of the form |aix|, 1 ≤ i ≤ m, where ai is a measurement row vector. Our main focus is on the case where the measurement vectors are unconstrained, and where x is exactly K-sparse, or the so-called general compressive phase-retrieval problem. We introduce PhaseCode, a novel family of fast and efficient algorithms (that includes Unicolor PhaseCode and Multicolor PhaseCode) that are based on a sparse-graph coding framework. As one instance, our Unicolor PhaseCode algorithm can provably recover, with high probability, all but a tiny 10-7 fraction of the significant signal components, using at most m = 14K measurements, which is a small constant factor from the fundamental limit, with an optimal O(K) decoding time and an optimal O(K) memory complexity. We provide extensive simulation results that validate the practical power of our proposed algorithms. A key contribution of our work is the novel use of coding-theoretic tools like density evolution methods for the design and analysis of fast and efficient algorithms for compressive phase-retrieval problems. This contrasts and complements popular approaches to the phase retrieval problem based on alternating-minimization, convex-relaxation, and semi-definite programming.

On Scheduling Redundant Requests With Cancellation Overheads

Reducing latency in distributed computing and data storage systems is gaining increasing importance. Several empirical works have reported on the efficacy of scheduling redundant requests in such systems. That is, one may reduce job latency by 1) scheduling the same job at more than one server and 2) waiting only until the fastest of them responds. Inspired by the empirically observed gains of such schemes, several theoretical models have been proposed to explain the power of using redundant requests. Although the proposed models in the literature provide useful insights such as when scheduling redundant requests can be beneficial, all these results rely heavily on a common assumption: all redundant requests of a job can be immediately cancelled as soon as one of them is completed. In this paper, we study how one should schedule redundant requests when such assumption does not hold. This is of great importance in practice since cancellation of running jobs typically incurs non-negligible delays. In order to bridge the gap between the existing models and practice, we propose a new queueing model that captures such cancellation delays. We then find how one can schedule redundant requests to achieve the optimal average job latency when cancellation delay is considered and accounted for. Our results show that even with a small cancellation overhead, the actual optimal scheduling policy differs significantly from the optimal scheduling policy when the overhead is zero. Further, we study optimal dynamic scheduling policies, which appropriately schedule redundant requests based on the number of jobs in the system. Our analysis reveals that for the twoserver case, the optimal dynamic scheduler can achieve 7% to 16% lower average job latency, compared with the optimal static scheduler. This observation is in stark contrast to the known fact that the optimal static scheduler performs as well as the optimal dynamic scheduler when cancellation overhead is ignored, affirming that misleading conclusions result if the cancellation overhead is ignored completely.

Fast and robust compressive phase retrieval with sparse-graph codes

In this paper, we tackle the compressive phase retrieval problem in the presence of noise. The noisy compressive phase retrieval problem is to recover a K-sparse complex signal s ∈ ℂⁿ, from a set of m noisy quadratic measurements: y_i = |a_i^Hs|² + w_i; where a_i^H ∈ ℂⁿ is the ith row of the measurement matrix A ∈ ℂ^m×n, and w_i is the additive noise to the ith measurement. We consider the regime where K = βn^δ, δ ∈ (0; 1). We use the architecture of PhaseCode algorithm [1], and robustify it using two schemes: the almost-linear scheme and the sublinear scheme. We prove that with high probability, the almost-linear scheme recovers s with sample complexity¹ Θ(K log(n)) and computational complexity Θ(n log(n)), and the sublinear scheme recovers s with sample complexity Θ(K log³(n)) and computational complexity Θ(K log³(n)). To the best of our knowledge, this is the first scheme that achieves sublinear computational complexity for compressive phase retrieval problem. Finally, we provide simulation results that support our theoretical contributions.

Robust scheduling in a flexible fork-join network

We consider a general flexible fork-join processing network, in which jobs are modeled as directed acyclic graphs with nodes representing tasks, and edges representing precedence constraints among tasks. Both servers and tasks are flexible in the sense that each task can be processed by several servers, which in turn can serve multiple task types. The system model is motivated by the problem of efficient scheduling of both sequential and parallel tasks in a flexible processing environment, which arises in application areas such as healthcare, cloud computing, and manufacturing. A major challenge in designing efficient scheduling policies is the lack of reliable estimates of system parameters such as arrival and/or service rates. We call a policy robust if it does not depend on system parameters such as arrival and service rates. In this paper, we propose a robust scheduling policy for the flexible fork-join network model, and prove that it is rate stable when service rates can be written as products of a task-dependent quantity and a server-dependent quantity. We also provide a detailed simulation study to demonstrate the performance of the proposed policy.

Scheduling tasks with precedence constraints on multiple servers

We consider the problem of scheduling jobs which are modeled by directed acyclic graphs (DAG). In such graphs, nodes represent tasks of a job and edges represent precedence constraints in processing these tasks. The DAG scheduling problem, also known as scheduling in fork-join processing networks, is motivated by examples such as job scheduling in data centers and cloud computing, patient flow scheduling in health systems and many other applications. We consider a flexible system, in which servers may process different, possibly overlapping, sets of task types. In this paper, we first discuss the difficulties in designing provably efficient policies for DAG scheduling, which arise due to interactions between the flexibility of the processing environment and the precedence constraints in the system. A major difficulty is the classical synchronization issue, which is further complicated in the presence of system flexibility. Then, we propose two queueing networks to model the scheduling problem that overcome this difficulty. These are virtual queues that enable us to design provably efficient scheduling policies. We show that the well-known Max-Weight policy for these queueing networks is throughput-optimal. Finally, to compare the delay performance of the two queueing networks, we consider a simplified model in which tasks and servers are identical. We characterize their delay performances under a simple first-come-first-serve policy, via a novel coupling argument.

On the DMT optimality of the rotate-and-forward scheme in a two-hop MIMO relay channel

The rotate-and-forward scheme was introduced in (Yang-Belfiore, 2010) to recover spatial diversity in multi-hop MIMO relay networks. It was shown that this scheme achieves the optimal diversity-multiplexing (DMT) trade-off in a two-hop relay network, with two antennas at the relay node. In this paper, it is shown that the scheme is DMT optimal for arbitrary number of antennas at the source, relay, and destination node.