IEEE Transactions on Communications | 2021
Coded Computing for Resilient, Secure, and Privacy-Preserving Distributed Matrix Multiplication
Abstract
Coded computing is a new framework to address fundamental issues in large scale distributed computing, by injecting structured randomness and redundancy. We first provide an overview of coded computing and summarize some recent advances. Then we focus on distributed matrix multiplication and consider a common scenario where each worker is assigned a fraction of the multiplication task. In particular, by partitioning two input matrices into <inline-formula> <tex-math notation= LaTeX >$m$ </tex-math></inline-formula>-by-<inline-formula> <tex-math notation= LaTeX >$p$ </tex-math></inline-formula> and <inline-formula> <tex-math notation= LaTeX >$p$ </tex-math></inline-formula>-by-<inline-formula> <tex-math notation= LaTeX >$n$ </tex-math></inline-formula> subblocks, a single multiplication task can be viewed as computing linear combinations of <inline-formula> <tex-math notation= LaTeX >$pmn$ </tex-math></inline-formula> submatrix products, which can be assigned to <inline-formula> <tex-math notation= LaTeX >$pmn$ </tex-math></inline-formula> workers. Such block-partitioning-based designs have been widely studied under the topics of secure, private, and batch computation, where the state of the arts all require computing at least “cubic” (<inline-formula> <tex-math notation= LaTeX >$pmn$ </tex-math></inline-formula>) number of submatrix multiplications. Entangled polynomial codes, first presented for straggler mitigation, provides a powerful method for breaking the cubic barrier. It achieves a subcubic recovery threshold, i.e., recovering the final product from <italic>any</italic> subset of multiplication results with a size order-wise smaller than <inline-formula> <tex-math notation= LaTeX >$pmn$ </tex-math></inline-formula>. We show that entangled polynomial codes can be further extended to also include these three important settings, providing unified frameworks that order-wise reduce the total computational costs by achieving subcubic recovery thresholds.