Yaoqing Yang

Computing Linear Transformations With Unreliable Components

We consider the problem of computing a binary linear transformation when all circuit components are unreliable. Two models of unreliable components are considered: probabilistic errors and permanent errors. We introduce the “ENCODED” technique that ensures that the error probability of the computation of the linear transformation is kept bounded below a small constant independent of the size of the linear transformation even when all logic gates in the computation are noisy. By deriving a lower bound, we show that in some cases, the computational complexity of the ENCODED technique achieves the optimal scaling in error probability. Further, we examine the gain in energy-efficiency from the use of a “voltage-scaling” scheme, where gate-energy is reduced by lowering the supply voltage. We use a gate energy-reliability model to show that tuning gate-energy appropriately at different stages of the computation (“dynamic” voltage scaling), in conjunction with ENCODED, can lead to orders of magnitude energy-savings over the classical “uncoded” approach. Finally, we also examine the problem of computing a linear transformation when noiseless decoders can be used, providing upper and lower bounds to the problem.

Can a noisy encoder be used to communicate reliably

In this paper the problem of reliable communication with a noisy encoder is examined. We explicitly provided the construction of the encoder and show that even when all logic gates that constitute the encoder are noisy, reliable communication with a positive rate is still possible. The encoding complexity is shown to be O(log 1/ptar/log1/ε) per bit to achieve a target bit error rate ptar, where ε denotes the error probability of each noisy gate. This complexity upper bound is shown to coincide with a lower bound in order sense, and is hence tight. The key technique in the proposed construction is to embed noisy decoders inside the noisy encoder, which are utilized repeatedly to prevent the bit error rate from escalating. The proposed noisy encoder has a direct application in noisy computing of a linear transform.

Fault-tolerant distributed logistic regression using unreliable components

We consider the problem of computing distributed logistic regression using unreliable components. We consider both faults in the memory units and faults in the processing units. We show that using a real-number-coding technique, we can suppress errors during the computation and ensure that logistic regression converges with bounded error if the number of faults that happen during each iteration of the logistic regression is bounded, even when the faults happen in an adversarial manner. Moreover, since the coding technique is based on computation with real numbers, we show that the error-correction can be carried out at the algorithmic level (or block-level) based on the results from intermediate steps of logistic regression. Therefore, we only need to add redundant hardware at block-level, not the circuit level, for achieving fault-tolerance in the computation of logistic regression.

Coding for lossy function computation: Analyzing sequential function computation with distortion accumulation

We consider the problem of lossy linear function computation for Gaussian sources in a tree network. The goal is to find the optimal tradeoff between the sum rate (the overall number of bits communicated in the network) and the achieved distortion (the overall mean-square error of estimating the function result) at a specified sink node. Using random Gaussian codebooks, an inner bound is obtained that is shown to match the information-theoretic outer bound (obtained in our earlier work [1]) in the limit of zero distortion. To compute the overall distortion for the random coding scheme, we applied the analysis of Distortion Accumulation which was quantified in [1] for MMSE estimates of intermediate computation variables instead of for the codewords of random Gaussian codebooks. The key in applying the analysis of Distortion Accumulation is showing that the random-coding based codeword on the receiver side is close in mean-square sense to the MMSE estimate of the source, even if the knowledge of the source distribution is not fully accurate.

Computing linear transforms with unreliable components

We consider the problem of computing a binary linear transform when all circuit components are unreliable. We propose a novel “ENCODED” technique that uses LDPC (low-density parity-check) codes and embedded noisy decoders to keep the error probability of the computation below a small constant independent of the size of the linear transform, even when all logic gates in the computation are prone to probabilistic errors. Unlike existing works on applying coding to computing with unreliable components, the “ENCODED” technique explicitly considers the errors that happen during both the encoding and the decoding phases. Further, we show that ENCODED requires fewer operations (in order sense) than repetition techniques.

Fault-tolerant parallel linear filtering using compressive sensing

In this paper, we study the problem of designing fault-tolerant parallel linear filters. We assume that a linear filter can either function perfectly or fail completely, i.e., generate arbitrary outputs. We use real-number error correcting codes based on linear programming decoding to introduce redundancy into the linear filters and detect faulty filters. We prove that all faulty filters can be detected and corrected if the number of faulty filters is smaller than a threshold value. We also obtain simulation results to support our statement. To the best of our knowledge, we believe that this work is the first to connect the information-theoretic idea of real-number error control coding with parallel linear filtering systems.

Information dissipation in noiseless lossy in-network function computation

We consider the problem of distributed lossy linear function computation in a tree network. We examine two cases: (i) data aggregation (only one sink node computes) and (ii) consensus (all nodes compute the same function). By quantifying the information dissipation in distributed computing, we obtain fundamental limits on network computation rate as a function of incremental distortions (and hence incremental information dissipation) along the edges of the network, and not just the overall distortions used classically. Combining this observation with an inequality on the dominance of mean-square measures over relative-entropy measures, we obtain lower bounds on the rate-distortion function that are tighter than classical cut-set bounds by a difference which can be arbitrarily large in both data aggregation and consensus.

Detecting Localized Categorical Attributes on Graphs

Do users from Carnegie Mellon University form social communities on Facebook? Do signal processing researchers tightly collaborate with each other? Do Chinese restaurants in Manhattan cluster together? These seemingly different problems share a common structure: an attribute that may be localized on a graph. In other words, nodes activated by an attribute form a subgraph that can be easily separated from other nodes. In this paper, we thus focus on the task of detecting localized attributes on a graph. We are particularly interested in categorical attributes such as attributes in online social networks, ratings in recommender systems, and viruses in cyber-physical systems because they are widely used in numerous data mining applications. To solve the task, we formulate a statistical hypothesis testing problem to decide whether a given attribute is localized or not. We propose two statistics: Graph wavelet statistic and graph scan statistic, both of which are provably effective in detecting localized attributes. We validate the robustness of the proposed statistics on both simulated data and two real-world applications: High air-pollution detection and keyword ranking in a coauthorship network collected from IEEE Xplore. Experimental results show that the proposed graph wavelet statistic and graph scan statistic are effective and efficient.

Rate Distortion for Lossy In-Network Linear Function Computation and Consensus: Distortion Accumulation and Sequential Reverse Water-Filling

We consider the problem of distributed lossy linear function computation in a tree network. We examine two cases: 1) data aggregation (only one sink node computes) and 2) consensus (all nodes compute the same function). By quantifying the accumulation of information loss in distributed computing, we obtain fundamental limits on network computation rate as a function of incremental distortions (and hence incremental loss of information) along the edges of the network. The above characterization, based on quantifying distortion accumulation, offers an improvement over classical cut-set type techniques, which are based on overall distortions instead of incremental distortions. This quantification of information loss qualitatively resembles information dissipation in cascaded channels [2]. Surprisingly, this accumulation effect of distortion happens even at infinite blocklength. Combining this observation with an inequality on the dominance of mean-square quantities over relative-entropy quantities, we obtain outer bounds on the rate distortion function that are tighter than classical cut-set bounds by a difference, which can be arbitrarily large in both data aggregation and consensus. We also obtain inner bounds on the optimal rate using random Gaussian coding, which differ from the outer bounds by

Energy efficient distributed coding for data collection in a noisy sparse network

\mathcal {O}(\sqrt {D})