Mohamed Adel Attia

Information Theoretic Limits of Data Shuffling for Distributed Learning

Data shuffling is one of the fundamental building blocks for distributed learning algorithms, that increases the statistical gain for each step of the learning process. In each iteration, different shuffled data points are assigned by a central node to a distributed set of workers to perform local computation, which leads to communication bottlenecks. The focus of this paper is on formalizing and understanding the fundamental information-theoretic tradeoff between storage (per worker) and the worst-case communication overhead for the data shuffling problem. We completely characterize the information theoretic tradeoff for K = 2, and K = 3 workers, for any value of storage capacity, and show that increasing the storage across workers can reduce the communication overhead by leveraging coding. We propose a novel and systematic data delivery and storage update strategy for each data shuffle iteration, which preserves the structural properties of the storage across the workers, and aids in minimizing the communication overhead in subsequent data shuffling iterations.

An efficient Intrusion Detection System against cyber-physical attacks in the smart grid

Abstract Without robust security mechanisms, the smart grid remains vulnerable to many attacks that can cause serious damages. Since state estimation is a critical entity to monitor and control electricity production and distribution, intruders are more attracted by this entity in order to disrupt the smart grid reliability. In this context, we propose an Intrusion Detection System (IDS) architecture to detect lethal attacks with a focus on two smart grid security issues: (i) Firstly, against integrity issue with price manipulation attack, we propose a Cumulative Sum (CUSUM) algorithm that detects this attack even with granular price changes; (ii) Secondly, the availability issue with Denial of Service (DoS) attack against which we develop an efficient method to monitor and detect any misbehaving node. Performance evaluations show the robustness of the proposed IDS system compared to existing mechanisms. The achieved detection rate is above 95% and the false positive rate is below 5%.

Towards using local energy to enhance smart meters privacy

Nowadays, smart meters play a primordial role in the Smart Grid (SG) in order to optimize electricity production, distribution as well as consumption. By exchanging more frequent information between the smart meter and Utility Provider (UP), SG will operate more efficiently with less loss. However, this frequent exchange poses a serious privacy issue which can invade individual privacy since the UP is able to collect a massive amount of smart meter-related data not necessary for billing purposes. In this paper, we propose an efficient sustainable model that uses a local energy to enhance the consumer privacy and reduce his electricity cost at the same time. Moreover, the proposed model is lightweight but also reliable to deal with the low computation capacity of the smart meter. Analytic results prove the efficiency of our model.

On the secure degrees-of-freedom of partially connected networks with no CSIT

In this work, we focus on the partially connected interference network with confidential messages, and study the secure degrees of freedom with no channel state information at the transmitters (CSIT). Prior works on fully connected interference networks with full CSIT have shown that the secure degrees of freedom scales linearly with the number of users. With no CSIT, however, the secure degrees of freedom of fully connected networks collapses to zero. In this work, we show that partial connectivity, a widely prevalent property of wireless networks, can be leveraged to provide secrecy even with no CSIT. We present a systematic approach to first understand the feasibility of secure communication in a partially connected network and develop achievable schemes for a class of regular partially connected networks. Finally, we also provide novel information theoretic outer bounds on the secure degrees of freedom for this class of regular partially connected networks, and approximately characterize the secure degrees of freedom.

On the worst-case communication overhead for distributed data shuffling

Distributed learning platforms for processing large scale data-sets are becoming increasingly prevalent. In typical distributed implementations, a centralized master node breaks the data-set into smaller batches for parallel processing across distributed workers to achieve speed-up and efficiency. Several computational tasks are of sequential nature, and involve multiple passes over the data. At each iteration over the data, it is common practice to randomly re-shuffle the data at the master node, assigning different batches for each worker to process. This random re-shuffling operation comes at the cost of extra communication overhead, since at each shuffle, new data points need to be delivered to the distributed workers. In this paper, we focus on characterizing the information theoretically optimal communication overhead for the distributed data shuffling problem. We propose a novel coded data delivery scheme for the case of no excess storage, where every worker can only store the assigned data batches under processing. Our scheme exploits a new type of coding opportunity and is applicable to any arbitrary shuffle, and for any number of workers. We also present information theoretic lower bounds on the minimum communication overhead for data shuffling, and show that the proposed scheme matches this lower bound for the worst-case communication overhead.