Masood Mortazavi

DCODE: A Distributed Column-Oriented Database Engine for Big Data Analytics

We propose a novel Distributed Column-Oriented Database Engine (DCODE) for efficient analytic query processing that combines advantages of both column storage and parallel processing. In DCODE, we enhance an existing open-source columnar database engine by adding the capability for handling queries over a cluster. Specifically, we studied parallel query execution and optimization techniques such as horizontal partitioning, exchange operator allocation, query operator scheduling, operator push-down, and materialization strategies, etc. The experiments over the TPC-H dataset verified the effectiveness of our system.

A Zero-Penalty Container-Based Execution Infrastructure for Hadoop Framework

With the growing need of sharing computing resoures in data-center, various resource management and scheduling schemes have been designed for Hadoop framework. Unfortunately, the Hadoops existing schedulers (such as Fair and Delayed Fair) suffer from the conflicting nature of fairness and data locality, which can jeopardize system performance. In this work, we propose a zero-penalty container-based execution infrastructure for Hadoop to address such a problem. Specifically, based on Linux container LXC, we designed the framework where map/reduce tasks can be executed inside their designated containers instead of in virtual machines as in the traditional approach. When a task needs to be cancelled due to system fairness requirements, unlike the conventional approach in Hadoop that kills the task, the corresponding container is frozen, which can continue its execution when CPU slots become available for the task at a later time. With such un/freeze operations of LXC containers, the proposed infrastructure can essentially provide preemptive executions of map/reduce tasks in Hadoop framework, where the partial computation of the enclosed tasks in containers can be preserved with zero-penalty. We illustrated the capabilities of the proposed execution infrastructure through a simple setting of two users, and the results show that the proposed scheme can remarkably reduce the waiting and completion time of tasks by up to 32.89% and 14.93%.

Privacy and Big Data

Issues related to privacy and Big Data came to broader academic scrutiny and greater public attention with Edward Snowden’s revelations regarding NSA’s Big Data surveillance programs and methods. A large number of academic conferences, government summits and probing legal, social and engineering studies have now tackled the subject of Big Data surveillance and its impact on people’s private lives. This chapter provides an overview of the concepts of privacy and Big Data. It begins with a review of the benefits and limitations of Big Data analysis techniques including some of the purely mathematical challenges such as the curse of dimensionality. It next reviews the more modern understanding of the concept of privacy, discussing various legal and ethical issues including those particular to Big Data systems. A general overview follows regarding the current privacy protection techniques and the challenges we face. The analysis of the modern conceptual understanding of privacy proves that much of the ancient and classical conceptualization of privacy and the taboos against eavesdropping, as discussed in the introduction, have survived into the current age but at a much more complicated manner. While researchers have articulated conceptual details that pay due attention to the impact of privacy violation on freedom and human beings’ personality development across the board, the broader ethical understanding seems to be fading away as privacy policies become harder to track and understand. Furthermore, privacy protection techniques are still in their infancy. While they have some applications in enterprise and health care, the challenges posed to privacy by Big Data surveillance capabilities can only best be met by architectural shifts such as trusted cloud architectures which will have direct business and other implications. Proposals for privacy-protecting architectures of the future are currently in early development by various researchers and technologists who share an interest in protecting what gives us our personalities and differences—our privacy. Most of these techniques point towards attempts to turn the Big Data cloud into storage machines for encrypted data.

Optimization Strategies for Column Materialization in Parallel Execution of Queries

All parallel query processing frameworks need to determine the optimality norms for column materialization. We investigate performance trade-off of alternative column materialization strategies. We propose a common parallel query processing approach that encapsulates varying column materialization strategies within exchange nodes in query execution plans. Our experimental observations confirm the theoretically deduced trade-offs that suggest optimality norms to be dependent on the scale of the cluster, data transmissions required for a query, and the predicate selectivities involved. Lastly, we have applied a probit statistical model to the experimental data in order to establish a systemdependent adhoc performance estimation method that can be used to select the optimal materialization strategy at runtime.

Novel approach to big data collaboration with network operators network function virtualisation (NFV)

The intersection of network function virtualisation (NFV) technologies and big data has the potential of revolutionising todays telecommunication networks from deployment to operations resulting in significant reductions in capital expenditure (CAPEX) and operational expenditure, as well as cloud vendor and additional revenue growths for the operators. One of the contributions of this article is the comparisons of the requirements for big data and network virtualisation and the formulation of the key performance indicators for the distributed big data NFVs at the operators infrastructures. Big data and virtualisation are highly interdependent and their intersections and dependencies are analysed and the potential optimisation gains resulted from open interfaces between big data and carrier networks NFV functional blocks for an adaptive environment are then discussed. Another contribution of this article is a comprehensive discussion on open interface recommendations which enables global collaborative and scalable virtualised big data applications.