Mostapha Zbakh

Performance comparison between Hadoop and Spark frameworks using HiBench benchmarks.

Big Data has become one of the major areas of research for cloud service providers due to a large amount of data produced every day and the inefficiency of traditional algorithms and technologies to handle these large amounts of data. Big Data with its characteristics such as volume, variety, and veracity (3V) requires efficient technologies to process in real time. To solve this problem and to process and analyze this vast amount of data, there are many powerful tools like Hadoop and Spark, which are mainly used in the context of Big Data. They work following the principles of parallel computing. The challenge is to specify which Big Datas tool is better depending on the processing context. In this paper, we present and discuss a performance comparison between two popular Big Data frameworks deployed on virtual machines. Hadoop MapReduce and Apache Spark are used to efficiently process a vast amount of data in parallel and distributed mode on large clusters, and both of them suit for Big Data processing. We also present the execution results of Apache Hadoop in Amazon EC2, a major cloud computing environment. To compare the performance of these two frameworks, we use HiBench benchmark suite, which is an experimental approach for measuring the effectiveness of any computer system. The comparison is made based on three criteria: execution time, throughput, and speedup. We test Wordcount workload with different data sizes for more accurate results. Our experimental results show that the performance of these frameworks varies significantly based on the use case implementation. Furthermore, from our results we draw the conclusion that Spark is more efficient than Hadoop to deal with a large amount of data in major cases. However, Spark requires higher memory allocation, since it loads the data to be processed into memory and keeps them in caches for a while, just like standard databases. So the choice depends on performance level and memory constraints.

Cloud computing and big data: Technologies and applications

During the last decade, cloud computing has gained great attention from academia, industry, and government as a new infrastructure requiring slighter investments in hardware platform, staff training, or licensing new software tools. It is defined in the work of Borko and Armando as a new computing paradigm in which resources are provided as services and considered as an infrastructure characterized by its availability, ease of use, and no installation or configuration required from end users. In other words, cloud computing can be seen as a collection of resources and applications that offer the following services. SaaS (Software‐as‐a‐Service) that allows end users to run applications from their PCs, laptops, PDAs, smartphones, or tablets. IaaS (Infrastructure‐as‐a‐Service) that allows users to use clouds resources as a service. PaaS (Platform‐as‐a‐Service) that gives the users a complete development platform including computing resources and operating systems to develop new services and applications. There are 4 types of cloud computing: public, private, hybrid, and community clouds. Public or external cloud allows all off‐site users to use available resources over the Internet via Web applications or Web services. Private or internal cloud could be built for the exclusive use of 1 client, which takes responsibility for its management and control. The hybrid cloud combines multiple public and private cloud platforms. Finally, community cloud is a cloud‐hosting type in which there is a mutual sharing of the setup among many organizations belonging to a particular community such as trading firms and banks. All these models are devoted to providing users with on‐demand resources while ensuring Quality of Service (QoS) in hardware/CPU performance, bandwidth, and memory capacity together with autonomous and transparent system management. However, despite various efforts to improve the cloud performances, there are still several challenges that need to be addressed. For example, the scalability issue that can be addressed by integrating high‐performance platforms and techniques to increase the computing performance and data storage. Furthermore, security and privacy issues and concerns such as Distributed Denial of Service (DDOS) and phishing attacks are considered among the biggest challenges against the widespread adoption of cloud computing services by organizations and customers. In parallel to the rapid development and deployment of cloud services that provide users with access to services (all the time, everywhere, and in a transparent way), the high volume of data that are generated from physical (ie, devices embedded in the surrounding physical environment and/or carried by the user) and Web sensors (ie, social media like Facebook and Twitter) reinforce the usefulness

CoderLabs: A cloud-based platform for real-time online labs with user collaboration

In this paper we describe the architecture of a real time environment for numerous remote experiments. The environment is created with web standards as HTML5 such that no plug-in needs to be installed by the user. Users are able to use the remote lab simultaneously and in collaboration. This collaboration between users is made feasible by adopting a message broker. Finally, by using Google Coder, developers can easily change or create the user interface of their remote experiments and share experiments in the cloud.

Performance Analysis of Preconditioned Conjugate Gradient Solver on Heterogeneous (Multi-CPUs/Multi-GPUs) Architecture

The solution of systems of linear equations is one of the most central processing unit-intensive steps in engineering and simulation applications and can greatly benefit from the multitude of processing cores and vectorisation on today’s parallel computers. Our objective is to evaluate the performance of one of them, the conjugate gradient method, on a hybrid computing platform (Multi-GPU/Multi-CPU). We consider the preconditioned conjugate gradient solver (PCG) since it exhibits the main features of such problems. Indeed, the relative performance of CPU and GPU highly depends on the sub-routine: GPUs are for instance much more efficient to process regular kernels such as matrix vector multiplications rather than more irregular kernels such as matrix factorization. In this context, one solution consists in relying on dynamic scheduling and resource allocation mechanisms such as the ones provided by StarPU. In this chapter we evaluate the performance of dynamic schedulers proposed by StarPU, and we analyse the scalability of PCG algorithm. We show how effectively we can choose the best combination of resources in order to improve their performance.

A Review of Green Cloud Computing Techniques

The information and communication technology became of important use but its impact on the environment became as important due to the large amount of CO2 emissions and energy consumption. Cloud computing is considered one of Information and communication technologies that managed to achieve efficient usage of resources and energy. However, data centers still represent a huge percentage of the companies energy cost since the usage is continuously growing. Ever since this issue took notice, the number of research on energy efficiency and the green field has being growing. Green cloud computing represents a solution to allow companies and users to use the Cloud and all its perks while reducing the negative environmental impact and general costs through energy efficiency, carbon footprint and e-waste reduction. Applications and practices to make companies more eco friendly are being developed or deployed day by day. Different aspects are treated in companies to achieve green cloud computing. This chapter presents different techniques to achieve green computing but focuses more on Cloud computing.

Threshold-based load balancing algorithm for Big Data on a Cloud environment

In this paper, we discuss a load balancing strategy in heterogeneous cloud environments. Load balancing of data processing takes a very important place in cloud computing for several years. Load balancing on cloud systems is critical problem that is difficult to cope with, especially on the emerging heterogeneous clusters. In this aspect, we propose a threshold-based load balancing algorithm that balances the load among datacenters in cloud environments as well as minimizing remote communication among datacenters. The proposed approach is divided into two phases. Firstly, we specify the load threshold of each datacenter based on its processing speed and storage capacity. Secondly, we maintain load balancing among datacenters based on this threshold while the proposed approach takes into consideration the heterogeneity of datacenters. The results show that our approach improve efficiently the load balancing among datacenters.

Performance Analysis of Intrusion Detection Systems in Cloud-Based Systems

Cloud computing services are widely used nowadays and need to be more secured for an effective exploitation by the users. One of the most challenging issues in these environments is the security of the hosted data. Many cloud computing providers offer web applications for their clients, this is why the most handling attacks in cloud computing are Distributed Denial of Service (DDoS). In this paper, we provide a comparative performance analysis of intrusion detection systems (IDSs) in a real world lab. The aim is to provide an up to date study for researchers and practitioners to understand the issues related to intrusion detection and to deal with DDoS attacks. This analysis includes intrusion detection rates, time running, etc. In the experiments, we configured a cloud platform using OpenStack and an IDS monitoring the whole network traffic of the web server configured. The results show that Suricata drops fewer packets than Bro and Snort successively when a DDoS attack is happening and detect more malicious packets.