Shin Gyu Kim

MRBench: A Benchmark for MapReduce Framework

MapReduce is Googles programming model for easy development of scalable parallel applications which process huge quantity of data on many clusters. Due to its conveniency and efficiency, MapReduce is used in various applications (e.g., Web search services and online analytical processing). However, there are only few good benchmarks to evaluate MapReduce implementations by realistic testsets. In this paper, we present MRBench that is a benchmark for evaluating MapReduce systems. MRBench focuses on processing business oriented queries and concurrent data modifications. To this end, we build MRBench to deal with large volumes of relational data and execute highly complex queries. By MRBench, users can evaluate the performance of MapReduce systems while varying environmental parameters such as data size and the number of (map/reduce) tasks. Our extensive experimental results show that MRBench is a useful tool to benchmark the capability of answering critical business questions.

A RESTful Approach to the Management of Cloud Infrastructure

Recently, REpresentational State Transfer (REST) has been proposed as an alternative architecture for Web services.In the era of Cloud and Web 2.0, many complex Web service-based systems such as e-Business an de-Government applications have adopted REST. Unfortunately, the REST approach has been applied to few cases in management systems, especially for a management system for cloud computing infrastructures.In this paper, we design and implement a RESTful Cloud Management System (CMS).Managed elements can be modeled as resources in REST and operations in existing systems can be evaluated using four methods of REST or a combination of them.We also show how components of existing management systems can be realized as REST-style Web services.

Adaptive delay-based congestion control for high bandwidth-delay product networks

The design of an end-to-end Internet congestion control protocol that could achieve high utilization, fair sharing of bottleneck bandwidth, and fast convergence while remaining TCP-friendly is an ongoing challenge that continues to attract considerable research attention. This paper presents ACP, an Adaptive end-to-end Congestion control Protocol that achieves the above goals in high bandwidth-delay product networks where TCP becomes inefficient. The main contribution of ACP is a new form of congestion window control, combining the estimation of the bottleneck queue size and a measure of fair sharing. Specifically, upon detecting congestion, ACP decreases the congestion window size by the exact amount required to empty the bottleneck queue while maintaining high utilization, while the increases of the congestion window are based on a “fairness ratio” metric of each flow, which ensures fast convergence to a fair equilibrium. We demonstrate the benefits of ACP using both ns-2 simulation and experimental measurements of a Linux prototype implementation. In particular, we show that the new protocol is TCP-friendly and allows TCP and ACP flows to coexist in various circumstances, and that ACP indeed behaves more fairly than other TCP variants under heterogeneous round-trip times (RTT).

Energy-centric DVFS controlling method for multi-core platforms

Dynamic voltage and frequency scaling (DVFS) is a well-known and effective technique for reducing energy consumption in modern processors. However, accurately predicting the effect of frequency scaling on system performance is a challenging problem in real environments. In this paper, we propose a realistic DVFS performance prediction method, and a practical DVFS control policy (eDVFS) that aims to minimize total energy consumption in multi-core platforms. We also present power consumption estimation models for CPU and DRAM by exploiting a hardware energy monitoring unit. We implemented eDVFS in Linux, and our evaluation results show that eDVFS can save a substantial amount of energy compared with Linux “on-demand” CPU governor in diverse environments.

Virtual machine consolidation based on interference modeling

Server consolidation is very attractive for cloud computing platforms to improve energy efficiency and resource utilization. Advances in multi-core processors and virtualization technologies have enabled many workloads to be consolidated in a physical server. However, current virtualization technologies do not ensure performance isolation among guest virtual machines, which results in degraded performance due to contention in shared resources along with violation of service level agreement (SLA) of the cloud service. In that sense, minimizing performance interference among co-located virtual machines is the key factor of successful server consolidation policy in the cloud computing platforms. In this work, we propose a performance model that considers interferences in the shared last-level cache and memory bus. Our performance interference model can estimate how much an application will hurt others and how much an application will suffer from others. We also present a virtual machine consolidation method called swim which is based on our interference model. Experimental results show that the average performance degradation ratio by swim is comparable to the optimal allocation.

Improving Hadoop performance in intercloud environments

Intercloud is a federated environment of private clusters and public clouds. The performance of Hadoop could be degraded significantly in intercloud environments. Because previous solutions for intercloud environments rely on speculative execution, they require additional cost in the cloud. In this paper, we propose a new task scheduler that improves performance without the help of speculative execution in intercloud environments.

Energy-Centric DVFS Controling Method for Multi-core Platforms

Saving data center energy consumption is a hot issue for the environment and the economy. CPU is the biggest energy consuming component in a server, and it has various energy saving technologies, such as C-state and P-state. Advances in multi-core processors and virtualization technologies have enabled many workloads to be consolidated in a physical server, and energy efficiency can be degraded due to the contention in shared resources. We observed that energy efficiency can be improved further by adjusting CPU frequency according to the degree of contention in shared resources, and the most energy-efficient CPU frequency is quite different for each situation. In this paper, we propose an energy-centric DVFS controlling method (eDVFS), which aims to minimize total energy consumption. Our experimental results show that eDVFS method is more energy-efficient and faster than current “on-demand” CPU governor.

Virtual machine scheduling for multicores considering effects of shared on-chip last level cache interference

As the cloud markets grow, the cloud providers are faced with new challenges such as reduction of power consumption and guaranteeing service level agreements (SLAs). One reason for these problems is the use of server consolidation policy based on virtualization technologies for maximizing the efficiency of resource usage. Because current virtualization technologies do not ensure performance isolation among active virtual machines (VMs), it is required to consider resource usage pattern of VMs to improve total throughput and quality of service. In this paper, we propose a virtual machine scheduler for multicore processors, which exploits the last-level cache (LLC) reference ratio. Specifically, we focus on the performance impact of contention in a shared LLC. We have found that the ratio of the number of LLC references to that of instructions (LLC reference ratio) is highly associated with the amount of cache demand, and a Performance-Maximizing VM (PMV) scheduling algorithm can be devised by using the ratio. We show that our PMV scheduler is effective by evaluation for various workloads.

Improving MapReduce Performance by Exploiting Input Redundancy

The proliferation of data parallel programming on large clusters has set a new research avenue: accommodating numerous types of data-intensive applications with a feasible plan. Behind the many research efforts, we can observe that there exists a nontrivial amount of redundant I/O in the execution of data-intensive applications. This redundancy problem arises as an emerging issue in the recent literature because even the locality-aware scheduling policy in a MapReduce framework is not effective in a cluster environment where storage nodes cannot provide a computation service. In this article, we introduce SplitCache for improving the performance of data-intensive OLAP-style applications by reducing redundant I/O in a MapReduce framework. The key strategy to achieve the goal is to eliminate such I/O redundancy especially when different applications read common input data within an overlapped time period; SplitCache caches the first input stream in the computing nodes and reuses them for future demands. We also design a cache-aware task scheduler that plays an important role in achieving high cache utilization. In execution of the TPC-H benchmark, we achieved 64.3% faster execution and 83.48% reduction in network traffic in average.

Large Graph Processing Based on Remote Memory System

This paper focuses on large graph processing based on the remote memory system. Using our remote memory system enables applications to deal with large data sets, especially graph data, which do not fit into the machines main memory. Although recent dramatic increases in DRAM capacity now allow us to build inexpensive computers with very large amounts of main memory, the rise in brand-new Internet services has resulted in rapid increases in data size. This is especially true for on-line social network services that generate various data sets that can be represented as graphs. On the other hand, high-speed networking technologies such as Infini Band, Myrinet and 10G Ethernet now enable us to transfer data with low latency and high throughput. The advanced networking technologies reduce the latency/bandwidth gap between main memory and remote memory. Thus, remote memory based processing could now be helpful in accelerating large-scale graph process when main memory space is insufficient to store application data. In this paper, we present our design and implementation of remote memory system that efficiently processes large graph data. We also evaluate a breadth-first search of various types of graphs using our system and show that our approach is good for large graph data processing.