In Kee Kim

CloudDRN: A Lightweight, End-to-End System for Sharing Distributed Research Data in the Cloud

The cloud has proven itself as a scalable platform for Web-based applications. However, scientists and medical researchers are still searching for a simple cloud-based architecture that enables secure collaboration and sharing of distributed datasets. To date, attempts at using the cloud for this purpose generally view the cloud as simply a pool of servers upon which to run their legacy software. This approach fails to leverage the unique platform capabilities of the cloud. In this paper, we describe our Cloud Distributed Research Network (CloudDRN). We leverage the cloud for availability, reliability, scalability, and improved security as compared to legacy distributed systems while still supporting site autonomy. Our philosophy is to adapt commercial software tooling that was originally designed for business use-cases, thereby benefiting from the large built-in user community. We describe our general architecture and show an example of our system created to share distributed clinical research data. We evaluate our system in Amazon Web Services (AWS) and in Microsoft Windows Azure and find that while each cloud achieves similar financial cost, representative queries are 3.5x slower on average in Windows Azure.

Empirical Evaluation of Workload Forecasting Techniques for Predictive Cloud Resource Scaling

Many predictive resource scaling approaches have been proposed to overcome the limitations of the conventional reactive approaches most often used in clouds today. In general, due to the complexity of clouds, these reactive approaches were often forced to make significant limiting assumptions in either the operating conditions/requirements or expected workload patterns. As such, it is extremely difficult for cloud users to know which – if any – existing workload predictor will work best for their particular cloud activity, especially when considering highly-variable workload patterns, non-trivial billing models, variety of resources to add/subtract, etc. To solve this problem, we conduct comprehensive evaluations for a variety of workload predictors under real-world cloud configurations. The workload predictors cover four classes of 21 predictors: naive, regression, temporal, and non-temporal methods. We simulate a cloud application under four realistic workload patterns, two different cloud billing models, and three different styles of predictive scaling. Our evaluation confirms that no workload predictor is universally best for all workload patterns, and shows that Predictive Scaling-out + Predictive Scaling-in has the best cost efficiency and the lowest job deadline miss rate in cloud resource management, on average providing 30% better cost efficiency and 80% less job deadline miss rate compared to other styles of predictive scaling.

Adaptive Distance Filter-based Traffic Reduction for Mobile Grid

The mobile grid introduces various research challenges distinguished from existing grid computing systems. They are low bandwidth, low processing power, low battery capacity, frequent disconnectivity, and mobility. Mobility of the grid node increases the system load of the mobile grid in a constrained operating environment by increasing the number of communication messages required to confirm the location between the grid broker and mobile grid node. Therefore, this paper proposes an adaptive distance filter that can effectively reduce communication traffic between the mobile grid node and grid broker. This filter constructs clusters based on the mobility and velocity of the grid node and filters the location updates. However, the reduction of location updates generates location errors, which occur when the grid broker cannot acquire the exact location of mobile nodes. To solve this problem, if the location updates are filtered, the grid broker can estimate the location of the mobile node using a statistical estimation method. For the performance evaluation of the adaptive distance filter, we modeled the mobility of the grid nodes. We then measured the reduction in location updates and location errors. In these experiments, we prove that the adaptive distance filter is an effective scheme for reducing location updates and the grid broker can reduce location errors through location estimation.

QLP-LBS: quantization and location prediction-based LBS for reduction of location update costs

This paper proposes the QLP-LBS (Quantization and Location Prediction-based LBS). This QLP-LBS system is based on quantization theory and uses statistical location prediction mechanism. This LBS applies the quantum range of quantization theory to each mobile user and reduces location update costs by comparing results between moving distance of mobile user and quantum range. But, this LBS system generates location errors from the quantization. In order to solve this problem, we apply statistical location prediction mechanism to LBS system. This prediction mechanism predicts location of mobile user using its historical path and decrease location errors by quantization and makes more reliable LBS system. For performance evaluation, this paper measures location accuracy and reduction rate of location update costs with various quantum ranges. This experiments show that QLP-LBS effectively reduces location update costs of LBS system. Also, QLP-LBS solves problem of location errors using location prediction mechanism which is problem of general quantized system. Therefore, QLP-LBS is solution for reduction of location update costs and has reliable location accuracy.

WDCloud: An end to end system for large-scale watershed delineation on cloud

Watershed delineation is a process to compute the drainage area for a point on the land surface, which is a critical step in hydrologic and water resources analysis. However, existing watershed delineation tools are still insufficient to support hydrologists and watershed researchers due to the lack of essential capabilities such as fully leveraging scalable and high performance computing infrastructure (public cloud), and providing predictable performance for the delineation tasks. To solve these problems, this paper reports on WDCloud, which is a system for large-scale watershed delineation on public cloud. For the design and implementation of WDCloud, we employ three main approaches: 1) an automated catchment search mechanism for a public data set, 2) three performance improvement strategies (Data-reuse, parallel-union, and MapReduce), and 3) local linear regression-based execution time estimator for watershed delineation. Moreover, WDCloud extensively utilizes several compute and storage capabilities from Amazon Web Services in order to maximize the performance, scalability, and elasticity of watershed delineation system. Our evaluations on WDCloud focus on two main aspects of WDCloud; the performance improvement for watershed delineation via three strategies and the estimation accuracy for watershed delineation time by local linear regression. The evaluation results show that WDCloud can achieve 18x-111x of speed-ups for delineating any scale of watersheds in the contiguous United States as compared to commodity laptop environments, and accurately predict execution time for watershed delineation with 85.6% of prediction accuracy, which is 23%-13% higher than other state-of-the-art approaches.

Adaptive and Mobility-predictive Quantization-based Communication Data Management for High Performance Distributed Computing

Communication data management (CDM) is an important issue in high performance distributed computing where a massive amount of data eXchange frequently occurs among geographically distributed components. In this paper, we review eXisting CDM schemes in distributed computing systems and we propose more efficient CDM schemes. Three types of quantization-based CDM schemes are proposed: the fiXed quantization-based CDM (FQ-CDM), the adaptive quantization-based CDM (AQ-CDM), and the mobility-predictive quantization-based CDM (MPQ-CDM). The FQ-CDM applies a basic theory of quantized systems to the distributed computing environment. The AQ-CDM uses a communication object clustering mechanism, which operates a pattern recognition clustering algorithm. The MPQ-CDM predicts the neXt states of communication objects by using past and current data and controls data communication among communication objects. The mobile object location monitoring system (MOLMS), based on High Level Architecture, is designed and developed to apply these CDM schemes to distributed computing. In this paper we conduct eXperiments by comparing these CDM schemes with each other on the MOLMS. The eXperimental results show that the AQ-CDM is the more effective scheme for communication message reduction and the MPQ-CDM is the more suitable scheme for mobile location error reduction.Communication data management (CDM) is an important issue in high performance distributed computing where a massive amount of data eXchange frequently occurs among geographically distributed components. In this paper, we review eXisting CDM schemes in distributed computing systems and we propose more efficient CDM schemes. Three types of quantization-based CDM schemes are proposed: the fiXed quantization-based CDM (FQ-CDM), the adaptive quantization-based CDM (AQ-CDM), and the mobility-predictive quantization-based CDM (MPQ-CDM). The FQ-CDM applies a basic theory of quantized systems to the distributed computing environment. The AQ-CDM uses a communication object clustering mechanism, which operates a pattern recognition clustering algorithm. The MPQ-CDM predicts the neXt states of communication objects by using past and current data and controls data communication among communication objects. The mobile object location monitoring system (MOLMS), based on High Level Architecture, is designed and developed to apply these CDM schemes to distributed computing. In this paper we conduct eXperiments by comparing these CDM schemes with each other on the MOLMS. The eXperimental results show that the AQ-CDM is the more effective scheme for communication message reduction and the MPQ-CDM is the more suitable scheme for mobile location error reduction.

Node Availability-Based Congestion Control Model Using Fuzzy Logic for Computational Grid

This paper deals with congestion control in which GRID network community has been interested. In order to solve problems of existing congestion control models like FOBS, this paper proposes the node availability-based congestion control model. The model monitors not only storage availability but also network availability of grid nodes to detect and control congestion. As using the fuzzy adaptive packet allocation mechanism, the model estimates congestion level of grid nodes and adjusts their packet transmission rate for stable and reliable data transfer in computational grid. Experiment results demonstrate that our model outperforms FOBS which is a typical congestion control model with the improved QoS.

Resource demand prediction-based grid resource transaction network model in grid computing environment

This paper reviews existing grid resource transaction models in grid computing environment and proposes an efficient market mechanism-based grid resource transaction model. This model predicts a future grid resource demand of grid users and suggests a reasonable transaction price of each resource to customers and resource providers. The suggestion of transaction price infers the more transactions between customers and providers and reduces a response time after ordering resource. In order to improve accuracy of transaction price prediction, microeconomics-based statistics approach is applied to this grid resource transaction model. For performance evaluation, this paper measures resource demand response time, and number of transactions. This model works on the less 72.39% of response time and the more 162.56% of the number of transactions than those of single auction model and double auction model.