Faruk Caglar

A cloud middleware for assuring performance and high availability of soft real-time applications

Applications are increasingly being deployed in the cloud due to benefits stemming from economy of scale, scalability, flexibility and utility-based pricing model. Although most cloud-based applications have hitherto been enterprise-style, there is an emerging need for hosting real-time streaming applications in the cloud that demand both high availability and low latency. Contemporary cloud computing research has seldom focused on solutions that provide both high availability and real-time assurance to these applications in a way that also optimizes resource consumption in data centers, which is a key consideration for cloud providers. This paper makes three contributions to address this dual challenge. First, it describes an architecture for a fault-tolerant framework that can be used to automatically deploy replicas of virtual machines in data centers in a way that optimizes resources while assuring availability and responsiveness. Second, it describes the design of a pluggable framework within the fault-tolerant architecture that enables plugging in different placement algorithms for VM replica deployment. Third, it illustrates the design of a framework for real-time dissemination of resource utilization information using a real-time publish/subscribe framework, which is required by the replica selection and placement framework. Experimental results using a case study that involves a specific replica placement algorithm are presented to evaluate the effectiveness of our architecture.

iOverbook: Intelligent Resource-Overbooking to Support Soft Real-Time Applications in the Cloud

Cloud service providers (CSPs) often overbook their resources with user applications despite having to maintain service-level agreements with their customers. Overbooking is attractive to CSPs because it helps to reduce power consumption in the data center by packing more user jobs in less number of resources while improving their profits. Overbooking becomes feasible because user applications tend to overestimate their resource requirements utilizing only a fraction of the allocated resources. Arbitrary resource overbooking ratios, however, may be detrimental to soft real-time applications, such as airline reservations or Netflix video streaming, which are increasingly hosted in the cloud. The changing dynamics of the cloud preclude an offline determination of overbooking ratios. To address these concerns, this paper presents iOverbook, which uses a machine learning approach to make systematic and online determination of overbooking ratios such that the quality of service needs of soft real-time systems can be met while still benefiting from overbooking. Specifically, iOverbook utilizes historic data of tasks and host machines in the cloud to extract their resource usage patterns and predict future resource usage along with the expected mean performance of host machines. To evaluate our approach, we have used a large usage trace made available by Google of one of its production data centers. In the context of the traces, our experiments show that iOverbook can help CSPs improve their resource utilization by an average of 12.5% and save 32% power in the data center.

A publish/subscribe middleware for dependable and real-time resource monitoring in the cloud

Providing scalable and QoS-enabled (i.e., real-time and reliable) monitoring of resources (both virtual and physical) in the cloud is essential to supporting application QoS properties in the cloud as well as identifying security threats. Existing approaches to resource monitoring in the cloud are based on web interfaces, such as RESTful APIs and SOAP, which cannot provide real-time information efficiently and scalably because of a lack of support for fine-grained and differentiated monitoring capabilities. Moreover, their implementation overhead results in a distinct loss in performance, incurs latency jitter, and degrades reliable delivery of time-sensitive information. To address these challenges this paper presents a novel lighter weight and scalable resource monitoring and dissemination solution based on the publish/subscribe (pub/sub) paradigm. Our solution called SQRT-C leverages the OMG Data Distribution Service (DDS) real-time pub/sub middleware, and uses effective software engineering principles to make it usable with multiple cloud platforms. Preliminary empirical results comparing SQRT-C with contemporary web-based resource usage monitoring services reveals that SQRT-C is significantly better than the conventional approaches in terms of latency, jitter and scalability.

A simulation as a service cloud middleware

Many seemingly simple questions that individual users face in their daily lives may actually require substantial number of computing resources to identify the right answers. For example, a user may want to determine the right thermostat settings for different rooms of a house based on a tolerance range such that the energy consumption and costs can be maximally reduced while still offering comfortable temperatures in the house. Such answers can be determined through simulations. However, some simulation models as in this example are stochastic, which require the execution of a large number of simulation tasks and aggregation of results to ascertain if the outcomes lie within specified confidence intervals. Some other simulation models, such as the study of traffic conditions using simulations may need multiple instances to be executed for a number of different parameters. Cloud computing has opened up new avenues for individuals and organizations with limited resources to obtain answers to problems that hitherto required expensive and computationally-intensive resources. This paper presents SIMaaS, which is a cloud-based Simulation-as-a-Service to address these challenges. We demonstrate how lightweight solutions using Linux containers (e.g., Docker) are better suited to support such services instead of heavyweight hypervisor-based solutions, which are shown to incur substantial overhead in provisioning virtual machines on-demand. Empirical results validating our claims are presented in the context of two case studies.

Cloud-hosted simulation-as-a-service for high school STEM education

Abstract Despite their advanced status, nations such as the United States of America continue to face a STEM (science, technology, engineering and mathematics) crisis in their education system. Lack of effective teaching modalities that can leverage real-world examples to stimulate student interest in STEM concepts are identified as one of the reasons for this crisis. To address these challenges, our research is investigating the use of innovative and attractive modeling and simulation frameworks for concurrent, interactive and collaborative STEM education where vehicular traffic serves as the real-world example to reify STEM concepts. Existing traffic-related tools, such as traffic simulators, however, do not provide: (1) intuitive abstractions to construct, refine, and simulate various traffic models that are commensurate to the level of high school students, (2) concurrent and scalable model execution, and (3) collaborative learning environments. On the other hand, although intuitive abstractions such as Google Maps exist, these abstractions do not support semantics for dynamic behavior, which is representative of real-world traffic scenarios. To overcome both these challenges and address the STEM problem, this paper presents a Cloud-based, Collaborative, and Scaled-up Modeling and Simulation Framework for STEM Education called C2SuMo. The key contribution of this paper lies in the design and implementation of a cloud-based, elastic modeling and simulation framework that provides an intuitive, model-driven, collaborative, and concurrent visual simulation environment for STEM education. The paper also reports on insights we gained conducting a user study involving over sixty high school students.

Teaching Computational Thinking Skills in C3STEM with Traffic Simulation

Computational thinking (CT) skills applied to Science, Technology, Engineering, and Mathematics (STEM) are critical assets for success in the 21st century workplace. Unfortunately, many K-12 students lack advanced training in these areas. C3STEM seeks to provide a framework for teaching these skills using the traffic domain as a familiar example to develop analysis and problem solving skills. C3STEM is a smart learning environment that helps students learn STEM topics in the context of analyzing traffic flow, starting with vehicle kinematics and basic driver behavior. Students then collaborate to produce a large city-wide traffic simulation with an expert tool. They are able to test specific hypotheses about improving traffic in local areas and produce results to defend their suggestions for the wider community.

iPlace: An Intelligent and Tunable Power- and Performance-Aware Virtual Machine Placement Technique for Cloud-Based Real-Time Applications

Power and performance tradeoffs are critical and challenging issues faced by cloud service providers (CSPs) while managing their data centers. On the one hand, CSPs strive to reduce power consumption of their data centers to not only decrease their energy costs but to also reduce adverse impact on the environment. On the other hand, CSPs must deliver performance expected by the applications hosted in their cloud in accordance with predefined Service Level Agreements (SLAs). Not doing so will lead to loss of customers and thereby major revenue losses for the CSPs. Addressing these dual set of challenges is hard for the CSPs because power management and performance assurance are conflicting objectives, particularly in the context of multi-tenant cloud systems where multiple virtual machines (VMs) may be hosted on a single physical server. The problem becomes even harder when real-time applications are hosted in these VMs. To address these challenges and make appropriate tradeoffs, we present iPlace, which is an intelligent and tunable power- and performance-aware VM placement middleware. The placement strategy is based on a two-level artificial neural network which predicts (1) CPU usage at the first level, and (2) power consumption and performance of a host machine at the second level that uses the predicted CPU usage. The efficacy of iPlace is evaluated in the context of a VM consolidation algorithm that is applied to running virtual machines and host machines in a private cloud.

Model-driven performance estimation, deployment, and resource management for cloud-hosted services

There is a growing trend towards migrating applications and services to the cloud. This trend has led to the emergence of different cloud service providers (CSPs), in turn leading to different cost models offered by these CSPs to lease their resources, variabilities in the granularity and specification of resources provided, and heterogeneous APIs offered by the CSPs to the users to program resource requests and deployment for their cloud-hosted services. These challenges make it hard for customers of the cloud to seamlessly transition their services to the cloud or migrate between different CSPs. To address these challenges, this paper presents a solution based on model-driven engineering (MDE). Specifically, we describe the design of the domain-specific modeling languages in our MDE framework and the associated generative mechanisms that address the challenges related to estimating performance and cost to host the services in the cloud, automated deployment and resource management.

iTune: Engineering the Performance of Xen Hypervisor via Autonomous and Dynamic Scheduler Reconfiguration

Despite the widespread use of server virtualization technologies in cloud data centers, system administrators experience multiple challenges in configuring the hypervisor’s scheduler parameters to optimize its performance. Manually tuning the scheduler’s parameters is a common practice, however, this approach is not effective particularly when dealing with dynamically changing workload and resource utilizations on the host machines. This problem becomes even harder if cloud resources are overbooked while hosting both latency-sensitive and batched applications. To address these issues, this paper presents iTune, which is a framework for engineering the performance of a hypervisor intelligently via autonomous scheduler configurations. Concretely, iTune optimizes the Xen hypervisor’s scheduler configuration parameters autonomously through a three phase process comprising: (1) Discoverer, which monitors and saves the resource usage history of the host machines and groups set of related host machine workloads, (2) Optimizer, where optimum Xen scheduler configuration parameters for each workload cluster are explored by employing a simulated annealing machine learning algorithm, and (3) Observer, where iTune monitors the resource usage of host machines online, classifies them into one of the categories found in the Discoverer phase, and loads the optimum scheduler parameters determined in the Optimizer phase. Experimental results validate our claims.

Intelligent, Performance Interference-Aware Resource Management for IoT Cloud Backends

Emerging Internet of Things (IoT) applications often demonstrate unpredictable Big Data processing workloads at the cloud backends making it hard for cloud service providers (CSPs) to employ existing resource overbooking schemes effectively. Ad hoc approaches to resource overbooking can lead to performance interference among the virtual machines (VMs) hosted on the physical resources causing performance unpredictability for VM-hosted performance-sensitive IoT applications. Balancing these conflicting needs requires an intelligent strategy for hosting applications such that the performance interference effects are minimized while still allowing resource overbooking. Such a strategy must be online because application workloads may change at run time. To address these challenges, this paper presents iSensitive, which is an intelligent, performance interference-aware resource management scheme for IoT cloud backends. iSensitive first classifies the VMs based on their historic mean CPU, memory, and network usage features. Subsequently, it learns the desired VM patterns of collocating the classified VMs by employing machine learning techniques. These extracted patterns document the lowest performance interference level on the specified host machines making them amenable to hosting performance-sensitive applications while still allowing resource overbooking. Our approach is validated by emulating a publicly available usage trace of a large data center owned by Google and benchmark tools running real-world applications. Experimental results evaluating iSensitive illustrates its advantages in deploying VMs to aptly-suited host machine compared to traditional schemes, such as first-fit bin packing.