Daniela Loreti

Enabling big data analytics in the hybrid cloud using iterative mapreduce

The cloud computing model has seen tremendous commercial success through its materialization via two prominent models to date, namely public and private cloud. Recently, a third model combining the former two service models as on-/off-premise resources has been receiving significant market traction: hybrid cloud. While state of art techniques that address workload performance prediction and efficient workload execution over hybrid cloud setups exist, how to address data-intensive workloads - including Big Data Analytics - in similar environments is nascent. This paper addresses this gap by taking on the challenge of bursting over hybrid clouds for the benefit of accelerating iterative MapReduce applications. We first specify the challenges associated with data locality and data movement in such setups. Subsequently, we propose a novel technique to address the locality issue, without requiring changes to the MapReduce framework or the underlying storage layer. In addition, we contribute with a performance prediction methodology that combines modeling with micro-benchmarks to estimate completion time for iterative MapReduce applications, which enables users to estimate cost-to-solution before committing extra resources from public clouds. We show through experimentation in a dual-Openstack hybrid cloud setup that our solutions manage to bring substantial improvement at predictable cost-control for two real-life iterative MapReduce applications: large-scale machine learning and text analysis.

Distributed Compliance Monitoring of Business Processes over MapReduce Architectures

In the era of IoT, large volumes of event data from different sources are collected in the form of streams. As these logs need to be online processed to extract further knowledge about the underlying business process, it is becoming more and more important to give support to run-time monitoring. In particular, increasing attention has been turned to conformance checking as a way to identify when a sequence of events deviates from the expected behavior. Albeit rather straightforward on a small log file, conformance verification techniques may show poor performance when dealing with big data, making increasingly attractive the possibility to improve scalability through distributed computation. In this paper, we adopt a previously implemented framework for compliance verification (which provides a high-level logic-based notation for the monitoring specification) and we show how it can be efficiently distributed on a set of computing nodes to support scalable run-time monitoring when dealing with large volumes of event logs.

A Hybrid Cloud Infrastructure for Big Data Applications

The trending evolution towards the Internet of things and the general increase in broadband are constantly creating large volumes of data that need to be processed to extract further knowledge. Recently, the cloud computing model has seen the evolution from the initial scenario of a public cloud offering its resources to customers through virtualization and Internet, toward the concept of hybrid cloud, where the classic scenario is enriched with a private (company owned) cloud e.g., for the management of sensible data. In this work, we propose a software layer for the deployment and dynamic scaling of virtual clusters on a hybrid cloud. This system can be used for cloud bursting in the context of big data applications. Our work shows that although the execution is significantly influenced by the inter-cloud bandwidth, a dynamic off-premise provisioning mechanism could allow the user to significantly increase the application performance.

A distributed self-balancing policy for virtual machine management in cloud datacenters

Cloud Computing is a crucial computational paradigm for modern companies because it can discharge them from managing their ever growing IT infrastructure. Dynamically offering a plenty of computational resources, the cloud can also simplify the execution of CPU-intensive applications. Modern data centers for cloud computing are facing the challenge of a growing complexity due to the increasing number of users and their augmenting resource requests. A lot of efforts are now concentrated on providing the cloud infrastructure with autonomic behavior, so that it can take decisions about virtual machine (VM) management across the datacenters nodes without human intervention. While the major part of these solutions is intrinsically centralized and suffers of scalability and reliability problems, we investigate the possibility to provide the cloud with a decentralized self-organizing behavior. To this purpose we present a novel VM migration policy suitable for a distributed environment, where hosts can exchange status information with each other according to a predefined protocol. The main goal of the policy is to balance the computational load on datacenters physical hosts by conveniently moving virtual machines (VMs). We tested the policy performance by means of an ad hoc built simulator.

Process Mining Monitoring for Map Reduce Applications in the Cloud

The adoption of mobile devices and sensors, and the Internet of Things trend, are making available a huge quantity of information that needs to be analyzed. Distributed architectures, such as Map Reduce, are indeed providing technical answers to the challenge of processing these big data. Due to the distributed nature of these solutions, it can be difficult to guarantee the Quality of Service: e.g., it might be not possible to ensure that processing tasks are performed within a temporal deadline, due to specificities of the infrastructure or processed data itself. However, relaying on cloud infrastructures, distributed applications for data processing can easily be provided with additional resources, such as the dynamic provisioning of computational nodes. In this paper, we focus on the step of monitoring Map Reduce applications, to detect situations where resources are needed to meet the deadlines. To this end, we exploit some techniques and tools developed in the research field of Business Process Management: in particular, we focus on declarative languages and tools for monitoring the execution of business process. We introduce a distributed architecture where a logic-based monitor is able to detect possible delays, and trigger recovery actions such as the dynamic provisioning of further resources.

A distributed approach to compliance monitoring of business process event streams

Abstract In recent years, the significant advantages brought to business processes by process mining account for its evolution as a major concern in both industrial and academic research. In particular, increasing attention has been turned to compliance monitoring as a way to identify when a sequence of events deviates from the expected behaviour. As we are entering the IoT era, an increasing variety of smart objects can be introduced in business processes (e.g., tags to track products in a plant, smartphones and badge swiping to draw the activities of customers and employees in a shopping centre, etc.). All these objects produce large volumes of log data in the form of streams, which need to be run-time analysed to extract further knowledge about the underlying business process and to identify unexpected, non-conforming events. Albeit rather straightforward on a small log file, compliance verification techniques may show poor performances when dealing with big data and streams, thus calling for scalable approaches. This work investigates the possibility of spreading the compliance monitoring task over a network of computing nodes, achieving the desired scalability. The monitor is realised through the existing SCIFF framework for compliance checking, which provides a high level logic-based language for expressing the properties to be monitored and nicely supports the partitioning of the monitoring task. The distributed computation is achieved through a MapReduce approach and the adoption of an existing general engine for large scale stream processing. Experimental results show the feasibility of the approach as well as the advantages in performance brought to the compliance monitoring task.

Map Reduce Autoscaling over the Cloud with Process Mining Monitoring

Over the last years, the traditional pressing need for fast and reliable processing solutions has been further exacerbated by the increase of data volumes – produced by mobile devices, sensors and almost ubiquitous internet availability. These big data must be analyzed to extract further knowledge.

Mapreduce over the hybrid cloud: a novel infrastructure management policy

Over the last few years, the context of big data has gained a significant traction due to many factors. While the public cloud model had been deeply studied to face the increasing demand for large-scale data processing capabilities, many organizations are now focusing on the hybrid cloud model, where the classic scenario is enriched with a private (company owned) cloud -- e.g., for the management of sensible data. In this work, we propose HyMR, a policy to enable autonomic cloud bursting for clusters of virtual machines operating MapReduce jobs over a hybrid cloud. This policy -- together with an infrastructure level system for resource provisioning in hybrid clouds -- can be used to face the temporary (or permanent) lack of computational resources on the private cloud, allowing cloud bursting in the context of big data applications. By means of an empirical evaluation of the system scale-up/-down performance, we show that HyMR policy allows the user to significantly reduce the data-processing time.

UCD, Ergonomics and Inclusive Design: The HABITAT Project

Recent forecasts about the European population have highlighted the fact that the number of elderly people will grow rapidly in the upcoming years and that the economic impact of aging society will be relevant in all EU countries. In this perspective, a healthy, active, and independent aging, for as long as possible, is a goal that involves the whole community, as it can lead to an improvement in the quality of life and a great cost savings. In this scenario, digital technology can put itself at the service of healthy ageing also by empowering available tools and devices, and allowing the development of new support paradigms, like seamless anywhere-anytime medical treatment and home assistance, with sustainable quality and costs. The article aims at describing the application of a Human Centered design tool, like the QFD, to the selection and development of technological solutions related to physical and cognitive ergonomics issues in the design of smart objects connected to the Internet of Things for elderly. The applied methods take also in account the needs of all the people involved in the care and assistance of the elderly, trying to define the most inclusive and less intrusive design solutions. The analysis is based on the first results obtained by the Habitat project, a multidisciplinary design research focused on the development of a IOT platform for the Home Assistance of self-sufficient and non-self-sufficient elderly users.

Abduction for Generating Synthetic Traces

In this paper we report our preliminary experience on the design of a generator of synthetic logs. Sometimes real logs might not be available, or their quality might not be good enough: synthetic logs instead can be generated with all the desired features and characteristics. Our tool takes as input a structured workflow model, encoded in the abductive declarative language SCIFF, and provides as output a log containing positive traces, i.e. traces deemed as conformant w.r.t. the model. Distinctive features of our approach are the capability of generating trace templates as well as grounded traces, the possibility of taking into account user-specified constraints on data and timestamps, and the capability of generating traces starting from a user-specified partial trace. Although our tool is still in its preliminary version, we have successfully exploited it to generate synthetic logs of different dimension, thus proving the viability of our approach.