Adnene Guabtni

An overview of the commercial cloud monitoring tools: research dimensions, design issues, and state-of-the-art

Cloud monitoring activity involves dynamically tracking the Quality of Service (QoS) parameters related to virtualized resources (e.g., VM, storage, network, appliances, etc.), the physical resources they share, the applications running on them and data hosted on them. Applications and resources configuration in cloud computing environment is quite challenging considering a large number of heterogeneous cloud resources. Further, considering the fact that at given point of time, there may be need to change cloud resource configuration (number of VMs, types of VMs, number of appliance instances, etc.) for meet application QoS requirements under uncertainties (resource failure, resource overload, workload spike, etc.). Hence, cloud monitoring tools can assist a cloud providers or application developers in: (i) keeping their resources and applications operating at peak efficiency, (ii) detecting variations in resource and application performance, (iii) accounting the service level agreement violations of certain QoS parameters, and (iv) tracking the leave and join operations of cloud resources due to failures and other dynamic configuration changes. In this paper, we identify and discuss the major research dimensions and design issues related to engineering cloud monitoring tools. We further discuss how the aforementioned research dimensions and design issues are handled by current academic research as well as by commercial monitoring tools.

A data model for processing financial market and news data

Due to immense amounts of data being generated from financial markets in many different formats, professionals and academics face interoperability problems when analysing such data. This paper proposes a data model that gives a coherent view of the information available from financial market data repositories. The novel features of this data model include: modelling the behaviour of an electronic market as an extensible event-based class hierarchy, and using ontologies to represent financial data as a set of inter-related and meaningful events. Using this data model, we develop interoperable web services that process the data at a high level of abstraction using a Service-Oriented Architecture.

Towards scientific workflow patterns

Scientific workflow management systems provide users with a set of design primitives for process modeling and execution features that have different semantics and capabilities comparing to traditional workflow management systems. The main limitation that prevents the democratization of scientific workflow management systems is the lack of appropriate guidelines and abstract constructs for the development of workflow models. This paper takes on the challenge of developing design patterns for scientific workflow modeling. We present the hybrid semantics of scientific workflow modeling that compose control and data dependencies. We discuss the appropriateness of standard modeling notations to scientific workflow modeling and present the basic scientific workflow patterns.

ADAGE: a framework for supporting user-driven ad-hoc data analysis processes

Data analysis is an important part of the scientific process carried out by domain experts in data-intensive science. Despite the availability of several software tools and systems, their use in combination with each other for conducting complex types of analyses is a very difficult task for non-IT experts. The main contribution of this paper is to introduce an open architectural framework based on service-oriented computing (SOC) principles called the Ad-hoc DAta Grid Environment (ADAGE) framework that can be used to guide the development of domain-specific problem-solving environments or systems to support data analysis activities. Through an application of the ADAGE framework and a prototype implementation that supports the analysis of financial news and market data, this paper demonstrates that systems developed based on the framework allow users to effectively express common analysis processes. This paper also outlines some limitations as well as avenues for future research.

A User-Driven SOA for Financial Market Data Analysis

This paper is concerned with an environment, referred to as Ad-hoc DAta Grid Environment (ADAGE), which facilitates the analysis of large financial datasets by expert end-users. The paper focuses on the design of a Service-Oriented Architecture (SOA) that makes it possible to define re-usable and interoperable software components as web services to manipulate entities of an underlying event-based data model. Such a model allows for a coherent representation of market activities as events, e.g., high-frequency market data like trade prices and quotes, and a subsequent analysis. The paper also describes an implementation of a user-driven composition tool based on the SOA which allows domain experts to conveniently compose services to execute individualised processes. The approach is illustrated on a case study about analysing the price setting behaviour of issuers in the market for structured products.

A workload-driven approach to database query processing in the cloud

This paper is concerned with data provisioning services (information search, retrieval, storage, etc.) dealing with a large and heterogeneous information repository. Increasingly, this class of services is being hosted and delivered through Cloud infrastructures. Although such systems are becoming popular, existing resource management methods (e.g. load-balancing techniques) do not consider workload patterns nor do they perform well when subjected to non-uniformly distributed datasets. If these problems can be solved, this class of services can be made to operate in more a scalable, efficient, and reliable manner.The main contribution of this paper is a approach that combines proprietary cloud-based load balancing techniques and density-based partitioning for efficient range query processing across relational database-as-a-service in cloud computing environments. The study is conducted over a real-world data provisioning service that manages a large historical news database from Thomson Reuters. The proposed approach has been implemented and tested as a multi-tier web application suite consisting of load-balancing, application, and database layers. We have validated our approach by conducting a set of rigorous performance evaluation experiments using the Amazon EC2 infrastructure. The results prove that augmenting a cloud-based load-balancing service (e.g. Amazon Elastic Load Balancer) with workload characterization intelligence (density and distribution of data; composition of queries) offers significant benefits with regards to the overall system’s performance (i.e. query latency and database service throughput).

A User-Driven Environment for Financial Market Data Analysis

This paper proposes a software development environment which facilitates the analysis of large financial datasets by end-users. This environment is based on an event-based data model that gives a coherent representation of market activities, particularly high-fequency market and news data. The model makes it possible to define software components and Web services to manipulate entities in the model. The paper also describes a prototype implementation which allows domain experts to compose components and services to build an application. This prototype uses the Triana scientific workflow system to define workflows of existing software components and Web Services. This approach is demonstrated on a realistic case study related to processing both news and financial market data.

Exploration of Discovered Process Views in Process Spaceship

Business processes are important for streamlining the operations of public and private enterprises. Over the last decade, capabilities arising from advances in online technologies, especially ServiceOrientedArchitectures (SOA), enabled enterprises to increase productivity, simplify automation, and extend the execution if business processes to various systems in the enterprise. While business process management systems, which allow formodeling, analysis, andmanagement of business processes, are relatively successful, currently, they only cover a fraction of business processes in the enterprise.One challenge inmodern enterprises is that information about business process execution is maintained over multiple heterogeneous systems (e.g., email systems, ERP, document management systems, etc), and rarely there exists a central workflow log, where all process execution information can be found. The next challenge is that the traditional one-view-fits-all fashion of process definition does not scale, as different users may have their own perspective of the business process execution in the enterprise. In such environments, not only one but a space of processes can be defined corresponding to the perspectives of different users or systems involved in the process.

Using graph aggregation for service interaction message correlation

Discovering the behavior of services and their interactions in an enterprise requires the ability to correlate service interaction messages into process instances. The service interaction logic (or process model) is then discovered from the set of process instances that are the result of a given way of correlating messages. However, sometimes, the Correlation Conditions (CC) allowing to identify correlations of messages from a service interaction log are not known. In such cases, and with a large number of messages correlator attributes, we are facing a large space of possible ways messages may be correlated which makes identifying process instances difficult. In this paper, we propose an approach based on message indexation and aggregation to generate a size-efficient Aggregated Correlation Graph (ACG) that exhibits all the ways messages correlate in a service interaction log not only for disparate pairs of messages but also for sequences of messages corresponding to process instances. Adapted filtering techniques based on user defined heuristics are then applied on such a graph to help the analysts efficiently identify the most frequently executed processes from their sequences of CCs. The approach has been implemented and experiments show its effectiveness to identify relevant sequences of CCs from large service interaction logs.

Management towards reducing cloud usage costs

Many organizations are attracted to cloud computing as an ICT (information and communications technology) sourcing model that can improve flexibility and total cost of ICT systems. However, it can be difficult for a prospective cloud customer to determine and manage cloud usage costs. We present an overview of several NICTA research projects that aim at providing information that can help ICT professionals determine various cloud usage costs and make decisions that are appropriate from the business viewpoint. Before migrating an application into a cloud, it is necessary to choose to which cloud to migrate, because there is a huge variety of cloud offerings, with significantly different pricing models. To accurately capture projected operating costs of an application in a particular cloud and enable side-by-side comparison of cloud offerings from different providers, NICTA developed a cost estimation tool that calculates the costs based on usage patterns and other characteristics of the application. This tool can also be used during runtime as an input into making adaptation/control decisions. To collect various runtime metrics (e.g., about the amount of transferred data or received quality of service - QoS) that are necessary for operational management and assessment of cloud usage costs, NICTA developed an innovative tool for flexible and integrated monitoring of applications in clouds and (in case of hybrid clouds) related local data centers. To help determine which runtime adaptation/control decisions are best from the business viewpoint (e.g., incur lowest cost), we extended the WS-Policy4MASC language and MiniZnMASC middleware for autonomic business-driven IT management with events and adaptation actions relevant for cloud management. The tools from the presented projects can be used separately or as parts of a powerful integrated cloud management system (which contains several additional tools).