Jan Schaffner

Predicting in-memory database performance for automating cluster management tasks

In Software-as-a-Service, multiple tenants are typically consolidated into the same database instance to reduce costs. For analytics-as-a-service, in-memory column databases are especially suitable because they offer very short response times. This paper studies the automation of operational tasks in multi-tenant in-memory column database clusters. As a prerequisite, we develop a model for predicting whether the assignment of a particular tenant to a server in the cluster will lead to violations of response time goals. This model is then extended to capture drops in capacity incurred by migrating tenants between servers. We present an algorithm for moving tenants around the cluster to ensure that response time goals are met. In so doing, the number of servers in the cluster may be dynamically increased or decreased. The model is also extended to manage multiple copies of a tenants data for scalability and availability. We validated the model with an implementation of a multi-tenant clustering framework for SAPs in-memory column database TREX.

Shared Table Access Pattern Analysis for Multi-Tenant Applications

With the rise of applications in the area of software as a service it becomes crucial to operate profitable even with small profit margins. The possibility of customers to switch between different hosted solutions makes the calculation of the total cost of ownership even more important. Not only the application design is an important factor in this calculation but as well the employed storage techniques used to create and maintain thousands of instances of the same application play one of the most important roles in this game. None of the well-known commercial of the shelf databases like MySQL, PostgreSQL, etc. provides an optimized solution for multi-tenancy problems. In this paper we analyze typical operations in the area of multi-tenant databases and show how they are wired together. Based on the analysis we propose an architecture for a multi-tenant storage engine capable to overcome the problems of typical relational databases in this area.

A Formal Model for Mixed Initiative Service Composition

Automated service composition has been investigated thoroughly during the last years. Although it promises to alleviate the difficulties of manual service composition, it will only work if complete and correct service specifications are available. In this paper, we present a third approach - semi-automated composition - based on three mixed initiative features which we have derived from an industry case study with SAP. These features, filter inappropriate services, check validity, and suggest partial plans, are definded on the basis of a common formal model. Related approaches, in contrast, are limited to supporting individual mixed initiative features. To show the applicability of our approach, we have developed a prototypical implementation. Our results show that our mixed initiative approach significantly eases the modeling of service compositions.

A semi-automated orchestration tool for service-based business processes

When creating service compositions from a very large number of atomic service operations, it is inherently difficult for the modeler to discover suitable operations for his particular goal. Automated service composition claims to solve this problem, but only works if complete and correct ontologies alongside with service descriptions are in place. In this paper, we present a semi-automated modeling environment for Web service compositions. At every step in the process of creating the composition, the environment suggests the modeler a number of relevant Web services. Furthermore, the environment summarizes the problems that would prevent the composed service from being invocable. The environment is also able to insert composed services into the composition at suitable places, with atomic services producing the required data artifacts to come to an invocable composition. Our results show that this mixed initiative approach significantly eases the creation of composed services. We validated our implementation with the leading vendor of business applications, using their processes and service repository, which spans across multiple functional areas of enterprise computing.

The effects of virtualization on main memory systems

Virtualization is mainly employed for increasing the utilization of a lightly-loaded system by consolidation, but also to ease the administration based on the possibility to rapidly provision or migrate virtual machines. These facilities are crucial for efficiently managing large data centers. At the same time, modern hardware --- such as Intels Nehalem microarchitecure --- change critical assumptions about performance bottlenecks and software systems explicitly exploiting the underlying hardware --- such as main memory databases --- gain increasing momentum. In this paper, we address the question of how these specialized software systems perform in a virtualized environment. To do so, we present a set of experiments looking at several different variants of in-memory databases: The MonetDB Calibrator, a fine-grained hybrid row/column in-memory database running an OLTP workload, and an in-memory column store database running a multi-user OLAP workload. We examine how memory management in virtual machine monitors affects these three classes of applications. For the multi-user OLAP experiment we also experimentally compare a virtualized Nehalem server to one of its predecessors. We show that saturation of the memory bus is a major limiting factor but is much less impactful on the new architecture.

Mixed initiative use cases for semi-automated service composition: a survey

Semi-automated service composition with mixed initiative interactions, where both user and machine jointly contribute to the creation of composed services, is currently subject to intensive research. In this paper, we give an overview over recent research approaches by presenting three different semi-automated service composition tools. As the main contribution of this paper, we introduce three mixed initiative use cases characteristic for semi-automated composition, which we have extracted and generalized from the presented approaches and then extended. Based on these use cases and additional distinctive properties, we give a qualitative evaluation of the presented approaches.

A Composite Benchmark for Online Transaction Processing and Operational Reporting

Up-to-date data is of immense importance for operational reporting. Global enterprises require such a high throughput during daily operations that reporting systems had to be separated from the transactional system to avoid inhibiting performance. These architectures, however, do not provide the required reporting flexibility as the data set is a pre-defined subset of the actual data and updated only at certain time intervals, e.g. nightly. The composite benchmark for online transaction processing (OLTP) and operational reporting, henceforth CBTR, provides means to evaluate the performance of enterprise systems for a mixed workload of OLTP and operational reporting queries. Such a system offers up-to-date information and the flexibility of the entire data set for reporting. CBTR provokes the conflicts that were the reason for separating the two workloads on different systems. In this paper we introduce the concepts of CBTR, which is based on the original data set and real workloads of an existing, globally operating enterprise.

Towards enterprise software as a service in the cloud

For traditional data warehouses, mostly large and expensive server and storage systems are used. In particular, for small- and medium size companies, it is often too expensive to run or rent such systems. These companies might need analytical services only from time to time, for example at the end of a billing period. A solution to overcome these problems is to use Cloud Computing. In this paper, we report on work-in-progress towards building an OLAP cluster of multi-tenant main memory column databases on the Amazon EC2 cloud computing environment, for which purpose we ported SAPs in-memory column database TREX to run in the Amazon cloud. We discuss early findings on cost/performance tradeoffs between reliably storing the data of a tenant on a single node using a highly-available network attached storage, such as Amazon EBS, vs. replication of tenant data to a secondary node where the data resides on less resilient storage. We also describe a mechanism to provide support for historical queries across older snapshots of tenant data which is lazy-loaded from Amazons S3 near-line archiving storage and cached on the local VM disks.

Analytics on historical data using a clustered insert-only in-memory column database

In the field of OLAP and Data Warehousing, column stores and compressed main-memory data storage technology have successfully been implemented in products that enable a significant speed improvement of analytical queries with special performance requirements. We could soon see the majority of analytical workloads move to such main-memory based systems. Having one specialized OLAP DBMS explicitly aimed at performing ad-hoc queries on an ever-growing database requires the capability of an in-memory database to retain historical states so that applications can calculate consistent values based on previous states of the database, a requirement often found in financial and production planning analytical applications. This paper describes Rock, an in-memory analytics cluster based on a column store database, and proposes an architecture for historical query support as well as the prototypical implementation in Rock.

Enterprise data management in mixed workload environments

Enterprise applications are presently built on a 20year old data management infrastructure that was designed to meet a specific set of requirements for OLTP systems. In the meantime, enterprise applications have become more sophisticated, data set sizes have increased, requirements on the freshness of input data have been strengthened, and the time allotted for completing business processes has been reduced. To meet these challenges, enterprise applications have become increasingly complicated to make up for short-comings in the data management infrastructure. This paper outlines the characteristics of enterprise application with regards to the underlying data management layer. We also propose a database design perfectly fit to the demanded requirements of enterprise applications.