Kai Herrmann

CoDEL – A Relationally Complete Language for Database Evolution

Software developers adapt to the fast-moving nature of software systems with agile development techniques. However, database developers lack the tools and concepts to keep pace. Data, already existing in a running product, needs to be evolved accordingly, usually by manually written SQL scripts. A promising approach in database research is to use a declarative database evolution language, which couples both schema and data evolution into intuitive operations. Existing database evolution languages focus on usability but did not aim for completeness. However, this is an inevitable prerequisite for reasonable database evolution to avoid complex and error-prone workarounds. We argue that relational completeness is the feasible expressiveness for a database evolution language. Building upon an existing language, we introduce CoDEL. We define its semantic using relational algebra, propose a syntax, and show its relational completeness.

Living in Parallel Realities: Co-Existing Schema Versions with a Bidirectional Database Evolution Language

We introduce end-to-end support of co-existing schema versions within one database. While it is state of the art to run multiple versions of a continuously developed application concurrently, it is hard to do the same for databases. In order to keep multiple co-existing schema versions alive -- which are all accessing the same data set -- developers usually employ handwritten delta code (e.g. views and triggers in SQL). This delta code is hard to write and hard to maintain: if a database administrator decides to adapt the physical table schema, all handwritten delta code needs to be adapted as well, which is expensive and error-prone in practice. In this paper, we present InVerDa: developers use the simple bidirectional database evolution language BiDEL, which carries enough information to generate all delta code automatically. Without additional effort, new schema versions become immediately accessible and data changes in any version are visible in all schema versions at the same time. InVerDa also allows for easily changing the physical table design without affecting the availability of co-existing schema versions. This greatly increases robustness (orders of magnitude less lines of code) and allows for significant performance optimization. A main contribution is the formal evaluation that each schema version acts like a common full-fledged database schema independently of the chosen physical table design.

InVerDa - co-existing schema versions made foolproof

In modern software landscapes multiple applications usually share one database as their single point of truth. All these applications will evolve over time by their very nature. Often former versions need to stay available, so database developers find themselves maintaining co-existing schema version of multiple applications in multiple versions. This is highly error-prone and accounts for significant costs in software projects, as developers realize the translation of data accesses between schema versions with hand-written delta code. In this demo, we showcase INVERDA, a tool for integrated, robust, and easy to use database versioning. We rethink the way of specifying the evolution to new schema versions. Using the richer semantics of a descriptive database evolution language, we generate all required artifacts automatically and make database versioning foolproof.

Online horizontal partitioning of heterogeneous data

Abstract In an increasing number of use cases, databases face the challenge of managing heterogeneous data. Heterogeneous data is characterized by a quickly evolving variety of entities without a common set of attributes. These entities do not show enough regularity to be captured in a traditional database schema. A common solution is to centralize the diverse entities in a universal table. Usually, this leads to a very sparse table. Although todays techniques allow efficient storage of sparse universal tables, query efficiency is still a problem. Queries that address only a subset of attributes have to read the whole universal table including many irrelevant entities. A solution is to use a partitioning of the table, which allows pruning partitions of irrelevant entities before they are touched. Creating and maintaining such a partitioning manually is very laborious or even infeasible, due to the enormous complexity. Thus an autonomous solution is desirable. In this article, we define the Online Partitioning Problem for heterogeneous data. We sketch how an optimal solution for this problem can be determined based on hypergraph partitioning. Although it leads to the optimal partitioning, the hypergraph approach is inappropriate for an implementation in a database system. We present Cinderella, an autonomous online algorithm for horizontal partitioning of heterogeneous entities in universal tables. Cinderella is designed to keep its overhead low by operating online; it incrementally assigns entities to partition while they are touched anyway during modifications. This enables a reasonable physical database design at runtime instead of static modeling.

Robust and simple database evolution

Software developers adapt to the fast-moving nature of software systems with agile development techniques. However, database developers lack the tools and concepts to keep the pace. Whenever the current database schema is evolved, the already existing data needs to be evolved as well. This is usually realized with manually written SQL scripts, which is error-prone and explains significant costs in software projects. A promising solution are declarative database evolution languages, which couple both schema and data evolution into intuitive operations. Existing database evolution languages focus on usability but do not strive for completeness. However, this is an inevitable prerequisite to avoid complex and error-prone workarounds. We present CoDEL which is based on an existing language but is relationally complete. We precisely define its semantic using relational algebra, propose a syntax, and formally validate its relational completeness. Having a complete and comprehensive database evolution language facilitates valuable support throughout the whole evolution of a database. As an instance, we present VaCo, a tool supporting developers with variant co-evolution. Given a variant schema derived from a core schema, VaCo uses the richer semantics of CoDEL to semi-automatically co-evolve this variant with the core.

Position Paper: Runtime Model for Role-Based Software Systems

In the increasingly dynamic realities of todays software systems, it is no longer feasible to always expect human developers to react to changing environments and changing conditions immediately. Instead, software systems need to be self-aware and autonomously adapt their behavior according to their experiences gathered from their environment. Current research provides role-based modeling as a promising approach to handle the adaptivity and self-awareness within a software system. There are established role-based systems e.g., for application development, persistence, and so on. However, these are isolated approaches using the role-based model on their specific layer and mapping to existing non-role-based layers. We present a global runtime model covering the whole stack of a software system to maintain a global view of the current system state and model the interdependencies between the layers. This facilitates building holistic role-based software systems using the role concept on every single layer to exploit its full potential, particularly adaptivity and self-awareness.

Database Evolution for Software Product Lines

Software product lines (SPLs) allow creating a multitude of individual but similar products based on one common software model. Software components can be developed independently and new products can be generated easily. Inevitably, software evolves, a new version has to be deployed, and the data already existing in the database has to be transformed accordingly. As independently developed components are compiled into an individual SPL product, the local evolution script of every involved component has to be weaved into a single global database evolution script for the product. In this paper, we report on the database evolution toolkit DAVE in the context of an industry project. DAVE solves the weaving problem and provides a feasible solution for database evolution in SPLs.

Multi-schema-version data management: data independence in the twenty-first century

Agile software development allows us to continuously evolve and run a software system. However, this is not possible in databases, as established methods are very expensive, error-prone, and far from agile. We present InVerDa, a multi-schema-version database management system (MSVDB) for agile database development. MSVDBs realize co-existing schema versions within one database, where each schema version behaves like a regular single-schema database and write operations are propagated between schema versions. Developers use a relationally complete and bidirectional database evolution language (BiDEL) to easily evolve existing schema versions to new ones. BiDEL scripts are more robust, orders of magnitude shorter, and cause only a small performance overhead compared to handwritten SQL scripts. We formally guarantee data independence: no matter how the data of the co-existing schema versions is physically materialized, each schema version is guaranteed to behave like a regular database. Since, the chosen physical materialization significantly determines the overall performance, we equip database administrators with an advisor that proposes an optimized materialization for the current workload, which can improve the performance by orders of magnitude compared to naïve solutions. To our best knowledge, we are the first to facilitate agile evolution of production databases with full support of co-existing schema versions and formally guaranteed data independence.

Seamless Database Evolution for Cloud Applications.

We present DECA (Database Evolution in Cloud Applications), a framework that facilitates fast, robust, and agile database evolution in cloud applications. Successful business requires to react to changing wishes and requirements of customers instantly. In today’s realities, this often boils down to adding new features or fixing bugs in a software system. We focus on cloud application platforms that allow seamless development and deployment of applications by operating both the old and the new version in parallel for the time of development/deployment. This does not work if a common database is involved, since they cannot co-exist in multiple versions. To overcome this limitation, we apply current advances in the field of agile database evolution. DECA equips developers with an intuitive Database Evolution Language to create a new co-existing schema version for development and testing. Meanwhile, users can continuously use the old version. With the click of a button, we migrate the database to the new version and move all the users without unpredictable downtime and without the risk of corrupting our data. So, DECA speeds up the evolution of information systems to the pace of modern business.

Listen to the customer: model-driven database design

In modern IT landscapes, databases are subject to a major role change. Especially in Service-Oriented Architectures, databases are more and more frequently dedicated to a single application. Therefore, it is even more important to reflect the application requirements in their design. Software developers and application experts formulate application requirements in software models. Hence, we obviously need to bridge the gap to the software world and directly derive a database design from the software models used in application development and maintenance. We introduce this concept as model-driven database design. In this paper, we present the architecture principles of a model-driven database design tool and details on the enumeration and evaluation of logical database designs.