James F. Terwilliger

Bidirectional Transformations: A Cross-Discipline Perspective

The GRACE International Meeting on Bidirectional Transformations was held in December 2008 near Tokyo, Japan. The meeting brought together researchers and practitioners from a variety of sub-disciplines of computer science to share research efforts and help create a new community. In this report, we survey the state of the art and summarize the technical presentations delivered at the meeting. We also describe some insights gathered from our discussions and introduce a new effort to establish a benchmark for bidirectional transformations.

Trill: a high-performance incremental query processor for diverse analytics

This paper introduces Trill -- a new query processor for analytics. Trill fulfills a combination of three requirements for a query processor to serve the diverse big data analytics space: (1) Query Model: Trill is based on a tempo-relational model that enables it to handle streaming and relational queries with early results, across the latency spectrum from real-time to offline; (2) Fabric and Language Integration: Trill is architected as a high-level language library that supports rich data-types and user libraries, and integrates well with existing distribution fabrics and applications; and (3) Performance: Trills throughput is high across the latency spectrum. For streaming data, Trills throughput is 2-4 orders of magnitude higher than comparable streaming engines. For offline relational queries, Trills throughput is comparable to a major modern commercial columnar DBMS. Trill uses a streaming batched-columnar data representation with a new dynamic compilation-based system architecture that addresses all these requirements. In this paper, we describe Trills new design and architecture, and report experimental results that demonstrate Trills high performance across diverse analytics scenarios. We also describe how Trills ability to support diverse analytics has resulted in its adoption across many usage scenarios at Microsoft.

Recent Advances in Schema and Ontology Evolution

Schema evolution is the increasingly important ability to adapt deployed schemas to changing requirements. Effective support for schema evolution is challenging since schema changes may have to be propagated, correctly and efficiently, to instance data and dependent schemas, mappings, or applications. We introduce the major requirements for effective schema and ontology evolution, including support for a rich set of change operations, simplicity of change specification, evolution transparency (e.g., by providing and maintaining views or schema versions), automated generation of evolution mappings, and predictable instance migration that minimizes data loss and manual intervention. We then give an overview about the current state of the art and recent research results for the evolution of relational schemas, XML schemas, and ontologies. For numerous approaches, we outline how and to what degree they meet the introduced requirements.

Dagstuhl seminar on bidirectional transformations (BX)

The Dagstuhl BX seminar, held January 16–21, 2011, brought together researchers from 13 countries across disciplines that study bidirectional transformations. It was a follow-up of the GRACE International Meeting on Bidirectional Transformations held in December 2008 near Tokyo, Japan [5]. This consisted of short introductions from each of the participants on their background and work, followed by presentations and demonstrations on representative technologies from each field, and some open discussion time. A major benefit of the GRACE meeting was the opportunity for the disciplines to get some initial exposure to each other. The Dagstuhl seminar intended to go a step further and begin to identify commonalities between the disciplines and start to set a cross-disciplinary research agenda. The first part of the seminar consisted of tutorials from each of the four represented disciplines. The second part consisted of cross-disciplinary working groups dedicated to investigating specific examples of commonality between solutions or identifying requirements, terminology, or scenarios that may reach across fields. There were also sessions in which participants gave position statements on their own work. Participants at both the Dagstuhl and GRACE seminars came from four disciplines: (1) Programming Languages, (2) Graph Transformations, (3) Software Engineering, and (4) Databases. At Dagstuhl, each of the first three disciplines made up about 2/7 of the participants, while databases took the remaining 1/7 out of about 45 participants. Representation from the database field was, nevertheless, an improvement over the turnout from the GRACE meeting.

Querying through a user interface

In contrast to a traditional setting where users express queries against the database schema, we assert that the semantics of data can often be understood by viewing the data in the context of the user interface (UI) of the software tool used to enter the data. That is, we believe that users will understand the data in a database by seeing the labels, drop-down menus, tool tips, or other help text that are built into the user interface. Our goal is to allow domain experts with little technical skill to understand and query data. In this paper, we present our GUi As View (Guava) framework and describe how we use forms-based UIs to generate a conceptual model that represents the information in the user interface. We then describe how we generate a query interface from the conceptual model. We characterize the resulting query language using a subset of the relational algebra. Since most application developers want to craft a physical database to meet desired performance needs, we present here a transformation channel that can be configured by instantiating one or more of our transformation operators. The channel, once configured, automatically transforms queries from our query interface into queries that address the underlying physical database and delivers query results that conform to our query interface. In this paper, we define and formalize our database transformation operators. The contributions of this paper are that first, we demonstrate the feasibility of creating a query interface based directly on the user interface and second, we introduce a general purpose database transformation channel that will likely shorten the application development process and increase the quality of the software by automatically generating software artifacts that are often made manually and are prone to errors.

Worry-free database upgrades: automated model-driven evolution of schemas and complex mappings

Schema evolution is an unavoidable consequence of the application development lifecycle. The two primary schemas in an application, the client conceptual object model and the persistent database model, must co-evolve or risk quality, stability, and maintainability issues. We present MoDEF, an extension to Visual Studio that supports automatic evolution of object-relational mapping artifacts in the Microsoft Entity Framework. When starting with a valid mapping between client and store, MoDEF translates changes made to a client model into incremental changes to the store as an upgrade script, along with a new valid mapping to the new store. MoDEF mines the existing mapping for mapping patterns which MoDEF reuses for new client artifacts.

How clean is your sandbox

Bidirectional transformations (bx) constitute an emerging mechanism for maintaining the consistency of interdependent sources of information in software systems. Researchers from many different communities have recently investigated the use of bxto solve a large variety of problems, including relational view update, schema evolution, data exchange, database migration, and model co-evolution, just to name a few. Each community leveraged and extended different theoretical frameworks and tailored their use for specific sub-problems. Unfortunately, the question of how these approaches actually relate to and differ from each other remains unanswered. This question should be addressed to reduce replicated efforts among and even within communities, enabling more effective collaboration and fostering cross-fertilization. To effectively move forward, a systematization of these many theories and systems is now required. This paper constitutes a first, humble yet concrete step towards a unified theoretical framework for a tractable and relevant subset of bx approaches and tools. It identifies, characterizes, and compares tools that allow the incremental definition of bidirectional mappings between software artifacts. Identifying similarities between such tools yields the possibility of developing practical tools with wide-ranging applicability; identifying differences allows for potential new research directions, applying the strengths of one tool to another whose strengths lie elsewhere.

Language-integrated querying of XML data in SQL server

Developers need to access persistent XML data programmatically. Object-oriented access is often the preferred method. Translating XML data into objects or vice-versa is a hard problem due to the data model mismatch and the difficulty of query translation. Our prototype addresses this problem by transforming object-based queries and updates into queries and updates on XML using declarative mappings between classes and XML schema types. Our prototype extends the ADO.NET Entity Framework and leverages its object-relational mapping capabilities. We demonstrate how a developer can interact with stored relational and XML data using the Language Integrated Query (LINQ) feature of .NET. We show how LINQ queries are translated into a combination of SQL and XQuery. Finally, we illustrate how explicit mappings facilitate data independence upon database refactoring.

Mapping XML to a Wide Sparse Table

XML is commonly supported by SQL database systems. However, existing mappings of XML to tables can only deliver satisfactory query performance for limited use cases. In this paper, we propose a novel mapping of XML data into one wide table whose columns are sparsely populated. This mapping provides good performance for document types and queries that are observed in enterprise applications but are not supported efficiently by existing work. XML queries are evaluated by translating them into SQL queries over the wide sparsely-populated table. We show how to translate full XPath 1.0 into SQL. Based on the characteristics of the new mapping, we present rewriting optimizations that dramatically reduce the number of joins. Experiments demonstrate that query evaluation over the new mapping delivers considerable improvements over existing techniques for the target use cases.

Automated co-evolution of conceptual models, physical databases, and mappings

Schema evolution is an unavoidable consequence of the application development lifecycle. The two primary schemas in an application, the conceptual model and the persistent database model, must co-evolve or risk quality, stability, and maintainability issues. We study application-driven scenarios, where the conceptual model changes and the database and mapping must evolve in kind. We present a technique that, in most cases, allows those evolutions to progress automatically. We treat the mapping as data, and mine that data for patterns. Then, given an incremental change to the conceptual model, we can derive the proper store and mapping changes without user intervention. We characterize the significant subset of mappings for which automatic evolution is possible, and present our techniques for evolution propagation.