Amy J. Lee

The EVE approach: view synchronization in dynamic distributed environments

The construction and maintenance of data warehouses (views) in large-scale environments composed of numerous distributed and evolving information sources (ISs) such as the WWW has received great attention recently. Such environments are plagued with changing information because ISs tend to continuously evolve by modifying not only their content but also their query capabilities and interface and by joining or leaving the environment at any time. We are the first to introduce and address the problem of schema changes of ISs, while previous work in this area, such as incremental view maintenance, has mainly dealt with data changes at ISs. We outline our solution approach to this challenging new problem of how to adapt views in such evolving environments. We identify a new view adaptation problem for view evolution in the context of ISs schema changes, which we call view synchronization. We also outline the Evolvable View Environment (EVE) approach that we propose as framework for solving the view synchronization problem, along with our decisions concerning the key design issues surrounding EVE. The main contributions of this paper are: 1) we provide an E-SQL view definition language with which the view definer can direct the view evolution process, 2) we introduce a model for information source description which allows a large class of ISs to participate in our system dynamically, 3) we formally define what constitutes a legal view rewriting, 4) we develop replacement strategies for affected view components which are designed to meet the preferences expressed by E-SQL, 5) we prove the correctness of the replacement strategies, and 6) we provide a set of view synchronization algorithms based on those strategies. A prototype of our EVE system has successfully been built using Java, JDBC, Oracle, and MS Access.

The CVS Algorithm for View Synchronization in Evolvable Large-Scale Information Systems

Current view technology supports only static views in the sense that views become undefined and hence obsolete as soon as the underlying information sources (ISs) undergo capability changes. We propose to address this new view evolution problem — which we call view synchronization — by a novel solution approach that allows affected view definitions to be dynamically evolved to keep them in synch with evolving ISs. We present in this paper a general strategy for the view synchronization process that guided by constraints imposed by the view evolution preferences embedded in the view definition achieves view preservation (i.e., view redefinition). We present the formal correctness, the CVS algorithm, as well as numerous examples to demonstrate the main concepts.

Consistent schema version removal: an optimization technique for object-oriented views

Powerful solutions enabling interoperability must allow applications to evolve and the requirements of shared databases to change, while minimizing such changes on other integrated applications. Several approaches have been proposed to make interoperability possible by using object-oriented techniques. These approaches may generate a large number of schema versions over time, resulting in an excessive build-up of classes and underlying object instances, not all being necessarily still in use. This results in degradation of system performance due to the view maintenance and the storage overhead costs. In this paper, we address the problem of removing obsolete view schemas. We characterize four potential problems of schema consistency that could be caused by the removal of a single derived class. We demonstrate that schema version removal is sensitive to the order in which individual classes are processed, and present a formal dependency model that captures all dependencies between classes as logic clauses and manipulates them to make decisions on class deletions and non-deletions while guaranteeing the consistency of the schema. We have also developed and proven consistent a dependency graph (DG) representation of the formal model. Lastly, we present a cost model for evaluating alternative removal patterns on a DG to assure selection of the optimal solution. The proposed techniques have been implemented in our Schema View Removal (SVR) tool. Lastly, we report experimental findings for applying our techniques for consistent schema version removal on the MultiView/TSE (Transparent Schema Evolution) system.

Capacity-Augmenting Schema Changes on Object-Oriented Databases: Towards Increased Interoperability

The realization of capacity-augmenting schema changes on a shared database while providing continued interoperability to active applications has been recognized as a hard open problem. A novel three-pronged process, called transparent object schema evolution (TOSE), is presented that successfully addresses this problem. TOSE uses the combination of views and versioning to simulate schema changes requested by one application without affecting other applications interoperating on a shared OODB. The approach is of high practical relevance as it builds upon schema evolution support offered by commercial OODBMSs.

An active OODB system for genome physical map assembly

Abstract In this paper, we describe the design and implementation of a scientific database for the map assembly tasks performed by the geneticists at the University of Michigan Human Genome Center. Our system manages complex genomic data and supports the automation of the associated map assembly tasks. For this purpose, we have developed an overlap refinement hierarchy characterizing the types of overlap and ordering relationships between DNA fragments. Based on this model, we designed an associated set of inference operators to automate some of the contig assembly steps, such as inferring overlap information using transitivity rules. In order to realize this map inferencing approach and to take advantage of the modeling power offered by object-oriented database technology, we have developed an active object-oriented database (OODB) system, called Crystal, on the GemStone OODB. Crystal seamlessly integrates rule inferencing with object modeling and other typical database capabilities, thus avoiding the overhead in moving data between systems for rule processing and data management, and eliminating data mismatch in these different representations. We also discuss the implementation of the physical map assembly (PMA) tool on top of Crystal, which includes the realization of the genome information model and the map assembly inferencing process. PMA executes inferencing automatically, without explicitly having to initiate rule execution, and thus can respond to the incremental addition of new experimental data. In conclusion, we provide a walk-through example and several experimental results that demonstrate how our approach can be used to support physical contig assembly.

Quality and Cost of Data Warehouse Views

Query rewriting has been used as a query optimization technique for several decades to reduce the computational cost of a query. Traditional problems in query rewriting include in particular query optimization au][28, 60, 6] and rewriting queries using views [40, 7]. Most of these works deal with the problem of maintaining the exact original interface (schema) and extent of a given query while optimizing performance.

Data warehouse evolution: consistent meta data management

Large information spaces such as the WWW face the problem of how to maintain data warehouses (views) defined over information sources (ISs) whenever capabilities of these ISs change. We have developed a novel solution approach to address this problem, called the evolvable view environment (EVE). In EVE, knowledge of both the capabilities of as well as (partial) containment relationships between ISs is collected in a meta knowledge base (MKB). We describe the meta knowledge management problem and focus on issues related to the MKB evolution process. The contributions of this paper are threefold: 1) formally define consistency criteria for the MKB evolution process; 2) use PC constraint evolution to demonstrate our MKB evolution process; and 3) discuss techniques of keeping the MKB as powerful as possible by deriving and preserving minimal implicit knowledge from the affected explicit meta knowledge, before retracting the latter.

Visualization and database tools for YAC and cosmid contig construction

Recently, construction of physical maps based on YAC and cosmid contigs has become increasingly common, since these maps serve as a fine-scale ordering basis for gene searches. We describe a combination of visualization and database tools that address this problem. The YAC-Cosmid contig tool is a map visualization and editing tool designed for large data sets. The Physical Map Assembler (PMA) uses a rule-based interval-ordering approach to infer ordering from experimental data. It is implemented as an active object-oriented database. The PMA visualizer displays interval orderings in a comprehensible fashion, and thus helps the scientist resolve conflicts and ambiguities identified by PMA. Both visualization tools let the user query the database to see the details behind the picture. We have found that a tight integration of visualization and analysis tools with the database enhances their usability, and speeds up the dual tasks of analyzing and understanding the experimental data. The use of our tools is illustrated by examples from a gene search at the University of Michigan Genome Center.<<ETX>>

Physical Map Assembler: an active OODB system for human genome applications

We describe the design and implementation of a scientific database for the map assembly tasks performed by the geneticists at the University of Michigan Human Genome Center. Our system manages complex genomic data and supports the automation of the associated map assembly tasks. For the former, we present a genomic object model that integrates both experimental and derived data. For the latter, we describe operators to automate some of the analysis steps. To develop a framework for implementing our rule-based approach to physical mapping, we have designed and implemented an active object-oriented database (OODB) system, called Crystal, on GemStone. Crystal seamlessly integrates inference capabilities with complex object modeling and other typical database capabilities as required for physical mapping. We also discuss the implementation of a physical map assembly tool on top of Crystal. In conclusion, we provide a walk-through example that demonstrates how our approach can be used to effectively support physical contig assembly.<<ETX>>