Thomas Ruf

Improving query response time in scientific databases using data aggregation -a case study

Although most state-of-the-art database systems have no inherent limitations w.r.t. the amount of data they can handle, the huge data quantities typically found in scientific database applications often exceed the feasibility level from a practical point of view when query performance is the issue. One theoretically well-known concept of improving query response time in scientific database applications is using the categorization and classification facilities often found in scientific computing domains for storing data aggregations that allow to substitute expensive access to raw data by the use of stored aggregated values. The results of an empirical performance study carried out in the application domain of market research are presented which substantiate the practical importance of such work. Using real market research data, it is shown that query response time can be shortened in an order of magnitude if a proper data aggregation concept is used. If the data aggregates are designed properly, the overhead of generating and managing materializations of data aggregates is by far outweighed by the improved query performance in realistic scenarios.

CROSS-DB: a feature-extended multidimensional data model for statistical and scientific databases

Statistical and scientifi computing applications exhibit characteristics that are finahmentally different fmm classical database system application domains. The CROSS-DB data model presented in this paper is optimized for use in such applications by providing advanced data mo&lling methods and application-oriented query fmilities, thus providing afiamework for optimized alzta nuuuzgement procedures. CROSS-DB (which stands for Classifiationoriented Redundancy-based Optimization of Statistical and Scientific DataBases) is based on a multidimenswnal data view. The model differs j?om other approaches by o~ering two complementary rnechanisrnsfor structuring qualifying information, classification and feature description. Using these ntechanisms results in a normalized, low-dimensional database schema which ensures both modelling uniqueness and understo.mhbility while providing enhanced modelling

Cooperative problem solving using database conversations

exibility.

Implementation of a distributed data management system for manufacturing applications-a feasibility study

Cooperative problem solving is a joint style of producing and consuming data. Unfortunately, most database mechanisms developed so far; are more appropriate for competitive usage than for a cooperative working style. They mostly adopt an operational point of view which binds data to applications. Data-oriented mechanisms like check-in/out avoid this binding but do not improve synchronization towards concurrent usage of data. Conversations are an application-independent, tight framework for jointly modifying common data. The idea is to create transaction-spanning conversational working stages that organize different contributions instead of serializing accesses. To illustrate the conversation concept, an extended query language with conversational operations is presented.<<ETX>>

Dealing with Complex Reports in OLAP Applications

A prototype implementation of a data management system primarily designed for technical applications is discussed. The system may be regarded as a superset of well-known data administration concepts like file systems and database systems, both centralized and distributed. The basic idea is to enrich the concept of consistency supported in a classical database system by a more flexible data distribution mechanism. The prototype makes it possible to validate the basic concepts it is then extended to a more powerful version.<<ETX>>

A Redundancy-Based Optimization Approach for Aggregation in Multidimensional Scientific and Atatistical Databases

Slice&dice and drilling operations are key concepts for ad-hoc data analysis in state-of-the-art data warehouse and OLAP (On-Line Analytical Processing) systems. While most data analysis operations can be executed on that basis from a functional point of view, the representation requirements of applications in the SSDB (Scientific&Statistical DataBase) area by far exceed the means typically provided by OLAP systems. In the first part of the paper, we contrast the data analysis and representation approaches in the OLAP and SSDB field and develop a generalized model for the representation of complex reports in data warehouse environments. The second part of the paper describes the implementation of this model from a report definition, management and execution perspective. The research and implementation work was executed in the data warehouse project at GfK Marketing Services, a top-ranked international market research company. Various examples from the market research application domain will demonstrate the benefits of the work over other approaches in the data warehouse and OLAP domain.

Concepts and methods for the optimization of distributed data processing

Large data volumes, flexible drill-down analysis and short query response times, which are predominant characteristics of Scientific and Statistical Data Base (SSDB) applications, require new optimization techniques as compared to traditional DBMS. This paper describes formally an optimization approach in the SSDB domain which is based on the re-use of materialized results offormer queries to process aggregate queries along a classi

Data refinement in a market research applications' data production process

cation hierarchy. The description of the approach is embedded in the CROSS-DB model (which stands for Classification-oriented, Redundancybased Optimization of Scientzj% and Statistical DataBases). It will be shown that the approach taken can improve query response time by orders of magnitude while adding tolerable storage and maintenance overhead to the database.

Flexible and reactive integration of manufacturing systems through dynamic resource allocation

In this paper we introduce and discuss a model of distributed data processing. For this purpose, a typical application system is analyzed and divided into sub-applications. To fulfill the task of the global application, the sub-applications have to communicate in an appropriate manner by exchanging data resp. information. In our model the communication between sub-applications is split up into two steps: the offering of information by sending sub-applications, and its acceptance by receiving sub-applications. For both communication steps synchronous and asynchronous processing modes are defined. Supporting those different communication modes the cooperation between sub-applications can be defined very closely to the specific demands of the application system. This optimizes distributed data processing. At last we demonstrate the prototype implementation of a distributed data management system, which is based on the flexible communication mechanism described in the paper.

Implementation of a distributed data management system for technical applications: a feasibility study

In this contribution, we will show how empirically collected field data for a market research application are refined in a stepwise manner and enriched into end-user market reports and charts. The collected data are treated by selections, transformations, enrichments, and aggregations to finally derive new market knowledge from the raw data material. Besides data-oriented aspects, process- and organization-related aspects have to be considered as well to ensure the required product quality for GfK Marketing Services’ customers, which have known GfK for decades as a top-10 player in the international market research area. Based on an ongoing example from the panel-based Retail & Technology application domain, we will show how de-centrally collected and pre-processed data are transformed into integrated, global market knowledge in a network of world-wide companies.