Darja Solodovnikova

Metadata to Support Data Warehouse Evolution

The focus of this chapter is metadata necessary to support data warehouse evolution. We present the data warehouse framework that is able to track evolution process and adapt data warehouse schemata and data extraction, transformation, and loading (ETL) processes. We discuss the significant part of the framework, the metadata repository that stores information about the data warehouse, logical and physical schemata and their versions. We propose the physical implementation of multiversion data warehouse in a relational DBMS. For each modification of a data warehouse schema, we outline the changes that need to be made to the repository metadata and in the database.

Evolution-Oriented User-Centric Data Warehouse

Data warehouses tend to evolve, because of changes in data sources and business requirements of users. All these kinds of changes must be properly handled, therefore, data warehouse development is never-ending process. In this paper we propose the evolution-oriented user-centric data warehouse design, which on the one hand allows to manage data warehouse evolution automatically or semi-automatically, and on the other hand it provides users with the understandable, easy and transparent data analysis possibilities. The proposed approach supports versions of data warehouse schemata and data semantics.

On Implicitly Discovered OLAP Schema-Specific Preferences in Reporting Tool

On Implicitly Discovered OLAP Schema-Specific Preferences in Reporting Tool We propose content-based methods for construction of recommendations for reports in the OLAP reporting tool. Recommendations are generated based on preference information in user profile, which is updated implicitly by collecting and analyzing user activity in the reporting tool. Taking advantage of data about user preferences for data warehouse schema elements, existing reports that potentially may be interesting to the user are distinguished and recommended. The approach used for recommending reports is composed of two methods - cold-start and hot-start.

Handling Evolving Data Warehouse Requirements

A data warehouse is a dynamic environment and its business requirements tend to evolve over time, therefore, it is necessary not only to handle changes in data warehouse data, but also to adjust a data warehouse schema in accordance with changes in requirements. In this paper, we propose an approach to propagate modified data warehouse requirements in data warehouse schemata. The approach supports versions of data warehouse schemata and employs the requirements formalization metamodel and multiversion data warehouse metamodel to identify necessary changes in a data warehouse.

Query-Driven Method for Improvement of Data Warehouse Conceptual Model

We propose a query-driven method that elicits the information requirements from existing queries on data sources and their usage statistics. Our method presumes that the queries against the source database reflect the analysis needs of users. We use this method to recommend changes to the existing data warehouse schemata. In our method, we take advantage of the schema versioning approach to reflect all changes that occur in the analysed process, and we analyse the activity of users in the source system, rather than changes in physical data structure, to infer the necessary improvements to the data warehouse schema.

Towards Introducing User Preferences in OLAP Reporting Tool

This paper presents an OLAP reporting tool and an approach for determining and processing user OLAP preferences, which are useful for generating recommendations on potentially interesting reports. We discuss the metadata layers of the reporting tool including our proposed OLAP preferences metamodel, which supports various scenarios of formulating preferences of two different types: schema-specific and report-specific. The process of semantic metadata usage at the stage of formulating user preferences is also considered. The methods for processing schema-specific and report-specific OLAP preferences are outlined.

Towards a Data Warehouse Architecture for Managing Big Data Evolution.

The problem of designing data warehouses in accordance with user requirements and adapting its data and schemata to changes in these requirements as well as data sources has been studied by many researchers worldwide in the context of relational database environments. However, due to the emergence of big data technologies and the necessity to perform OLAP analysis over big data, innovative methods must be developed also to support evolution of data warehouse that is used to analyse big data. Therefore, the main objective of this paper is to propose a data warehousing architecture over big data capable of automatically or semi-automatically adapting to user needs and requirements as well as to changes in the underlying data

Architecture Enabling Adaptation of Data Integration Processes for a Research Information System

Abstract Today, many efforts have been made to implement information systems for supporting research evaluation activities. To produce a good framework for research evaluation, the selection of appropriate measures is important. Quality aspects of the systems’ implementation should also not be overlooked. Incomplete or faulty data should not be used and metric computation formulas should be discussed and valid. Correctly integrated data from different information sources provide a complete picture of the scientific activity of an institution. Knowledge from the data integration field can be adapted in research information management. In this paper, we propose a research information system for bibliometric indicator analysis that is incorporated into the adaptive integration architecture based on ideas from the data warehousing framework for change support. A data model of the integrated dataset is also presented. This paper also provides a change management solution as a part of the data integration framework to keep the data integration process up to date. This framework is applied for the implementation of a publication data integration system for excellence-based research analysis at the University of Latvia.

Publication Data Integration as a Tool for Excellence-Based Research Analysis at the University of Latvia

The evaluation of research results can be carried out with different purposes aligned with strategic goals of an institution, for example, to decide upon distribution of research funding or to recruit or promote employees of an institution involved in research. Whereas quantitative measures such as number of scientific papers or number of scientific staff are commonly used for such evaluation, the strategy of the institution can be set to achieve ambitious scientific goals. Therefore, a question arises as to how more quality oriented aspects of the research outcomes should be measured. To supply an appropriate dataset for evaluation of both types of metrics, a suitable framework should be provided, that ensures that neither incomplete, nor faulty data are used, that metric computation formulas are valid and the computed metrics are interpreted correctly. To provide such a framework with the best possible features, data from various available sources should be integrated to achieve an overall view on the scientific activity of an institution along with solving data quality issues. The paper presents a publication data integration system for excellence-based research analysis at the University of Latvia. The system integrates data available at the existing information systems at the university with data obtained from external sources. The paper discusses data integration flows and data integration problems including data quality issues. A data model of the integrated dataset is also presented. Based on this data model and integrated data, examples of quality oriented metrics and analysis results of them are provided.

Can SQ and EQ Values and Their Difference Indicate Programming Aptitude to Reduce Dropout Rate

A crucial problem that we are currently facing at the Faculty of Computing of the University of Latvia is that during the first study semester on average 30% of the first-year students drop out, whereas after the first year of studies the number of dropouts increases up to nearly 50%. Thus, our overall goal is to determine in advance applicants that most likely will not finish the first study year successfully. A hypothesis formulated in another research study was that programming aptitude could be predicted based on the results of two personality self-report questionnaires − Systemizing Quotient (SQ) and Empathy Quotient (EQ) − taken by students. The difference between the SQ and EQ scores had a strong correlation with grades received for programming test. We reproduced the circumstances of mentioned empirical study with our first-year students using similar tests to calculate SQ and EQ, and semester grades in introductory programming course as a quantitative measure to evaluate programming ability. In this paper, we elaborate on the empirical setting, measures, and estimation methods of our study, which produced the results that made us call the stated hypothesis into question and disprove it.