Jose-Norberto Mazón

An MDA approach for the development of data warehouses

Different modeling approaches have been proposed to overcome every design pitfall of different data warehouse (DW) components. However, most of them offer partial solutions that deal only with isolated aspects of the DW and do not provide developers with an integrated and standard framework for designing all DW relevant components, such as ETL processes, data sources, DW repository and so on. To overcome this problem, this paper describes how to align the whole DW development process with a Model Driven Architecture (MDA) framework. We then focus on describing one part of it: an MDA approach for the development of the DW repository, because it is the cornerstone of any DW system. Therefore, we describe how to build the different MDA models for the DW repository by using an extension of the Unified Modeling Language (UML) and the Common Warehouse Metamodel (CWM). Transformations between models are also clearly and formally established by using the Query/View/Transformation (QVT) language. Finally, a case study is provided to exemplify the benefits of our MDA framework.

A model-driven goal-oriented requirement engineering approach for data warehouses

The development of a data warehouse has been traditionally guided by an in-depth analysis of the underlying operational data sources, thus overlooking an explicit development phase in which information requirements of decision makers are addressed. This scenario has prompted that the deployed data warehouse often fails in delivering the expected support of the decision making process. To overcome this problem, we propose to use the i* modeling framework and the model driven architecture (MDA) in order to describe (i) how to model goals and information requirements for data warehouses, and (ii) how to derive a conceptual multidimensional model that provides the required information to support the decision making process.

A survey on summarizability issues in multidimensional modeling

The development of a data warehouse (DW) system is based on a conceptual multidimensional model, which provides a high level of abstraction in accurately and expressively describing real-world situations. Once this model is designed, the corresponding logical representation must be obtained as the basis of the implementation of the DW according to one specific technology. However, even though a good conceptual multidimensional model is designed underneath a DW, there is a semantic gap between this model and its logical representation. In particular, this gap complicates an adequate treatment of summarizability issues, which in turn may lead to erroneous results of data analysis tools. Research addressing this topic has produced only partial solutions, and individual terminology used by different parties hinders further progress. Consequently, based on a unifying vocabulary, this survey sheds light on (i) the weak and strong points of current approaches for modeling complex multidimensional structures that reflect real-world situations in a conceptual multidimensional model and (ii) existing mechanisms to avoid summarizability problems when conceptual multidimensional models are being implemented.

Reconciling requirement-driven data warehouses with data sources via multidimensional normal forms

Successful data warehouse (DW) design needs to be based upon a requirement analysis phase in order to adequately represent the information needs of DW users. Moreover, since the DW integrates the information provided by data sources, it is also crucial to take these sources into account throughout the development process to obtain a consistent reconciliation of data sources and information needs. In this paper, we start by summarizing our approach to specify user requirements for data warehouses and to obtain a conceptual multidimensional model capturing these requirements. Then, we make use of the multidimensional normal forms to define a set of Query/View/Transformation (QVT) relations to assure that the conceptual multidimensional model obtained from user requirements agrees with the available data sources that will populate the DW. Thus, we propose a hybrid approach to develop DWs, i.e., we firstly obtain the conceptual multidimensional model of the DW from user requirements and then we verify and enforce its correctness against data sources by using a set of QVT relations based on multidimensional normal forms. Finally, we provide some snapshots of the CASE tool we have used to implement our QVT relations.

Applying MDA to the development of data warehouses

Different modeling approaches have been proposed to overcome every design pitfall of the development of the different parts of a data warehouse (DW) system. However, they are all partial solutions which deal with isolated aspects of the DW and do not provide designers with an integrated and standard method for designing the whole DW (ETL processes, data sources, DW repository and so on). On the other hand, the Model Driven Architecture (MDA) is a standard framework for software development that addresses the complete life cycle of designing, deploying, integrating, and managing applications by using models in software development. In this paper, we describe how to align the whole DW development process to MDA. Then, we define MD2A (MultiDimensional Model Driven Architecture), an approach for applying the MDA framework to one of the stages of the DW development: multidimensional (MD) modeling. First, we describe how to build the different MDA artifacts (i.e. models) by using extensions of the Unified Modeling Language (UML). Secondly, transformations between models are clearly and formally established by using the Query/View/Transformation (QVT) approach. Finally, an example is provided to better show how to apply MDA and its transformations to the MD modeling.

A model-driven framework for ETL process development

ETL processes are the backbone component of a data warehouse, since they supply the data warehouse with the necessary integrated and reconciled data from heterogeneous and distributed data sources. However, the ETL process development, and particularly its design phase, is still perceived as a time-consuming task. This is mainly due to the fact that ETL processes are typically designed by considering a specific technology from the very beginning of the development process. Thus, it is difficult to share and reuse methodologies and best practices among projects implemented with different technologies. To the best of our knowledge, no attempt has been yet dedicated to harmonize the ETL process development by proposing a common and integrated development strategy. To overcome this drawback, in this paper, a framework for model-driven development of ETL processes is introduced. The benefit of our framework is twofold: (i) using vendor-independent models for a unified design of ETL processes, based on the expressive and well-known standard for modeling business processes, the Business Process Modeling Notation (BPMN), and (ii) automatically transforming these models into the required vendor-specific code to execute the ETL process into a concrete platform.

BPMN-based conceptual modeling of ETL processes

Business Intelligence (BI) solutions require the design and implementation of complex processes (denoted ETL) that extract, transform, and load data from the sources to a common repository. New applications, like for example, real-time data warehousing, require agile and flexible tools that allow BI users to take timely decisions based on extremely up-to-date data. This calls for new ETL tools able to adapt to constant changes and quickly produce and modify executable code. A way to achieve this is to make ETL processes become aware of the business processes in the organization, in order to easily identify which data are required, and when and how to load them in the data warehouse. Therefore, we propose to model ETL processes using the standard representation mechanism denoted BPMN (Business Process Modeling and Notation). In this paper we present a BPMN-based metamodel for conceptual modeling of ETL processes. This metamodel is based on a classification of ETL objects resulting from a study of the most used commercial and open source ETL tools.

Automatic generation of ETL processes from conceptual models

Data warehouses (DW) integrate different data sources in order to give a multidimensional view of them to the decision-maker. To this aim, the ETL (Extraction, Transformation and Load) processes are responsible for extracting data from heterogeneous operational data sources, their transformation (conversion, cleaning, standardization, etc.), and its load in the DW. In recent years, several conceptual modeling approaches have been proposed for designing ETL processes. Although these approaches are very useful for documenting ETL processes and supporting the designer tasks, these proposals fail to give mechanisms to carry out an automatic code generation stage. Such a stage should be required to both avoid fails and save development time in the implementation of complex ETL process. Therefore, in this paper we define an approach for the automatic code generation of ETL processes. To this aim, we align the modeling of ETL processes in DW with MDA (Model Driven Architecture) by formally defining a set of QVT (Query, View, Transformation) transformations.

A hybrid model driven development framework for the multidimensional modeling of data warehouses

Developing a multidimensional (MD) model of a data warehouse (DW) is a highly complex, prone to fail, and time consuming task, due to the fact that (i) the information needs of decision makers and the available operational data sources that will populate the DW must both be considered in a conceptual MD model, and (ii) complex mappings must be performed to obtain an implementation of this conceptual MD model. However, no significant effort has been made to take these issues into account in a systematic, well structured and comprehensive development process. To overcome the lack of such a process, a framework based on the Model Driven Architecture (MDA) is proposed for the development of a hybrid MD model at the conceptual level and for the automatic derivation of its logical representation. Also, a running example is shown throughout this paper.

Extending OCL for OLAP querying on conceptual multidimensional models of data warehouses

The development of data warehouses begins with the definition of multidimensional models at the conceptual level in order to structure data, which will facilitate decision makers with an easier data analysis. Current proposals for conceptual multidimensional modelling focus on the design of static data warehouse structures, but few approaches model the queries which the data warehouse should support by means of OLAP (on-line analytical processing) tools. OLAP queries are, therefore, only defined once the rest of the data warehouse has been implemented, which prevents designers from verifying from the very beginning of the development whether the decision maker will be able to obtain the required information from the data warehouse. This article presents a solution to this drawback consisting of an extension to the object constraint language (OCL), which has been developed to include a set of predefined OLAP operators. These operators can be used to define platform-independent OLAP queries as a part of the specification of the data warehouse conceptual multidimensional model. Furthermore, OLAP tools require the implementation of queries to assure performance optimisations based on pre-aggregation. It is interesting to note that the OLAP queries defined by our approach can be automatically implemented in the rest of the data warehouse, in a coherent and integrated manner. This implementation is supported by a code-generation architecture aligned with model-driven technologies, in particular the MDA (model-driven architecture) proposal. Finally, our proposal has been validated by means of a set of sample data sets from a well-known case study.