Daniel L. Moody

Improving the quality of data models: empirical validation of a quality management framework

This paper describes the results of a 5-year research programme into evaluating and improving the quality of data models. The theoretical base for this work was a data model quality management framework proposed by Moody and Shanks (In: P. Loucopolous (Ed.), Proceedings of the 13th International Conference on the Entity Relationship Approach, Manchester, England, December 14-17, 1994). A combination of field and laboratory research methods (action research, laboratory experiments and systems development) was used to empirically validate the framework. This paper describes how the framework was used to: (a) quality assure a data model in a large application development project (product quality); (b) reengineer application development processes to build quality into the data analysis process (process quality); (c) investigate differences between data models produced by experts and novices; (d) provide automated support for the evaluation process (the Data Model Quality Advisor). The results of the research have been used to refine and extend the framework, to the point that it is now a stable and mature approach.

Metrics for Evaluating the Quality of Entity Relationship Models

This paper defines a comprehensive set of metrics for evaluating the quality of Entity Relationship models. This is an extension of previous research which developed a conceptual framework and identified stakeholders and quality factors for evaluating data models. However quality factors are not enough to ensure quality in practice, because different people will have different interpretations of the same concept. The objective of this paper is to refine these quality factors into quantitative measures to reduce subjectivity and bias in the evaluation process. A total of twenty five candidate metrics are proposed in this paper, each of which measures one of the quality factors previously defined. The metrics may be used to evaluate the quality of data models, choose between alternatives and identify areas for improvement.

What Makes a Good Data Model? Evaluating the Quality of Entity Relationship Models

This paper develops a framework for evaluating the quality of data models and choosing between alternative representations of requirements. For any particular set of user requirements there are many possible models, each of which has drastically different implications for database and systems design. In the absence of formally defined and agreed criteria, the choice of an appropriate representation is usually made in an ad hoc way, based on personal opinion. The evaluation framework proposed consists of four major constructs: qualities (desirable properties of a data model), metrics (ways of measuring each quality), weightings (relative importance of each quality) and strategies (ways of improving data models). Using this framework, any two data models may be compared in an objective and comprehensive manner. The evaluation framework also builds commitment to the model by involving all stakeholders in the process: end users, management, the data administrator and application developers.

Evaluating the Quality of Process Models: Empirical Testing of a Quality Framework

This paper conducts an empirical analysis of a conceptual model quality framework for evaluating the quality of process models. 194 participants were trained in the concepts of the quality framework, and then used it to evaluate models represented in a workflow modelling language. A randomised, double-blind design was used, and the results evaluated using a combination of quantitative and qualitative techniques. An analysis was also conducted of the frameworks likelihood of adoption in practice, which is an issue rarely addressed in IS design research. The study provides strong support for the validity of the framework and suggests that it is likely to be adopted in practice, but raises questions about its reliability. The research findings provide clear direction for further research to improve the framework.

Cognitive Load Effects on End User Understanding of Conceptual Models: An Experimental Analysis

According to Cognitive Load Theory (CLT), presenting information in a way that cognitive load falls within the limitations of working memory can improve speed and accuracy of understanding, and facilitate deep understanding of information content. This paper describes a laboratory experiment which investigates the effects of reducing cognitive load on end user understanding of conceptual models. Participants were all naive users, and were given a data model consisting of almost a hundred entities, which corresponds to the average-sized data model encountered in practice. One group was given the model in standard Entity Relationship (ER) form and the other was given the same model organised into cognitively manageable “chunks”. The reduced cognitive load representation was found to improve comprehension and verification accuracy by more than 50%, though conflicting results were found for time taken. The practical significance of this research is that it shows that managing cognitive load can improve end user understanding of conceptual models, which will help reduce requirements errors. The theoretical significance is that it provides a theoretical insight into the effects of complexity on understanding of conceptual models, which have previously been unexplored. The research findings have important design implications for all conceptual modelling notations.

Improving the Quality of Entity Relationship Models - Experience in Research and Practice

This paper is an extension of previous research which developed a framework for evaluating and improving the quality of Entity Relationship models. The framework has now been used extensively in research and practice, including application in two of the largest commercial organisations in Australia. The experiences gained have been used to further develop and refine the framework. This paper describes how the framework has been used to: (a) quality assure data models as part of application development projects (product quality); (b) reengineer application development procedures to build quality into the data modelling process (process quality); (c) provide automated support for the evaluation process (Data Model Quality Advisor); (d) investigate the differences between data models produced by expert and novice data modellers. The results show that use of the framework has the potential to significantly improve research, practice and teaching of data modelling.

Graphical Entity Relationship Models: Towards a More User Understandable Representation of Data

The Entity Relationship Model was originally proposed as a way of representing user requirements in a way that non-technical users could understand. However anecdotal evidence and empirical studies both indicate that users have major difficulties understanding Entity Relationship models in practice. This paper proposes a number of modifications to the Entity Relationship Model to make it more understandable to business users. These include the use of an enhanced graphical representation, levels of abstraction and the use of business scenarios. This method has been used successfully in a wide range of organisational contexts, and has been particularly successful at the corporate level, where understandability of models has been found to be a major barrier to their acceptance and use. In addition, an automated tool has been developed to support the technique, which allows users to interact directly with the model and understand how it works through the use of animation.

Theoretical and Practical Issues in Evaluating the Quality of Conceptual Models

Recently, an international standard has been proposed for evaluating software quality (ISO/IEC 9126). However there is no equivalent stan-dard for evaluating the quality of conceptual models. There are a number of reasons for this. Firstly, during analysis, the notion of software development as an art rather than an engineering discipline is strongest, and quality is therefore most difficult to assess. Secondly, it is easier to evaluate the quality of a finished product than a logical specification. Finally, the conceptual modelling field is less mature and there has been less time for a consensus to emerge. However an international standard for conceptual model quality should be a long term objective for both researchers and practitioners alike.

A Multi-Level Architecture for Representing Enterprise Data Models

One of the most serious limitations of the Entity Relationship Model in practice is its inability to cope with complexity. With large numbers of entities, data models become difficult to understand and maintain. The problem becomes unmanageable at the enterprise level, where models typically consist of hundreds of entities. A number of approaches have been proposed in the literature to address this problem, but none have achieved widespread acceptance in practice. This paper proposes a simple and natural extension to the Entity Relationship Model which allows enterprise data models to be represented at multiple levels of abstraction, from a one page overview down to primitive entities and relationships. The model may be organised into any number of levels, depending on its complexity. The technique is based on the organisation of a city street directory, which is a practical solution to the problem of representing a large and complex model in everyday life.

Comparative Evaluation of Large Data Model Representation Methods: The Analyst's Perspective

One of the most serious limitations of the Entity Relationship (ER) Model in practice is its inability to cope with complexity. A number of approaches have been proposed in the literature to address this problem, but so far there has been no systematic empirical research into the effectiveness of these methods. This paper describes a laboratory experiment which compares the effectiveness of different representation methods for documentation and maintenance of large data models (analysts viewpoint). The methods are compared using a range of performance-based and perception-based variables, including time taken, documentation correctness, consistency, perceived ease of use, perceived usefulness and intention to use. An important theoretical contribution of this paper is the development and empirical testing of a theoretical model (the Method Evaluation Model) for evaluating IS design methods. This model may help to bridge the gap between research and practice in IS design research, as it addresses the issue of method adoption in practice, which has largely been ignored by IS design researchers.