Geert Poels

Defining and validating metrics for assessing the understandability of entity-relationship diagrams

Database and data model evolution cause significant problems in the highly dynamic business environment that we experience these days. To support the rapidly changing data requirements of agile companies, conceptual data models, which constitute the foundation of database design, should be sufficiently flexible to be able to incorporate changes easily and smoothly. In order to understand what factors drive the maintainability of conceptual data models and to improve conceptual modelling processes, we need to be able to assess conceptual data model properties and qualities in an objective and cost-efficient manner. The scarcity of early available and thoroughly validated maintainability measurement instruments motivated us to define a set of metrics for Entity-Relationship (ER) diagrams. In this paper we show that these easily calculated and objective metrics, measuring structural properties of ER diagrams, can be used as indicators of the understandability of the diagrams. Understandability is a key factor in determining maintainability as model modifications must be preceded by a thorough understanding of the model. The validation of the metrics as early understandability indicators opens up the way for an in-depth study of how structural properties determine conceptual data model understandability. It also allows building maintenance-related prediction models that can be used in conceptual data modelling practice.

A conceptual modeling quality framework

The goal of any modeling activity is a complete and accurate understanding of the real-world domain, within the bounds of the problem at hand and keeping in mind the goals of the stakeholders involved. High-quality representations are critical to that understanding. This paper proposes a comprehensive Conceptual Modeling Quality Framework, bringing together two well-known quality frameworks: the framework of Lindland, Sindre, and Sølvberg (LSS) and that of Wand and Weber based on Bunge’s ontology (BWW). This framework builds upon the strengths of the LSS and BWW frameworks, bringing together and organizing the various quality cornerstones and then defining the many quality dimensions that connect one to another. It presents a unified view of conceptual modeling quality that can benefit both researchers and practitioners.

Evaluating quality of conceptual modelling scripts based on user perceptions

This paper presents the development of a user evaluations based quality model for conceptual modelling scripts applying Seddons variant of the well-known model of DeLone and McLean [W.H. DeLone, E.R. McLean, Information systems success: the quest for the dependent variable, Information Systems Journal 3(1) (1992) 60-95] for evaluating the success of information systems. Given the growing awareness among researchers and practitioners about the importance of high-quality conceptual modelling scripts, it is surprising that there is no practical evaluation framework that considers the quality of conceptual modelling scripts from the user perspective. A first research goal is therefore to determine what the appropriate dimensions are for evaluating the success or quality of conceptual modelling scripts from the user point of view. A second goal is to investigate the relationships between these quality dimensions. The paper also presents an empirical test of the proposed model of quality dimensions and their relationships. Results are presented of two experiments with 187 and 124 business students respectively, that were designed to test a set of hypotheses generated from the proposed model. The results largely support the proposed model and have implications for both theory and practice of quality evaluation of conceptual modelling scripts.

Defining business process maturity. A journey towards excellence

Organisations are increasingly focusing on their business processes (BPs) to excel. Since the journey towards process excellence is challenging, various authors have proposed a business process maturity model (BPMM) to gradually improve maturity and capability. Nonetheless, a comprehensive definition for BPMM, maturity and capability is missing. Moreover, the differences between maturity and capability are frequently neglected. Therefore, this study elaborates on the lacking foundation. In particular, a thorough literature study is conducted to address three gaps regarding the BPMM scope, terminology and design. It reveals six basic BPMM components within a BP lifecycle (scope gap): (1) modelling, (2) deployment, (3) optimisation, (4) management, (5) culture and (6) structure. Their overall growth refers to maturity, whereas the growth of an individual component includes capability (terminology gap). To increase its usability, a BPMM gives guidance on both assessing and improving BPs (design gap). The findings allow clear communication about BPMMs, and new models based on the identified components. This unambiguous understanding helps to avoid confusion and inappropriate assumptions in the BP literature. Future research will further validate the definitions by comparing BPMMs.

The pragmatic quality of Resources- Events-Agents diagrams: an experimental evaluation

The Resources‐Events‐Agents (REA) model is a semantic data model for the development of enterprise information systems. Although this model has been proposed as a benchmark for enterprise information modelling, only few studies have attempted to empirically validate the claimed benefits of REA modelling. Moreover, these studies focused on the evaluation of REA‐based system implementations rather than directly assessing the REA‐modelled conceptual schemas that these systems are based on. This paper presents a laboratory experiment that measured the user understanding of diagrammatic conceptual schemas developed using the REA model. The theoretical foundation for the hypotheses are cognitive theories that explain pattern recognition phenomena and the resulting reduction in cognitive effort for understanding conceptual schemas. The results of the experiment indicate a more accurate understanding of the business processes and policies modelled when users recognize the REA model’s core pattern of enterprise information in the diagram. The implication for modelling practice is that the use of the REA model improves the requirements engineering process by facilitating the user validation of conceptual schemas produced by analysts, and thus helps ensuring the quality of the enterprise information system that is developed or implemented.

A functional size measurement method for object-oriented conceptual schemas: design and evaluation issues

Functional Size Measurement (FSM) methods are intended to measure the size of software by quantifying the functional user requirements of the software. The capability to accurately quantify the size of software in an early stage of the development lifecycle is critical to software project managers for evaluating risks, developing project estimates and having early project indicators. In this paper, we present OO-Method Function Points (OOmFP), which is a new FSM method for object-oriented systems that is based on measuring conceptual schemas. OOmFP is presented following the steps of a process model for software measurement. Using this process model, we present the design of the measurement method, its application in a case study, and the analysis of different evaluation types that can be carried out to validate the method and to verify its application and results.

Tying process model quality to the modeling process: the impact of structuring, movement, and speed

In an investigation into the process of process modeling, we examined how modeling behavior relates to the quality of the process model that emerges from that. Specifically, we considered whether (i) a modelers structured modeling style, (ii) the frequency of moving existing objects over the modeling canvas, and (iii) the overall modeling speed is in any way connected to the ease with which the resulting process model can be understood. In this paper, we describe the exploratory study to build these three conjectures, clarify the experimental set-up and infrastructure that was used to collect data, and explain the used metrics for the various concepts to test the conjectures empirically. We discuss various implications for research and practice from the conjectures, all of which were confirmed by the experiment.

The impact of structural complexity on the understandability of UML statechart diagrams

The effectiveness of current software development strategies, such as Model-Driven Development (MDD), depends largely on the quality of their primary artefacts, i.e. software models. As the standard modelling language for software systems is the Unified Modelling Language (UML), quality assurance of UML models is a major research field in Computer Science. Understandability, i.e. a models ability to be easily understood, is one model quality property that is currently heavily under investigation. In particular, researchers are searching for the factors that determine an UML models understandability and are looking for ways to manipulate these factors. This paper presents an empirical study investigating the effect that structural complexity has on the understandability of one particular type of UML model, i.e. the statechart diagram. Based on data collected in a family of three experiments, we have identified three dimensions of structural complexity that affect understandability: (i) the size and control flow complexity of the statechart in terms of features such as the number of states, events, guards and state transitions; (ii) the actions that are performed when entering or leaving a state; (iii) the sequence of actions that is performed while staying within a state. Based on these structural complexity dimensions we have built an understandability prediction model using a regression technique that is specifically recommended for data obtained through a repeated measures design. Our test results show that each of the underlying structural complexity dimensions has a significant impact on the understandability of a statechart diagram.

A conceptual framework and classification of capability areas for business process maturity

The article elaborates on business process maturity, which indicates how well an organisation can perform based on its business processes, i.e. on its way of working. This topic is of paramount importance for managers who try to excel in todays competitive world. Hence, business process maturity is an emerging research field. However, no consensus exists on the capability areas (or skills) needed to excel. Moreover, their theoretical foundation and synergies with other fields are frequently neglected. To overcome this gap, our study presents a conceptual framework with six main capability areas and 17 sub areas. It draws on theories regarding the traditional business process lifecycle, which are supplemented by recognised organisation management theories. The comprehensiveness of this framework is validated by mapping 69 business process maturity models (BPMMs) to the identified capability areas, based on content analysis. Nonetheless, as a consensus neither exists among the collected BPMMs, a classification of different maturity types is proposed, based on cluster analysis and discriminant analysis. Consequently, the findings contribute to the grounding of business process literature. Possible future avenues are evaluating existing BPMMs, directing new BPMMs or investigating which combinations of capability areas (i.e. maturity types) contribute more to performance than others.

Case Studies in research: Choosing the right business process maturity model

We have built and tested a decision tool which will help organisations properly select one business process maturity model (BPMM) over another. This prototype consists of a novel questionnaire with decision criteria for BPMM selection, linked to a unique data set of 69 BPMMs. Fourteen criteria (questions) were elicited from an international Delphi study, and weighed by the analytical hierarchy process. Case studies have shown (non-)profit and academic applications. Our purpose was to describe criteria that enable an informed BPMM choice (conform to decision-making theories, rather than ad hoc). Moreover, we propose a design process for building BPMM decision tools.