Hamed Shariat Yazdi

Generating realistic test models for model processing tools

Test models are needed to evaluate and benchmark algorithms and tools in model driven development. Most model generators randomly apply graph operations on graph representations of models. This approach leads to test models of poor quality. Some approaches do not guarantee the basic syntactic correctness of the created models. Even if so, it is almost impossible to guarantee, or even control, the creation of complex structures, e.g. a subgraph which implements an association between two classes. Such a subgraph consists of an association node, two association end nodes, and several edges, and is normally created by one user command. This paper presents the SiDiff Model Generator, which can generate models, or sets of models, which are syntactically correct, contain complex structures, and exhibit defined statistical characteristics.

A procedure with stepsize control for solving n one-dimensional IVPs

Finite precision computations may affect the stability of algorithms and the accuracy of computed solutions. In this paper we first obtain a relation for computing the number of common significant digits between the exact solution and a computed solution of a one-dimensional initial-value problem obtained by using a single-step or multi-step method. In fact, by using the approximate solutions obtained with stepsizes h and h /2, the number of common significant digits between approximate solution with stepsize h and exact solution is estimated. Then by using the stochastic arithmetic, the CESTAC method, and the CADNA library we propose an algorithm to control the round-off error effect on the computed solution. This method can easily apply to a system of n one-dimensional initial-value problems. Finally some numerical examples are given to show the efficiency of the method.

Synthesizing realistic test models

Tools and methods in the context of Model Driven Engineering (MDE) have to be evaluated and tested using appropriate models as test cases. Unfortunately, adequate test models are scarcely available in many application domains and have to be artificially created. In this regard, model generators have been proposed recently. Principally, they generate test models by modifying a base model using a specified set of edit operations. The modification process should be done in a way that the resulting test models are as “realistic” as possible, i.e. the applied changes should resemble the real evolution that one observes in real software systems at the abstraction level of models. Therefore, we have to (1) properly capture the evolution of real software models, (2) statistically model the evolution (changes) and (3) finally properly reproduce it in the generated test models. To this end, we reversed engineered all revisions of nine typical Java systems into their class diagrams (totally 6,559 distinct models). We compared the subsequent models using a state-of-the-art model differencing tool and we computed the changes between them in terms of applied edit operations. We investigated the fitness of 60 promising distributions on the observed frequencies of edit operations in order to statistically model the changes. Four of our candidate distributions were successful to statistically model the changes with very good success rates. Since it was not known how to implement them, i.e. produce their random variates, we developed a practical implementation. The implemented distributions are then used to reproduce the real evolution of software systems in order to synthesize more realistic test models for MDE tools.

Analysis and Prediction of Design Model Evolution Using Time Series

Tools which support Model-Driven Engineering have to be evaluated and tested. In the domain of model differencing and model versioning, sequences of software models (model histories), in which a model is obtained from its immediate predecessor by some modification, are of special interest. Unfortunately, in this application domain adequate real test models are scarcely available and must be artificially created. To this end, model generators were proposed in recent years. Generally, such model generators should be configured in a way that the generated sequences of models are as realistic as possible, i.e. they should mimic the changes that happen in real software models. Hence, it is a necessary prerequisite to analyze and to stochastically model the evolution (changes) of real software systems at the abstraction level of models. In this paper, we present a new approach to statistically analyze the evolution of models. Our approach uses time series as a statistical method to capture the dynamics of the evolution. We applied this approach to several typical projects and we successfully modeled their evolutions. The time series models could predict the future changes of the next revisions of the systems with good accuracies. The obtained time series models are used to create more realistic model histories for model versioning and model differencing tools.

Assessing the Quality of Model Differencing Engines

In recent years many tools and algorithms for model comparison and differencing were proposed. Typically, the main focus of the research laid on being able to compute the difference in the first place. Only very few papers addressed the quality of the delivered differences sufficiently. Hence, this is a general shortcoming in the state-of-the-art. Currently, there are no established community standards how to assess the quality of differences and it is neither possible to compare the quality of different algorithms, nor can developers decide whether or not an algorithm is able to produce adequate results in a given application scenario. We propose a parallel working session to be held to discuss this general problem and its implications. The goal of the working session is to achieve a common understanding of what the crucial factors in assessing the quality of differences are. Furthermore, it is planed to discuss possible solutions that help the research community as whole, e.g. by drafting the design of an initial benchmark corpus which later could be turned into a standardized, openly available benchmark set.

A framework for capturing, statistically modeling and analyzing the evolution of software models

New framework for capturing, statistically modeling and simulating evolution of modelsEvolution formulated at 2 abstraction levels: low and high level changes between modelsEmpirical study of evolution of Java models using 3 kinds of time series modelsMixed ARMA-GARCH models were superior, but ARMA models performed well in practiceStatistical models were used to generate more realistic test models for MDE tools This paper presents a new methodological framework for capturing and statistically modeling the evolution of models in model-driven software development. The framework captures the changes between revisions of models in terms of both low-level (internal) and high-level (developer-visible) edit operations applied between revisions. In our approach, evolution is modeled statistically by using ARMA, GARCH and mixed ARMA-GARCH models. Forecasting and simulation aspects of these time series models are thoroughly assessed. The suitability of the framework is shown by applying it to a large set of design models of real Java systems. Our analysis shows that mixed ARMA-GARCH models are superior to ARMA models.A main motivation for, and application of, the resulting statistical models is to control the generation of realistic model histories which are intended to be used for testing model versioning tools. We present the architecture of the model generator and show how to generate random sequences from the statistical models which control the generation process. Further usages of the statistical models include various forecasting and simulation tasks.