Juan F. Ramil

Software evolution: background, theory, practice

This paper opens with a brief summary of some 30 years of study of the software evolution phenomenon. The results of those studies include the SPE program classification, a principle of software uncertainty and laws of E-type software evolution. The laws were termed so because they encapsulate phenomena largely independent of the people, the organisations and the domains involved in the evolution of the E-type systems studied. Recent studies have refined earlier conclusions, yielded practical guidelines for software evolution management and provide a basis for the formation of a theory of software evolution. Given the volume of published material and the extent of recent discussions on the topic (see, e.g., [Proc. ICSM, Montreal, 2002, p. 66]), this paper is restricted to an overview that exposes the significance of the evolution phenomenon and its study to the wider community, providing a basis for the future and, in particular, development of a theory of software evolution.

Software Evolution and Software Evolution Processes

Most of the software in regular use in businesses and organisations all over the world cannot be completely specified. It cannot be implemented, once and for all. Both the original implementation and the inevitable subsequent evolution (maintenance) are a continual learning experience driven, inter alia, by feedback from the results of the behaviour under execution of the software, as perceived by various stakeholders, by advances and growth in the user organisations and by adaptation to changes in the external world, both independent and as a result of installation and use of the software. Real world, termed type-E, software is essentially evolutionary in nature. The study of the processes of evolution of such software is of considerable interest, as is that of the domains that co-evolve with the software. After briefly discussing the meaning of the term evolution in the context of software, its technology, the software process and related domains, this paper describes some of the facets of the evolution phenomenon and implications to the evolution process as identified during many years of active interest in the topic.

Agent-based Simulation of Open Source Evolution

We present an agent-based simulation model developed to study how size, complexity and effort relate to each other in the development of open source software (OSS). In the model, many developer agents generate, extend, and re-factor code modules independently and in parallel. This accords with empirical observations of OSS development. To our knowledge, this is the first model of OSS evolution that includes the complexity of software modules as a limiting factor in productivity, the fitness of the software to its requirements, and the motivation of developers.nValidation of the model was done by comparing the simulated results against four measures of software evolution (system size, proportion of highly complex modules, level of complexity control work, and distribution of changes) for four large OSS systems. The simulated results resembled the observed data, except for system size: three of the OSS systems showed alternating patterns of super-linear and sub-linear growth, while the simulations produced only super-linear growth. However, the fidelity of the model for the other measures suggests that developer motivation and the limiting effect of complexity on productivity have a significant effect on the development of OSS systems and should be considered in any model of OSS development.

Structural evolution of an open source system: a case study

Software evolution empirical studies are made possible only when data is available and easily collectable. Open source software provides a good opportunity for observing software products in various phases of their evolution, growth and maturity. One of the aspects that have not been analyzed yet through empirical studies is the relation that exists between code components growth and the evolving code structure. In this paper, we look at the growth of code elements by measuring the total number of files per release and also by visualizing the folder structure as a tree. Both, the number of files and the shape of the folder tree structure offer proxies to study the evolving complexity of the system. In this study, a number of hypotheses about the evolutionary patterns in the size of files and folders, in the folder tree structure, reflecting common assumptions on software engineering and open source are tested against empirical data reflecting the evolution of the ARLA system. We also relate some of the observed patterns to the arrival rate of new developers. Results show that the number of code components (files and folders) over releases can be interpreted as a linear trend with a superimposed ripple. We also describe the more apparent characteristics of the evolution of the folder tree structure. Furthermore, we observe that the average size of files and folders tends to stabilize over releases. The cumulative number of people who have been involved as developers of the system follows a trend which resembles that of the size in files over releases, suggesting that the latter could provide a good indicator for rate of work and productivity. There was no apparent relation between the arrival rate of developers and the changes observed in the code structure.

Evolution in software and related areas

After briefly discussing the meaning of the term evolution in the context of software, its technology, the software process and related domains, the paper describes some of the facets and implications of the evolution phenomenon as identified during many years of active interest in the topic, most recently during the FEAST (Feedback, Evolution And Software Technology) projects.

Qualitative simulation of models of software evolution

Lehmans laws and the quantitative models based on them seek to encapsulate empirical generalizations about E-type program evolution. Such modelling is hampered by insufficient knowledge about the mechanisms at work and their parameters. Qualitative reasoning is a body of work that handles a lack of precise knowledge by reasoning at a more abstract level than with quantitative models. This paper describes the introduction of qualitative reasoning to the study of software evolution. It reports on the derivation of qualitative versions of two existing quantitative models of the software evolution process, leading to identification of previously unrecognized behaviours. A third qualitative model is also discussed. The paper also shows how qualitative trend abstraction enables a high level of abstraction analysis of empirical data and that, at this level, the empirical patterns observed in several different software systems display similarities. Finally, we compare the qualitative simulation outputs of the three systems to the abstracted empirical trends.

A study of open source software evolution data using qualitative simulation

Simulation modelling of growth trends and other empirical data from software systems can reveal the main drivers of software evolution and help improve understanding and management of the software evolution phenomenon. This article reports on the application of qualitative abstraction techniques to data from 25 open source software (OSS) systems. After justifying the use of qualitative simulation techniques for the software process domain, the article presents an analysis of the support from OSS-derived data to previously developed qualitative simulation models, which were inspired by observations from the domain of proprietary systems. The analysis involved comparison of model output and qualitatively abstracted growth trends. The analysis also involved the comparison of the transitions in trends of functional size and complexity to those predicted by the models in a sub-set of 21 systems for which complexity data was available. The results indicate that models are able to replicate some of the features in the data. This, in turn, suggests that the study of the relationship between size and complexity and its interaction via feedback loops has a role in explaining the long-term evolutionary behaviour of OSS systems. This is not surprising since it is well known that the evolution of all type of real-world software systems, proprietary and OSS, is driven by feedback from their stakeholders

Studying the evolution of open source systems at different levels of granularity: two case studies

This work presents a study of several evolutionary attributes of two open source software systems: the distributed file system Arla and the stable branch of the Web browser Mozilla. The attributes (size, activity rate and complexity) are visualized over releases using measures at various levels of granularity: folders, files and procedures. A number of hypotheses about the evolutionary behaviour of the systems are considered and, in general, supported by the data from the 2 systems.

The evolution of source folder structure in actively evolved open source systems

Predicting when and how a software system evolves is one of the most fascinating challenges of software engineering. No matter what approach one is using to study such evolution, empirical studies, including observations of systems used in the real world, and of their processes, are needed in order to define correlations, find recurring patterns, and eventually predict how systems are likely to evolve. In the empirical study presented in this paper, we take 25 software systems released as open source, and observe their evolution. Our focus is not only on how much systems grow in size, but rather on how code structure is adapted and gets modified over time and releases. The goal here is to recognize recurring patterns and practices used in evolving long-lived real world software systems. In our study we find three dominant patterns of code structure evolution of open source systems: horizontal expansion, vertical expansion, vertical shrinkage By detailed studied of exemplars of these three patterns one can identify under which conditions a particular pattern is more likely to prevail than the others.

Behavioural modelling of long-lived evolution processes: some issues and an example

For reasons now well understood, application software that is regularly used for real-world problem solving must be continually adapted and enhanced to maintain its fitness to an ever-changing real world, its applications and application domains. This type of activity is termed progressive. As evolution continues, the complexity (functional, structural) of the evolving system is likely to increase unless work, termed antiregressive , is undertaken to control and even reduce it. However, with progressive and anti-regressive work naturally competing for the same pool of resources, management requires means to estimate the amount of work and resources to be applied to each of the two types. After providing a necessary background, the paper describes a systems dynamics model that can serve as the core of a tool to support decision making regarding the optimal personnel allocation over the system lifetime. The model is provided as an example of the use of formalisms in modelling the behaviour of the evolution process.