Tihana Galinac Grbac

A Second Replicated Quantitative Analysis of Fault Distributions in Complex Software Systems

To contribute to the body of empirical research on fault distributions during development of complex software systems, a replication of a study of Fenton and Ohlsson is conducted. The hypotheses fr...

On the probability distribution of faults in complex software systems

Abstract Context There are several empirical principles related to the distribution of faults in a software system (e.g. the Pareto principle) widely applied in practice and thoroughly studied in the software engineering research providing evidence in their favor. However, the knowledge of the underlying probability distribution of faults, that would enable a systematic approach and refinement of these principles, is still quite limited. Objective In this paper we study the probability distribution of faults detected during verification in four consecutive releases of a large-scale complex software system for the telecommunication exchanges. This is the first such study analyzing closed software system, replicating two previous studies for open source software. Method We take into consideration the Weibull, lognormal, double Pareto, Pareto, and Yule–Simon probability distributions, and investigate how well these distributions fit our empirical fault data using the non-linear regression. Results The results indicate that the double Pareto distribution is the most likely choice for the underlying probability distribution. This is not consistent with the previous studies on open source software. Conclusion The study shows that understanding the probability distribution of faults in complex software systems is more complicated than previously thought. Comparison with previous studies shows that the fault distribution strongly depends on the environment, and only further replications would make it possible to build up a general theory for a given context.

Co-evolutionary multi-population genetic programming for classification in software defect prediction

Evolving diverse ensembles using genetic programming has recently been proposed for classification problems with unbalanced data. Population diversity is crucial for evolving effective algorithms. Multilevel selection strategies that involve additional colonization and migration operations have shown better performance in some applications. Therefore, in this paper, we are interested in analysing the performance of evolving diverse ensembles using genetic programming for software defect prediction with unbalanced data by using different selection strategies. We use colonization and migration operators along with three ensemble selection strategies for the multi-objective evolutionary algorithm. We compare the performance of the operators for software defect prediction datasets with varying levels of data imbalance. Moreover, to generalize the results, gain a broader view and understand the underlying effects, we replicated the same experiments on UCI datasets, which are often used in the evolutionary computing community. The use of multilevel selection strategies provides reliable results with relatively fast convergence speeds and outperforms the other evolutionary algorithms that are often used in this research area and investigated in this paper. This paper also presented a promising ensemble strategy based on a simple convex hull approach and at the same time it raised the question whether ensemble strategy based on the whole population should also be investigated.

Software defect prediction with Bug-Code analyzer - A data collection tool demo

Empirical software engineering research community aims to accumulate knowledge in software engineering community based on the empirical studies on datasets obtained from the real software projects. Limiting factor to building the theory over thus accumulated knowledge is often related to dataset bias. One solution to this problem is developing a systematic data collection procedure through standard guidelines that would be available to open community and thus enable reducing data collection bias. In this paper we present a tool demonstration that implements a systematic data collection procedure for software defect prediction datasets from the open source bug tracking and the source code management repositories. Main challenging issue that the tool addresses is linking the information related to the same entity (e.g. class file) from these two sources. The tool implements interfaces to bug and source code repositories and even other tools for calculating the software metrics. Finally, it offers the user to create software defect prediction datasets even if he is unaware of all the details behind this complex task.

Software structure evolution and relation to system defectiveness

We still do not have clear figure about how software systems evolve and how we may control its evolution process. Software structure has been identified that may have the biggest impact, especially because it may be represented from numerous perspectives. Novelty introduced in this paper is the way how we define the structure of evolving complex software systems. The structure is represented with help of graph representations, and subgraph frequencies, the concept reused from the network analysis theory. The graph structure of a software system and its evolution over the system versions, as well as its relation to defectiveness, is empirically studied in terms of subgraph frequencies and motifs for more than 30 releases of three large open source software systems. We identified that the same set of subgraphs of software system is present across the system version, but different sets, although overlapping, are present in different software systems. Furthermore, we confirmed the continuous system evolution in terms of continuous structure change and we find some evidence for its relation to system defectiveness.

A quantitative analysis of the unit verification perspective on fault distributions in complex software systems: an operational replication

Unit verification, including software inspections and unit tests, is usually the first code verification phase in the software development process. However, principles of unit verification are weakly explored, mostly due to the lack of data, since unit verification data are rarely systematically collected and only a few studies have been published with such data from industry. Therefore, we explore the theory of fault distributions, originating in the quantitative analysis by Fenton and Ohlsson, in the weakly explored context of unit verification in large-scale software development. We conduct a quantitative case study on a sequence of four development projects on consecutive releases of the same complex software product line system for telecommunication exchanges. We replicate the operationalization from earlier studies, analyzed hypotheses related to the Pareto principle of fault distribution, persistence of faults, effects of module size, and quality in terms of fault densities, however, now from the perspective of unit verification. The patterns in unit verification results resemble those of later verification phases, e.g., regarding the Pareto principle, and may thus be used for prediction and planning purposes. Using unit verification results as predictors may improve the quality and efficiency of software verification.

A systematic data collection procedure for software defect prediction

Software defect prediction research relies on data that must be collected from otherwise separate repositories. To achieve greater generalization of the results, standardized protocols for data collection and validation are necessary. This paper presents an exhaustive survey of techniques and approaches used in the data collection process. It identifies some of the issues that must be addressed to minimize dataset bias and also provides a number of measures that can help researchers to compare their data collection approaches and evaluate their data quality. Moreover, we present a data collection procedure that uses a bug-code linking technique based on regular expression. The detailed comparison and root cause analysis of inconsistencies with a number of popular data collection approaches and their publicly available datasets, reveals that our procedure achieves the most favorable results. Finally, we implement our data collection procedure in a data collection tool we name the Bug-Code (BuCo) Analyzer.

Transforming Vertical Web Applications into Elastic Cloud Applications

There exists a huge amount of vertical applications that are developed for isolated computing environments. Due to increasing demand for additional resources there is a clear need to adapt these applications to the distributed environments. However, this is not an easy task and numerous variants are possible. Moreover, in this transition a new quality requirements become important, such as application elasticity. Application elasticity has to be built into a software system to enable smooth cost optimization at the run-time. In this paper, we provide a framework for evaluating different transformation variants of vertical Java EE multi-tiered applications into elastic cloud applications. With support of this framework the software developer is guided how to transform its application achieving optimal elasticity strategy. The framework is evaluated on slicing and evaluating elasticity of existing SaaS multi-tiered Java application used in Croatian market.

A quality cost reduction model for large-scale software development

Understanding quality costs is recognized as a prerequisite for decreasing the variability of the success of software development projects. This paper presents an empirical quality cost reduction (QCR) model to support the decision-making process for additional investment in the early phases of software verification. The main idea of the QCR model is to direct additional investment into software units that have some fault-slip potential in their later verification phases, with the aim of reducing costs and increasing product quality. The fault-slip potential of a software unit within a system is determined by analogy with historical projects. After a preliminary study on a sample of software units, which proves that we can lower quality costs with additional investment into particular verification activities, we examine the effectiveness of the proposed QCR model using real project data. The results show that applying the model produces a positive business case, meaning that the model lowers quality costs and increases quality, resulting in economic benefit. The potential to reduce quality costs is growing significantly with the evolution of software systems and the reuse of their software units. The proposed model is the result of a research project performed at Ericsson.

Data collection for Software Defect Prediction - An exploratory case study of open source software projects

Software Defect Prediction (SDP) empirical studies are highly biased with the quality of data and widely suffer from limited generalizations. The main reasons are the lack of data and its systematic data collection procedures. Our research aims at producing the first systematically defined data collection procedure for SDP datasets that are obtained by linking separate development repositories. This paper is the first step to achieving that objective, performing an exploratory study. We review the existing literature on approaches and tools used in the collection of SDP datasets, derive a detailed collection procedure and test it in this exploratory study. We quantify the bias that may be caused by the issues we identified and we review 35 tools for software product metrics collection. The most critical issues are many-to-many relation between bug-file links, duplicated bug-file links and the issue of untraceable bugs. Our research provides more detailed, experience based data collection procedure, crucial for further development of SDP body of knowledge. Furthermore, our findings enabled us to develop the automatic data collection tool.