Marco D'Ambros

An extensive comparison of bug prediction approaches

Reliably predicting software defects is one of software engineerings holy grails. Researchers have devised and implemented a plethora of bug prediction approaches varying in terms of accuracy, complexity and the input data they require. However, the absence of an established benchmark makes it hard, if not impossible, to compare approaches.

Evaluating defect prediction approaches: a benchmark and an extensive comparison

Reliably predicting software defects is one of the holy grails of software engineering. Researchers have devised and implemented a plethora of defect/bug prediction approaches varying in terms of accuracy, complexity and the input data they require. However, the absence of an established benchmark makes it hard, if not impossible, to compare approaches. We present a benchmark for defect prediction, in the form of a publicly available dataset consisting of several software systems, and provide an extensive comparison of well-known bug prediction approaches, together with novel approaches we devised. We evaluate the performance of the approaches using different performance indicators: classification of entities as defect-prone or not, ranking of the entities, with and without taking into account the effort to review an entity. We performed three sets of experiments aimed at (1) comparing the approaches across different systems, (2) testing whether the differences in performance are statistically significant, and (3) investigating the stability of approaches across different learners. Our results indicate that, while some approaches perform better than others in a statistically significant manner, external validity in defect prediction is still an open problem, as generalizing results to different contexts/learners proved to be a partially unsuccessful endeavor.

On the Relationship Between Change Coupling and Software Defects

Change coupling is the implicit relationship between two or more software artifacts that have been observed to frequently change together during the evolution of a software system. Researchers have studied this dependency and have observed that it points to design issues such as architectural decay. It is still unknown whether change coupling correlates with a tangible effect of design issues, i.e., software defects.In this paper we analyze the relationship between change coupling and software defects on three large software systems. We investigate whether change coupling correlates with defects, and if the performance of bug prediction models based on software metrics can be improved with change coupling information.

Visualizing Co-Change Information with the Evolution Radar

Software evolution analysis provides a valuable source of information that can be used both to understand a systems design and predict its future development. While for many program comprehension purposes, it is sufficient to model a single version of a system, there are types of information that can only be recovered when the history of a system is taken into account. Logical coupling, the implicit dependency between software artifacts that have been changed together, is an example of such information. Previous research has dealt with low-level couplings between files, leading to an explosion of the data to be analyzed, or has abstracted the logical couplings to the level of modules, leading to a loss of detailed information. In this paper, we present a visualization-based approach that integrates logical coupling information at different levels of abstraction. This facilitates an in-depth analysis of the logical couplings, and at the same time, leads to a characterization of a systems modules in terms of their logical coupling. The presented approach supports the retrospective analysis of a software system and maintenance activities such as restructuring and redocumentation. We illustrate retrospective analysis on two large open-source software systems.

On the Impact of Design Flaws on Software Defects

The presence of design flaws in a software system has a negative impact on the quality of the software, as they indicate violations of design practices and principles, which make a software system harder to understand, maintain, and evolve. Software defects are tangible effects of poor software quality. In this paper we study the relationship between software defects and a number of design flaws. We found that, while some design flaws are more frequent, none of them can be considered more harmful with respect to software defects. We also analyzed the correlation between the introduction of new flaws and the generation of defects.

Content classification of development emails

Emails related to the development of a software system contain information about design choices and issues encountered during the development process. Exploiting the knowledge embedded in emails with automatic tools is challenging, due to the unstructured, noisy, and mixed language nature of this communication medium. Natural language text is often not well-formed and is interleaved with languages with other syntaxes, such as code or stack traces. We present an approach to classify email content at line level. Our technique classifies email lines in five categories (i.e., text, junk, code, patch, and stack trace) to allow one to subsequently apply ad hoc analysis techniques for each category. We evaluated our approach on a statistically significant set of emails gathered from mailing lists of four unrelated open source systems.

Reverse Engineering with Logical Coupling

Evolutionary information about software systems has proven to be a good resource to complement existing reverse engineering approaches, because it helps in giving a historical perspective of the system to be reverse engineered. Moreover, it provides additional types of information that are not present when only one version of a system is considered. Logical coupling, the implicit dependency between artifacts which changed together, is one example of such information. However, the recurrent problem is that such information comes in large amounts and must be processed to be useful for the reverse engineering of a system. In this paper we propose an approach to use logical coupling information at different levels of abstraction to detect areas in the system which may lead to maintenance problems. They represent a good starting point to decrease the coupling in the system. Our approach uses an interactive visualization technique called the Evolution Radar, which can effectively break down the amount and complexity of the logical coupling information. We present our technique in detail and apply it on a large open-source software system

Software bugs and evolution: a visual approach to uncover their relationship

Versioning systems such as CVS exhibit a large potential to investigate and understand the evolution of large software systems. Bug reporting systems such as Bugzilla help to understand which parts of the system are affected by problems. In this article, we present a novel visual approach to uncover the relationship between evolving software and the way it is affected by software bugs. By visually putting the two aspects close to each other, we can characterize the evolution of software artifacts. We validate our approach on 3 very large open source software systems

Method-level bug prediction

Researchers proposed a wide range of approaches to build effective bug prediction models that take into account multiple aspects of the software development process. Such models achieved good prediction performance, guiding developers towards those parts of their system where a large share of bugs can be expected. However, most of those approaches predict bugs on file-level. This often leaves developers with a considerable amount of effort to examine all methods of a file until a bug is located. This particular problem is reinforced by the fact that large files are typically predicted as the most bug-prone. In this paper, we present bug prediction models at the level of individual methods rather than at file-level. This increases the granularity of the prediction and thus reduces manual inspection efforts for developers. The models are based on change metrics and source code metrics that are typically used in bug prediction. Our experiments-performed on 21 Java open-source (sub-)systems-show that our prediction models reach a precision and recall of 84% and 88%, respectively. Furthermore, the results indicate that change metrics significantly outperform source code metrics.

A Bug's Life Visualizing a Bug Database

Visualization has long been accepted as a viable means to comprehend large amounts of information. Especially in the context of software evolution a well-designed visualization is crucial to be able to cope with the sheer data that needs to be analyzed. Many approaches have been investigated to visualize evolving systems, but most of them focus on structural data and are useful to answer questions about the structural evolution of a system. In this paper we consider an often neglected type of information, namely the one provided by bug tracking systems, which store data about the problems that various people, from developers to end users, detected and reported. We first briefly introduce the context by reporting on the particularities of the present data, and then propose two visualizations to render bugs as first-level entities.