Brian S. Mitchell

Using automatic clustering to produce high-level system organizations of source code

We describe a collection of algorithms that we developed and implemented to facilitate the automatic recovery of the modular structure of a software system from its source code. We treat automatic modularization as an optimization problem. Our algorithms make use of traditional hill-climbing and genetic algorithms.

Bunch: a clustering tool for the recovery and maintenance of software system structures

Software systems are typically modified in order to extend or change their functionality, improve their performance, port them to different platforms, and so on. For developers, it is crucial to understand the structure of a system before attempting to modify it. The structure of a system, however, may not be apparent to new developers, because the design documentation is non-existent or, worse, inconsistent with the implementation. This problem could be alleviated if developers were somehow able to produce high-level system decomposition descriptions from the low-level structures present in the source code. We have developed a clustering tool called Bunch that creates a system decomposition automatically by treating clustering as an optimization problem. The paper describes the extensions made to Bunch in response to feedback we received from users. The most important extension, in terms of the quality of results and execution efficiency, is a feature that enables the integration of designer knowledge about the system structure into an otherwise fully automatic clustering process. We use a case study to show how our new features simplified the task of extracting the subsystem structure of a medium size program, while exposing an interesting design flaw in the process.

On the automatic modularization of software systems using the Bunch tool

Since modern software systems are large and complex, appropriate abstractions of their structure are needed to make them more understandable and, thus, easier to maintain. Software clustering techniques are useful to support the creation of these abstractions by producing architectural-level views of a systems structure directly from its source code. This paper examines the Bunch clustering system which, unlike other software clustering tools, uses search techniques to perform clustering. Bunch produces a subsystem decomposition by partitioning a graph of the entities (e.g., classes) and relations (e.g., function calls) in the source code. Bunch uses a fitness function to evaluate the quality of graph partitions and uses search algorithms to find a satisfactory solution. This paper presents a case study to demonstrate how Bunch can be used to create views of the structure of significant software systems. This paper also outlines research to evaluate the software clustering results produced by Bunch.

Comparing the decompositions produced by software clustering algorithms using similarity measurements

Decomposing source code components and relations into subsystem clusters is an active area of research. Numerous clustering approaches have been proposed in the reverse engineering literature, each one using a different algorithm to identify subsystems. Since different clustering techniques may not produce identical results when applied to the same system, mechanisms that can measure the extent of these differences are needed. Some work to measure the similarity between decompositions has been done, but this work considers the assignment of source code components to clusters as the only criterion for similarity. We argue that better similarity measurements can be designed if the relations between the components are considered. The authors propose two similarity measurements that overcome certain problems in existing measurements. We also provide some suggestions on how to identify and deal with source code components that tend to contribute to poor similarity results. We conclude by presenting experimental results, and by highlighting some of the benefits of our similarity measurements.

On the evaluation of the Bunch search-based software modularization algorithm

The first part of this paper describes an automatic reverse engineering process to infer subsystem abstractions that are useful for a variety of software maintenance activities. This process is based on clustering the graph representing the modules and module-level dependencies found in the source code into abstract structures not in the source code called subsystems. The clustering process uses evolutionary algorithms to search through the enormous set of possible graph partitions, and is guided by a fitness function designed to measure the quality of individual graph partitions. The second part of this paper focuses on evaluating the results produced by our clustering technique. Our previous research has shown through both qualitative and quantitative studies that our clustering technique produces good results quickly and consistently. In this part of the paper we study the underlying structure of the search space of several open source systems. We also report on some interesting findings our analysis uncovered by comparing random graphs to graphs representing real software systems.

An architecture for distributing the computation of software clustering algorithms

Collections of general purpose networked workstations offer processing capability that often rivals or exceeds supercomputers. Since networked workstations are readily available in most organizations, they provide an economic and scalable alternative to parallel machines. The authors discuss how individual nodes in a computer network can be used as a collection of connected processing elements to improve the performance of a software engineering tool that we developed. Our tool, called Bunch, automatically clusters the structure of software systems into a hierarchy of subsystems. Clustering helps developers understand complex systems by providing them with high-level abstract (clustered) views of the software structure. The algorithms used by Bunch are computationally intensive and, hence, we would like to improve our tools performance in order to cluster very large systems. The paper describes how we designed and implemented a distributed version of Bunch, which is useful for clustering large systems.

Search based reverse engineering

In this paper we describe a two step process for reverse engineering the software architecture of a system directly from its source code. The first step involves clustering the modules from the source code into abstract structures called subsystems. The second step involves reverse engineering the subsystem-level relations using a formal (and visual) architectural constraint language. We use search techniques to accomplish both of these steps, and have implemented a suite of integrated tools to support the reverse engineering process. Through a case study, we demonstrate how our tools can be used to extract the software architecture of an open-source software package from its source code without having any a priori knowledge about its design.

Using Interconnection Style Rules to Infer Software Architecture Relations

Software design techniques emphasize the use of abstractions to help developers deal with the complexity of constructing large and complex systems. These abstractions can also be used to guide program- mers through a variety of maintenance, reengineering and enhancement activities. Unfortunately, recovering design abstractions directly from a systems implementation is a difficult task because the source code does not contain them. In this paper we describe an automatic process to in- fer architectural-level abstractions from the source code. The first step uses software clustering to aggregate the systems modules into abstract containers called subsystems. The second step takes the output of the clustering process, and infers architectural-level relations based on formal style rules that are specified visually. This two step process has been im- plemented using a set of integrated tools that employ search techniques to locate good solutions to both the clustering and the relationship in- ferencing problem quickly. The paper concludes with a case study to demonstrate the effectiveness of our process and tools.

CRAFT: a framework for evaluating software clustering results in the absence of benchmark decompositions [Clustering Results Analysis Framework and Tools]

Software clustering algorithms are used to create high-level views of a systems structure using source code-level artifacts. Software clustering is an active area of research that has produced many clustering algorithms. However, we have so far seen very little work that investigates how the results of these algorithms can be evaluated objectively in the absence of a benchmark decomposition or without the active participation of the original designers of the system. Ideally, for a given system, art agreed upon reference (benchmark) decomposition of the systems structure would exist, allowing the results of various clustering algorithms to be compared against it. Since such benchmarks seldom exist, we seek alternative methods to gain confidence in the quality of results produced by software clustering algorithms. In this paper, we present a tool that supports the evaluation of software clustering results in the absence of a benchmark decomposition.