Onaiza Maqbool

Hierarchical Clustering for Software Architecture Recovery

Gaining an architectural level understanding of a software system is important for many reasons. When the description of a systems architecture does not exist, attempts must be made to recover it. In recent years, researchers have explored the use of clustering for recovering a software systems architecture, given only its source code. The main contributions of this paper are given as follows. First, we review hierarchical clustering research in the context of software architecture recovery and modularization. Second, to employ clustering meaningfully, it is necessary to understand the peculiarities of the software domain, as well as the behavior of clustering measures and algorithms in this domain. To this end, we provide a detailed analysis of the behavior of various similarity and distance measures that may be employed for software clustering. Third, we analyze the clustering process of various well-known clustering algorithms by using multiple criteria, and we show how arbitrary decisions taken by these algorithms during clustering affect the quality of their results. Finally, we present an analysis of two recently proposed clustering algorithms, revealing close similarities in their apparently different clustering approaches. Experiments on four legacy software systems provide insight into the behavior of well-known clustering algorithms and their characteristics in the software domain.

The weighted combined algorithm: a linkage algorithm for software clustering

Software systems need to evolve as business requirements, technology and environment change. As software is modified to accommodate the required changes, its structure deteriorates. There is increased deviation from the actual design and architecture. Very often, documentation is not updated to reflect these changes thus making it more and more difficult to understand, manage and maintain these systems. Researchers have applied various techniques to recover the components and architecture of such software systems. The use of clustering techniques has recently been explored for reverse engineering and software architecture recovery. There is a need to tailor clustering algorithms and similarity measures to cater to software. We present a new algorithm for finding intercluster distance. We compare the performance of some popular similarity measures for this algorithm using two test systems and suggest variations of the similarity measures which show better results for software clustering.

Improved Similarity Measures for Software Clustering

Software clustering is a useful technique to recover architecture of a software system. The results of clustering depend upon choice of entities, features, similarity measures and clustering algorithms. Different similarity measures have been used for determining similarity between entities during the clustering process. In software architecture recovery domain the Jaccard and the Unbiased Ellenberg measures have shown better results than other measures for binary and non-binary features respectively. In this paper we analyze the Russell and Rao measure for binary features to show the conditions under which its performance is expected to be better than that of Jaccard. We also show how our proposed Jaccard-NM measure is suitable for software clustering and propose its counterpart for non-binary features. Experimental results indicate that our proposed Jaccard-NM measure and Russell & Rao measure perform better than Jaccard measure for binary features, while for non-binary features, the proposed Unbiased Ellenberg-NM measure produces results which are closer to the decomposition prepared by experts.

Bug Prioritization to Facilitate Bug Report Triage

The large number of new bug reports received in bug repositories of software systems makes their management a challenging task. Handling these reports manually is time consuming, and often results in delaying the resolution of important bugs. To address this issue, a recommender may be developed which automatically prioritizes the new bug reports. In this paper, we propose and evaluate a classification based approach to build such a recommender. We use the Naïve Bayes and Support Vector Machine (SVM) classifiers, and present a comparison to evaluate which classifier performs better in terms of accuracy. Since a bug report contains both categorical and text features, another evaluation we perform is to determine the combination of features that better determines the priority of a bug. To evaluate the bug priority recommender, we use precision and recall measures and also propose two new measures, Nearest False Negatives (NFN) and Nearest False Positives (NFP), which provide insight into the results produced by precision and recall. Our findings are that the results of SVM are better than the Naïve Bayes algorithm for text features, whereas for categorical features, Naïve Bayes performance is better than SVM. The highest accuracy is achieved with SVM when categorical and text features are combined for training.

Cooperative clustering for software modularization

Clustering is a useful technique to group data entities. Many different algorithms have been proposed for software clustering. To combine the strengths of various algorithms, researchers have suggested the use of Consensus Based Techniques (CBTs), where more than one actors (e.g. algorithms) work together to achieve a common goal. Although the use of CBTs has been explored in various disciplines, no work has been done for modularizing software. In this paper, the main research question we investigate is whether the Cooperative Clustering Technique (CCT), a type of CBT, can improve software modularization results. The main contributions of this paper are as follows. First, we propose our CCT in which more than one similarity measure cooperates during the hierarchical clustering process. To this end, we present an analysis of well-known measures. Second, we present a cooperative clustering approach for two types of well-known agglomerative hierarchical software clustering algorithms, for binary as well as non-binary features. Third, to evaluate our proposed CCT, we conduct modularization experiments on five software systems. Our analysis identifies certain cases that reveal weaknesses of the individual similarity measures. The experimental results support our hypothesis that these weaknesses may be overcome by using more than one measure, as our CCT produces better modularization results for test systems in which these cases occur. We conclude that CCTs are capable of showing significant improvement over individual clustering algorithms for software modularization.

Automated software clustering: An insight using cluster labels

Abstract Clustering techniques have shown promising results for the architecture recovery and re-modularization of legacy software systems. Clusters that are obtained as a result of the clustering process may not be easy to interpret until they are assigned appropriate labels. Automatic labeling of clusters reduces the time required to understand them and can also be used to evaluate the effectiveness of the clustering process, if the assigned labels are meaningful and convey the purpose of each cluster effectively. In this paper, we present a labeling scheme based on identifiers of an entity. As the clustering process proceeds, keywords within identifiers are ranked using two ranking schemes: frequency and inverse frequency. We present experimental results to demonstrate the effectiveness of our labeling approach. A comparison between the ranking schemes reveals the inverse frequency scheme to form more meaningful labels, especially for small clusters. A comparison of clustering results of the complete and weighted combined algorithms based on labels of the clusters produced by them during clustering shows that the latter produces a more understandable cluster hierarchy with easily identifiable software sub-systems.

Evaluating relationship categories for clustering object-oriented software systems

Various techniques have been proposed for the automatic modularisation and architecture recovery of software systems. These techniques usually employ an algorithm to form clusters of similar entities. Similarity between entities is based on their characteristics, and is often determined by the relationships that exist between them. When using automatic techniques, selecting a suitable algorithm and appropriate relationships are challenging issues, and have a significant influence on the quality of results. Although researchers have employed different algorithms for modularising object-oriented software systems, there has been relatively little work to determine which relationships produce better modularisation results. The authors evaluate in this study a large number of relationships that may exist between entities in an object-oriented system, by dividing the relationships into different categories. For modularisation, experiments are conducted using multiple hierarchical clustering algorithms. The experimental results indicate the relationships that improve the quality of results for the algorithms, and thus may be considered more important for software clustering.

Application of Artificial Neural Networks for monsoon rainfall prediction

Prediction of monsoon rainfall in a timely manner can be highly beneficial for Pakistan, where monsoon is the major source of rain. Presently, Multiple Linear Regression and Statistical Downscaling Models are being used for monsoon rainfall prediction. In spite of making use of a large number of resources and having dependency on a number of parameters, the results of these models have not been satisfactory. In this paper, we explore the use of Artificial Neural Networks for monsoon rainfall prediction. The techniques investigated include Backpropagation (BP) and Learning Vector Quantization (LVQ). We use 45 years real monsoon rainfall data from 1960 to 2004 for training of neural network models and evaluate the performance of these models over a test period of five years from 2005 to 2009. Comparison with Multiple Linear Regression and Statistical Downscaling Models reveals better performance of neural network techniques in terms of accuracy, and also in terms of greater lead time and fewer required resources.

Monitoring software evolution using multiple types of changes

Software systems require gradual changes to survive in an environment where they are implemented. Several reasons are a cause of change in software e.g. error fixing, enhancement in functionality, performance improvement. This behaviour of gradual change in software is known as software evolution. The study of software evolution is an active area of research. Researchers have monitored software evolution in different ways. The method of monitoring evolution is a key point, because different methods may reflect different evolutionary picture of software. In this paper, we studied changes that occurred in software systems for software evolution. Our experimental study focuses on three different types of changes i.e. addition, deletion and modification, and is helpful for detailed analysis of software evolution. Furthermore, on the basis of different type of changes, we investigated Lehmans 5th Law (Conservation of Familiarity) for small scale open source software systems. Our experimental study shows that different measures reflect different evolutionary picture of the software systems.

Metarule-guided association rule mining for program understanding

Software systems are expected to change over their lifetime in order to remain useful. Understanding a software system that has undergone changes is often difficult owing to the unavailability of up-to-date documentation. Under these circumstances, source code is the only reliable means of information regarding the system. In the paper, association rule mining is applied to the problem of software understanding i.e. given the source files of a software system, association rule mining is used to gain an insight into the software. To make association rule mining more effective, constraints are placed on the mining process in the form of metarules. Metarule-guided mining is carried out to find associations which can be used to identify recurring problems within software systems. Metarules are related to re-engineering patterns which present solutions to these problems. Association rule mining is applied to five legacy systems and results presented show how extracted association rules can be helpful in analysing the structure of a software system and modifications to improve the structure are suggested. A comparison of the results obtained for the five systems also reveals legacy system characteristics, which can lead to understanding the nature of open source legacy software and its evolution.