Manoel G. Mendonça

Replicating software engineering experiments: addressing the tacit knowledge problem

Recently the awareness of the importance of replicating studies has been growing in the empirical software engineering community. The results of any one study cannot simply be extrapolated to all environments because there are many uncontrollable sources of variation between different environments. In our work, we have reasoned that the availability of laboratory packages for experiments can encourage better replications and complementary studies. However, even with effectively specified laboratory packages, transfer of experimental know-how can still be difficult. A cooperation between Brazilian and American researchers addressing effective running of replications was formed in 1999. One of the specific issues being addressed is the problem of transferring tacit knowledge. We discuss what we learned about the tacit knowledge transfer problem and the evolution of laboratory packages in the description of a replication performed in Brazil using a PBR (Perspective Based Reading) laboratory package; also how further issues are addressed.

A Visual Text Mining approach for Systematic Reviews

The software engineering research community has been adopting systematic reviews as an unbiased and fair way to assess a research topic. Despite encouraging early results, a systematic review process can be time consuming and hard to conduct. Thus, tools that help on its planning or execution are needed. This article suggests the use of visual text mining (VTM) to aid systematic reviews. A feasibility study was conducted comparing the proposed approach with a manual process. We observed that VTM can contribute to systematic review and we propose a new strategy called VTM-Based systematic review.Commercial software development is a complex task that requires a thorough understanding of the architecture of the software system. We analyze the Windows Server 2003 operating system in order to assess the relationship between its software dependencies, churn measures and post-release failures. Our analysis indicates the ability of software dependencies and churn measures to be efficient predictors of post-release failures. Further, we investigate the relationship between the software dependencies and churn measures and their ability to assess failure-proneness probabilities at statistically significant levels.

Validation of an approach for improving existing measurement frameworks

Software organizations are in need of methods to understand, structure, and improve the data their are collecting. We have developed an approach for use when a large number of diverse metrics are already being collected by a software organization (M.G. Mendonca et al., 1998; M.G. Mendonca, 1997). The approach combines two methods. One looks at an organizations measurement framework in a top-down goal-oriented fashion and the other looks at it in a bottom-up data-driven fashion. The top-down method is based on a measurement paradigm called Goal-Question-Metric (GQM). The bottom-up method is based on a data mining technique called Attribute Focusing (AF). A case study was executed to validate this approach and to assess its usefulness in an industrial environment. The top-down and bottom-up methods were applied in the customer satisfaction measurement framework at the IBM Toronto Laboratory. The top-down method was applied to improve the customer satisfaction (CUSTSAT) measurement from the point of view of three data user groups. It identified several new metrics for the interviewed groups, and also contributed to better understanding of the data user needs. The bottom-up method was used to gain new insights into the existing CUSTSAT data. Unexpected associations between key variables prompted new business insights, and revealed problems with the process used to collect and analyze the CUSTSAT data. The paper uses the case study and its results to qualitatively compare our approach against current ad hoc practices used to improve existing measurement frameworks.

Identifying Code Smells with Multiple Concern Views

Code smells are anomalies often caused by the way concerns are realized in the source code. Their identification might depend on properties governing the structure of individual concerns and their inter-dependencies in the system implementation. Although code visualization tools are increasingly applied to support anomaly detection, they are mostly limited to represent modular structures, such as methods, classes and packages. This paper presents a multiple views approach that enriches four categories of code views with concern properties, namely: (i) concern’s package-class method structure, (ii) concern’s inheritance-wise structure, (iii)concern dependency, and (iv) concern dependency weight. An exploratory study was conducted to assess the extent to which visual views support code smell detection. Developers identified a set of well-known code smells on five versions of an open source system. Two important results came out of this study. First, the concern-driven views provided useful support to identify God Class and Divergent Change smells. Second, strategies for smell detection supported by the multiple concern views were uncovered.

Perspective-Based Reading: A Replicated Experiment Focused on Individual Reviewer Effectiveness

This paper describes a replication conducted to compare the effectiveness of inspectors using Perspective Based Reading (PBR) to the effectiveness of inspectors using a checklist. The goal of this replication was to better understand the complementary aspects of the PBR perspectives. To this end, a brief discussion of the original study is provided as well as a more detailed description of the replication. A detailed statistical analysis is then provided along with analysis of the PBR perspectives.For the individual PBR perspectives, we saw an interesting dichotomy: In the original study there was little overlap among the sets of defects found by each of the three perspectives, while in the replication two of the three perspectives found similar sets of defects on one of the two documents used in the study. Interestingly this document was the only case where the users of PBR were not more effective than the users of a checklist. This result leads to a new hypothesis that the complementary aspect of the PBR perspectives is the characteristic that provides the benefit over other defect detection techniques.

Software evolution visualization: A systematic mapping study

Background: Software evolution is an important topic in software engineering. It generally deals with large amounts of data, as one must look at whole project histories as opposed to their current snapshot. Software visualization is the field of software engineering that aims to help people to understand software through the use of visual resources. It can be effectively used to analyze and understand the large amount of data produced during software evolution. Objective: This study investigates Software Evolution Visualization (SEV) approaches, collecting evidence about how SEV research is structured, synthesizing current evidence on the goals of the proposed approaches and identifying key challenges for its use in practice. Methods: A mapping study was conducted to analyze how the SEV area is structured. Selected primary studies were classified and analyzed with respect to nine research questions. Results: SEV has been used for many different purposes, especially for change comprehension, change prediction and contribution analysis. The analysis identified gaps in the studies with respect to their goals, strategies and approaches. It also pointed out to a widespread lack of empirical studies in the area. Conclusion: Researchers have proposed many SEV approaches during the past years, but some have failed to clearly state their goals, tie them back to concrete problems, or formally validate their usefulness. The identified gaps indicate that there still are many opportunities to be explored in the area.

Identification and management of technical debt

ContextThe technical debt metaphor describes the effect of immature artifacts on software maintenance that bring a short-term benefit to the project in terms of increased productivity and lower cost, but that may have to be paid off with interest later. Much research has been performed to propose mechanisms to identify debt and decide the most appropriate moment to pay it off. It is important to investigate the current state of the art in order to provide both researchers and practitioners with information that enables further research activities as well as technical debt management in practice. ObjectiveThis paper has the following goals: to characterize the types of technical debt, identify indicators that can be used to find technical debt, identify management strategies, understand the maturity level of each proposal, and identify what visualization techniques have been proposed to support technical debt identification and management activities. MethodA systematic mapping study was performed based on a set of three research questions. In total, 100 studies, dated from 2010 to 2014, were evaluated. ResultsWe proposed an initial taxonomy of technical debt types, created a list of indicators that have been proposed to identify technical debt, identified the existing management strategies, and analyzed the current state of art on technical debt, identifying topics where new research efforts can be invested. ConclusionThe results of this mapping study can help to identify points that still require further investigation in technical debt research.

A Framework for Software Engineering Experimental Replications

Experimental replications are very important to the advancement of empirical software engineering. Replications are one of the key mechanisms to confirm previous experimental findings. They are also used to transfer experimental knowledge, to train people, and to expand a base of experimental evidence. Unfortunately, experimental replications are difficult endeavors. It is not easy to transfer experimental know-how and experimental findings. Based on our experience, this paper discusses this problem and proposes a Framework for Improving the Replication of Experiments (FIRE). The FIRE addresses knowledge sharing issues both at the intra-group (internal replications) and inter-group (external replications) levels. It encourages coordination of replications in order to facilitate knowledge transfer for lower cost, higher quality replications and more generalizable results.

An approach to improving existing measurement frameworks

Software organizations are in need of methods for understanding, structuring, and improving the data they are collecting. This paper discusses an approach for use when a large number of diverse metrics are already being collected by a software organization. The approach combines two methods. One looks at an organizations measurement framework in a top-down fashion and the other looks at it in a bottom-up fashion. The top-down method, based on the goal-question-metric (GQM) paradigm, is used to identify the measurement goals of data users. These goals are then mapped to the metrics being used by the organization, allowing us to: (1) identify which metrics are and are not useful to the organization, and (2) determine whether the goals of data user groups can be satisfied by the data that are being collected by the organization. The bottom-up method is based on a data mining technique called attribute focusing (AF). Our method uses this technique to identify useful information in the data that the data users were not aware of. We describe our experience in analyzing data from a software customer satisfaction survey at IBM to illustrate how the AF technique can be combined with the GQM paradigm to improve measurement and data use inside software organizations.

Extracting Information from Experimental Software Engineering Papers

Experiments have been conducted to investigate analysis, design, implementation, testing, maintenance, quality assurance and reuse techniques, but, a body of evidence has not yet been built that enables a project manager to know with confidence what software processes produce what product characteristics and under what conditions. This paper extends an approach we proposed earlier to extract information from papers so that systematically analyzing results from several papers is possible. It also describes an in-vitro experiment we did with graduate students to validate the approach. The results show that the approach is feasible and can be taught to less experienced researchers.