Reginaldo Ré

A comprehensive view of Hadoop research—A systematic literature review

Context: In recent years, the valuable knowledge that can be retrieved from petabyte scale datasets – known as Big Data – led to the development of solutions to process information based on parallel and distributed computing. Lately, Apache Hadoop has attracted strong attention due to its applicability to Big Data processing. Problem: The support of Hadoop by the research community has provided the development of new features to the framework. Recently, the number of publications in journals and conferences about Hadoop has increased consistently, which makes it difficult for researchers to comprehend the full body of research and areas that require further investigation. Solution: We conducted a systematic literature review to assess research contributions to Apache Hadoop. Our objective was to identify gaps, providing motivation for new research, and outline collaborations to Apache Hadoop and its ecosystem, classifying and quantifying the main topics addressed in the literature. Results: Our analysis led to some relevant conclusions: many interesting solutions developed in the studies were never incorporated into the framework; most publications lack sufficient formal documentation of the experiments conducted by authors, hindering their reproducibility; finally, the systematic review presented in this paper demonstrates that Hadoop has evolved into a solid platform to process large datasets, but we were able to spot promising areas and suggest topics for future research within the framework.

Minimizing stub creation during integration test of aspect-oriented programs

A problem related to the integration test of Object-Oriented programs is the order in which classes are integrated and tested. This problems also appears in Aspect-Oriented programs. The incremental integration strategy, which suggests that classes are tested first and then integrated to the aspects, is often proposed as the more adequate strategy to integrate classes and aspects. This work presents a study about ordering classes and aspects in Aspect-Oriented programming to minimize the number of stubs in integration test. A dependency type model among classes and aspects is defined considering the syntax constructions and the semantics of AspectJ. An algorithm for class ordering is adapted and applied to an AOP program and the result obtained is analyzed and discussed, showing that a more refined strategy than the plain incremental approach is better in several situations.

Social metrics included in prediction models on software engineering: a mapping study

Context: Previous work that used prediction models on Software Engineering included few social metrics as predictors, even though many researchers argue that Software Engineering is a social activity. Even when social metrics were considered, they were classified as part of other dimensions, such as process, history, or change. Moreover, few papers report the individual effects of social metrics. Thus, it is not clear yet which social metrics are used in prediction models and what are the results of their use in different contexts. Objective: To identify, characterize, and classify social metrics included in prediction models reported in the literature. Method: We conducted a mapping study (MS) using a snowballing citation analysis. We built an initial seed list adapting strings of two previous systematic reviews on software prediction models. After that, we conducted backward and forward citation analysis using the initial seed list. Finally, we visited the profile of each distinct author identified in the previous steps and contacted each author that published more than 2 papers to ask for additional candidate studies. Results: We identified 48 primary studies and 51 social metrics. We organized the metrics into nine categories, which were divided into three groups - communication, project, and commit-related. We also mapped the applications of each group of metrics, indicating their positive or negative effects. Conclusions: This mapping may support researchers and practitioners to build their prediction models considering more social metrics.

An Empirical Study of the Relation Between Strong Change Coupling and Defects Using History and Social Metrics in the Apache Aries Project

Change coupling is an implicit relationship observed when artifacts change together during software evolution. The literature leverages change coupling analysis for several purposes. For example, researchers discovered that change coupling is associated with software defects and reveals relationships between software artifacts that cannot be found by scanning code or documentation. In this paper, we empirically investigate the strongest change couplings from the Apache Aries project to characterize and identify their impact in software development. We used historical and social metrics collected from commits and issue reports to build classification models to identify strong change couplings. Historical metrics were used because change coupling is a phenomenon associated with recurrent co-changes found in the software history. In turn, social metrics were used because developers often interact with each other in issue trackers to accomplish the tasks. Our classification models showed high accuracy, with 70−99 % F-measure and 88−99 % AUC. Using the same set of metrics, we also predicted the number of future defects for the artifacts involved in strong change couplings. More specifically, we were able to predict 45.7 % of defects where these strong change couplings reoccurred in the post-release. These findings suggest that developers and projects managers should detect and monitor strong change couplings, because they can be associated with defects and tend to happen again in the subsequent release.

An exploratory study about the cross-project defect prediction: Impact of using different classification algorithms and a measure of performance in building predictive models

Predicting defects in software projects is a complex task, especially in the initial phases of software development because there are a few available data. The use of cross-project defect prediction is indicated in such situation because it enables to reuse data of similar projects. In order to find and group similar projects, this paper proposes the construction of cross-project prediction models using a measure of performance achieved through the application of classification algorithms. To do so, we studied the combined application of different algorithms of classification, of feature selection, and clustering data, applied to 1270 projects aiming to building different cross-project prediction models. In this study we concluded that Naive Bayes algorithm obtained the best performance, with 31.58 % of satisfactory predictions in 19 models created with its use. This proposal seems to be promise, once the local predictions considered satisfactory reached 31.58%, against 26.31 % of global predictions.

Using Structural Holes Metrics from Communication Networks to Predict Change Dependencies

Conway’s Law describes that software systems are structured according to the communication structures of their developers. These developers when working on a feature or correcting a bug commit together a set of source code artifacts. The analysis of these co-changes makes it possible to identify change dependencies between artifacts. Influenced by Conway’s Law, we hypothesize that Structural Hole Metrics (SHM) are able to identify strong and weak change coupling. We used SHM computed from communication networks to predict co-changes among files. Comparing SHM against process metrics using six well-known classification algorithms applied to Rails and Node.js projects, we achieved recall and precision values near 80% in the best cases. Mathews Correlation metric was used to verify if SHM was able to identify strong and weak co-changes. We also extracted rules to provide insights about the metrics using classification tree. To the best of our knowledge, this is the first study that investigated social aspects to predict change dependencies and the results obtained are very promising.

Predicting Change Propagation from Repository Information

Change propagation occurs when a change in an artifact leads to changes in other artifacts. Previous research has used frequency of past changes between artifacts and different types of artifacts coupling to build prediction models of change propagation. To improve the accuracy of the prediction, we explored the combination of different data from software development repository, such as change requests, communication data, and artifacts modifications. This information can capture different dimensions of software development, what can lead to improvements on the accuracy of the models. We conducted an empirical study in four open source projects, namely Cassandra, Camel, Hadoop, and Lucene. Classifiers were constructed for each pair of artifacts that change together to predict if the change propagation between two files occurs in a certain change request. The models obtained values of area under the curve (AUC) of 0.849 on average. Furthermore, the sensitivity (recall) obtained is almost 4 times higher (57.06% vs. 15.70%) when compared our models to a baseline model built using association rules. With a reduced number of false positives, the models could be used in practice to help developers during software evolution.

An Extensible Service for Experts Recommendation on Distributed Software Development Projects

A common challenge in Distributed software development (DSD) is the identification and ranking of experts that can provide help to team members on their tasks. This paper presents a survey conducted with distributed software developers to identify a set of requirements to improve collaboration among team members by providing experts location features. We submit an architecture that was proposed based on the survey results. Therefore, the developed architecture enables the addition of new mining and ranking methods in one search engine, also enabling the use of different ways to present the ranking.

A catalogue of stub and driver patterns to support integration testing of aspect-oriented programs

This work presents a pattern collection aiming at supporting the integration testing design of aspect-oriented software. Such patterns provide a reduction in the effort of implementing stubs and drivers required for the integration testing. Furthermore, guidelines to implement these software artifacts are proposed to support the testing of aspect-oriented programs.