Daniela S. Cruzes

Research synthesis in software engineering

ContextComparing and contrasting evidence from multiple studies is necessary to build knowledge and reach conclusions about the empirical support for a phenomenon. Therefore, research synthesis is at the center of the scientific enterprise in the software engineering discipline. ObjectiveThe objective of this article is to contribute to a better understanding of the challenges in synthesizing software engineering research and their implications for the progress of research and practice. MethodA tertiary study of journal articles and full proceedings papers from the inception of evidence-based software engineering was performed to assess the types and methods of research synthesis in systematic reviews in software engineering. ResultsAs many as half of the 49 reviews included in the study did not contain any synthesis. Of the studies that did contain synthesis, two thirds performed a narrative or a thematic synthesis. Only a few studies adequately demonstrated a robust, academic approach to research synthesis. ConclusionWe concluded that, despite the focus on systematic reviews, there is limited attention paid to research synthesis in software engineering. This trend needs to change and a repertoire of synthesis methods needs to be an integral part of systematic reviews to increase their significance and utility for research and practice.

The evolution and impact of code smells: A case study of two open source systems

Code smells are design flaws in object-oriented designs that may lead to maintainability issues in the further evolution of the software system. This study focuses on the evolution of code smells within a system and their impact on the change behavior (change frequency and size). The study investigates two code smells, God Class and Shotgun Surgery, by analyzing the historical data over several years of development of two large scale open source systems. The detection of code smells in the evolution of those systems was performed by the application of an automated approach using detection strategies. The results show that we can identify different phases in the evolution of code smells during the system development and that code smell infected components exhibit a different change behavior. This information is useful for the identification of risk areas within a software system that need refactoring to assure a future positive evolution.

Are all code smells harmful? A study of God Classes and Brain Classes in the evolution of three open source systems

Code smells are particular patterns in object-oriented systems that are perceived to lead to difficulties in the maintenance of such systems. It is held that to improve maintainability, code smells should be eliminated by refactoring. It is claimed that classes that are involved in certain code smells are liable to be changed more frequently and have more defects than other classes in the code. We investigated the extent to which this claim is true for God Classes and Brain Classes, with and without normalizing the effects with respect to the class size. We analyzed historical data from 7 to 10 years of the development of three open-source software systems. The results show that God and Brain Classes were changed more frequently and contained more defects than other kinds of class. However, when we normalized the measured effects with respect to size, then God and Brain Classes were less subject to change and had fewer defects than other classes. Hence, under the assumption that God and Brain Classes contain on average as much functionality per line of code as other classes, the presence of God and Brain Classes is not necessarily harmful; in fact, such classes may be an efficient way of organizing code.

Understanding the High-Performance-Computing Community: A Software Engineer's Perspective

Computational scientists developing software for HPC systems face unique software engineering issues. Attempts to transfer SE technologies to this domain must take these issues into account.

The impact of global dispersion on coordination, team performance and software quality – A systematic literature review

Abstract Context Global software development (GSD) contains different context setting dimensions, which are essential for effective teamwork and success of projects. Although considerable research effort has been made in this area, as yet, no agreement has been reached about the impact of these dispersion dimensions on team coordination and project outcomes. Objective This paper summarizes empirical evidence on the impact of global dispersion dimensions on coordination, team performance and project outcomes. Method We performed a systematic literature review of 46 publications from 25 journals and 19 conference and workshop proceedings, which were published between 2001 and 2013. Thematic analysis was used to identify global dimensions and their measures. Vote counting was used to decide on the impact trends of dispersion dimensions on team performance and software quality. Results Global dispersion dimensions are consistently conceptualized, but quantified in many different ways. Different dispersion dimensions are associated with a distinct set of coordination challenges. Overall, geographical dispersion tends to have a negative impact on team performance and software quality. Temporal dispersion tends to have a negative impact on software quality, but its impact on team performance is inconsistent and can be explained by type of performance. Conclusion For researchers, we reveal several opportunities for future research, such as coordination challenges in inter-organizational software projects, impact of processes and practices mismatches on project outcomes, evolution of coordination needs and mechanism over time and impact of dispersion dimensions on open source project outcomes. For practitioners, they should consider the tradeoff between cost and benefits while dispersing tasks, alignment impact of dispersion dimensions with individual and organizational objectives, coordination mechanisms as situational approaches and collocation of development activities of high quality demand components in GSD projects.

Agile Team Perceptions of Productivity Factors

In this paper, we investigate agile team perceptions of factors impacting their productivity. Within this overall goal, we also investigate which productivity concept was adopted by the agile teams studied. We here conducted two case studies in the industry and analyzed data from two projects that we followed for six months. From the perspective of agile team members, the three most perceived factors impacting on their productivity were appropriate team composition and allocation, external dependencies, and staff turnover. Teams also mentioned pair programming and collocation as agile practices that impact productivity. As a secondary finding, most team members did not share the same understanding of the concept of productivity. While some known factors still impact agile team productivity, new factors emerged from the interviews as potential productivity factors impacting agile teams.

Synthesizing evidence in software engineering research

Synthesizing the evidence from a set of studies that spans many countries and years, and that incorporates a wide variety of research methods and theoretical perspectives, is probably the single most challenging task of performing a systematic review. In this paper, we perform a tertiary review to assess the types and methods of research synthesis in systematic reviews in software engineering. Almost half of the 31 studies included in our review did not contain any synthesis; of the ones that did, two thirds performed a narrative or a thematic synthesis. The results show that, despite the focus on systematic reviews, there is, currently, limited attention to research synthesis in software engineering. This needs to change and a repertoire of synthesis methods needs to be an integral part of systematic reviews to increase their significance and utility for research and practice.

Case studies synthesis: a thematic, cross-case, and narrative synthesis worked example

Case studies are largely used for investigating software engineering practices. They are characterized by their flexible nature, multiple forms of data collection, and are mostly informed by qualitative data. Synthesis of case studies is necessary to build a body of knowledge from individual cases. There are many methods for such synthesis, but they are yet not well explored in software engineering. The objective of this research is to demonstrate the similarities and differences of the results and conclusions when applying three different methods of synthesis, and to discuss the challenges of synthesizing evidence from reported case studies in SE. We describe a worked example of three such methods where three independent teams synthesized two studies that investigated critical factors of trust in outsourced projects through thematic synthesis and cross-case analysis, and compared these to each other and also to an already published narrative synthesis. In addition, despite that the primary studies were well presented for synthesis, we identified challenges in the use of case studies synthesis methods related to the goals and research questions of the synthesis, the types and number of case studies, variations in context, limited access to raw data, and quality of the case studies. Thus, we recommend that the analysts should be aware of these challenges and try to account for them during the execution of the synthesis. We also recommend that analysts consider using more than one method of synthesis for sake of reliability of the results and conclusions.

Challenges of applying ethnography to study software practices

Ethnography is about the adoption of a cultural lens to observe and interpret events, actions, and behaviors, ensuring that they are placed in a relevant and meaningful context. Using this approach, it is possible to capture and analyze software development practices. Our aims are to illustrate the use of an ethnographic approach in a case study of agile software development adoption, to discuss the methodological challenges involved, and to provide support to others who conduct ethnographic studies of software practice. An ethnographic case study was conducted, employing participant observation, interviews, and document analysis. Difficulties and decisions were recorded and compared with those encountered in the literature. Finally, key challenges and guidelines to tackle them were discussed and documented. We identified five key challenges of applying ethnography to the study of software practices: (a) working in collaboration with and having something to offer to the participating company; (b) the insider/outsider dynamic of participant observation; (c) the balance between participant listening and participant observation; (d) the researchers relationship with the participants; and (e) the rigor in qualitative work that involves the dilemma of the contextualization to be sufficiently broad and detailed. This study shows that ethnographic methods are indispensible when trying to understand software practice, and that the fundamental challenge for the researcher is to balance the role of participant observer with rigorous fieldwork.

A study of cyclic dependencies on defect profile of software components

Background: Empirical evidence shows that dependency cycles among software components are pervasive in real-life software systems, although such cycles are known to be detrimental to software quality attributes such as understandability, testability, reusability, build-ability and maintainability. Research goals: Can the use of extended object-oriented metrics make us better understand the relationships among cyclic related components and their defect-proneness? Approach: First, we extend such metrics to mine and classify software components into two groups - the cyclic and the non-cyclic ones. Next, we have performed an empirical study of six software applications. Using standard statistical tests on four different hypotheses, we have determined the significance of the defect profiles of both groups. Results: Our results show that most defects and defective components are concentrated in cyclic-dependent components, either directly or indirectly. Discussion and conclusion: These results have important implications for software maintenance and system testing. By identifying the most defect-prone set in a software system, it is possible to effectively allocate testing resources in a cost efficient manner. Based on these results, we demonstrate how additional structural properties could be collected to understand components defect proneness and aid decision process in refactoring defect-prone cyclic related components.