Hussan Munir

Considering rigor and relevance when evaluating test driven development: A systematic review

Context: Test driven development (TDD) has been extensively researched and compared to traditional approaches (test last development, TLD). Existing literature reviews show varying results for TDD. Objective: This study investigates how the conclusions of existing literature reviews change when taking two study quality dimension into account, namely rigor and relevance. Method: In this study a systematic literature review has been conducted and the results of the identified primary studies have been analyzed with respect to rigor and relevance scores using the assessment rubric proposed by Ivarsson and Gorschek 2011. Rigor and relevance are rated on a scale, which is explained in this paper. Four categories of studies were defined based on high/low rigor and relevance. Results: We found that studies in the four categories come to different conclusions. In particular, studies with a high rigor and relevance scores show clear results for improvement in external quality, which seem to come with a loss of productivity. At the same time high rigor and relevance studies only investigate a small set of variables. Other categories contain many studies showing no difference, hence biasing the results negatively for the overall set of primary studies. Given the classification differences to previous literature reviews could be highlighted. Conclusion: Strong indications are obtained that external quality is positively influenced, which has to be further substantiated by industry experiments and longitudinal case studies. Future studies in the high rigor and relevance category would contribute largely by focusing on a wider set of outcome variables (e.g. internal code quality). We also conclude that considering rigor and relevance in TDD evaluation is important given the differences in results between categories and in comparison to previous reviews.

Open innovation in software engineering: a systematic mapping study

Open innovation (OI) means that innovation is fostered by using both external and internal influences in the innovation process. In software engineering (SE), OI has existed for decades, while we currently see a faster and broader move towards OI in SE. We therefore survey research on how OI takes place and contributes to innovation in SE. This study aims to synthesize the research knowledge on OI in the SE domain. We launched a systematic mapping study and conducted a thematic analysis of the results. Moreover, we analyzed the strength of the evidence in the light of a rigor and relevance assessment of the research. We identified 33 publications, divided into 9 themes related to OI. 17/33 studies fall in the high–rigor/high–relevance category, suggesting the results are highly industry relevant. The research indicates that start-ups have higher tendency to opt for OI compared to incumbents. The evidence also suggests that firms assimilating knowledge into their internal R&D activities, have higher likelihood of gaining financial advantages. We concluded that OI should be adopted as a complementary approach to facilitate internal innovation and not to substitute it. Further research is advised on situated OI strategies and the interplay between OI and agile practices.

Evaluating the Governance Model of Hardware-Dependent Software Ecosystems – A Case Study of the Axis Ecosystem

Ecosystem governance becomes gradually more relevant for a set of companies or actors characterized by symbiotic relations evolved on the top of a technological platform, i.e. a software ecosystem. In this study, we focus on the governance of a hardware-dependent software ecosystem. More specifically, we evaluate the governance model applied by Axis, a network video and surveillance camera producer, that is the platform owner and orchestrator of the Application Development Partner (ADP) software ecosystem. We conduct an exploratory case study col- lecting data from observations and interviews and apply the governance model for prevention and improvement of the software ecosystem health proposed by Jansen and Cusumano. Our results reveal that although the governance actions do not address the majority of their governance model, the ADP ecosystem is considered a growing ecosystem provid- ing opportunities for its actors. This can be explained by the fact that Axis, as the orchestrator and the platform owner, does not address the productivity and robustness of the ecosystem adequately, but has a net- work of vendors and resellers to support it and some of the governance activities (e.g. communication) are achieved by non-formal means. The current governance model does not take into consideration.

Open innovation using open source tools: a case study at Sony Mobile

Despite growing interest of Open Innovation (OI) in Software Engineering (SE), little is known about what triggers software organizations to adopt it and how this affects SE practices. OI can be realized in numerous of ways, including Open Source Software (OSS) involvement. Outcomes from OI are not restricted to product innovation but also include process innovation, e.g. improved SE practices and methods. This study explores the involvement of a software organization (Sony Mobile) in OSS communities from an OI perspective and what SE practices (requirements engineering and testing) have been adapted in relation to OI. It also highlights the innovative outcomes resulting from OI. An exploratory embedded case study investigates how Sony Mobile use and contribute to Jenkins and Gerrit; the two central OSS tools in their continuous integration tool chain. Quantitative analysis was performed on change log data from source code repositories in order to identify the top contributors and triangulated with the results from five semi-structured interviews to explore the nature of the commits. The findings of the case study include five major themes: i) The process of opening up towards the tool communities correlates in time with a general adoption of OSS in the organization. ii) Assets not seen as competitive advantage nor a source of revenue are made open to OSS communities, and gradually, the organization turns more open. iii) The requirements engineering process towards the community is informal and based on engagement. iv) The need for systematic and automated testing is still in its infancy, but the needs are identified. v) The innovation outcomes included free features and maintenance, and were believed to increase speed and quality in development. Adopting OI was a result of a paradigm shift of moving from Windows to Linux. This shift enabled Sony Mobile to utilize the Jenkins and Gerrit communities to make their internal development process better for its software developers and testers.

Requirements engineering in open innovation: a research agenda

In recent years Open Innovation (OI) has gained much attention and made firms aware that they need to consider the open environment surrounding them. To facilitate this shift Requirements Engineering (RE) needs to be adapted in order to manage the increase and complexity of new requirements sources as well as networks of stakeholders. In response we build on and advance an earlier proposed software engineering framework for fostering OI, focusing on stakeholder management, when to open up, and prioritization and release planning. Literature in open source RE is contrasted against recent findings of OI in software engineering to establish a current view of the area. Based on the synthesized findings we propose a research agenda within the areas under focus, along with a framing-model to help researchers frame and break down their research questions to consider the different angles implied by the OI model.

An experimental evaluation of test driven development vs. test-last development with industry professionals

Test-Driven Development (TDD) is a software development approach where test cases are written before actual development of the code in iterative cycles. Context: TDD has gained attention of many software practitioners during the last decade since it has contributed several benefits to the software development process. However, empirical evidence of its dominance in terms of internal code quality, external code quality and productivity is fairly limited. Objective: The aim behind conducting this controlled experiment with professional Java developers is to see the impact of Test-Driven Development (TDD) on internal code quality, external code quality and productivity compared to Test-Last Development (TLD). Results: Experiment results indicate that values found related to number of acceptance test cases passed, McCabes Cyclomatic complexity, branch coverage, number of lines of code per person hours, number of user stories implemented per person hours are statistically insignificant. However, static code analysis results were found statistically significant in the favor of TDD. Moreover, the results of the survey revealed that the majority of developers in the experiment prefer TLD over TDD, given the lesser required level of learning curve as well as the minimum effort needed to understand and employ TLD compared to TDD.

A Survey on the Perception of Innovation in a Large Product-focused Software Organization

Context. Innovation is promoted in companies to help them stay competitive. Four types of innovation are defined: product, process, business, and organizational. Objective. We want to understand the perception of the innovation concept in industry, and particularly how the innovation types relate to each other. Method. We launched a survey at a branch of a multi-national corporation. Results. From a qualitative analysis of the 229 responses, we see that the understanding of the innovation concept is somewhat narrow, and mostly related to product innovation. A majority of respondents indicate that product innovation triggers process, business, and organizational innovation, rather than vice versa. However, there is a complex interdependency between the types. We also identify challenges related to each of the types. Conclusion. Increasing awareness and knowledge of different types of innovation, may improve the innovation. Further, they cannot be handled one by one, but in their interdependent relations.

Software testing in open innovation: an exploratory case study of the acceptance test harness for jenkins

Open Innovation (OI) has gained significant attention since the term was introduced in 2003. However, little is known whether general software testing processes are well suited for OI. An exploratory case study on the Acceptance Test Harness (ATH) is conducted to investigate OI testing activities of Jenkins. As far as the research methodology is concerned, we extracted the change log data of ATH followed by five interviews with key contributors in the development of ATH. The findings of the study are threefold. First, it highlights the key stakeholders involved in the development of ATH. Second, the study compares the ATH testing activities with ISO/IEC/IEEE testing process and presents a tailored process for software testing in OI. Finally, the study underlines some key challenges that software intensive organizations face while working with the testing in OI.

Motivating the contributions: An Open Innovation perspective on what to share as Open Source Software

Abstract Open Source Software (OSS) ecosystems have reshaped the ways how software-intensive firms develop products and deliver value to customers. However, firms still need support for strategic product planning in terms of what to develop internally and what to share as OSS. Existing models accurately capture commoditization in software business, but lack operational support to decide what contribution strategy to employ in terms of what and when to contribute. This study proposes a Contribution Acceptance Process (CAP) model from which firms can adopt contribution strategies that align with product strategies and planning. In a design science influenced case study executed at Sony Mobile, the CAP model was iteratively developed in close collaboration with the firm’s practitioners. The CAP model helps classify artifacts according to business impact and control complexity so firms may estimate and plan whether an artifact should be contributed or not. Further, an information meta-model is proposed that helps operationalize the CAP model at the organization. The CAP model provides an operational OI perspective on what firms involved in OSS ecosystems should share, by helping them motivate contributions through the creation of contribution strategies. The goal is to help maximize return on investment and sustain needed influence in OSS ecosystems.

A theory of openness for software engineering tools in software organizations

Abstract Context The increased use of Open Source Software (OSS) affects how software-intensive product development organizations (SIPDO) innovate and compete, moving them towards Open Innovation (OI). Specifically, software engineering tools have the potential for OI, but require better understanding regarding what to develop internally and what to acquire from outside the organization, and how to cooperate with potential competitors. Aim This paper aims at synthesizing a theory of openness for software engineering tools in SIPDOs, that can be utilized by managers in defining more efficient strategies towards OSS communities. Method We synthesize empirical evidence from a systematic mapping study, a case study, and a survey, using a narrative method. The synthesis method entails four steps: (1) Developing a preliminary synthesis, (2) Exploring the relationship between studies, (3) Assessing the validity of the synthesis, and (4) Theory formation. Result We present a theory of openness for OSS tools in software engineering in relation to four constructs: (1) Strategy, (2) Triggers, (3) Outcomes, and (4) Level of openness. Conclusion The theory reasons that openness provides opportunities to reduce the development cost and development time. Furthermore, OI positively impacts on the process and product innovation, but it requires investment by organizations in OSS communities. By betting on openness, organizations may be able to significantly increase their competitiveness.