Adnan Causevic

Factors Limiting Industrial Adoption of Test Driven Development: A Systematic Review

Test driven development (TDD) is one of the basic practices of agile software development and both academia and practitioners claim that TDD, to a certain extent, improves the quality of the code produced by developers. However, recent results suggest that this practice is not followed to the extent preferred by industry. In order to pinpoint specific obstacles limiting its industrial adoption we have conducted a systematic literature review on empirical studies explicitly focusing on TDD as well as indirectly addressing TDD. Our review has identified seven limiting factors viz., increased development time, insufficient TDD experience/knowledge, lack of upfront design, domain and tool specific issues, lack of developer skill in writing test cases, insufficient adherence to TDD protocol, and legacy code. The results of this study is of special importance to the testing community, since it outlines the direction for further detailed scientific investigations as well as highlights the requirement of guidelines to overcome these limiting factors for successful industrial adoption of TDD.

An Industrial Survey on Contemporary Aspects of Software Testing

Software testing is a major source of expense in software projects and a proper testing process is a critical ingredient in the cost-efficient development of high-quality software. Contemporary aspects, such as the introduction of amore lightweight process, trends towards distributed development, and the rapid increase of software in embedded and safety-critical systems, challenge the testing process in unexpected manners. To our knowledge, there are very few studies focusing on these aspects in relation to testing as perceived by different contributors in the software development process. This paper qualitatively and quantitatively analyses data from an industrial questionnaire survey, with a focus on current practices and preferences on contemporary aspects of software testing. Specifically, the analysis focuses on perceptions of the software testing process in different categories of respondents. Categorization of respondents is based on safety-criticality, agility, distribution of development, and application domain. While confirming some of the commonly acknowledged facts, our findings also reveal notable discrepancies between preferred and actual testing practices. We believe continued research efforts are essential to provide guidelines in the adaptation of the testing process to take care of these discrepancies, thus improving the quality and efficiency of the software development.

Automated test generation using model checking: an industrial evaluation

In software development, testers often focus on functional testing to validate implemented programs against their specifications. In safety-critical software development, testers are also required to show that tests exercise, or cover, the structure and logic of the implementation. To achieve different types of logic coverage, various program artifacts such as decisions and conditions are required to be exercised during testing. Use of model checking for structural test generation has been proposed by several researchers. The limited application to models used in practice and the state space explosion can, however, impact model checking and hence the process of deriving tests for logic coverage. Thus, there is a need to validate these approaches against relevant industrial systems such that more knowledge is built on how to efficiently use them in practice. In this paper, we present a tool-supported approach to handle software written in the Function Block Diagram language such that logic coverage criteria can be formalized and used by a model checker to automatically generate tests. To this end, we conducted a study based on industrial use-case scenarios from Bombardier Transportation AB, showing how our toolbox CompleteTest can be applied to generate tests in software systems used in the safety-critical domain. To evaluate the approach, we applied the toolbox to 157 programs and found that it is efficient in terms of time required to generate tests that satisfy logic coverage and scales well for most of the programs.

Reuse with Software Components - A Survey of Industrial State of Practice

Software is often built from pre-existing, reusable components, but there is a lack of knowledge regarding how efficient this is in practice. In this paper we therefore present qualitative results from an industrial survey on current practices and preferences, highlighting differences and similarities between development with reusable components, development without reusable components, and development of components for reuse. Component reuse does happen, but the findings are still partly disappointing: currently, many potential benefits are not achieved. Still, the findings are encouraging: there are indeed good, reusable components properly verified and documented, and mature organizations who manage to reuse these components efficiently, e.g. by leveraging the previous component verification. We also find that replacing one component for another is not necessarily complicated and costly.

Test case quality in test driven development: A study design and a pilot experiment

Background: Test driven development, as a side-effect of developing software, will produce a set of accompanied test cases which can protect implemented features during code refactoring. However, recent research results point out that successful adoption of test driven development might be limited by the testing skills of developers using it. Aim: Main goal of this paper is to investigate if there is a difference between the quality of test cases created while using test-first and test-last approaches. Additional goal of this paper is to measure the code quality produced using test-first and test-last approaches. Method: A pilot study was conducted during the master level course on Software Verification & Validation at Malardalen University. Students were working individually on the problem implementation by being randomly assigned to a test-first or a test-last (control) group. Source code and test cases created by each participant during the study, as well as their answers on a survey questionnaire after the study, were collected and analysed. The quality of the test cases is analysed from three perspectives: (i) code coverage, (ii) mutation score and (iii) the total number of failing assertions. Results: The total number of test cases with failing assertions (test cases revealing an error in the code) was nearly the same for both test-first and test-last groups. This can be interpreted as “test cases created by test-first developers were as good as (or as bad as) test cases created by test-last developers”. On the contrary, solutions created by test-first developers had, on average, 27% less failing assertions when compared to solutions created by the test-last group. Conclusions: Though the study provided some interesting observations, it needs to be conducted as a fully controlled experiment wit

Impact of Test Design Technique Knowledge on Test Driven Development: A Controlled Experiment

Agile development approaches are increasingly being followed and favored by the industry. Test Driven Development (TDD) is a key agile practice and recent research results suggest that the successf ...

Interoperability in heterogeneous Low-Power Wireless Networks for Health Monitoring Systems

Ensuring interoperability in the future Internet of Things applications can be a challenging task, especially in mission-critical applications such as Health Monitoring Systems. Existing low-power wireless network architectures are designed in isolated networks, and ensure a satisfying level of performance in homogeneous networks. However, with co-existence of different low-power networks, the interoperability related problems arise. To bridge this gap in this paper, we study various protocol stacks (i.e., Bluetooth, Bluetooth Low Energy, IEEE 802.15.4, ZigBee, 6LoWPAN and IEEE 802.15.6), and explain their specific features. Furthermore, we provide a generic protocol stack design that facilitates multiple radios with different protocol stacks, regardless of being IP-based or non-IP-based networks. We see this approach as a possibility to enhance network performance in terms of reliability, timeliness, and security, while providing higher levels of scalability and connectivity.

Enablers and impediments for collaborative research in software testing: an empirical exploration

When it comes to industrial organizations, current collaboration efforts in software engineering research are very often kept in-house, depriving these organizations of the skills necessary to build independent collaborative research. The current trend, towards empirical software engineering research, requires certain standards to be established which would guide these collaborative efforts in creating a strong partnership that promote independent, evidence-based, software engineering research. This paper examines key enabling factors for an efficient and effective industry-academia collaboration in the software testing domain. A major finding of the research was that while technology is a strong enabler to better collaboration, it must be complemented with industrial openness to disclose research results and the use of a dedicated tooling platform. We use as an example an automated test generation approach that has been developed in the last two years collaboratively with Bombardier Transportation AB in Sweden.

Effects of Negative Testing on TDD: An Industrial Experiment

In our recent academic experiments, an existence of positive test bias, that is lack of negative test cases, was identified when a test driven development approach was used. At the same time, when ...

A Black-Box Approach to Latency and Throughput Analysis

To enable fast and reliable delivery of mobile text messages (SMS), special bidirectional protocols are often used. Measuring the achieved throughput and involved latency is however non-trivial, due to the complexity of these protocols. Modifying an existing system would incur too much of a risk, so instead a new tool was created to analyse the log files containing information about this traffic in a black-box fashion. When the produced raw data was converted into graphs, they gave new insights into the behaviour of both the protocols and the remote systems involved.