Ossi Taipale

Research Issues for Software Testing in the Cloud

Cloud computing is causing a paradigm shift in the provision and use of computing services, away from the traditional desktop form to online services. This implies that the manner in which these computing services are tested should also change. This paper discusses the research issues that cloud computing imposes on software testing. These issues were gathered during interviews with industry practitioners from eleven software organizations. The interviews were analyzed using qualitative grounded theory method. Findings of the study were compared with existing literature. The research issues were categorized according to application, management, legal and financial issues. By addressing these issues, researchers can offer reliable recommendation for practitioners in the industry.

Empirical Observations on Software Testing Automation

This study explores the factors that affect the use of software testing automation in different types of organizations. A case study was conducted in five organizations that develop and test technical software for the automation or telecommunication domains. The data was collected from interviews with managers, testers, and developers in each organization. The grounded theory was used as the data analysis method. According to the results, the factors which emphasize the wide use of testing automation include that the tested products are generic and independent of third-party systems. Low human involvement in testing, steady underlying technology, and reusability support the use of testing automation.

Software Testing as an Online Service: Observations from Practice

The objective of this qualitative study was to explore and understand the conditions that influence software testing as an online service and elicit important research issues. Interviews were conducted with managers from eleven organizations. The study used qualitative grounded theory as its research method. The results indicate that the demand for software testing as an online service is on the rise and is influenced by conditions such as the level of domain knowledge needed to effectively test an application, flexibility and cost effectiveness as benefits, security and pricing as top requirements, cloud computing as the delivery mode and the need for software testers to hone their skills. Potential research areas suggested include application areas best suited for online software testing, pricing and handling of test data among others.

Software test automation in practice: empirical observations

The objective of this industry study is to shed light on the current situation and improvement needs in software test automation. To this end, 55 industry specialists from 31 organizational units were interviewed. In parallel with the survey, a qualitative study was conducted in 12 selected software development organizations. The results indicated that the software testing processes usually follow systematic methods to a large degree, and have only little immediate or critical requirements for resources. Based on the results, the testing processes have approximately three fourths of the resources they need, and have access to a limited, but usually sufficient, group of testing tools. As for the test automation, the situation is not as straightforward: based on our study, the applicability of test automation is still limited and its adaptation to testing contains practical difficulties in usability. In this study, we analyze and discuss these limitations and difficulties.

Testing in the Cloud: Exploring the Practice

As applications and services migrate to the cloud, testing will follow the same trend. Therefore, organizations must understand the dynamics of cloud-based testing. This article presents interviews with eight organizations that use cloud computing. The results suggest that cloud computing can make testing faster and enhance the delivery of testing services. Cloud computing also highlights important aspects of testing that require attention, such as integration and interoperability. This article includes a Web extra that provides additional references for further study.

Improving software testing by observing practice

The objective of this qualitative study was to understand the complex practice of software testing, and based on this knowledge, to develop process improvement propositions that could concurrently reduce development and testing costs and improve software quality. First, a survey of testing practices was onducted and 26 organizational units (OUs) were interviewed. From this sample, five OUs were further selected for an in-depth case study. The study used grounded theory as its research method and the data was collected from 41 theme-based interviews. The analysis yielded improvement propositions that included enhanced testability of software components, efficient communication and interaction between development and testing, early involvement of testing, and risk-based testing. The connective and central improvement proposition was that testing ought to adapt to the business orientation of the OU. Other propositions were integrated around this central proposition. The results of this study can be used in improving development and testing processes.

A study on agility and testing processes in software organizations

In this paper, we studied the differences in testing activities between software organizations which apply agile development methods and organizations which take the traditional plan-driven approach. Our focus was on the concepts which allow the software organization to successfully apply agile development methods or plan-driven methods. We also observed the test process enhancements and hindrances, which originate in the selected development method. We interviewed 12 organizations, which were selected to represent different polar types of software production. The interviews were tape-recorded and transcribed for further analysis. The study yielded hypotheses which were derived by applying the qualitative grounded theory method. The results indicated that in practice, agile methods can improve the position of testing through the early involvement of testing activities in development, and also have a positive influence on end-product satisfaction. By applying these results, organizations can improve their processes and avoid pitfalls when transitioning to agile methods.

Analysis of Problems in Testing Practices

The objective of this qualitative study was to explore and understand the problems of software testing in practice and find improvement proposals for these issues. The study focused on organizational units that develop and test technical software for automation or telecommunication domains, for which a survey of testing practices was conducted and 26 organizations were interviewed. From this sample, five organizations were further selected for an in-depth grounded theory case study. The analysis yielded hypotheses indicating that a software project design should promote testability as architectural attribute and apply specialized personnel to enhance testing implementation. Testing tools should also be selected based on usability and configurability criteria. These results of this study can be used in developing the efficiency of software testing and in development of the testing strategy for organization.

Finding and ranking research directions for software testing

The percentage of software testing in software development is high; 50 % is often mentioned in the literature. Moreover, the amount of testing is increasing because of the demand for better quality. A survey was carried out to find out where software testing research should be directed. The experts of the participating industry and the research institutes evaluated the issues of testing research. The research method was a derivative of the Delphi method. The most prominent research issues in ranking included process improvement, testing automation with testing tools, and standardization of testing. The process and results of the survey are presented and the need for further research is discussed. This study also shows an example how project steering groups can be used in resolving research priorities.

Test case selection and prioritization: risk-based or design-based?

The objective of this qualitative study was to observe and empirically study how software organizations decide on which test cases to select for their software projects. As the software test processes are limited in resources such as time or money, a selection process usually exists for tested features. In this study we conducted a survey on 31 software-producing organizations, and interviewed 36 software professionals from 12 focus organizations to gain a better insight into testing practices. Our findings indicated that the basic approaches to test case selection are usually oriented towards two possible objectives. One is the risk-based selection, where the aim is to focus testing on those parts that are too expensive to fix after launch. The other is design-based selection, where the focus is on ensuring that the software is capable of completing the core operations it was designed to do. These results can then be used to develop testing organizations and to identify better practices for test case selection.