Karen Smiley

Applying regression test selection for COTS-based applications

ABB incorporates a variety of commercial-off-the-shelf (COTS) components in its products. When new releases of these components are made available for integration and testing, source code is often not provided. Various regression test selection processes have been developed and have been shown to be cost effectiveness. However, the majority of these test selection techniques rely on access to source code for change identification. In this paper we present the application of the lightweight Integrated - Black-box Approach for Component Change Identification (I-BACCI) Version 3 process that select regression tests for applications that use COTS components. Two case studies, examining a total of nine new component releases, were conducted at ABB on products written in C/C++ to determine the effectiveness of I-BACCI. The results of the case studies indicate this process can reduce the required number of regression tests at least 70% without sacrificing the regression fault exposure.

Regression Test Selection for Black-box Dynamic Link Library Components

Software products are often configured with commercial-off-the-shelf (COTS) components. When new releases of these components are made available for integration and testing, source code is usually not provided. Various regression test selection processes have been developed and have been shown to be cost effective. However, the majority of these test selection techniques rely on access to source code for change identification. Based on our prior work, we are studying the solution to regression testing COTS-based applications that incorporate components of dynamic link library (DLL) files. We evolved the Integrated - Black-box Approach for Component Change Identification (I-BACCI) process that selects regression tests for applications based upon static binary code analysis to Version 4 to support DLL components. A feasibility case study was conducted at ABB on products written in C/C++ to determine the effectiveness of the I-BACCI process. The results of the case study indicate this process can reduce the required number of regression tests by as much as 100% if our analysis indicates the changes to the component are not called by the glue code of the application using the COTS component. Similar to other regression test selection techniques, when there are many changes in the new component I-BACCI suggests a retest-all regression test strategy.

An initial study of a lightweight process for change identification and regression test selection when source code is not available

Various regression test selection techniques have been developed and have shown to improve testing cost effectiveness via improving efficiency. The majority of these test selection techniques rely on access to source code for change identification. However, when new releases of COTS components are made available for integration and testing, source code is often not available to guide in regression test selection. In this paper we describe a lightweight integrated-black-box approach for component change identification (I-BACCI) process for selection of regression tests for user/glue code that uses COTS components. I-BACCI is applicable when component licensing agreements do not preclude binary code analysis. A case study of the process was conducted on an ABB product that uses a medium-scale internal ABB software component. Six releases of the component were examined to evaluate the efficacy of the proposed process. The result of the case study indicates that this process can reduce the required regression tests by 40% on average

A lightweight process for change identification and regression test selection in using COTS components

Various regression test selection techniques have been developed and have shown fault detection effectiveness. The majority of these test selection techniques rely on access to source code for change identification. However, when new releases of COTS components are made available for integration and testing, source code is often not available. In this paper, we present a lightweight integrated-black-box approach for component change identification (I-BACCI) process for selection of regression tests for user/glue code that uses COTS components. I-BACCI is applicable when component licensing agreements do not preclude analysis of the binary files. A case study of the process was conducted on an ABB product that uses a medium-scale internal ABB software component. Five releases of the component were examined to evaluate the efficacy of the proposed process. The result of the case study indicates this process can reduce the required number of regression tests by 54% on average.

A process for identifying changes when source code is not available

Various regression test selection techniques have been developed and shown to improve fault detection effectiveness. The majority of these test selection techniques rely on access to source code for change identification. However, when new releases of COTS components are made available for integration and testing, source code is often not available to guide regression test selection. This paper describes a process for identifying changed functions when code is not available. This change information is beneficial for selecting white-box regression tests of customer/glue code. This process is applicable when COTS licensing agreements do not preclude decompilation. A feasibility study of the process was conducted with four releases of a medium-scale internal ABB product. The results of the feasibility study indicate that this process can be effective in identifying changed functions.

Causal Analysis of Factors Governing Collaboration in Global Software Development Teams

Globally distributed software development (GSD) is increasing in popularity in industry. However, as it is coupled with challenges of distance, time, and culture, it increases the importance of identifying and understanding the specific factors that enable and hinder GSD teams. This paper presents the approach and preliminary findings from an exploratory study of the enabling and inhibiting factors that affected several globally distributed projects in a large commercial organization. Our quantitative analysis includes grouping these factors to reduce the dimensional complexity, studying their underlying causal relationships, and identifying the most influential factors using factor analysis and structural equation modeling. The paper concludes by presenting preliminary findings, limitations, and directions for future work.

Global Software Project Management: A Case Study

Global software development (GSD) is a growing phenomenon in industry, including the ABB Group of companies, which has a long history of executing globally distributed development projects. Geographic and temporal separation, culturally-based misunderstandings, and language effects are well-described complications for GSD teams. These factors aggravate issues (on both a practical and a leadership level) in communication, trust, and coordination, impeding the effective sharing and management of team knowledge, and creating risks to project success. In the interest of continually improving our business performance, ABB has joined the research community in exploring these issues and ways to increase awareness and tactical support for GSD project managers. In this paper, we present aggregate findings from qualitative interviews with people across different sites in the organization, and describe how identifying, measuring, and actively managing GSD-related risks can help project managers and leaders in planning and executing projects more effectively.

Architectural requirements prioritization and analysis applied to software technology evaluation

In this short paper, we summarize an industrial project in which we developed and applied the Attribute Hierarchy-based Evaluation of Architectural Designs (AHEAD) method for selecting a software technology to form the basis for the next-generation architecture of a complex commercial software application. AHEAD leverages the Software Engineering Institutes Attribute-Driven Design (ADD) method and the Analytic Hierarchy Process (AHP) for evaluating software technologies that have important architectural impact. The core activities of AHEAD include elicitation, prioritization, and analysis of architectural requirements. The goal of these requirements activities was to establish and apply objective criteria for selecting, prototyping, and evaluating software technology alternatives. We found that using AHEAD brought greater objectivity to prioritization of architectural requirements and to the technical judgments of the software technology options.

Evolving an industrial analytics product line architecture

This paper focuses on an industrial experience with software product lines of analytics-enabled solutions, specifically the evolution of the software product line architecture for a Subject Matter Expert Workbench toolset which supports analytic plugins for multiple software product lines. As context, the toolset product line was intended for integration of expert knowledge into a family of industrial asset health applications at runtime. The toolset architecture is now being evolved to build and manage plugins for multiple Industrial Analytics solutions (software systems and services) beyond asset health. This evolution is driving changes in the desired architecture qualities of the toolset; widening the stakeholder pool and influencing priorities; affecting the architecture tradeoffs and decisions; and triggering updates to the product line architecture, the guidance for applying it, and the current prototype of the toolset. We describe our experiences in handling this evolution, assess lessons learned, and discuss potential relevance to other product line scenarios.