Odd Petter N. Slyngstad

Development with Off-the-Shelf Components: 10 Facts

Empirical studies have revealed a discrepancy between academic theory and industrial practices regarding the selection and integration of commercial off-the-shelf and open source software components in software system development.

Reflections on conducting an international survey of software engineering

Component-based software engineering (CBSE) with commercial-off-the-shelf (COTS) or open source software (OSS) components are more and more frequently being used in industrial software development. We therefore need to issue experience-based guidelines for the evaluation, selection and integration of such components. We have performed a survey on industrial COTS/OSS development in three countries - Norway, Italy and Germany. Concrete survey results, e.g. on risk management policies and process tailoring, are not being described here, but in other papers. This is a method paper, reporting on the challenges, approaches and experiences gained by conducting the main survey. The main contributions are as follows: At best we can achieve a stratified-random sample of ICT companies, followed by a convenience sample of relevant projects. This is probably the ftirst software engineering survey using census type data, and has revealed that the entire sampling and contact process can be unexpectedly expensive. It is also hard to avoid national variations in the total process, possibly leading to uncontrollable method biases

An empirical study on decision making in off-the-shelf component-based development

Component-based software development (CBSD) is becoming more and more important since it promotes reuse to higher levels of abstraction. As a consequence, many components are available being either open-source software (OSS) or commercial-off-the-shelf (COTS). However, it is still unclear how the decision for acquiring OSS or COTS components is made in practice. This paper describes an empirical study on why project decision-makers selected COTS instead of OSS components, or vice versa. The study was performed as an international survey in Norway, Italy and Germany. It focused on decision making on using off-the-shelf (OTS) components. We have gathered answers from 83 projects using only COTS components and 44 projects using only OSS components. Results of this study show significant differences and commonalities of integrating OSS or COTS components. Moreover, the study illustrates several research questions that warrant future research.

Validation of New Theses on Off-the-Shelf Component Based Development

Using OTS (Off-The-Shelf) components in software development has become increasingly popular in the IT industry. OTS components can be either COTS (Commercial-Off-The-Shelf), or OSS (Open-Source-Software) components. A recent study with seven structured interviews concluded with six theses, which contradicted widely accepted (or simply undisputed) insight. Since the sample size of that study was very small, it is necessary to investigate these theses in a larger and randomized sample. A state-of-the-practice survey in three countries — Norway, Italy, and Germany — has been performed to validate these new theses. Data from 133 OTS component-based projects has been collected. Results of this survey support four and contradict two of the initial theses. The supported theses are: OSS components were mainly used without modification in practice; custom code mainly provided additional functionality; formal OTS selection processes were seldom used; OTS component users managed to get required changes from vendors. The unsupported theses are: standard mismatches were more frequent than architecture mismatches; OTS components were mainly selected based on architecture compliance instead of function completeness.

The Impact of Test Driven Development on the Evolution of a Reusable Framework of Components An Industrial Case Study

Test driven development (TDD) is a software engineering technique to promote fast feedback, task-oriented development, improved quality assurance and more comprehensible low-level software design. Benefits have been shown for non-reusable software development in terms of improved quality (e.g. lower defect density). We have carried out an empirical study of a framework of reusable components, to see whether these benefits can be shown for reusable components. The framework is used in building new applications and provides services to these applications during runtime. The three first versions of this framework were developed using traditional test-last development, while for the two latest versions TDD was used. Our results show benefits in terms of reduced mean defect density (35.86%), when using TDD, over two releases. Mean change density was 76.19% lower for TDD than for test-last development. Finally, the change distribution for the TDD approach was 33.3% perfective, 5.6% adaptive and 61.1% preventive.

Preliminary results from a state-of-the-practice survey on risk management in off-the-shelf component-based development

Software components, both Commercial-Off-The-Shelf and Open Source, are being increasingly used in software development. Previous studies have identified typical risks and related risk management strategies for what we will call OTS-based (Off-the-Shelf) development. However, there are few effective and well-proven guidelines to help project managers to identify and manage these risks. We are performing an international state-of-the-practice survey in three countries – Norway, Italy, and Germany – to investigate the relative frequency of typical risks, and the effect of the corresponding risk management methods. Preliminary results show that risks concerning changing requirements and effort estimation are the most frequent risks. Risks concerning traditional quality attributes such as reliability and security of OTS component seem less frequent. Incremental testing and strict quality evaluation have been used to manage the possible negative impact of poor component quality. Realistic effort estimation on OTS quality evaluation helped to mitigate the possible effort estimation biases in OTS component selection and integration.

An empirical study on off-the-shelf component usage in industrial projects

Using OTS (Off-The-Shelf) components in software projects has become increasing popular in the IT industry. After project managers opt for OTS components, they can decide to use COTS (Commercial-Off-The-Shelf) components or OSS (Open Source Software) components instead of building these themselves. This paper describes an empirical study on why project decision-makers use COTS components instead of OSS components, or vice versa. The study was performed in form of an international survey on motivation and risks of using OTS components, conducted in Norway, Italy and Germany. We have currently gathered data on 71 projects using only COTS components and 39 projects using only OSS components, and 5 using both COTS and OSS components. Results show that both COTS and OSS components were used in small, medium and large software houses and IT consulting companies. The overall software system also covers several application domains. Both COTS and OSS were expected to contribute to shorter time-to-market, less development effort and the application of newest technology. However, COTS users believe that COTS component should have good quality, technical support, and will follow the market trend. OSS users care more about the free ownership and openness of the source code. Projects using COTS components had more difficulties in estimating selection effort, following customer requirement changes, and controlling the components negative effect on system security. On the other hand, OSS user had more difficulties in getting the support reputation of OSS component providers.

A Case Study of Defect-Density and Change-Density and their Progress over Time

We have performed an empirical case study, investigating defect-density and change-density of a reusable framework compared with one application reusing it over time at a large Oil and Gas company in Norway, Statoil ASA. The framework, called JEF, consists of seven components grouped together, and the application, called DCF, reuses the framework, without modifications to the framework. We analyzed all trouble reports and change requests from three releases of both. Change requests in our study covered any changes (not correcting defects) in the requirements, while trouble reports covered any reported defects. Additionally, we have investigated the relation between defect-density and change-density both for the reusable JEF framework and the application. The results revealed that the defect-density of the reusable framework was lower than the application. The JEF framework had higher change-density in the first release, but lower change-density than the DCF application over the successive releases. For the DCF application, on the other hand, a slow increase in change-density appeared. On the relation between change-density and defect-density for the JEF framework, we found a decreasing defect-density and change-density. The DCF application here showed a decreasing defect-density, with an increasing change-density. The results show that the quality of the reusable framework improves and it becomes more stable over several releases, which is important for reliability of the framework and assigning resources

Barriers to disseminating off-the-shelf based development theories to IT industry

In this position paper, we have reported results of an industrial seminar. The seminar was intended to show our findings in an international survey, conducted in Norway, Italy and Germany, on off-the-shelf component-based development. Discussion in the second section of the seminar revealed several obstacles of popularizing the OTS based development theories into IT industry.