Takaji Fujiwara

TESTING-DOMAIN DEPENDENT SOFTWARE RELIABILITY GROWTH MODELS AND THEIR COMPARISONS OF GOODNESS-OF-FIT

A software developer has to test to verify the implemented functions based on its requirement specification. We use many various test-cases for testing. Then, there is a set of the modules and functions in the software system to be influenced by the executed test-cases. The set is called a testing-domain and it spreads with the progress of testing. The growth rate of testing-domain in the software system is closely related to the quality and quantity of the executed test-cases by testing. Further, the quality of test-cases is related to the testing-skill of test-case designers. In this paper, we discuss testing-domain dependent software reliability growth models. The models are formulated by a nonhomogeneous Poisson process. Further, we propose three kinds of testing-domain, i.e., the basic testing-domain, the testing-domain with skill-factor, and the testing-domain with imperfect debugging. Finally, these models are applied to fault data observed in actual development projects, the software reliability analysis results are shown, and the comparisons of goodness-of-fit with the conventional software reliability growth models are performed.

Towards quantitative software reliability assessment in incremental development processes

The iterative and incremental development is becoming a major development process model in industry, and allows us for a good deal of parallelism between development and testing. In this paper we develop a quantitative software reliability assessment method in incremental development processes, based on the familiar non-homogeneous Poisson processes. More specifically, we utilize the software metrics observed in each incremental development and testing, and estimate the associated software reliability measures. In a numerical example with a real incremental developmental project data, it is shown that the estimate of software reliability with a specific model can take a realistic value, and that the reliability growth phenomenon can be observed even in the incremental development scheme.

C0 COVERAGE-MEASURE AND TESTING-DOMAIN METRICS BASED ON A SOFTWARE RELIABILITY GROWTH MODEL

Generally, a software development manager is performing the testing-progress management by using various CASE tools. These CASE tools support testing activities of Windows applications or embedded software systems. Then, by using various testing coverage-measures, the testing-progress management can be recognized visually. However, in the system-testing phase, it is impossible to measure them of software systems which have been becoming large-scale and complicated recently. Therefore, the manager has to determine the testing termination time in consideration of the convergence situation of the cumulative number of detected faults and the prespecified delivery time to the users. These determination method have ambiguity based on the managers experience or intuition. In this paper, we investigate the relationship between the testing-domain rate derived from a testing-domain dependent software reliability growth model and the testing-path coverage. Then, we show that the testing-domain rate, which is defined as the increasing ratio of the testing-paths in the modules and functions in the software system to be influenced by executed test-cases, is useful as an alternative measure of testing-coverage metrics. Further, this model is applied to the fault data observed in an actual software project, and the comparisons of goodness-of-fit with the conventional software reliability growth models are performed. Finally, in numerical illustrations, we discuss the software reliability analysis and the usefulness of the testing-domain rate, and propose a new testing-path coverage rate.

A Software Accelerated Life Testing Model

Software system developed for a specific user under contract undergoes a period of testing by the user before acceptance. This is known as user acceptance testing and is useful to debug the software in the users operational circumstance. In this paper we first present a simple non-homogeneous Poisson process (NHPP)-based software reliability model to assess the quantitative software reliability under the user acceptance test, where the idea of an accelerated life testing model is introduced to represent the users operational phase and to investigate the impact of users acceptance test. This idea is applied to the reliability assessment of web applications in a different testing environment, where two stress tests with normal and higher workload conditions are executed in parallel. Through numerical examples with real software fault data observed in actual user acceptance and stress tests, we show the applicability of the software accelerated life testing model to two different software testing schemes.

A bootstrap software reliability assessment method to squeeze out remaining faults

This paper develops a bootstrap software reliability assessment method which can evaluate the number of remaining software faults at the final stage of the software testing process. The bootstrap method for reliability assessment problems has been already developed in the literature. However the method has a weak point which affects the applicability to the data set to be analyzed. We propose a new calculation formula in order to overcome this weak point. After showing the reliability assessment method by the traditional NHPP (nonhomogeneous Poisson process) models, we compare the performance of software reliability prediction with the bootstrap-based method by using a real software fault data set.

BIVARIATE EXTENSION OF SOFTWARE RELIABILITY MODELING WITH NUMBER OF TEST CASES

In this paper we consider a software reliability model (SRM) depending on the number of test cases executed in software testing. The resulting SRM is based on a two-dimensional discrete non-homogeneous Poisson process (NHPP) and is considered as a bivariate extension of the usual NHPP-based SRM by taking account of two time scales; calendar time and number of test cases executed. We apply the Marshall and Olkins bivariate geometric distribution and develop a two-dimensional discrete geometric SRM. In a numerical example with real software fault data observed in four real development projects, we investigate the goodness-of-fit for the proposed SRM and refer to an applicability to the actual software reliability assessment.

Software reliability growth modeling based on testing‐skill characteristics: Model and application

In order to test a software system, many test-cases are executed in the final stage of software development. Then, the set of modules and functions are influenced by the executed test-cases. The set is called the testing-domain and it expands with the progress of testing. The growth rate of the testing-domain in the software system is closely related to the quality and quantity of executed test-cases. Further, the quality of test-cases is related to the testing skill of test-case designers. In this paper, we discuss a software reliability growth model which describes the time-dependent growth behavior of the isolated testing-domain. This model based on a nonhomogeneous Poisson process is used for describing a software fault detection phenomenon during the software testing. Finally, this model is applied to fault data observed in actual development projects, the software reliability assessment results are shown, and comparisons of goodness-of-fit with existing software reliability growth models are performed.

A calculation method for software safety integrity level

In the functional safety standards (IEC 61508 and ISO/DIS 26262), development methods and quantitative analytical methods are defined for establishment of safety-related systems. However, only development methods are recommended to establish the software of safety-related systems. That is, the safety integrity level for software is determined only by the number of the development methods applied to practical safety-related system development. This is not reasonable to evaluate the safety integrity level, because various risk factors should be taken up. In this paper, we propose how to calculate the safety integrity level for software. Especially, we propose the calculation method based on the software reliability growth model that has long been used in the large-scale system development.

Evaluation and Application of MVFs in Coverage for Coverage-Based NHPP SRGM Frameworks

Many non-homogeneous Poisson process software reliability growth models are characterized by their mean value functions. Mean value functions of coverage-based models are usually obtained as composite functions of the coverage growth function and the function relating the number of detected faults to the coverage. This paper performs empirical evaluation of the relationships between the number of detected faults and the coverage embedded in the coverage- based software reliability growth models. It is also illustrated that integration of well-performing coverage growth functions and relationships between the number of detected faults and the coverage produces well-performing mean value functions.

INTEGRATION OF IMPERFECT DEBUGGING IN GENERAL TESTING-DOMAIN DEPENDENT NHPP SRGM

Recently the general testing-domain dependent NHPP SRGM is developed to reflect repeated execution of constructs and location of detected faults. It assumes that debugging is perfect. Since realistic models need to reflect imperfect debugging, this paper integrates imperfect debugging in the general testing-domain dependent NHPP SRGM. Differential equations representing the mean value function are first derived for general testing strategy and then realized for the uniform testing. Specific mean value functions are obtained for some selected fault detection rate functions and constant fault reduction rate. Finally empirical performance evaluation is fulfilled.