Adalberto Nobiato Crespo

Sensitivity of reliability-growth models to operational profile errors vs. testing accuracy [software testing]

This paper investigates: 1) the sensitivity of reliability-growth models to errors in the estimate of the operational profile (OP); and 2) the relation between this sensitivity and the testing accuracy for computer software. The investigation is based on the results of a case study in which several reliability-growth models are applied during the testing phase of a software system. The faults contained in the system are known in advance; this allows measurement of the software reliability-growth and comparison with the estimates provided by the models. Measurement and comparison are repeated for various OPs, thus giving information about the effect of a possible error in the estimate of the OP. The results show that: 1) the predictive accuracy of the models is not heavily affected by errors in the estimate of the OP; and 2) this relation depends on the accuracy with which the software system has been tested.

A binomial software reliability model based on coverage of structural testing criteria

A new approach to software reliability modeling is discussed where variables indirectly related with software reliability are used to provide additional information for the modeling process. Previous studies, empirical and theoretical evidences, and results from experiments indicate that there is a strong relationship between software reliability and coverage of program elements required to be exercised by structural testing criteria. This paper develops a binomial type coverage-based software reliability model through the definition of a coverage-based failure rate function. The Binomial software reliability Model Based on Coverage—BMBC—is proposed and discussed. In the BMBC test data between failures is used instead of time as independent variable; the model was assessed with test data from a real application, making use of the following structural testing criteria: all-nodes, all-edges, and potential-uses—a data-flow based family of testing criteria. The results from our experiments have shown that our modeling approach has some advantages over some traditional reliability models and points to a very promising research direction in software reliability.

Sensitivity of Two Coverage-Based Software Reliability Models to Variations in the Operational Profile

Software in field use may be utilized by users with diverse profiles. The way software is used affects the reliability perceived by its users, that is, software reliability may not be the same for different operational profiles. Two software reliability growth models based on structural testing coverage were evaluated with respect to their sensitivity to variations in operational profile. An experiment was performed on a real program (SPACE) with real defects, submitted to three distinct operational profiles. Distinction among the operational profiles was assessed by applying the Kolmogorov-Smirnov test. Testing coverage was measured according to the following criteria: all-nodes, all-arcs, all-uses, and all-potential-uses. Reliability measured for each operational profile was compared to the reliabilities estimated by the two models, estimated reliabilities were obtained using the coverage for the four criteria. Results from the experiment show that the predictive ability of the two models is not affected by variations in the operational profile of the program.

Applying Code Coverage Approach to an Infinite Failure Software Reliability Model

An approach to software reliability modeling based on code coverage is used to derive the Infinite Failure software reliability Model Based on Code Coverage - IFMBC. Our aim was to verify the soundness of the approach under different assumptions. The IFMBC was assessed with test data from a real application, making use of the following structural testing criteria: all-nodes, all-edges, and potential-uses - a data-flow based family of criteria. The IFMBC was shown to be as good as the Geometric Model - GEO, found to be the best traditional time-based model that fits the data. Results from the analysis also show that the IFMBC is as good as the BMBC - Binomial software reliability Model Based on Coverage - a model previously derived using the code coverage approach, indicating it to be effective under different modeling assumptions

Experimental Comparison of Software Reliability Models Based on Code Coverage and on Time Domain

The study of software reliability has its origin in 1967 when the first experiments were performed. Since then, diverse models that use as parameter the time of execution of software or the time between failures have been considered. Another approach indicates that the use of code coverage, instead of execution time, yields better estimates of the reliability of the software. This paper presents the results observed in an experiment carried out to compare the software reliability models Binomial Model based on Coverage (BMBC) and Infinite Failure Software Reliability Model Based on Code Coverage (IFMBC) with the following models based on time domain: Geometric (GEO), Littlewood-Linear Verral (LAV-L), Littlewood-Quadratic Verral (LAVQ), Musa Basic (MS-B), Musa Logarithmic (MS-L) and Non-Homogeneous Poisson Model (NHP). These models, in addition to being much quoted in the literature, are implemented by SMERFS^3, used to calculate its parameters and to estimate reliability. As a result, it has been observed that traditional models do not provide robustness when the profile was changed. On the other hand, code coverage-based models have produced good results for all operational profiles. In addition to the good fit, coverage-based models have generated better estimates of software reliability, as shown by the Kolmogorov-Smirnov test.