Stephen R. Schach

Validation of the coupling dependency metric as a predictor of run-time failures and maintenance measures

The coupling dependency metric (CDM) is a successful design quality metric. Here we apply it to four case studies: run-time failure data for a COBOL registration system; maintenance data for a C text-processing utility; maintenance data for a C++ patient collaborative care system; and maintenance data for a Java electronic file transfer facility. CDM outperformed a wide variety of competing metrics in predicting run-time failures and a number of different maintenance measures. These results imply that coupling metrics may be good predictors of levels of interaction within a software product.

Determining the Distribution of Maintenance Categories: Survey versus Measurement

In 1978, Lientz, Swanson, and Tompkins published the results of a survey on software maintenance. They found that 17.4% of maintenance effort was categorized as corrective in nature, 18.2% as adaptive, 60.3% as perfective, and 4.1% was categorized as other. We refer to this as the “LST” result. We contrast this survey-based result with our empirical results from the analysis of data for the repeated maintenance of three software products: a commercial real-time product, the Linux kernel, and GCC. For all three products and at both levels of granularity we considered, our observed distributions of maintenance categories were statistically very highly significantly different from LST. In particular, corrective maintenance was always more than twice the LST value. For the summed data, the percentage of corrective maintenance was more than three times the LST value. We suggest various explanations for the observed differences, including inaccuracies on the part of the maintenance managers who responded to the LST survey.

Open-Source Change Logs

A recent editorial in Empirical Software Engineering suggested that open-source software projects offer a great deal of data that can be used for experimentation. These data not only include source code, but also artifacts such as defect reports and update logs. A common type of update log that experimenters may wish to investigate is the ChangeLog, which lists changes and the reasons for which they were made. ChangeLog files are created to support the development of software rather than for the needs of researchers, so questions need to be asked about the limitations of using them to support research. This paper presents evidence that the ChangeLog files provided at three open-source web sites were incomplete. We examined at least three ChangeLog files for each of three different open-source software products, namely, GNUJSP, GCC-g++, and Jikes. We developed a method for counting changes that ensures that, as far as possible, each individual ChangeLog entry is treated as a single change. For each ChangeLog file, we compared the actual changes in the source code to the entries in the ChangeLog> file and discovered significant omissions. For example, using our change-counting method, only 35 of the 93 changes in version 1.11 of Jikes appear in the ChangeLog file—that is, over 62% of the changes were not recorded there. The percentage of omissions we found ranged from 3.7 to 78.6%. These are significant omissions that should be taken into account when using ChangeLog files for research. Before using ChangeLog files as a basis for research into the development and maintenance of open-source software, experimenters should carefully check for omissions and inaccuracies.

A maintenance-oriented approach to software construction

SUMMARY Maintenance is not only performed after a software product has been delivered to the client. On the contrary, the requirements frequently change during development, thereby necessitating reconstruction of the artifacts that have been developed to date. In this paper we present a process for software construction that recognizes maintenance as an essential aspect of the entire life cycle of the software product, considered from the very first steps of the initial development. The process may be used in conjunction with any software development methodology. Our process consists of two components: A procedure that is uniformly applied at every step of the chosen methodology, whether development or maintenance; and a data structure, the propagation graph, which is updated at every step. When requirements change, the propagation graph is used to determine which artifacts of the software product are impacted by the change in requirements. From the viewpoint of changes in requirements, the process treats development as a special case of software maintenance.  2000 by John Wiley & Sons, Ltd.

Testing: principles and practice

Testing is the process of determining whether a task has been correctly carried out. Testing should be performed throughout the software life cycle; it should not be restricted to verification (testing whether a phase has been carried out correctly) and validation (testing whether the completed product satisfies its specifications). The correction of a fault exposed by testing is termed debugging. The goal of testing is to reveal faults. There are two types of testing: execution-based testing and nonexecutionbased testing. It is impossible to execute a specification document; instead, it has to be reviewed carefully. Once executable code has been written it is possible to run test cases, that is, perform execution-based testing. Nevertheless, the existence of code does not preclude nonexecution-based testing because carefully reviewing code will uncover at least as many faults as running test cases. In this survey, the principles and practice of both execution-based and nonexecution-based testing are described.

Quality Impacts of Clandestine Common Coupling

The increase in maintenance of software and the increased amounts of reuse are having major positive impacts on the quality of software, but are also introducing some rather subtle negative impacts on the quality. Instead of talking about existing problems (faults), developers now discuss “potential problems,” that is, aspects of the program that do not affect the quality initially, but could have deleterious consequences when the software goes through some maintenance or reuse. One type of potential problem is that of common coupling, which unlike other types of coupling can be clandestine. That is, the number of instances of common coupling between a module M and the other modules can be changed without any explicit change to M. This paper presents results from a study of clandestine common coupling in 391 versions of Linux. Specifically, the common coupling between each of 5332 kernel modules and the rest of the product as a whole was measured. In more than half of the new versions, a change in common coupling was observed, even though none of the modules themselves was changed. In most cases where this clandestine common coupling was observed, the number of instances of common coupling increased. These results provide yet another reason for discouraging the use of common coupling in software products.

Maintainability of the kernels of open-source operating systems: A comparison of Linux with FreeBSD, NetBSD, and OpenBSD

We compared and contrasted the maintainability of four open-source operating systems: Linux, FreeBSD, NetBSD, and OpenBSD. We used our categorization of common coupling in kernel-based software to highlight future maintenance problems. An unsafe definition is a definition of a global variable that can affect a kernel module if that definition is changed. For each operating system we determined a number of measures, including the number of global variables, the number of instances of global variables in the kernel and overall, as well as the number of unsafe definitions in the kernel and overall. We also computed the value of each our measures per kernel KLOC and per KLOC overall. For every measure and every ratio, Linux compared unfavorably with FreeBSD, NetBSD, and OpenBSD. Accordingly, we are concerned about the future maintainability of Linux.

Measuring the maintainability of open-source software

An editorial in empirical software engineering suggested that open-source software projects offer a great deal of data that can be used for experimentation. These data include artifacts such as source code and defect reports. In this paper we show that sources of open-source maintenance data, such as defect-tracking systems, change logs, and source code, cannot, in general, be used for measuring maintainability. We further show that approaches such as using defect distributions and the average lag time to fix a defect can be equally unusable. We conclude that, despite the plethora of open-source maintenance data, it is extremely hard to find data for determining the maintainability of open-source software.

Quantitatively measuring object-oriented couplings

One key to several quality factors of software is the way components are connected. Software coupling can be used to estimate a number of quality factors, including maintainability, complexity, and reliability. Object-oriented languages are designed to reduce the number of dependencies among classes, which encourages separation of concerns and should reduce the amount of coupling. At the same time, the object-oriented language features change the way the connections are made, how they must be analyzed, and how they are measured. This paper discusses software couplings based on object-oriented relationships between classes, specifically focusing on types of couplings that are not available until after the implementation is completed, and presents a static analysis tool that measures couplings among classes in Java packages. Data from evaluating the tool on several open-source projects are provided. The coupling measurement is based on source code, which has the advantage of being quantitative and more precise than previous measures, but the disadvantage of not being available before implementation, and thus not useful for some predictive efforts.

Constraint animation using an object-oriented declarative language

Prototypes can be an effective way of interacting with an end-user to validate that the users requirements have been correctly captured. In the formal methods community, specification animation has been investigated as a way of creating a kind of prototype that is generated from a formal specification. Enriching UML diagrams with OCL constraints can provide the formality that is needed to animate the diagrams without the need for a more rigorous formal specification language. This paper provides an overview of issues concerning specification animation and describes an initial attempt at an animation environment for UML/OCL. We translate the UML/OCL into an object-oriented declarative language, Prolog++, and utilize a primitive animation environment that allows both a developer and client to explore the validity of the specification. In particular, in this paper we focus on animating the effect of constraints.