Sylvia B. Sheppard

Measuring the Psychological Complexity of Software Maintenance Tasks with the Halstead and McCabe Metrics

Three software complexity measures (Halsteads E, McCabes u(G), and the length as measured by number of statements) were compared to programmer performance on two software maintenance tasks. In an experiment on understanding, length and u(G) correlated with the percent of statements correctly recalled. In an experiment on modification, most significant correlations were obtained with metrics computed on modified rather than unmodified code. All three metrics correlated with both the accuracy of the modification and the time to completion. Relationships in both experiments occurred primarily in unstructured rather than structured code, and in code with no comments. The metrics were also most predictive of performance for less experienced programmers. Thus, these metrics appear to assess psychological complexity primarily where programming practices do not provide assistance in understanding the code.

Experimental evaluation of software documentation formats

Abstract Four controlled experiments investigated the effectiveness of different documentation formats for presenting information about computer programs. Nine different documentation formats (some quite novel) were created by varying three different forms of symbology (natural language, constrained language, and ideograms) and three different spatial arrangements (sequential, branching, and hierarchical). Professional programmers used these formats as aids to comprehend (Experiment 1), code (Experiment 2), debug (Experiment 3), and modify (Experiment 4) modular-sized programs. For each programming task, a model of the cognitive activities a programmer would perform generated hypotheses about the effectiveness of different formats. Natural language was found to be less effective in assisting most of the tasks studied than a constrained language or ideograms. A smaller effect was observed (less frequently than expected) for the spatial arrangment in situations where control flow information aided the task. The largest effect in the experiments, individual differences among the participants, accounted for between a third and one-half of the variation in performance.

First-year results from a research program on human factors in software engineering

For the past two years the Software Management Research U nit at General Electric has been investigating several areas of human factors in software engineering with support from Engineering Psychology Programs of the Office of Naval Research. There have been two major thrusts in this research. The first thrust investigated the effects of several modern programming practices on programmer efficiency. The second thrust investigated the prediction of programmer performance from software complexity metrics such as those proposed by Halstead and McCabe. This research program consisted of separate experiments on the understanding, modification, debugging, and construction of software, each using professional programmers. Each experiment investigated both the effects of experimentally manipulated programming practices, and the values of complexity metrics computed from the programs employed. Structured coding techniques, mnemonic variable names and commenting are programming practices which supposedly reduce the complexity of software. Dijkstra4 contended that program construction should proceed in a structured, top-down fashion. By limiting the control structures allowed, he assumed that the simplified control flow would make functions performed by the program easier to trace. Mnemonic variable names supposedly simplify the cognitive task of understanding a program by reducing the memory load on a programmer. The inclusion of comments purportedly simplifies modification tasks, although there are different methods of commenting. Global comments preceding a program summarize what objectives are accomplished, whiie in-line comments deiineate how and where the objectives are fulfilled. In 1972 Halstead first published his theory of software physics (renamed software science) stating that algorithms have measurable characteristics analogous to physical laws. These characteristics provide one assessment of program complexity. According to Halstead, 13.14.16.18 the amount of mentai effort required to generate a program can be calculated from simple counts of distinct operators and uperands and the total frequencies of operators and operands. From these four quantities Halstead derives the number of mental comparisons required to generate a program. Correlations

An Empirical Evaluation of Language-Tailored PDLS

Recent research in the area of program documentation has suggested that using a detailed design written in a Program Design Language (PDL) will lead to superior performance over other formats due to the small amount of translation that is required in mapping from the design to the code. This hypothesis was tested by designing PDLs which reflected the syntax and features of particular programming languages and by examining the performance of programmers coding from these various PDLs in one of two coding languages. The data showed that programmers produced code most quickly from the form of documentation that was closest to the code.

Monitoring an Ada software development project

Abstract : As any science matures, the role of measurement, analysis and experimentation grows. The software engineering community has seen the continued development of new software development methods and tools and their use in various environments. The evaluation of methods and tools began with subjective criteria and has been developing toward more objective data collection, measurement and controlled experiments. While this trend is encouraging, these evaluation studies have largely been on a one-shot basis. What has been missing is a systematic approach which defines a longe-range program for the study, analysis and evaluation of a specific method or tool. The emergence of Ada provides a focal point for developing such a systematic study. As a first step, research teams from the University of Maryland and General Electric have embarked upon an eighteen-month collaborative effort. The purpose of this effort is to monitor the use of Ada on a realistically large and complex software development project within industry.

Monitoring an Ada software development

Abstract : Ada evolved from a desire within the Department of Defense to have a standard language for the development of real-time and large scale systems. In addition to providing features needed by those types of systems, Ada supports structured programming, data abstraction, modularity, and information hiding. Research with these techniques indicates that their use should improve the quality of the software development process and its product. While, programmers who are most familiar with various assembly languages and FORTRAN may use structured programming, generally they are not familiar with the other concepts. The problems with training programmers in Ada and its associated design and programming methods and then redeveloping current systems in Ada is unknown. In order to understand the effect of using Ada, the University of Maryland and the General Electric Company began a joint project. The purpose of the project is to monitor the use of Ada in an industrial software development project. In particular, we identify areas of success and difficulty in learning and using Ada as both a design and coding language. Our results indicate where emphasis should be placed in Ada training and in the development of tools and techniques for use with Ada. We also identify metrics used to evaluate and predict the cost, quality, and maintainability of Ada programs. (Author)

Program design languages: How much detail should they include?

Abstract This experiment evaluated the effectiveness of using a program design language (PDL) specifically designed to aid in coding a corresponding programming language. PDLs were designed to reflect the constructs and level of detail of three particular programming languages (i.e. MACRO-11, FORTRAN and APL). We measured the performance of programmers coding from these various PDLs in MACRO-11 and FORTRAN. Each participant was presented with three programs in one of the two programming languages. Several lines had been deleted from each program. A participants task, performed online, was to complete the code using the PDLs. For programmers coding in MACRO-11, the MACRO-like PDL was associated with the shortest coding times. Further, the participants said they found the MACRO-like PDL easiest to use, and they relied on it most heavily. For programmers coding in FORTRAN, the FORTRAN-like PDL was associated with the shortest coding times; the participants said they found the FORTRAN-like PDL easiest to use, and they relied on it most heavily. From these data we conclude that optimal use of a PDL requires that it be tailored to the target programming language in terms of type of construct and level of detail.