John W. Bailey

Experimental evaluation of software documentation formats

Abstract Four controlled experiments investigated the effectiveness of different documentation formats for presenting information about computer programs. Nine different documentation formats (some quite novel) were created by varying three different forms of symbology (natural language, constrained language, and ideograms) and three different spatial arrangements (sequential, branching, and hierarchical). Professional programmers used these formats as aids to comprehend (Experiment 1), code (Experiment 2), debug (Experiment 3), and modify (Experiment 4) modular-sized programs. For each programming task, a model of the cognitive activities a programmer would perform generated hypotheses about the effectiveness of different formats. Natural language was found to be less effective in assisting most of the tasks studied than a constrained language or ideograms. A smaller effect was observed (less frequently than expected) for the spatial arrangment in situations where control flow information aided the task. The largest effect in the experiments, individual differences among the participants, accounted for between a third and one-half of the variation in performance.

An Empirical Evaluation of Language-Tailored PDLS

Recent research in the area of program documentation has suggested that using a detailed design written in a Program Design Language (PDL) will lead to superior performance over other formats due to the small amount of translation that is required in mapping from the design to the code. This hypothesis was tested by designing PDLs which reflected the syntax and features of particular programming languages and by examining the performance of programmers coding from these various PDLs in one of two coding languages. The data showed that programmers produced code most quickly from the form of documentation that was closest to the code.

Monitoring an Ada software development project

Abstract : As any science matures, the role of measurement, analysis and experimentation grows. The software engineering community has seen the continued development of new software development methods and tools and their use in various environments. The evaluation of methods and tools began with subjective criteria and has been developing toward more objective data collection, measurement and controlled experiments. While this trend is encouraging, these evaluation studies have largely been on a one-shot basis. What has been missing is a systematic approach which defines a longe-range program for the study, analysis and evaluation of a specific method or tool. The emergence of Ada provides a focal point for developing such a systematic study. As a first step, research teams from the University of Maryland and General Electric have embarked upon an eighteen-month collaborative effort. The purpose of this effort is to monitor the use of Ada on a realistically large and complex software development project within industry.

The software-cycle models for re-engineering and reuse

This paper reports on the progress of a study which will contribute to our ability to perform high-level, component-based programming by describing means to obtain useful components, methods for the configuration and integration of those components, and an underlying economic model of the costs and benefits associated with this approach to reuse. One goal of the study is to develop and demonstrate methods to recover reusable components from domain-specific software through a combination of tools, to perform the identification, extraction, and re-engineering of components, and domain experts, to direct the applications of those tools. A second goal of the study is to enable the reuse of those components by identifying techniques for configuring and recombining the re-engineered software. This component-recovery or software-cycle model addresses not only the selection and re-engineering of components, but also their recombination into new programs. Once a model of reuse activities has been developed, the quantification of the costs and benefits of various reuse options will enable the development of an adaptable economic model of reuse, which is the principal goal of the overall study. This paper reports on the conception of the software-cycle model and on several supporting techniques of software recovery, measurement, and reuse which will lead to the development of the desired economic model.

ADA Reusability Analysis and Measurement

The demand for software has exceeded the industry’s capacity to supply it. Projects are frequently scaled down, delayed or even cancelled because of the time and effort required to develop the software for them. Further, the demand for software will continue to increase in the foreseeable future. Software reuse provides an answer to this dilemma. Although process and tool reuse is common practice, lifecycle product reuse is still in its infancy. Ultimately, reuse of early lifecycle products might provide the largest payoff, however for the near term, gains can be realized and further work can be guided by understanding how software can be developed with a minimum of newly-generated source lines of code.

Monitoring an Ada software development

Abstract : Ada evolved from a desire within the Department of Defense to have a standard language for the development of real-time and large scale systems. In addition to providing features needed by those types of systems, Ada supports structured programming, data abstraction, modularity, and information hiding. Research with these techniques indicates that their use should improve the quality of the software development process and its product. While, programmers who are most familiar with various assembly languages and FORTRAN may use structured programming, generally they are not familiar with the other concepts. The problems with training programmers in Ada and its associated design and programming methods and then redeveloping current systems in Ada is unknown. In order to understand the effect of using Ada, the University of Maryland and the General Electric Company began a joint project. The purpose of the project is to monitor the use of Ada in an industrial software development project. In particular, we identify areas of success and difficulty in learning and using Ada as both a design and coding language. Our results indicate where emphasis should be placed in Ada training and in the development of tools and techniques for use with Ada. We also identify metrics used to evaluate and predict the cost, quality, and maintainability of Ada programs. (Author)

The impact of Ada and object-oriented design in NASA Goddard's Flight Dynamics Division

This paper presents the highlights and key findings of 10 years of use and study of Ada and object-oriented design in NASA Goddards Flight Dynamics Division (FDD). In 1985, the Software Engineering Laboratory (SEL) began investigating how the Ada language might apply to FDD software development projects. Although they began cautiously using Ada on only a few pilot projects, they expected that, if the Ada pilots showed promising results, the FDD would fully transition its entire development organization from FORTRAN to Ada within 10 years. However, 10 years later, the FDD still produced 80 percent of its software in FORTRAN and had begun using C and C++, despite positive results on Ada projects. This paper presents the final results of a SEL study to quantify the impact of Ada in the FDD, to determine why Ada has not flourished, and to recommend future directions regarding Ada. Project trends in both languages are examined as are external factors and cultural issues that affected the infusion of this technology. The detailed results of this study were published in a formal study report [1] in March of 1995. This paper supersedes the preliminary results of this study that were presented at the Eighteenth Annual Software Engineering Workshop in 1993 [2].

Program design languages: How much detail should they include?

Abstract This experiment evaluated the effectiveness of using a program design language (PDL) specifically designed to aid in coding a corresponding programming language. PDLs were designed to reflect the constructs and level of detail of three particular programming languages (i.e. MACRO-11, FORTRAN and APL). We measured the performance of programmers coding from these various PDLs in MACRO-11 and FORTRAN. Each participant was presented with three programs in one of the two programming languages. Several lines had been deleted from each program. A participants task, performed online, was to complete the code using the PDLs. For programmers coding in MACRO-11, the MACRO-like PDL was associated with the shortest coding times. Further, the participants said they found the MACRO-like PDL easiest to use, and they relied on it most heavily. For programmers coding in FORTRAN, the FORTRAN-like PDL was associated with the shortest coding times; the participants said they found the FORTRAN-like PDL easiest to use, and they relied on it most heavily. From these data we conclude that optimal use of a PDL requires that it be tailored to the target programming language in terms of type of construct and level of detail.

Evaluating the effect of inter-programmer communication: levels on productivity by varying programming team structure

Various methods exist for estimating the complexity of single modules and the complexity of interacting modules. We propose to demonstrate how the degree of programmer inter-communication in a multi-programmer project also has a significant effect on productivity. Specifically, projects which use programming team structures which require more inter-programmer communication will experience lower productivities than projects which use team structures which minimize the need for inter-programmer communications.The design proposed here involves the simultaneous re-implementation of an existing, well-documented design by teams of seven graduate-level students. It offers the advantage of alternate levels of work depending upon the amount of data collection and analysis to be underwritten. The results of even the least demanding study will be of significance to any medium to large scale development environment as a guide to selecting programming team structure. The more involved study will give more general results as to the effects of unconstrained communication paths in a team environment.