David J. Gilmore

Comprehension and recall of miniature programs

Abstract Differences in the comprehensibility of programming notations can arise because their syntax can make them cognitively unwieldy in a generalized way ( Mayer, 1976 ), because all notations are translated into the same “mental language“ but some are easier to translate than others (Shneiderman & Mayer, 1979 ), or because the mental operations demanded by certain tasks are harder in some notations than in others ( Green, 1977 ). The first two hypotheses predict that the relative comprehensibility of two notations will be consistent across all tasks, whereas the mental operations hypothesis suggests that particular notations may be best suited to particular tasks. The present experiment used four notations and 40 non-programmers to test these hypotheses. Two of the notations were procedural and two were declarative, and one of each pair contained cues to declarative or procedural information, respectively. Different types of comprehension question were used (“sequential“ and “circumstantial“); a mental operations analysis predicted that procedural languages would be “matched” with sequential questions, and declarative languages with circumstantial questions. Questions were answered first from the printed text, and then from recall. Subjects performed best on “matched pairs” of tasks and languages. Perceptually-based cues improved the performance on “unmatched pairs” better than non-perceptual cues when answering from the text, and both types of cues improved performance on “unmatched pairs” in the recall stage. These results support the mental operations explanation. They also show that the mental representation of a program preserves some features of the original notation; a comprehended program is not stored in a uniform “mental language”.

Programming plans and programming expertise

This paper addresses issues of the nature of expertise in programming and asks whether “programming plans” represent the underlying deep structure of a program. It reports an experiment that investigated the effect, on experienced programmers, of highlighting the plan structure of a computer program, while they were performing both plan-related and unrelated tasks. The effect was examined in both Pascal and BASIC. For Pascal programmers, perceptual cues to the plan structure were useful only for plan-related tasks, but the same cues were of no benefit to experienced BASIC programmers in any of the tasks. These results suggest that the actual content of programming plans does not generalise across different languages, although it is possible that the BASIC programmers can use other plans. From these results a more detailed description of programming plans and their role in programming expertise can be developed. The fact that BASIC programmers were not sensitive to the same plans as Pascal programmers implies that plans cannot represent the underlying deep structure of the programming problem.

Models of debugging

Abstract This paper proposes a view of computer program debugging, which tackles some of the simplifying short-comings of existing models. The paper begins by reviewing some of the existing models of debugging and their assumptions, before looking in more detail at one of the dominant paradigms for investigating debugging, that of predicting bug detection success. A reanalysis of the bug detection data from Gilmore and Green (1988) provides evidence that the assumptions of existing models are not valid. The important part of this result is the realisation that these assumptions have been derived from a view of debugging as fault diagnosis, rather than as a critical component of design. In conclusion, the paper describes the important features of debugging as a design activity, before outlining some predictions and implications which can be derived from the model.

Learning graphical programming: An evaluation of KidSim™

This paper presents part of an evaluation of a new children’s programming environment, developed by Apple Computer Inc. for 10–13 year old children. We studied 56 children, generally working in groups of 2–3, using KidSimTM for between 2–12 hours, over a period of between 2 days and 3 weeks. The results show that children of this age can readily learn to master the programming environment, and that they greatly enjoy using the system — indeed in most cases it clearly fired their imaginations. However, questions remain about the level of programming abstractions that they were able to understand.

Towards a cognitive browser for OOPS

Software engineers have developed sophisticated “object‐oriented” programming environments that are intended to make the reuse of program code easy. Experience has shown that these environments can be improved: Even very experienced programmers have problems in locating and comprehending code for reuse. Programs cannot be modified as readily as had been anticipated. We describe the problems in terms of “cognitive dimensions” of notational systems and show how improved support for opportunistic design may be achieved. A central tenet is that programmers are not at present able to externalize enough of their knowledge about a program. We propose a scheme for attaching a “description level” in which arbitrary attributes and relationships can be recorded in a “browsable” form. Our conclusions stress improving the means for programmers to represent facts rather than the provision of predefined knowledge bases.

Methodological Issues in the Study of Programming

Abstract The primary purpose of this chapter is to provide readers with a sufficient understanding of methodological issues to enable them to consider critically the numerous experimental results presented elsewhere in the volume, particularly given the potential conflicts between controlled laboratory studies and real-world observations. The chapter commences with a review of the many reasons why data might be collected in the study of programming, before looking at the choice of programming tasks for investigation. This is followed by an introduction to the key concepts of experimental design – statistical significance, effect size, sample size and power – since understanding the distinctions between these is critical in applied research, even when behaviour is being observed in real-world contexts. Methods for observing programming behaviour in situ are presented next, followed by a brief discussion of some special issues which arise when we try to generalize from applied research. The chapter concludes with some example case studies which are intended to emphasize the complementary nature of controlled methods versus observation and artificiality versus the real world.

Interface Design: Have we got it wrong?

Human-Computer Interaction (HCI) has gathered many guidelines for interface design and has discovered the strengths of direct manipulation as an interaction technique. However, to date it has been generally assumed that these guidelines apply generically across all applications. In this paper I intend to question this assumption.

Selection through Rejection: Reconsidering the Invariant Learning Paradigm:

Two experiments are reported that investigate the nature of selections in the McGeorge and Burton (1990) invariant learning paradigm. McGeorge and Burton suggest that subjects implicitly acquire abstract knowledge of an invariant feature (usually the presence of the digit “3”) in a set of 30 stimuli. McGeorge and Burtons analysis has recently been challenged by Cock, Berry, and Gaffan (1994) and by Wright and Burton (1995). In this paper, we demonstrate that performance is based on knowledge of other aspects of the learning set besides the invariant digit, but that this knowledge is still implicit. Altering the nature of the learning stimuli to highlight these co-varying features enhances the effects and increases the reporting of explicit knowledge. Our results indicate that performance within this paradigm is more easily characterized as rejection of salient negatives than selection of positive instances, but that salience is not based simply on similarity.

Factors affecting the perceived value of coins

Abstract Subjects were asked to judge the relative values or assign absolute values to coins similar to the proposed new UK 20p and £1 coins along with other metal blanks, prior to the announcement of the new coins. Preliminary studies indicated that while a ‘yellow’ (gold-coloured) coin would not normally be rated higher in value than the blank 5p coin, the addition of extra thickness and an elaborate edge enhanced its apparent relative value. It was also shown that seven-sided ‘white’ (silver-coloured) coins were rated as more valuable than 12-sided or circular coins of about the same diameter. The absolute values assigned to such coins followed a similar trend to the relative values. Seven-sided white coins were assigned higher values than 12-sided white coins, and thicker yellow coins were assigned higher value characteristics and colour were examined independently. For smooth circular, milled circular and seven-sided coins, ‘red’ (cooper-coloured) coins were perceived as least valuable, and white and yellow coins were equally often assigned to intermediate and high values. For red, white and yellow coins, smooth circular coins were rated least valuable and seven-sided coins as most valuable. It appears that people assign learned ‘rules’ about features indicating value in a systematic and independent manner.

Breaking the Rules of Direct Manipulation

There is a clear conflict between making an interface easier to use and making computer-based problems easier to solve. Making the control of the problem-solving domain more complex helps users to reach their solutions in less steps. Previous explanations of this phenomena have been centred in how the interface impacts on learning or planning of the problem solving domain. This paper adds a new element to this work by drawing on observational analysis. Interface differences are closely related to the types of strategy a user employs. A more complex interface makes the user more sensitive to their problem solving situation. Explanations of the phenomena are reconsidered in the light of strategy analysis. The effect occurs in problem solving interfaces because the domain specific requirements of the interface cannot be divorced from the domain independent goals. The application of the effect to design is discussed in relation to several areas.