Kate Ehrlich

Empirical Studies of Programming Knowledge

We suggest that expert programmers have and use two types of programming knowledge: 1) programming plans, which are generic program fragments that represent stereotypic action sequences in programming, and 2) rules of programming discourse, which capture the conventions in programming and govern the composition of the plans into programs. We report here on two empirical studies that attempt to evaluate the above hypothesis. Results from these studies do in fact support our claim.

Pointing the way: active collaborative filtering

Collaborative filtering is based on the premise that people looking for information should be able to make use of what others have already found and evaluated. Current collaborative filtering systems provide tools for readers to filter documents based on which ones were read and liked by previous readers. This paper describes a different type of collaborative filtering system in which people who find interesting documents actively send pointers to those documents to their colleagues. A pointer contains a hypertext link to the source document as well as contextual information intended to help the recipient determine the potential interest and relevance of the document prior to accessing it. A preliminary version of our system has already proven easy t o use, with people using it to bookmark documents, send pointers to their colleagues and create digests that combine pointers with original text. Based on our experience we discuss the benefits of this form of filtering as well as its limitations.

Cognitive strategies and looping constructs: an empirical study

Authors Present Addresses: Elliot Soloway and Kate Ehrlich, Department of Computer Science, Yale University, P.O. Box 2158, New Haven, CT 06520; Jeffrey Bonar, Learning Research and Development Center, Univ. of Pittsburgh, Pittsburgh, PA 15260. This work was supported by the Army Research Institute for the Behavioral and Social Sciences. under ARI Grant No. MDA903-80-C-0508. This work was also supported by the National Science Foundation under NSF Grant SED-81-12403. Any opinions, findings, conclusions or recommendations expressed in this report are those of the authors, and do not necessarily reflect the views of the U.S. Government. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission.

Tapping into tacit programming knowledge

The Cognition and Programming Group at Yale University is engaged in two complementary efforts:n 1. exploring the programming process empirically, paying special attention to the knowledge and strategies which expert and non-experts employ, andn 2. building computer-based environments which aid novices learning to program.n In this extended abstract we will focus on the empirical strand of our research program; in particular, we will describe an experimental technique we have just begun to use to more carefully study what it is that expert and novice programmers do—and dont—know. In [19, 20, 22, 18, 7] we describe additional empirical studies, while [21] describes MENO-II, our intelligent programming tutor for Pascal.

Beyond numbers: Don't ask “how many” ... ask “why”

While programmers may differ in their assessment of the comprehensibility of a program, there are nonetheless some clear cut cases of programs that are truly difficult to understand. In this paper, we analyze three programs—two of which are relatively incomprehensible—using Halsteads Volume Metric, Propositional Analysis and Plan Analysis. We argue that only Plan Analysis provides a satisfactory explanation for why the programs in question differ with respect to understandability. Moreover, we suggest that a qualitative analysis, such as provided by Plan Analysis, is the desired type of evaluation: rather than simply providing a numerical ranking for programs, the qualitative analysis can pinpoint the troublesome area in the code and provide prescriptive information for correcting the difficulty.

Design: the what of XFR: eXperiments in the future of reading

Step up to the joystick on the exhibit to the right. A cartoon image of a young boy is projected on the screen. Kids crowd in around you as you read about Henry and his world. (See Figure 1.) There is a world of cartoon images. Lines trail off to small drawings falling away as though seen through a fisheye. As you push the joystick, another image of Henry rolls into view along one of the lines, and a comic-book dialog bubble appears. The story Henry tells is of the things in his imagination and his everyday world. One image leads to the next and then to the next. Over to the left of Henry is a sort of work bench with a touch-screen workstation sitting on it. No pictures on the screen this time, just a title (“Harry the Ape”) and a long paragraph of text. The story is about the creatures that live in Harry’s fur. Sprinkled around A Glimpse of XFR

Psychological perspectives for software science

Abstract In developing his theory of software science, Maurice Halstead borrowed heavily from psychology for theoretical explanations of the equations he was developing. Other software researchers have used psychological theory to defend their techniques or conclusions. Unfortunately, the psychological principles they have employed have not always been the most recent developments in the field. In fact, most of the psychological explanations found in software science are based on research that is one to three decades old. Some of these principles have been taken out of context and used inappropriately. Others do not represent the most powerful psychological principles that could be used in developing programming theory, techniques, and measures. This paper will first critique the current use of psychological theory in software science. Then, it will review recent developments in cognitive psychology which are relevant to programming. Finally, it will propose ways of using these newer findings in developing improved measures for software science.

Collecting and analyzing on-line protocols from novice programmers

Methodology for collecting and analyzing on-line protocols from novice programmers is described. On-line protocols are copies of all syntactically correct programs that students have written using an interactive computer system. Since the number of on-line protocols collected is quite large, we have developed a computer program, called the Bug Finder, which can auto-matically identify semantic and pragmatic bugs in subjects’ programs. In this paper, we describe the theory upon which the Bug Finder is built and provide an example of the Bug Finder in operation.

Just so stories: how the program got that bug

Teachers see many more buggy programs than they do correct programs. We view this volume of buggy programs as riches to be mined: as windows into the thought processes of the students. In this paper we will describe one of the methods we employ in analyzing buggy programs: just so bug stories, in which we generate hypotheses as to the misconceptions on the students part that may have led to the observed program bugs. Our goal in presenting this discussion is to convince the reader of the potential for improved programming instruction that can come from paying attention to the bugs in a students program: by truly understanding what the student was thinking about, we are in a much better position to offer counsel to the student.

Factors influencing technology transfer

This paper treats technology transfer as primarily a communication activity. Barriers to technology transfer can be erected by (1) organizational structures which inhibit the flow of communication between different groups; (2) the technology imposing specialized knowledge on the people who work with it; and, (3) the individuals themselves who have cultural biases which inhibit communication with people from different professional and experiential backgrounds. Some of these barriers can be overcome by creating small cross-organizational groups or partnerships, by increasing exposure to the technology, by rewarding joint work and by promoting people who are willing to champion technology transfer efforts within a corporation.