Laurie Murphy

Debugging: a review of the literature from an educational perspective

This paper reviews the literature related to the learning and teaching of debugging computer programs. Debugging is an important skill that continues to be both difficult for novice programmers to learn and challenging for computer science educators to teach. These challenges persist despite a wealth of important research on the subject dating back as far as the mid 1970s. Although the tools and languages novices use for writing programs today are notably different from those employed decades earlier, the basic problem-solving and pragmatic skills necessary to debug them effectively are largely similar. Hence, an understanding of the previous work on debugging can offer computer science educators insights into how to improve contemporary learning and teaching of debugging and may suggest directions for future research into this important area. This overview of the debugging literature is organized around four questions relevant to computer science educators and education researchers: What causes bugs to occur? What types of bugs occur? What is the debugging process? How can we improve the learning and teaching of debugging? We conclude with suggestions on using the existing literature both to facilitate pedagogical improvements to debugging education and to offer guidance for future research.

Debugging: finding, fixing and flailing, a multi-institutional study of novice debuggers

Debugging is often difficult and frustrating for novices. Yet because students typically debug outside the classroom and often in isolation, instructors rarely have the opportunity to closely observe students while they debug. This paper describes the details of an exploratory study of the debugging skills and behaviors of contemporary novice Java programmers. Based on a modified replication of Katz and Andersons study of novices, we sought to broadly survey the modern landscape of novice debugging abilities. As such, this study reports general quantitative results and fills in the picture with qualitative detail from a relatively small, but varied sample. Comprehensive interviews involving both a programming and a debugging task, followed by a semi-structured interview and a questionnaire, were conducted with 21 CS2 students at seven colleges and universities. While many subjects successfully debugged a representative set of typical CS1 bugs, there was a great deal of variation in their success at the programming and debugging tasks. Most of the students who were good debuggers were good novice programmers, although not all of the good programmers were successful at debugging. Students employed a variety of strategies to find 70% of all bugs and of the bugs they found they were able to fix 97% of them. They had the most difficulty with malformed statements, such as arithmetic errors and incorrect loop conditions. Our results confirm many findings from previous studies (some quite old) – most notably that once students find bugs, they can fix them. However, the results also suggest that some changes have occurred in the student population, particularly an increased use of debugging tools and online resources, as well as the use of pattern matching, which has not previously been reported.

Pair programming in education: a literature review

This article provides a review of educational research literature focused on pair programming in the undergraduate computer science curriculum. Research suggests that the benefits of pair programming include increased success rates in introductory courses, increased retention in the major, higher quality software, higher student confidence in solutions, and improvement in learning outcomes. Moreover, there is some evidence that women, in particular, benefit from pair programming. The literature also provides evidence that the transition from paired to solo programming is easy for students. The greatest challenges for paired students appear to concern scheduling and partner compatibility. This review also considers practical issues such as assigning partners, teaching students to work in pairs, and assessing individual contributions, and concludes with a discussion of open research questions.

Debugging From the Student Perspective

Learning to debug is a difficult, yet essential, aspect of learning to program. Students in this multi-institutional study report that finding bugs is harder than fixing them. They use a wide variety of debugging strategies, some of them unexpected. Time spent on understanding the problem can be effective. Pattern matching, particularly at the syntactic level, is an important technique for beginners. The Web has emerged as an obvious first place to look for similar examples. Lack of Web materials at an appropriate beginner level leads to flailing. Hypothesizing about the cause of bugs is an underdeveloped skill.

Learning computer science: perceptions, actions and roles

This phenomenographic study opens the classroom door to investigate teachers’ experiences of students learning difficult computing topics. Three distinct themes are identified and analysed. Why do students succeed or fail to learn these concepts? What actions do teachers perceive will ameliorate the difficulties facing students? Who is responsible, and for what, in the learning situation? Theoretical work on threshold concepts and conceptual change deals with mechanisms and processes associated with learning difficult material [Meyer, J. and Land, R., 2005. Threshold concepts and troublesome knowledge (2): epistemological considerations and a conceptual framework for teaching and learning. Higher Education, 49 (3), 373–388; Entwistle, N., 2007. Conceptions of learning and the experience of understanding: thresholds, contextual influences, and knowledge objects. In: S. Vosniadou, A. Baltas and X. Vamvakoussi, eds. Re-framing the conceptual change approach in learning and instruction. Amsterdam, The Netherlands: Elsevier, chap. 11]. With this work as a background, we concentrate on the perceptions of teachers. Where do teachers feel that the difficulties lie when studying the troublesome knowledge in computing? Student and teacher-centric views of teaching reported in other literature are also to be seen in our results. The first two categories in the ‘what’ and ‘who’ themes are teacher-centric. Higher level categories in all themes show increasingly learner centred conceptions of the instructional role. However, the nature of the categories in the ‘why’ theme reveals a new dimension dealing with teacher beliefs specific to the nature of troublesome knowledge in computing. A number of prior studies in tertiary teaching concentrate on approaches to teaching [Trigwell, K. and Prosser, M., 2004. Development and use of the approaches to teaching inventory. Educational Psychology Review, 16 (4), 409–424], and attitudes to scholarship of teaching and learning [Ashwin, P. and Trigwell, K., 2004. Investigating educational development. In: Making sense of staff and educational development, 117–131]. Our focus on learning difficult topics extends this work, investigating teacher conceptions of causality in relation to learning difficulties. We argue that teacher conceptions of enabling factors, for learning difficult computing topics, can act to limit the nature and scope of academics’ pedagogical responses. Improved awareness of teachers beliefs regarding student learning difficulties both extends and complements existing efforts to develop a more student-centred computing pedagogy.

Knowing what I know: An investigation of undergraduate knowledge and self-knowledge of data structures

This paper describes an empirical study that investigated the knowledge that Computer Science students have about the extent of their own previous learning. The study compared self-generated estimates of performance with actual performance on a data structures quiz taken by undergraduate students in courses requiring data structures as a prerequisite. The study was contextualized and grounded within a research paradigm in Psychology called calibration of knowledge that suggests that self-knowledge across a range of disciplines is highly unreliable. Such self-knowledge is important because of its role in meta-cognition, particularly in cognitive self-regulation and monitoring, as well as in the credence that instructors give to student self-reports. Our results indicated that Computer Science student self-estimates are highly correlated with performance, more so for estimates provided after the performance than before. This high level of calibration, however, was likely the result of a number of conditions that do not always hold: that the students already had domain expertise, that the quiz had unambiguous and verifiable answers, and that students expected their estimates to be validated. When these conditions are not met, it becomes more important for students to have direct feedback about their performance so as to uncover those areas where their intuitions might mislead them. Students also had weak knowledge about their standing relative to their peers, particularly those in the lower performance quartiles, exhibiting the well known better-than-average heuristic. There was, additionally, no correlation between calibration ability and degree of liking or difficulty with the data structures material, suggesting that instructors and researchers should not treat liking or difficulty as reliable indicators of the learning that has occurred.

Do computer science students know what they know?: a calibration study of data structure knowledge

This paper describes an empirical study that investigates the knowledge that Computer Science students have about the extent of their own previous learning. The study compares self-generated estimates of performance with actual performance on a data structures quiz taken by undergraduate students in courses requiring data structures as a pre-requisite. The study is contextualized and grounded within a research paradigm in Psychology called calibration of knowledge that suggests that self-knowledge across a range of disciplines is highly unreliable. Such self-knowledge is important because of its role in meta-cognition, particularly in cognitive self-regulation and monitoring. It is also important because of the credence that faculty give to student self-reports. Our results indicate that Computer Science student self-estimates correlate moderately with their performance on a quiz, more so for estimates provided after they have taken the quiz than before. The pedagogical implications are that students should be provided with regular opportunities for empirical validation of their knowledge as well as being taught the metacognitive skills of regular self-testing in order to overcome validation bias.

'explain in plain english' questions revisited: data structures problems

Recent studies have linked the ability of novice (CS1) programmers to read and explain code with their ability to write code. This study extends earlier work by asking CS2 students to explain object-oriented data structures problems that involve recursion. Results show a strong correlation between ability to explain code at an abstract level and performance on code writing and code reading test problems for these object-oriented data structures problems. The authors postulate that there is a common set of skills concerned with reasoning about programs that explains the correlation between writing code and explaining code. The authors suggest that an overly exclusive emphasis on code writing may be detrimental to learning to program. Non-code writing learning activities (e.g., reading and explaining code) are likely to improve student ability to reason about code and, by extension, improve student ability to write code. A judicious mix of code-writing and code-reading activities is recommended.

Teaching and Learning Recursive Programming: A Review of the Research Literature.

Hundreds of articles have been published on the topics of teaching and learning recursion, yet fewer than 50 of them have published research results. This article surveys the computing education research literature and presents findings on challenges students encounter in learning recursion, mental models students develop as they learn recursion, and best practices in introducing recursion. Effective strategies for introducing the topic include using different contexts such as recurrence relations, programming examples, fractal images, and a description of how recursive methods are processed using a call stack. Several studies compared the efficacy of introducing iteration before recursion and vice versa. The paper concludes with suggestions for future research into how students learn and understand recursion, including a look at the possible impact of instructor attitude and newer pedagogies.

Bug Infestation!: A Goal-Plan Analysis of CS2 Students' Recursive Binary Tree Solutions

A goal-plan analysis was conducted to examine the variety of plans students use in writing a recursive method for an operation on a binary search tree. Students were asked to write a recursive method to count the nodes in a binary search tree with exactly one child. The problem incorporated two goals: traversing the tree and counting nodes with one child. Three traversal plans and four counting plans were observed in student solutions. Over half of the students used the arms-length recursion plan, which involves testing for the base case before it is actually reached in order to avoid making recursive calls. This strategy creates complex and error prone code. Making students aware of arms-length recursion may help them avoid introducing bugs into their recursive code. Although nearly all of the 18 participants demonstrated viable plans for solving the problem, their solutions contained a variety of errors: 55 total errors of 15 types. Students had particular difficulty with base cases, misplaced calculations, and missing method calls. Knowledge of these errors can be useful for instructors when developing lecture examples, identifying distractors for peer instruction multiple-choice questions and for designing homework exercises. Instructors can counteract these problems by providing a variety of recursive examples.