Lynda Thomas

A multi-national, multi-institutional study of assessment of programming skills of first-year CS students

In computer science, an expected outcome of a students education is programming skill. This working group investigated the programming competency students have as they complete their first one or two courses in computer science. In order to explore options for assessing students, the working group developed a trial assessment of whether students can program. The underlying goal of this work was to initiate dialog in the Computer Science community on how to develop these types of assessments. Several universities participated in our trial assessment and the disappointing results suggest that many students do not know how to program at the conclusion of their introductory courses. For a combined sample of 216 students from four universities, the average score was 22.89 out of 110 points on the general evaluation criteria developed for this study. From this trial assessment we developed a framework of expectations for first-year courses and suggestions for further work to develop more comprehensive assessments.

A cognitive approach to identifying measurable milestones for programming skill acquisition

Traditional approaches to programming education, as exemplified by the typical CS1/CS2 course sequence, have not taken advantage of the long record of psychological and experimental studies on the development of programming skills. These studies indicate a need for a new curricular strategy for developing programming skills and indicate that a cognitive approach would be a promising starting point. This paper first reviews the literature on studies of programming skills, cognition and learning, then within that context reports on a new formal structure, called an anchor graph, that supports curricular design and facilitates the setting of measurable milestones.

Checklists for grading object-oriented CS1 programs: concepts and misconceptions

In this paper, we begin by considering object-oriented programming concepts and typical novice misconceptions as identified in the literature. We then present the results of a close examination of student programs, in an objects-first CS1 course, in which we find concrete evidence of students learning these concepts while also displaying some of these misconceptions. This leads to the development of two checklists that educators can use when designing or grading student programs.

Debugging: a review of the literature from an educational perspective

This paper reviews the literature related to the learning and teaching of debugging computer programs. Debugging is an important skill that continues to be both difficult for novice programmers to learn and challenging for computer science educators to teach. These challenges persist despite a wealth of important research on the subject dating back as far as the mid 1970s. Although the tools and languages novices use for writing programs today are notably different from those employed decades earlier, the basic problem-solving and pragmatic skills necessary to debug them effectively are largely similar. Hence, an understanding of the previous work on debugging can offer computer science educators insights into how to improve contemporary learning and teaching of debugging and may suggest directions for future research into this important area. This overview of the debugging literature is organized around four questions relevant to computer science educators and education researchers: What causes bugs to occur? What types of bugs occur? What is the debugging process? How can we improve the learning and teaching of debugging? We conclude with suggestions on using the existing literature both to facilitate pedagogical improvements to debugging education and to offer guidance for future research.

Debugging: finding, fixing and flailing, a multi-institutional study of novice debuggers

Debugging is often difficult and frustrating for novices. Yet because students typically debug outside the classroom and often in isolation, instructors rarely have the opportunity to closely observe students while they debug. This paper describes the details of an exploratory study of the debugging skills and behaviors of contemporary novice Java programmers. Based on a modified replication of Katz and Andersons study of novices, we sought to broadly survey the modern landscape of novice debugging abilities. As such, this study reports general quantitative results and fills in the picture with qualitative detail from a relatively small, but varied sample. Comprehensive interviews involving both a programming and a debugging task, followed by a semi-structured interview and a questionnaire, were conducted with 21 CS2 students at seven colleges and universities. While many subjects successfully debugged a representative set of typical CS1 bugs, there was a great deal of variation in their success at the programming and debugging tasks. Most of the students who were good debuggers were good novice programmers, although not all of the good programmers were successful at debugging. Students employed a variety of strategies to find 70% of all bugs and of the bugs they found they were able to fix 97% of them. They had the most difficulty with malformed statements, such as arithmetic errors and incorrect loop conditions. Our results confirm many findings from previous studies (some quite old) – most notably that once students find bugs, they can fix them. However, the results also suggest that some changes have occurred in the student population, particularly an increased use of debugging tools and online resources, as well as the use of pattern matching, which has not previously been reported.

Scaffolding with object diagrams in first year programming classes: some unexpected results

This paper reports on an experiment in which first year programming students were given explicit encouragement to use Object (Instance) diagrams when tracing code in multiple-choice questions. We conjectured that by providing scaffolding in this technique, students would be helped to understand the code better and that they would then continue to draw their own diagrams in similar situations. This turned out not to be the case. Although generally students who draw diagrams do better in questions that test their understanding of code behaviour and object referencing, our intervention does not appear to have helped students and the students who were exposed to the intervention were not more likely to go on to use the technique themselves.

Learning computer science: perceptions, actions and roles

This phenomenographic study opens the classroom door to investigate teachers’ experiences of students learning difficult computing topics. Three distinct themes are identified and analysed. Why do students succeed or fail to learn these concepts? What actions do teachers perceive will ameliorate the difficulties facing students? Who is responsible, and for what, in the learning situation? Theoretical work on threshold concepts and conceptual change deals with mechanisms and processes associated with learning difficult material [Meyer, J. and Land, R., 2005. Threshold concepts and troublesome knowledge (2): epistemological considerations and a conceptual framework for teaching and learning. Higher Education, 49 (3), 373–388; Entwistle, N., 2007. Conceptions of learning and the experience of understanding: thresholds, contextual influences, and knowledge objects. In: S. Vosniadou, A. Baltas and X. Vamvakoussi, eds. Re-framing the conceptual change approach in learning and instruction. Amsterdam, The Netherlands: Elsevier, chap. 11]. With this work as a background, we concentrate on the perceptions of teachers. Where do teachers feel that the difficulties lie when studying the troublesome knowledge in computing? Student and teacher-centric views of teaching reported in other literature are also to be seen in our results. The first two categories in the ‘what’ and ‘who’ themes are teacher-centric. Higher level categories in all themes show increasingly learner centred conceptions of the instructional role. However, the nature of the categories in the ‘why’ theme reveals a new dimension dealing with teacher beliefs specific to the nature of troublesome knowledge in computing. A number of prior studies in tertiary teaching concentrate on approaches to teaching [Trigwell, K. and Prosser, M., 2004. Development and use of the approaches to teaching inventory. Educational Psychology Review, 16 (4), 409–424], and attitudes to scholarship of teaching and learning [Ashwin, P. and Trigwell, K., 2004. Investigating educational development. In: Making sense of staff and educational development, 117–131]. Our focus on learning difficult topics extends this work, investigating teacher conceptions of causality in relation to learning difficulties. We argue that teacher conceptions of enabling factors, for learning difficult computing topics, can act to limit the nature and scope of academics’ pedagogical responses. Improved awareness of teachers beliefs regarding student learning difficulties both extends and complements existing efforts to develop a more student-centred computing pedagogy.

Threshold concepts and threshold skills in computing

Threshold concepts can be used to both organize disciplinary knowledge and explain why students have difficulties at certain points in the curriculum. Threshold concepts transform a students view of the discipline; before being learned, they can block a students progress. In this paper, we propose that in computing, skills, in addition to concepts, can sometimes be thresholds. Some students report finding skills more difficult than concepts. We discuss some computing skills that may be thresholds and compare threshold skills and threshold concepts.

Liminal spaces and learning computing

‘Threshold concepts’ are concepts that, among other things, transform the way a student looks at a discipline. Although the term ‘threshold’ might suggest that the transformation occurs at a specific point in time, an ‘aha’ moment, it seems more common (at least in computing) that a longer time period is required. This time period is referred to as the ‘liminal space’. In this paper, we summarise our findings concerning how computing students experience the liminal space and discuss how this might affect teaching. Most of our findings so far relate to software engineering. As it is likely that similar liminal spaces occur in other engineering disciplines, these findings have relevance across engineering education.

Student transformations: are they computer scientists yet?

We examine the changes in the ways computing students view their field as they learn, as reported by the students themselves in short written biographies. In many ways, these changes result in students thinking and acting more like computer scientists and identifying more with the computing community. Most of the changes are associated with programming and software engineering, rather than theoretical computer science, however.