Beth Simon

Not seeing the forest for the trees: novice programmers and the SOLO taxonomy

This paper reports on the authors use of the SOLO taxonomy to describe differences in the way students and educators solve small code reading exercises. SOLO is a general educational taxonomy, and has not previously been applied to the study of how novice programmers manifest their understanding of code. Data was collected in the form of written and think-aloud responses from students (novices) and educators (experts), using exam questions. During analysis, the responses were mapped to the different levels of the SOLO taxonomy. From think-aloud responses, the authors found that educators tended to manifest a SOLO relational response on small reading problems, whereas students tended to manifest a multistructural response. These results are consistent with the literature on the psychology of programming, but the work in this paper extends on these findings by analyzing the design of exam questions.

Ubiquitous presenter: increasing student access and control in a digital lecturing environment

The University of Washingtons Classroom Presenter lecturing system enables an active lecturing environment by combining a standard electronic slide presentation format with the capability for extemporaneous ink annotations by instructors and students using Tablet PCs. Thus, it can promote more interactive, student-centered learning. While many students may own laptops, few are yet Tablet devices. Also, Presenter uses multicast networking, which has availability and reliability issues. Ubiquitous Presenter (UP) expands Presenter via common web technologies to support non-Tablet audiences and enhance student control. UP enables students, using internet web browsers, to (a) synchronously or asynchronously view the slides and ink that are broadcast by the instructor, as well as (b) provide contextual submissions via text overlaid on the instructors slides. The only compromises are that non-Tablet students cannot produce ink, and that professor ink is provided after a small time delay.

Novice software developers, all over again

Transitions from novice to expert often cause stress and anxiety and require specialized instruction and support to enact efficiently. While many studies have looked at novice computer science students, very little research has been conducted on professional novices. We conducted a two-month in-situ qualitative case study of new software developers in their first six months working at Microsoft. We shadowed them in all aspects of their jobs: coding, debugging, designing, and engaging with their team, and analyzed the types of tasks in which they engage. We can explain many of the behaviors revealed by our analyses if viewed through the lens of newcomer socialization from the field of organizational man-agement. This new perspective also enables us to better understand how current computer science pedagogy prepares students for jobs in the software industry. We consider the implications of this data and analysis for developing new processes for learning in both university and industrial settings to help accelerate the transition from novice to expert software developer.

Preliminary experiences with a tablet PC based system to support active learning in computer science courses

There has been much research on the benefits of active and collaborative learning and on its use in computer science courses. As classroom technology becomes more prevalent it is natural to develop systems that support the use of these techniques. We have developed such a system as an extension to Classroom Presenter [2], a Tablet PC-based presentation system. In our system students (or groups of students) are equipped with tablet computers and at various points during the lecture, are asked to solve a problem or respond to a question. Students respond by writing their solution on the tablet and submitting it wirelessly to the instructor. The instructor can view all student responses, select one or more to display to the class, and annotate responses with ink as they are being displayed. Student responses can be saved for review after lecture by the instructor or made available electronically to the students. In this paper we describe the system and our initial experiences using the system in two classes (CS1 and Computer Ethics) at the University of San Diego in Fall 2003.

Exploring the potential of mobile phones for active learning in the classroom

Research has shown that educational technology can broaden and enhance the use of active learning in large classrooms. An educational technology platform often relies on students to bring laptops or specialized wireless devices like clickers to interact through the system. Mobile phones are an attractive alternative, as most students already possess them, they have more capabilities than dedicated clickers, and yet are small enough to minimize interference with note taking on a classroom desk.This paper presents the design and use of a mobile phone extension to Ubiquitous Presenter, which allows students to submit solutions to active learning exercises in the form of text or photo messages. In an exploratory study, students found that text messaging worked well for exercises with multiple choice or short answers. Entering symbols common to computer science was difficult. Many problems were more suitable to photo messaging of a handwritten answer, although image quality must be managed. The phones small size left space for the use of a notebook. The students had concerns about the message charges that would accrue in use. In conclusion, we offer recommendations to instructors and system designers interested in leveraging mobile phones to increase communication in the classroom.

Halving fail rates using peer instruction: a study of four computer science courses

Peer Instruction (PI) is a teaching method that supports student-centric classrooms, where students construct their own understanding through a structured approach featuring questions with peer discussions. PI has been shown to increase learning in STEM disciplines such as physics and biology. In this report we look at another indicator of student success the rate at which students pass the course or, conversely, the rate at which they fail. Evaluating 10 years of instruction of 4 different courses spanning 16 PI course instances, we find that adoption of the PI methodology in the classroom reduces fail rates by a per-course average of 61% (20% reduced to 7%) compared to standard instruction (SI). Moreover, we also find statistically significant improvements within-instructor. For the same instructor teaching the same course, we find PI decreases the fail rate, on average, by 67% (from 23% to 8%) compared to SI. As an in-situ study, we discuss the various threats to the validity of this work and consider implications of wide-spread adoption of PI in computing programs.

Success in introductory programming: what works?

How pair programming, peer instruction, and media computation have improved computer science education.

Retaining nearly one-third more majors with a trio of instructional best practices in CS1

Beginning in 2008, we introduced a new CS1 incorporating a trio of best practices intended to improve the quality of the course, appeal to a broader student body, and, hopefully, improve retention in the major. This trio included Media Computation, Pair Programming, and Peer Instruction. After 3 and 1/2 years (8 CS1 classes, 3 different instructors, and 1011 students passing the course) we find that 89% of the majors who pass the course are still studying computing one year later. This is an improvement of 18% over our average retention of 71% for the previous version of the course (measured since Fall 2001). If the focus shifts from retention of passing CS1 majors to retention of CS1 initially enrolled majors, multiple improvements--fewer students drop, more students pass, and more passing students are retained--compound to increase retention by 31% (from 51% to 82%). In this paper we analyze further aspects of these results, detail the three instructional design choices, and consider how they impact issues known to affect retention.

Debugging: a review of the literature from an educational perspective

This paper reviews the literature related to the learning and teaching of debugging computer programs. Debugging is an important skill that continues to be both difficult for novice programmers to learn and challenging for computer science educators to teach. These challenges persist despite a wealth of important research on the subject dating back as far as the mid 1970s. Although the tools and languages novices use for writing programs today are notably different from those employed decades earlier, the basic problem-solving and pragmatic skills necessary to debug them effectively are largely similar. Hence, an understanding of the previous work on debugging can offer computer science educators insights into how to improve contemporary learning and teaching of debugging and may suggest directions for future research into this important area. This overview of the debugging literature is organized around four questions relevant to computer science educators and education researchers: What causes bugs to occur? What types of bugs occur? What is the debugging process? How can we improve the learning and teaching of debugging? We conclude with suggestions on using the existing literature both to facilitate pedagogical improvements to debugging education and to offer guidance for future research.

Debugging: finding, fixing and flailing, a multi-institutional study of novice debuggers

Debugging is often difficult and frustrating for novices. Yet because students typically debug outside the classroom and often in isolation, instructors rarely have the opportunity to closely observe students while they debug. This paper describes the details of an exploratory study of the debugging skills and behaviors of contemporary novice Java programmers. Based on a modified replication of Katz and Andersons study of novices, we sought to broadly survey the modern landscape of novice debugging abilities. As such, this study reports general quantitative results and fills in the picture with qualitative detail from a relatively small, but varied sample. Comprehensive interviews involving both a programming and a debugging task, followed by a semi-structured interview and a questionnaire, were conducted with 21 CS2 students at seven colleges and universities. While many subjects successfully debugged a representative set of typical CS1 bugs, there was a great deal of variation in their success at the programming and debugging tasks. Most of the students who were good debuggers were good novice programmers, although not all of the good programmers were successful at debugging. Students employed a variety of strategies to find 70% of all bugs and of the bugs they found they were able to fix 97% of them. They had the most difficulty with malformed statements, such as arithmetic errors and incorrect loop conditions. Our results confirm many findings from previous studies (some quite old) – most notably that once students find bugs, they can fix them. However, the results also suggest that some changes have occurred in the student population, particularly an increased use of debugging tools and online resources, as well as the use of pattern matching, which has not previously been reported.