Michelle Craig

Reviewing CS1 exam question content

Many factors have been cited for poor performance of students in CS1. To investigate how assessment mechanisms may impact student performance, nine experienced CS1 instructors reviewed final examinations from a variety of North American institutions. The majority of the exams reviewed were composed predominantly of high-value, integrative code-writing questions, and the reviewers regularly underestimated the number of CS1 concepts required to answer these questions. An evaluation of the content and cognitive requirements of individual questions suggests that in order to succeed, students must internalize a large amount of CS1 content. This emphasizes the need for focused assessment techniques to provide students with the opportunity to demonstrate their knowledge.

Plagiarism detection using feature-based neural networks

This paper focuses on the use of code features for automatic plagiarism detection. Instead of the text-based analyses employed by current plagiarism detectors, we propose a system that is based on properties of assignments that course instructors use to judge the similarity of two submissions. This system uses neural network techniques to create a feature-based plagiarism detector and to measure the relevance of each feature in the assessment. The system was trained and tested on assignments from an introductory computer science course, and produced results that are comparable to the most popular plagiarism detectors.

Gr8 designs for Gr8 girls: a middle-school program and its evaluation

In order to address the under-representation of women in Computer Science, we have created a program for middle-school girls that specifically aims to change their attitudes about CS and encourages them to see it as a potential career. Our assessment of the program shows that it did indeed have a significant, positive impact and suggests that this was still in effect three months later. This paper describes the program and its assessment, and makes suggestions for those considering offering a similar program.

Evaluating an inverted CS1

This case study explores an inverted classroom offering of an introductory programming course (CS1). Students prepared for lecture by watching short lecture videos and completing required in-video quiz questions. During lecture, the students worked through exercises with the support of the instructor and teaching assistants. We describe the course implementation and its assessment, including pre- and post-course surveys. We also discuss lessons learned, modifications that we plan to make for the next offering, and recommendations for others teaching inverted courses.

Comparing outcomes in inverted and traditional CS1

We compare a traditional CS1 offering with an inverted offering delivered the following year to a comparable student population. We measure student attitudes, grades, and final course outcomes and find that, while students in the inverted offering do not report increased enjoyment and are no more likely to pass, learning as measured by final exam performance increases significantly. This increase is not simply a function of a more onerous inverted offering, as students report spending similar time per week in the traditional and inverted offerings. Contrary to our hypotheses, however, we find no evidence that the the inverted offering disproportionally helps beginners or those not fully fluent in English.

Drop, Fail, Pass, Continue: Persistence in CS1 and Beyond in Traditional and Inverted Delivery

Much attention has been paid to the failure rate in CS1 and attrition between CS1 and CS2. In our study of 1236 CS1 students, we examine subgroups of students, to find out how characteristics such as prior experience and reason for taking the course influence who drops, fails, or passes, and who continues on to CS2. We also examine whether student characteristics influence outcomes differently in traditional vs. inverted offerings of the course. We find that more students in the inverted offering failed the midterm test, but those who failed were much more likely to either drop the course or recover and ultimately pass the course. While we find no difference between the offerings in the overall drop-fail-pass rates or in the percentage and types of students who go on to take CS2, there is a significant, widely felt, boost in exam grades in the inverted offering.

Forming reasonably optimal groups: (FROG)

Instructors often put students into groups for coursework. Several tools exist to facilitate this process, but they typically limit the criteria one can use for forming groups. We have defined a general mathematical model for group formation: a set of attribute types, group-formation criteria, and fitness measures. We have implemented an optimizer that uses an evolutionary algorithm to create groups according to the instructors criteria. Our experiments support the hypothesis that, even with a general model, reasonably optimal solutions to the group-formation problem can be found in reasonable time. Several instructors have used the tool to form groups for their courses. In all cases, they were impressed by the expressiveness of the model and pleased with the quality of the groups produced.

Comparing and contrasting different algorithms leads to increased student learning

Comparing and contrasting different solution approaches is known in math education and cognitive science to increase student learning -- what about CS? In this experiment, we replicated work from Rittle-Johnson and Star, using a pretest--intervention--posttest--follow-up design (n=241). Our intervention was an in-class workbook in CS2. A randomized half of students received questions in a compare-and-contrast style, seeing different code for different algorithms in parallel. The other half saw the same code questions sequentially, and evaluated them one at a time. Students in the former group performed better with regard to procedural knowledge (code reading & writing), and flexibility (generating, recognizing & evaluating multiple ways to solve a problem). The two groups performed equally on conceptual knowledge. Our results agree with those of Rittle-Johnson and Star, indicating that the existing work in this area generalizes to CS education.

Facilitated student discussions for evaluating teaching

Trying to improve undergraduate teaching based on feedback collected by traditional student course evaluations can be a frustrating experience. Unclear, contradictory and ill-informed student comments leave instructors confused and discouraged. We designed and then implemented an evaluation mechanism where an independent CS faculty peer visits a lecture and holds an evaluation discussion with the students. These facilitated discussions begin by looking at overall strengths and weaknesses for the course but quickly focus on the key student concerns and suggestions for improvement. After conducting thirty four facilitated discussions, we find them appreciated by students who feel heard and valued. A survey of participating faculty indicates that the written discussion report is more useful to them than standard student survey results. Faculty report that they have made changes based on the recommendations and limited quantitative data suggests that teaching has improved and its value in the departmental culture has increased. In this paper we describe the evaluation process, discuss our experiences and offer some concrete suggestions for those who might want to try this approach in their own department.

Revisiting why students drop CS1

This paper describes a qualitative study of the factors that contribute to a students decision to withdraw from CS1. Individual interviews were held with 18 students in a majors-focused CS1 at a large, research-intensive North American university, and results both validate and extend previous work on the experience of students who struggle in introductory computer science. In particular, our analysis confirms the complexity of the decision to drop, with students citing a combination of interrelated factors that contribute to the decision. Lack of time, combined with ineffective study strategies or with a prioritization of other courses, were the most commonly cited combinations of factors. Interestingly, when compared to the experience of students who chose to complete the course, there is evidence that students encounter a decision point when they realize they are or soon will be behind. Students who drop speak of focusing on other priorities or being unable to catch up, while students who complete speak of understanding the need to use new techniques for learning and increasing their efforts.