Leo Porter

Halving fail rates using peer instruction: a study of four computer science courses

Peer Instruction (PI) is a teaching method that supports student-centric classrooms, where students construct their own understanding through a structured approach featuring questions with peer discussions. PI has been shown to increase learning in STEM disciplines such as physics and biology. In this report we look at another indicator of student success the rate at which students pass the course or, conversely, the rate at which they fail. Evaluating 10 years of instruction of 4 different courses spanning 16 PI course instances, we find that adoption of the PI methodology in the classroom reduces fail rates by a per-course average of 61% (20% reduced to 7%) compared to standard instruction (SI). Moreover, we also find statistically significant improvements within-instructor. For the same instructor teaching the same course, we find PI decreases the fail rate, on average, by 67% (from 23% to 8%) compared to SI. As an in-situ study, we discuss the various threats to the validity of this work and consider implications of wide-spread adoption of PI in computing programs.

Success in introductory programming: what works?

How pair programming, peer instruction, and media computation have improved computer science education.

Retaining nearly one-third more majors with a trio of instructional best practices in CS1

Beginning in 2008, we introduced a new CS1 incorporating a trio of best practices intended to improve the quality of the course, appeal to a broader student body, and, hopefully, improve retention in the major. This trio included Media Computation, Pair Programming, and Peer Instruction. After 3 and 1/2 years (8 CS1 classes, 3 different instructors, and 1011 students passing the course) we find that 89% of the majors who pass the course are still studying computing one year later. This is an improvement of 18% over our average retention of 71% for the previous version of the course (measured since Fall 2001). If the focus shifts from retention of passing CS1 majors to retention of CS1 initially enrolled majors, multiple improvements--fewer students drop, more students pass, and more passing students are retained--compound to increase retention by 31% (from 51% to 82%). In this paper we analyze further aspects of these results, detail the three instructional design choices, and consider how they impact issues known to affect retention.

Fast thread migration via cache working set prediction

The most significant source of lost performance when a thread migrates between cores is the loss of cache state. A significant boost in post-migration performance is possible if the cache working set can be moved, proactively, with the thread. This work accelerates thread startup performance after migration by predicting and prefetching the working set of the application into the new cache. It shows that simply moving cache state performs poorly, and that moving the instruction working set can be even more critical than data. This paper demonstrates a technique that captures the access behavior of a thread, summarizes that behavior into a compact form for transfer between cores, and then prefetches appropriate data into the new caches based on the summary. It presents a detailed study of single-thread migration effects, and then demonstrates its utility on a speculative multithreading architecture. Working set prediction as much as doubles the performance of short-lived threads, and in a full speculative multithreading implementation, the technique is also shown to nearly double the effectiveness of the spawned threads.

Peer Instruction in computing: The value of instructor intervention

Research has demonstrated that Peer Instruction (PI) is an attractive pedagogical practice in computer science classes. PI has been shown to improve final exam performance over standard lecture, reduce failure rates, contribute to increased retention, and be widely valued by students. In addition, a recent study using isomorphic (same-concept) questions found that students are learning during peer discussion and not merely copying from neighbors. Though this prior work is useful for evaluating peer discussion, it does not capture learning that takes place after peer discussion when the instructor further expands on the concept through a whole-class discussion. In the present work, isomorphic questions were used to determine the value of a PI question from start to finish: solo vote, group discussion, group vote, and instructor-led classwide discussion. The analysis revealed that the value of the instructor-led classwide discussion was evident in increased student performance over peer-discussion alone (raw gains of 22% compared to 14%). Moreover, the instructor-led discussion was highly valuable for all groups of students (weak, average, and strong) and was of particular value for weak students. Importantly, the largest gains were associated with more challenging PI questions, further suggesting that instructor expertise was valuable when students struggled.

Predicting student success using fine grain clicker data

Recent research suggests that the first weeks of a CS1 course have a strong influence on end-of-course student performance. The present work aims to refine the understanding of this phenomenon by using in-class clicker questions as a source of student performance. Clicker questions generate per-lecture and per-question data with which to assess student understanding. This work demonstrates that clicker question performance early in the term predicts student outcomes at the end of the term. The predictive nature of these questions applies to code-writing questions, multiple choice questions, and the final exam as a whole. The most predictive clicker questions are identified and the relationships between these questions and final exam performance are examined.

Can peer instruction be effective in upper-division computer science courses?

Peer Instruction (PI) is an active learning pedagogical technique. PI lectures present students with a series of multiple-choice questions, which they respond to both individually and in groups. PI has been widely successful in the physical sciences and, recently, has been successfully adopted by computer science instructors in lower-division, introductory courses. In this work, we challenge readers to consider PI for their upper-division courses as well. We present a PI curriculum for two upper-division computer science courses: Computer Architecture and Theory of Computation. These courses exemplify several perceived challenges to the adoption of PI in upper-division courses, including: exploration of abstract ideas, development of high-level judgment of engineering design trade-offs, and exercising advanced mathematical sophistication. This work includes selected course materials illustrating how these challenges are overcome, learning gains results comparing these upper-division courses with previous lower-division results in the literature, student attitudinal survey results (N = 501), and pragmatic advice to prospective developers and adopters. We present three main findings. First, we find that these upper-division courses achieved student learning gains equivalent to those reported in successful lower-division computing courses. Second, we find that student feedback for each class was overwhelmingly positive, with 88% of students recommending PI for use in other computer science classes. Third, we find that instructors adopting the materials introduced here were able to replicate the outcomes of the instructors who developed the materials in terms of student learning gains and student feedback.

Mapping Out a Path from Hardware Transactional Memory to Speculative Multithreading

This research demonstrates that coming support for hardware transactional memory can be leveraged to significantly reduce the cost of implementing true speculative multithreading. In particular, it explores the path from eager conflict detection HTM to full support of efficient speculative multithreading, focusing on the case where frequent memory dependencies exist between speculative threads. The result is a unified memory architecture capable of effective support for transactional parallel workloads and efficient speculative multithreading.

Computer science concept inventories: past and future

Concept Inventories (CIs) are assessments designed to measure student learning of core concepts. CIs have become well known for their major impact on pedagogical techniques in other sciences, especially physics. Presently, there are no widely used, validated CIs for computer science. However, considerable groundwork has been performed in the form of identifying core concepts, analyzing student misconceptions, and developing CI assessment questions. Although much of the work has been focused on CS1 and a CI has been developed for digital logic, some preliminary work on CIs is underway for other courses. This literature review examines CI work in other STEM disciplines, discusses the preliminary development of CIs in computer science, and outlines related research in computer science education that contributes to CI development.

Accurate branch prediction for short threads

Multi-core processors, with low communication costs and high availability of execution cores, will increase the use of execution and compilation models that use short threads to expose parallelism. Current branch predictors seek to incorporate large amounts of control flow history to maximize accuracy. However, when that history is absent the predictor fails to work as intended. Thus, modern predictors are almost useless for threads below a certain length. Using a Speculative Multithreaded (SpMT) architecture as an example of a system which generates shorter threads, this work examines techniques to improve branch prediction accuracy when a new thread begins to execute on a different core. This paper proposes a minor change to the branch predictor that gives virtually the same performance on short threads as an idealized predictor that incorporates unknowable pre-history of a spawned speculative thread. At the same time, strong performance on long threads is preserved. The proposed technique sets the global history register of the spawned thread to the initial value of the program counter. This novel and simple design reduces branch mispredicts by 29% and provides as much as a 13% IPC improvement on selected SPEC2000 benchmarks.