Brett A. Becker

An Effective Approach to Enhancing Compiler Error Messages

One of the many challenges novice programmers face from the time they write their first program is inadequate compiler error messages. These messages report details on errors the programmer has made and are the only feedback the programmer gets from the compiler. For students they play a particularly essential role as students often have little experience to draw upon, leaving compiler error messages as their primary guidance on error correction. However these messages are frequently inadequate, presenting a barrier to progress and are often a source of discouragement. We have designed and implemented an editor that provides enhanced compiler error messages and conducted a controlled empirical study with CS1 students learning Java. We find a reduced frequency of overall errors and errors per student. We also identify eight frequent compiler error messages for which enhancement has a statistically significant effect. Finally we find a reduced number of repeated errors. These findings indicate fewer students struggling with compiler error messages.

Towards Data Partitioning for Parallel Computing on Three Interconnected Clusters

We present a new data partitioning strategy for parallel computing on three interconnected clusters. This partitioning has two advantages over existing partitionings. First it can reduce communication time due to a lower total volume of communication and a more efficient communication schedule. When the network topology is a linear array this partitioning always results in a lower total volume of communication compared to existing partitionings, provided the most powerful node is at the center of the array. When the topology is fully connected this partitioning results in a lower total volume of communication for all but a few power ratios. Second, it allows for the overlapping of communication and computation. These two inherent advantages work together to reduce overall execution time significantly.

A New Metric to Quantify Repeated Compiler Errors for Novice Programmers

Encountering the same compiler error repeatedly, particularly several times consecutively, has been cited as a strong indicator that a student is struggling with important programming concepts. Despite this, there are relatively few studies which investigate repeated errors in isolation or in much depth. There are also few data-driven metrics for measuring programming performance, and fewer for measuring repeated errors. This paper makes two contributions. First we introduce a new metric to quantify repeated errors, the repeated error density (RED). We compare this to Jaduds Error Quotient (EQ), the most studied metric, and show that RED has advantages over EQ including being less context dependent, and being useful for short sessions. This allows us to answer two questions posited by Jadud in 2006 that have until now been unanswered. Second, we compare the EQ and RED scores using data from an empirical control/intervention group study involving an editor which enhances compiler error messages. This intervention group has been previously shown to have a reduced overall number of student errors, number of errors per student, and number of repeated student errors per compiler error message. In this research we find a reduction in EQ, providing further evidence that error message enhancement has positive effects. In addition we find a significant reduction in RED providing evidence that this metric is valid.

Matrix Multiplication on Two Interconnected Processors

This paper presents a new partitioning algorithm to perform matrix multiplication on two interconnected heterogeneous processors. Data is partitioned in a way which minimizes the total volume of communication between the processors compared to more general partitionings, resulting in a lower total execution time whenever the power ratio between the processors is greater than 3:1. The algorithm has interesting and important applicability, particularly as the top-level partitioning in a hierarchal algorithm that is to perform matrix multiplication on two interconnected clusters of computers

Effective Compiler Error Message Enhancement for Novice Programming Students.

Programming is an essential skill that many computing students are expected to master. However, programming can be difficult to learn. Successfully interpreting compiler error messages (CEMs) is crucial for correcting errors and progressing toward success in programming. Yet these messages are often difficult to understand and pose a barrier to progress for many novices, with struggling students often exhibiting high frequencies of errors, particularly repeated errors. This paper presents a control/intervention study on the effectiveness of enhancing Java CEMs. Results show that the intervention group experienced reductions in the number of overall errors, errors per student, and several repeated error metrics. These results are important as the effectiveness of CEM enhancement has been recently debated. Further, generalizing these results should be possible at least in part, as the control group is shown to be comparable to those in several studies using Java and other languages.

Max-plus algebra and discrete event simulation on parallel hierarchical heterogeneous platforms

In this paper we explore computing max-plus algebra operations and discrete event simulations on parallel hierarchal heterogeneous platforms. When performing such tasks on heterogeneous platforms parameters such as the total volume of communication and the top-level data partitioning strategy must be carefully taken into account. Choice of the partitioning strategy is shown to greatly affect the overall performance of these applications due to different volumes of inter-partition communication that various strategies impart on these operations. One partitioning strategy in particular is shown to reduce the execution times of these operations more than other, more traditional strategies. The main goal of this paper is to present benefits waiting to be exploited by the use of max-plus algebra operations on these platforms and thus speeding up more complex and quite common computational topic areas such as discrete event simulation.

The Effects of Enhanced Compiler Error Messages on a Syntax Error Debugging Test

There is an active strand of research in the literature exploring the effects of Enhanced Compiler Error Messages on student programming behavior, however many results seem conflicting. This is compounded by the fact that directly comparing these results is difficult as these studies utilize different metrics, and what metrics are best suited to measure the effects of enhanced compiler error messages is not known. Common to most studies to-date is that the metrics employed measure how many errors students produce, and/or rectify while writing programs. This study takes a different approach by measuring how many pre-existing syntax errors are rectified by students while debugging programs. Specifically, we measured the effect of enhanced compiler error messages in an empirical control/intervention experiment where students were given the task of removing syntax errors from non-compiling source code they did not write. We find a significant positive effect on the overall number of errors rectified, as well as the number of certain specific error types, but no significant effect on the number of non-compiling submissions or student scores. These results (in different ways) support the findings of several recent studies and suggest that their results may not be as conflicting as they seem. This is evidence that enhanced error messages may be effective, but also that the signal of these effects are relatively weak, indicating that how and what is measured when attempting to observe these effects is important.

Developing Assessments to Determine Mastery of Programming Fundamentals

Current CS1 learning outcomes are relatively general, specifying tasks such as designing, implementing, testing and debugging programs that use some fundamental programming constructs. These outcomes impact what we teach, our expectations, and our assessments. Although prior work has demonstrated the utility of single concept assessments, most assessments used in formal examinations combine numerous heterogeneous concepts, resulting in complex and difficult tasks. The exclusive use of these traditional assessments results in two major problems. Firstly, teachers who are administering the assessments find it difficult to quantify what students are struggling with. A program that fails to compile or fails to pass test cases provides little information about what a student can successfully achieve. Secondly, it deprives the students of feedback about what they do know and what they can achieve, which we know is one of the most significant factors that influence student learning. Our traditional compound assessment tasks limit the nature and quality of feedback to teachers and students alike, and may contribute to the perceived difficulty and high drop out rates observed in introductory programming courses. This working group aims to decompose existing CS1 learning outcomes into their component parts, and develop assessment items focused on these individual components. We aim to create exemplar assessments that may be used for formative or summative feedback on student understanding of specific components of programming knowledge. Focusing on independent components of programming increases opportunities for students to demonstrate what they can achieve, and may improve diagnosis of student difficulties. It is likely that this project would be of particular relevance to teachers and researchers interested in mastery learning, or those who wish to demonstrate tight coupling between learning outcomes and assessment tasks. It may also be of relevance to teachers and researchers interested in promoting a more positive mindset in the classroom, focusing on what students do know and what they can achieve during a typical CS1 course.

How statistics are used in computing education research

Conferences such as ITiCSE have recently seen an increase in the number of papers presenting empirical research in computing education. While empirical research need not be quantitative, there has been a corresponding increase in the number of papers that present some level of statistical analysis to support their arguments. This poster introduces a project that is exploring how -- and how well -- statistics are used and reported in computing education research.

A review of introductory programming research 2003–2017

A broad review of research on the teaching and learning of programming was conducted by Robins et al. in 2003. Since this work there have been several reviews of research concerned with the teaching and learning of programming, in particular introductory programming. However, these reviews have focused on highly specific aspects, such as student misconceptions, teaching approaches, program comprehension, potentially seminal papers, research methods applied, automated feedback for exercises, competency-enhancing games, and program visualisation. While these aspects encompass a wide range of issues, they do not cover the full scope of research into novice programming. Some notable areas that have not been reviewed are assessment, academic integrity, and novice student attitudes to programming. There does not appear to have been a comprehensive review of research into introductory programming since that of Robins et al. It is therefore timely to conduct and present such a review in order to gain an understanding of the research focuses, to highlight advances in knowledge since 2003, and to indicate possible future directions for research. The working group will conduct a systematic literature review based on the guidelines proposed by Kitchenham et al. This research project is well suited to an ITiCSE working group as the synthesis and discussion of the literature will benefit from input from a variety of researchers drawn from different backgrounds and countries.