Christopher D. Hundhausen

A Meta-Study of Algorithm Visualization Effectiveness

Algorithm visualization (AV) technology graphically illustrates how algorithms work. Despite the intuitive appeal of the technology, it has failed to catch on in mainstream computer science education. Some have attributed this failure to the mixed results of experimental studies designed to substantiate AV technologys educational effectiveness. However, while several integrative reviews of AV technology have appeared, none has focused specifically on the softwares effectiveness by analyzing this body of experimental studies as a whole. In order to better understand the effectiveness of AV technology, we present a systematic meta-study of 24 experimental studies. We pursue two separate analyses: an analysis ofindependent variables , in which we tie each study to a particular guiding learning theory in an attempt to determine which guiding theory has had the most predictive success; and an analysis of dependent variables, which enables us to determine which measurement techniques have been most sensitive to the learning benefits of AV technology. Our most significant finding is that how students use AV technology has a greater impact on effectiveness than what AV technology shows them. Based on our findings, we formulate an agenda for future research into AV effectiveness.

Exploring the role of visualization and engagement in computer science education

Visualization technology can be used to graphically illustrate various concepts in computer science. We argue that such technology, no matter how well it is designed, is of little educational value unless it engages learners in an active learning activity. Drawing on a review of experimental studies of visualization effectiveness, we motivate this position against the backdrop of current attitudes and best practices with respect to visualization use. We suggest a new taxonomy of learner engagement with visualization technology. Grounded in Blooms well-recognized taxonomy of understanding, we suggest metrics for assessing the learning outcomes to which such engagement may lead. Based on these taxonomies of engagement and effectiveness metrics, we present a framework for experimental studies of visualization effectiveness. Interested computer science educators are invited to collaborate with us by carrying out studies within this framework.

Deictic Roles of External Representations in Face-to-face and Online Collaboration.

This research explores how shared, learner-constructed representations serve as resources for conversation in face-to-face and online situations. An important role of shared representations in collaborative learning is to facilitate the ease of reference to previously introduced ideas. Complex ideas are more easily expressed when their component ideas can be indicated with simple gestures. Yet gesture does not have the same immediacy in typical online learning environments. We examined the extent to which gestural deixis is inhibited online, and how shared representations serve as conversational resources in other ways. Results show that gesture was almost never used online, and was partially replaced with verbal deixis and direct manipulation of the shared representation. Verbal deixis almost always referenced ideas already in the focus of attention, posing a potential problem for reflection on prior information. These results suggest the importance of better integration between communicative tools and shared representations and the inclusion of prompts for reflection.

Low Fidelity Algorithm Visualization

Computer science educators have traditionally used algorithm visualization (AV) software to create graphical representations of algorithms for use as visual aids in lectures, or as the basis for interactive labs. Typically, such visualizations are high-fidelity in the sense that (a) they depict the target algorithm for arbitrary input, and (b) they tend to have the polished look of textbook figures. In contrast, low-fidelity visualizations illustrate the target algorithm for a few, carefully chosen input data sets, and tend to have a sketched, unpolished appearance. Drawing on ethnographic field studies of a junior-level undergraduate algorithms course, we motivate the use of low-fidelity AV technology as the basis for an alternative learning paradigm in which students construct their own visualizations, and then present those visualizations to their instructor and peers for feedback and discussion. To explore the design space of low-fidelity AV technology, we present SALSA (Spatial ALgorithmic Language for StoryboArding) and ALVIS (ALgorithmVI sualization Storyboarder), a prototype end-user language and system firmly rooted in empirical studies in which students constructed and presented visualizations made out of simple art supplies. Our prototype end-user language and system pioneer a novel technique for programming of visualizations based on spatial relations, and a novel presentation interface that supports human discussions about algorithms by enabling reverse execution and dynamic mark-up and modification. Moreover, the prototype provides an ideal foundation for what we see as the algorithms classroom of the future: the interactive ‘algorithms studio’.

Toward effective algorithm visualization artifacts: designing for participation and negotiation in an undergraduate algorithms course

Despite their intuitive appeal, computer-based algorithm visuahzation (AV) artifacts have failed to enter mainstream computer science education. I argue that past research into the design, evaluation, and pedagogical use of AV artifacts has been guided by an underlying theory of effectiveness that is fundamentally deficient. Inspired by an alternative pedagogy in which students construct their own AVs, and by recent research into the situated nature of communication and learning, my dissertation develops an alternative theory that stresses the value of AV artitacts both in facilitating students’ participation in the Community of Algorithmaticians, and in providing students and instructors with resources for negotiating a shared understanding of algorithms.

Can direct manipulation lower the barriers to computer programming and promote transfer of training?: An experimental study

Novices face many barriers when learning to program a computer, including the need to learn both a new syntax and a model of computation. By constraining syntax and providing concrete visual representations on which to operate, direct manipulation programming environments can potentially lower these barriers. However, what if the ultimate learning goal of the novice is to be able to program in conventional textual languages, as is the case for introductory computer science students? Can direct manipulation programming environments lower the initial barriers to programming, and, at the same time, facilitate positive transfer to textual programming? To address this question, we designed a new direct manipulation programming interface for novices, and conducted an experimental study to compare the programming processes and outcomes promoted by the direct manipulation interface against those promoted by a textual programming interface. We found that the direct manipulation interface promoted significantly better initial programming outcomes, positive transfer to the textual interface, and significant differences in programming processes. Our results show that direct manipulation interfaces can provide novices with a “way in” to traditional textual programming.

Toward empirically-based software visualization languages

Underlying any single-user software visualization (SV) system is a visualization language onto which its users must map the computations they would like to visualize with the system. We hypothesize that the usability of such systems turns on their ability to provide an underlying visualization language that accords with the ways in which their users conceptualize the computations to be visualized. To explore the question of how to design visualization languages grounded in human conceptualization, we present an empirical study that made use of a research method called visualization storyboarding to investigate the human conceptualization of the bubblesort algorithm. Using an analytical framework based on entities, attributes, and transformations, we derive a semantic-level visualization language for bubblesort, in terms of which all visualizations observed in our study can be expressed. Our empirically-based visualization language provides a means for predicting the usability of the visualization language defined by Lens (Mukherjea and Stasko, 1994), a prototypical single-user SV system. We draw from a follow-up usability study of Lens to substantiate our predictions.

Talking about code: Integrating pedagogical code reviews into early computing courses

Given the increasing importance of soft skills in the computing profession, there is good reason to provide students with more opportunities to learn and practice those skills in undergraduate computing courses. Toward that end, we have developed an active learning approach for computing education called the Pedagogical Code Review (PCR). Inspired by the code inspection process used in the software industry, a PCR is a collaborative activity in which a small team of students, led by a trained moderator: (a) walk through segments of each others programming solutions, (b) check the code against a list of best coding practices, and (c) discuss and log issues that arise. To evaluate the viability and effectiveness of this approach, we conducted a series of four mixed-method empirical studies of various implementations of PCRs in CS1 courses at Washington State University. The first study validated the viability of the PCR activity. Using a quasi-experimental design, the final three studies evaluated two alternative implementations of PCRs—face-to-face and online. Our results provide evidence that PCRs can promote positive attitudinal shifts, and hone skills in critical review, teamwork, and communication. Based on our findings, we present a set of best practices for implementing PCRs.

On the design of an educational infrastructure for the blind and visually impaired in computer science

The blind and visually impaired community is significantly underrepresented in computer science. Students who wish to enter the discipline must overcome significant technological and educational barriers to succeed. In an attempt to help this population, we are engaged in a three-year research project to build an educational infrastructure for blind and visually impaired middle and high school students. Our primary research goal is to begin forging a multi-sensory educational infrastructure for the blind across the United States. We present here two preliminary results from this research: 1) a new auditory programming environment called Sodbeans, a programming language called Hop, and a multi-sensory (sound and touch) curriculum, and 2) an empirical study of our first summer workshop with the blind students. Results show that students reported a significant increase in programming self-efficacy after participating in our camp.

A methodology for analyzing the temporal evolution of novice programs based on semantic components

Empirical studies of novice programming typically rely on code solutions or test responses as the basis of their analyses. While such data can provide insight into novice programming knowledge, they say little about the programming processes in which novices engage. For those interested in improving novice programming environments, a key research question arises: How can we collect and analyze data on novice programming that will enable us (a) to analyze and compare the programming processes promoted by alternative novice programming environments, and (b) ultimately to build better novice programming environments? To address this question, we have collected a large video corpus of novices as they construct code solutions in various versions of ALVIS Live! [17], a novice programming environment. Through detailed post-hoc analyses of our video corpus, we have developed a methodology for compiling the moment-by-moment evolution of novice code solutions. Based on an analysis of a model code solutions key semantic components, our methodology enables researchers to document, on a second-by-second basis, (a) what part of a code solution a programmer is focusing on, and (b) where the semantic feedback provided by the programming environment is helping. Although it is time and labor intensive, our methodology provides researchers with a standard set of data and representations for comparing the programming processes promoted by alternative programming environments.