Andrew Head

Lamello: Passive Acoustic Sensing for Tangible Input Components

We describe Lamello, an approach for creating tangible input components that recognize user interaction via passive acoustic sensing. Lamello employs comb-like structures with varying-length tines at interaction points (e.g., along slider paths). Moving a component generates tine strikes; a real-time audio processing pipeline analyzes the resultant sounds and emits high-level interaction events. Our main contributions are in the co-design of the tine structures, information encoding schemes, and audio analysis. We demonstrate 3D printed Lamello-powered buttons, sliders, and dials.

Writing Reusable Code Feedback at Scale with Mixed-Initiative Program Synthesis

In large introductory programming classes, teacher feedback on individual incorrect student submissions is often infeasible. Program synthesis techniques are capable of fixing student bugs and generating hints automatically, but they lack the deep domain knowledge of a teacher and can generate functionally correct but stylistically poor fixes. We introduce a mixed-initiative approach which combines teacher expertise with data-driven program synthesis techniques. We demonstrate our novel approach in two systems that use different interaction mechanisms. Our systems use program synthesis to learn bug-fixing code transformations and then cluster incorrect submissions by the transformations that correct them. The MistakeBrowser system learns transformations from examples of students fixing bugs in their own submissions. The FixPropagator system learns transformations from teachers fixing bugs in incorrect student submissions. Teachers can write feedback about a single submission or a cluster of submissions and propagate the feedback to all other submissions that can be fixed by the same transformation. Two studies suggest this approach helps teachers better understand student bugs and write reusable feedback that scales to a massive introductory programming classroom.

Tutorons: Generating context-relevant, on-demand explanations and demonstrations of online code

Programmers frequently turn to the web to solve problems and find example code. For the sake of brevity, the snippets in online instructions often gloss over the syntax of languages like CSS selectors and Unix commands. Programmers must compensate by consulting external documentation. In this paper, we propose language-specific routines called Tutorons that automatically generate context-relevant, on-demand micro-explanations of code. A Tutoron detects explainable code in a web page, parses it, and generates in-situ natural language explanations and demonstrations of code. We build Tutorons for CSS selectors, regular expressions, and the Unix command “wget”. We demonstrate techniques for generating natural language explanations through template instantiation, synthesizing code demonstrations by parse tree traversal, and building compound explanations of co-occurring options. Through a qualitative study, we show that Tutoron-generated explanations can reduce the need for reference documentation in code modification tasks.

Exploring the Design Space of Automatically Synthesized Hints for Introductory Programming Assignments

For massive programming classrooms, recent advances in program synthesis offer means to automatically grade and debug student submissions, and generate feedback at scale. A key challenge for synthesis-based autograders is how to design personalized feedback for students that is as effective as manual feedback given by teachers today. To understand the state of hint-giving practice, we analyzed 132 online Q&A posts and conducted a semi-structured interview with a teacher from a local massive programming class. We identified five types of teacher hints that can also be generated by program synthesis. These hints describe transformations, locations, data, behavior, and examples. We describe our implementation of three of these hint types. This work paves the way for future deployments of automatic, pedagogically-useful programming hints driven by program synthesis.

ToneWars: Connecting Language Learners and Native Speakers through Collaborative Mobile Games

In this paper, we present ToneWars, a collaborative mobile game for learning Chinese as a Second Language (CSL). ToneWars provides a learning experience that combines mastery learning, microlearning, and group-based interaction between CSL learners and native speakers. The game explores how unique input modalities, like touch gestures and speech recognition, can improve language acquisition tasks on mobile devices. We report the design motivations and lessons learned through the iterative design process. We believe many insights from developing ToneWars are generalizable to designing productive language learning technology. Through a 24-participant (12 CSL and native speaker pairs) user study, we found ToneWars provides learning benefits for second-language learners and engages native speakers.

WiFröst: Bridging the Information Gap for Debugging of Networked Embedded Systems

The rise in prevalence of Internet of Things (IoT) technologies has encouraged more people to prototype and build custom internet connected devices based on low power microcontrollers. While well-developed tools exist for debugging network communication for desktop and web applications, it can be difficult for developers of networked embedded systems to figure out why their network code is failing due to the limited output affordances of embedded devices. This paper presents WiFröst, a new approach for debugging these systems using instrumentation that spans from the device itself, to its communication API, to the wireless router and back-end server. WiFröst automatically collects this data, displays it in a web-based visualization, and highlights likely issues with an extensible suite of checks based on analysis of recorded execution traces.

Can Human Development be Measured with Satellite Imagery

In many developing country environments, it is difficult or impossible to obtain recent, reliable estimates of human development. Nationally representative household surveys, which are the standard instrument for determining development policy and priorities, are typically too expensive to collect with any regularity. Recently, however, researchers have shown the potential for remote sensing technologies to provide a possible solution to this data constraint. In particular, recent work indicates that satellite imagery can be processed with deep neural networks to accurately estimate the sub-regional distribution of wealth in sub-Saharan Africa. In this paper, we explore the extent to which the same approach--- of using convolutional neural networks to process satellite imagery--- can be used to measure a broader set of human development indicators, in a broader range of geographic contexts. Our analysis produces three main results: First, we successfully replicate prior work showing that satellite images can accurately infer a wealth-based index of poverty in sub-Saharan Africa. Second, we show that this approach can generalize to predicting poverty in other countries and continents, but that the performance is sensitive to the hyperparameters used to tune the learning algorithm. Finally, we find that this approach does not trivially generalize to predicting other measures of development such as educational attainment, access to drinking water, and a variety of health-related indicators. We discuss in detail whether these findings represent a fundamental limitation of this approach, or could be fixed through more concerted adaptations of the machine learning environment.