Sébastien Cuendet

Designing augmented reality for the classroom

Augmented reality (AR) has recently received a lot of attention in education. Multiple AR systems for learning have been developed and tested through empirical studies often conducted in lab settings. While lab studies can be insightful, they leave out the complexity of a classroom environment. We developed three AR learning environments that have been used in genuine classroom contexts, some of them being now part of classroom regular practices. These systems and the learning activities they provide have been co-designed with teachers, for their own classrooms, through multiple cycles of prototyping and testing. We present here the features that emerged from these co-design cycles and abstract them into design principles.

VideoKheti: making video content accessible to low-literate and novice users

Designing ICT systems for rural users in the developing world is difficult for a variety of reasons ranging from problems with infrastructure to wide differences in user contexts and capabilities. Developing regions may include huge variability in spoken languages, and users are often low- or non-literate, with very little experience interacting with digital technologies. Researchers have explored the use of text-free graphical interfaces as well as speech-based applications to overcome some of the issues related to language and literacy. While there are benefits and drawbacks to each of these approaches, they can be complementary when used together. In this work, we present VideoKheti, a mobile system using speech, graphics, and touch interaction for low-literate farmers in rural India. VideoKheti helps farmers to find and watch agricultural extension videos in their own language and dialect. In this paper, we detail the design and development of VideoKheti and report on a field study with 20 farmers in rural India who were asked to find videos based on a scenario. The results show that farmers could use VideoKheti, but their success still greatly depended on their education level. While participants were enthusiastic about using the system, the multimodal interface did not overcome many obstacles for low-literate users.

Task performance vs. learning outcomes: a study of a tangible user interface in the classroom

Tangible User Interfaces (TUIs) offer the potential to facilitate collaborative learning in new ways. This paper presents an empirical study that investigated the effects of a TUI in a classroom setting on task performance and learning outcomes. In the tangible condition, apprentices worked together around an interactive tabletop warehouse simulation using tangible inputs. In the paper condition, they performed the same activity with only paper and pens. Results showed that the tangible condition resulted in better task performance (more alternative solutions explored and better final solution) but did not affect learning outcomes, i.e. understanding of important concepts and applying them to a problem-solving question. We discuss reasons for this in terms of task structure and type, nature of tangible user interfaces and effective interaction requirements.

Tangible vs. virtual representations: when tangibles benefit the training of spatial skills

Tangible user interfaces (TUIs) have been the focus of much attention in the HCI and learning communities because of their many potential benefits for learning. However, there have recently been debates about whether TUIs can actually increase learning outcomes and if so, under which conditions. In this article, we investigate the effect of object representation (physical vs. virtual) on learning in the domain of spatial skills. We ran a comparative study with 46 participants to measure the effects of the object representation on the ability to establish a link between 2D and 3D representations of an object. The participants were split into two conditions: in the first one, the 3D representation of the object was virtual; in the second one, it was tangible. Findings show that in both conditions the TUI led to a significant improvement of the spatial skills. The learning outcomes were not different between the two conditions, but the performance during the activities was significantly higher when using the tangible representation as opposed to the virtual one, and even more so in for difficult cases.

An integrated way of using a tangible user interface in a classroom

Despite many years of research in CSCL, computers are still scarcely used in classrooms today. One reason for this is that the constraints of the classroom environment are neglected by designers. In this contribution, we present a CSCL environment designed for a classroom usage from the start. The system, called TapaCarp, is based on a tangible user interface (TUI) and was designed to help train carpenter apprentices. A previous study (Cuendet and Dillenbourg 2013) showed that the tangible nature of TapaCarp helped integrate it in the classroom environment, but that this did not guarantee a meaningful learning activity. In this article, we describe the process that led us to design a new learning classroom activity for the particular context of dual carpentry apprenticeships. One innovative aspect of the activity is that TapaCarp is used only for a small part of it. This contrasts with the mainstream CSCL approach that assumes that the system must be used from beginning to end of the activity. This new activity was used in a classroom study with 3 classes of carpenter apprentices over two days for each class. Despite its many steps, the activity proved usable and fostered many connections to the workplace, which was one of its main purposes. The teacher and the students were positive and showed high engagement in the activity. The learning gain results were mixed: the performance of the students improved from day 1 to day 2, but the learning gain measured with a pre-test/post-test mechanism did not show any significant difference compared to that of a control group.

A Hindi speech recognizer for an agricultural video search application

Voice user interfaces for ICTD applications have immense potential in their ability to reach to a large illiterate or semi-literate population in these regions where text-based interfaces are of little use. However, building speech systems for a new language is a highly resource intensive task. There have been attempts in the past to develop techniques to circumvent the need for large amounts of data and technical expertise required to build such systems. In this paper we present the development and evaluation of an application specific speech recognizer for Hindi. We use the Salaam method [4] to bootstrap a high quality speech engine in English to develop a mobile speech based agricultural video search for farmers in India. With very little training data for a 79 word vocabulary we are able to achieve >90% accuracies for test and field deployments. We report some observations from field that we believe are critical to the effective development and usability of a speech application in ICTD.

Paper interface design for classroom orchestration

Designing computer systems for educational purpose is a difficult task. While many of them have been developed in the past, their use in classrooms is still scarce. We make the hypothesis that this is because those systems take into account the needs of individuals and groups, but ignore the requirements inherent in their use in a classroom. In this work, we present a computer system based on a paper and tangible interface that can be used at all three levels of interaction: individual, group, and classroom. We describe the current state of the interface design and why it is appropriate for classroom orchestration, both theoretically and through two examples for teaching geometry.

Using Mobile Eye-Trackers to Unpack the Perceptual Benefits of a Tangible User Interface for Collaborative Learning

In this study, we investigated the way users memorize, analyze, collaborate, and learn new concepts on a Tangible User Interface (TUI). Twenty-seven pairs of apprentices in logistics (N = 54) interacted with an interactive simulation of a warehouse. Their task was to discover efficient design principles for building storehouses. In a between-subjects experimental design, half of the participants used 3D physical shelves, whereas the other half used 2D paper shelves. This manipulation allowed us to control for the “representational effect” of 3D tangibles: the first group saw the warehouse as a small-scale model with realistic shelves, whereas the second group had access to a more abstract layout with rectangular pieces of paper. Both groups interacted with the system in the same way. We found that participants in the first group (i.e., who used 3D realistic shelves) better memorized a warehouse layout, built a more efficient model, and scored higher on a learning test. Additionally, students wore eye-tracking goggles while completing those tasks; preliminary results suggest that 3D interfaces increased joint visual attention, which was found to be a significant predictor for participants’ task performance and learning gains. Implications for designing TUIs in collaborative settings are discussed.

Leveraging mobile eye-trackers to capture joint visual attention in co-located collaborative learning groups

This paper describes a promising methodology for studying co-located groups: mobile eye-trackers. We provide a comprehensive description of our data collection and analysis processes so that other researchers can take advantage of this cutting-edge technology. Data were collected in a controlled experiment where 27 student dyads (N = 54) interacted with a Tangible User Interface. They first had to define some design principles for optimizing a warehouse layout by analyzing a set of Contrasting Cases, and build a small-scale layout based on those principles. The contributions of this paper are that: 1) we replicated prior research showing that levels of Joint Visual Attention (JVA) are correlated with collaboration quality across all groups; 2) we then qualitatively analyzed two dyads with high levels of JVA and show that it can hide a free-rider effect (Salomon and Globerson 1989); 3) in conducting this analysis, we additionally developed a new visualization (augmented cross-recurrence graphs) that allows researchers to distinguish between high JVA groups that have balanced and unbalanced levels of participations; 4) finally, we generalized this effect to the entire sample and found a significant negative correlation between dyads’ learning gains and unbalanced levels of participation (as computed from the eye-tracking data). We conclude by discussing implications for automatically analyzing students’ interactions using dual eye-trackers.

Grand Challenge Problem 4: Supporting Teacher Decision-Making Through Appropriate Feedback

The quality of teaching is significantly enhanced through feedback to teachers about their teaching. Whereas systems to show student learning exist, those showing the emotional state of the classroom do not. We argue that such systems could greatly improve teaching, and in consequence, pupil attainment. Data for such system could come from biometric sensors or from cameras and microphone monitoring the gaze of students, their restlessness, and the level of noise in the classroom. Such systems would be valuable in particular for teachers with less experience.