Daniel Szafir

Pay attention!: designing adaptive agents that monitor and improve user engagement

Embodied agents hold great promise as educational assistants, exercise coaches, and team members in collaborative work. These roles require agents to closely monitor the behavioral, emotional, and mental states of their users and provide appropriate, effective responses. Educational agents, for example, will have to monitor student attention and seek to improve it when student engagement decreases. In this paper, we draw on techniques from brain-computer interfaces (BCI) and knowledge from educational psychology to design adaptive agents that monitor student attention in real time using measurements from electroencephalography (EEG) and recapture diminishing attention levels using verbal and nonverbal cues. An experimental evaluation of our approach showed that an adaptive robotic agent employing behavioral techniques to regain attention during drops in engagement improved student recall abilities 43% over the baseline regardless of student gender and significantly improved female motivation and rapport. Our findings offer guidelines for developing effective adaptive agents, particularly for educational settings.

ARTFul: adaptive review technology for flipped learning

Internet technology is revolutionizing education. Teachers are developing massive open online courses (MOOCs) and using innovative practices such as flipped learning in which students watch lectures at home and engage in hands-on, problem solving activities in class. This work seeks to explore the design space afforded by these novel educational paradigms and to develop technology for improving student learning. Our design, based on the technique of adaptive content review, monitors student attention during educational presentations and determines which lecture topic students might benefit the most from reviewing. An evaluation of our technology within the context of an online art history lesson demonstrated that adaptively reviewing lesson content improved student recall abilities 29% over a baseline system and was able to match recall gains achieved by a full lesson review in less time. Our findings offer guidelines for a novel design space in dynamic educational technology that might support both teachers and online tutoring systems.

Communication of intent in assistive free flyers

Assistive free-flyers (AFFs) are an emerging robotic platform with unparalleled flight capabilities that appear uniquely suited to exploration, surveillance, inspection, and telepresence tasks. However, unconstrained aerial movements may make it difficult for colocated operators, collaborators, and observers to understand AFF intentions, potentially leading to difficulties understanding whether operator instructions are being executed properly or to safety concerns if future AFF motions are unknown or difficult to predict. To increase AFF usability when working in close proximity to users, we explore the design of natural and intuitive flight motions that may improve AFF abilities to communicate intent while simultaneously accomplishing task goals. We propose a formalism for representing AFF flight paths as a series of motion primitives and present two studies examining the effects of modifying the trajectories and velocities of these flight primitives based on natural motion principles. Our first study found that modified flight motions might allow AFFs to more effectively communicate intent and, in our second study, participants preferred interacting with an AFF that used a manipulated flight path, rated modified flight motions as more natural, and felt safer around an AFF with modified motion. Our proposed formalism and findings highlight the importance of robot motion in achieving effective human-robot interactions. Categories and Subject Descriptors H.1.2 [Models and Principles]: User/Machine Systems— human factors, software psychology; H.5.2 [Information Interfaces and Presentation]: User Interfaces— evaluation/ methodology, user-centered design General Terms Design, Human Factors

Communicating Directionality in Flying Robots

Small flying robots represent a rapidly emerging family of robotic technologies with aerial capabilities that enable unique forms of assistance in a variety of collaborative tasks. Such tasks will necessitate interaction with humans in close proximity, requiring that designers consider human perceptions regarding robots flying and acting within human environments. We explore the design space regarding explicit robot communication of flight intentions to nearby viewers. We apply design constraints to robot flight behaviors, using biological and airplane flight as inspiration, and develop a set of signaling mechanisms for visually communicating directionality while operating under such constraints. We implement our designs on two commercial flyers, requiring little modification to the base platforms, and evaluate each signaling mechanism, as well as a no-signaling baseline, in a user study in which participants were asked to predict robot intent. We found that three of our designs significantly improved viewer response time and accuracy over the baseline and that the form of the signal offered tradeoffs in precision, generalizability, and perceived robot usability. Categories and Subject Descriptors H.1.2 [Models and Principles]: User/Machine Systems—human factors, software psychology; H.5.2 [Information Interfaces and Presentation]: User Interfaces— evaluation/methodology, usercentered design General Terms Design, Human Factors

An exploration of the utilization of electroencephalography and neural nets to control robots

It has long been known that as neurons fire within the brain they produce measurable electrical activity. Electroencephalography (EEG) is the measurement and recording of these electrical signals using sensors arrayed across the scalp. The idea of Brain-Computer interfaces (BCIs), which allow the control of devices using brain signals, naturally present themselves to many extremely useful applications including prosthetic devices, restoring or aiding in communication and hearing, military applications, video gaming and virtual reality, and robotic control, and have the possibility of significantly improving the quality of life of many disabled individuals. The purpose of this research is to examine an off the shelf EEG system, the Emotiv EPOC© System, as a costeffective gateway to non-invasive portable EEG measurements and to build a BCI to control a robot, the Parallax Scribbler®. We built middleware to interpret the outputs from the Emotiv and map them into commands for the Scribbler robot.

GUI Robots: Using Off-the-Shelf Robots as Tangible Input and Output Devices for Unmodified GUI Applications

Traditional GUI applications provide limited support for tangible interaction, as most applications are not programmed to support tangible input, and most input devices do not provide haptic feedback. To address this limitation, we introduce GUI Robots, a software framework that enables developers to repurpose off-the-shelf robots as tangible input and haptic output devices, and to connect them to unmodified desktop applications. We introduce the GUI Robots framework and present several proof-of-concept applications, including a haptic scroll wheel, force feedback game controllers, a 3D mouse, and a self-driving notification robot. To evaluate whether GUI Robots can be used to prototype tangible interfaces for existing applications, we conducted a user study in which developers created customized tangible interfaces for two applications. Study participants were able to create tangible user interfaces for these applications in less than an hour. GUI Robots allows developers to easily extend applications with tangible input and haptic output.

SculptUp: A rapid, immersive 3D modeling environment

The SculptUp system enables the rapid creation of 3D models. All of the models in Figure 3 were created in under five minutes. Scenes such as Figure 1 and 4 could be easily created in ways that would be extremely difficult in traditional modeling systems.

Designing planning and control interfaces to support user collaboration with flying robots

Robots are becoming increasingly prevalent and are already providing assistance in a variety of activities, ranging from space exploration to domestic housework. Recent advances in the design of sensors, motors, and microelectromechanical systems have enabled the development of a new class of small aerial robots. These free-flying robots hold great promise in assisting humans by acting as mobile sensor platforms to collect data in areas that are difficult to access or infeasible to instrument. In this work, we explored the design of interfaces that support users in working with free-flying robots to accomplish tasks including inventory logistics and management, environmental data collection, and visual inspection. Extending prior work in control interfaces for ground robots, we conducted a formative study in order to identify key design requirements for free-flyer interfaces. We designed several realistic tasks for use in evaluating human–robot interaction within the context of indoor free-flyer operation. We implemented three prototype interfaces that each provide varying degrees of support in enabling remote users to work with a flying robot to plan, communicate goals, accomplish tasks, and respond to changes in a dynamic environment. An experimental evaluation of each interface found that the interface designed to support collaborative planning and replanning using an interactive timeline and three-dimensional spatial waypoints significantly improved users’ efficiency in accomplishing tasks, their ability to intervene in response to spontaneous changes in task demands, and their ratings of the robot as a teammate compared to interfaces that support low-level teleoperation or waypoint-based supervisory control. Our results demonstrate the utility of a data-driven design process and show the need for free-flyer interfaces to consider planning phases in addition to task execution. In addition, we demonstrate the importance of providing interface support for interrupting robot operations as unplanned events arise.

Improving Collocated Robot Teleoperation with Augmented Reality

Robot teleoperation can be a challenging task, often requiring a great deal of user training and expertise, especially for platforms with high degrees-of-freedom (e.g., industrial manipulators and aerial robots). Users often struggle to synthesize information robots collect (e.g., a camera stream) with contextual knowledge of how the robot is moving in the environment. We explore how advances in augmented reality (AR) technologies are creating a new design space for mediating robot teleoperation by enabling novel forms of intuitive, visual feedback. We prototype several aerial robot teleoperation interfaces using AR, which we evaluate in a 48-participant user study where participants completed an environmental inspection task. Our new interface designs provided several objective and subjective performance benefits over existing systems, which often force users into an undesirable paradigm that divides user attention between monitoring the robot and monitoring the robot»s camera feed(s).

From 9 to 90: Engaging Learners of All Ages

This paper details the creation of a two-day computer science and robotics outreach course aimed at simultaneously engaging youth (children, ages 9-14) and senior (their grandparents, ages 55+) students. Our goal is to encourage enthusiasm for science and technology in students of all ages as well as provide practical instruction regarding common computer science concepts, including variables, loops, and boolean logic. To this end, we ground our course in the emerging field of social robotics, which enables the design of several multidisciplinary hands-on activities for students. We report on a four-year experience in the development of our course, which has been offered twelve times and involved over 210 youth and senior students. Our work presents a discussion regarding the challenges in designing a course for students from diverse ages, guidelines for creating similar courses, and a reflection on how we might improve our own class. The activities and project code developed for our course are available online as open-source resources.