Shuo Niu

Augmenting Undergraduate Computer Science Education With Programmable Smartwatches

Smartwatches are emerging as wrist-based computers capable of complex calculation and communication, and the computer science curriculum should reflect the challenges and opportunities that they provide in the education domain. This paper puts forth an experience report focused on efforts to incorporate smartwatches in an upper-level undergraduate mobile application development class during two academic terms. Lectures, in-class activities, homeworks, and projects were tailored toward providing rich design and implementation experiences for the students that engaged them in developing for the smartwatch and a paired mobile device. Our experiences highlighted how incorporating smartwatches into a mobile app development class adds a valuable dimension in terms of design and implementation challenges and allowed students to exercise some of the fundamental computer science topics.

Tongible: a non-contact tongue-based interaction technique

Using tongue to access computer for people with none or minimal upper limb function has been studied in recent years. These studies mainly focus on utilizing mechanical or electromagnetic devices. These devices, however, must contact to peoples oral cavity and cause hygiene problems or accidental ingestion. This work presents an interaction technique named Tongible that employs tongue as input without any mechanical or electromagnetic assistive device. In Tongible, six gestures of tongue are captured by an RGB camera and used as basic controlling gestures. Preliminary usability testing suggests that Tongible is effective in pointing and text entry for people with dexterity impairment.

Collaborative design for young children with autism: Design tools and a user study

This paper provides an overview of a collaborative design effort that involves computer scientists, psychologists, and designers working together to investigate design methods to help in the creation of technology for people with neurodevelopmental disabilities. The focus of this effort was in developing techniques to help novice designers create technology interfaces to support anger management in young people with autism spectrum disorder (ASD). The primary output for designers is a card set for which each card has a claim about an anger management technique that can help young people. Design activities leveraging scenarios and personas are suggested that leverage the card set in the creation of technology interfaces. This paper introduces the card set and supporting techniques, describes a design session in an undergraduate classroom setting, and speculates about future directions for this work.

Predicting Vehicle Recalls with User-Generated Contents: A Text Mining Approach

Vehicle safety issues and component defects result in property losses and fatalities. Our study proposes a new method to predict vehicle recalls based on user generated contents in online discussion forums. Vehicle defects can cause bodily injuries and sometimes deadly consequences. However, vehicle recalls will not be issued until damage has occurred. Online vehicle discussion forums usually contain traits of vehicle defects long before manufacturers and government agencies take investigative actions. We find overlapping components in user generated contents and official recall notices. Our proposed recall prediction method can correctly predict vehicle recalls once in every two recall events. It is our hope that our proposed technique can be used to monitor online vehicle discussion forums and prompt the manufacturers and government agencies to issue recalls before catastrophic accidents occur. Our research has significant practical implications to vehicle and transportation safety.

Learning with interactive tabletop displays

Groupwork is core to the educational process at all levels, with peer learning activities demonstrated to be an important component in exploring new idea spaces. Recent advances in technology encourage a re-examination of ways that technology can support and expand groupwork. This paper examines how large multi-touch tabletop displays can be used by students in group information exploration, creation, and sharing situations. The multi-user, multi-touch features enable multiple people to transition between individual and group interactions with the display, rather than the turn-taking interactions common in older technologies. While this provides opportunities, it also presents challenges in the ways that activities for the shared space are crafted, executed, and evaluated. This paper describes how our software solutions encourage idea exchange, sharing, and awareness of the activities of others, toward crafting better artifact-centered educational experiences. Specifically, the paper describes three techniques for card-based explorations that help enable groupwork: semantic zoom, dynamic grouping and info glow. Semantic zoom considers how pinch zooming can vary not only artifact size but also artifact content, showing more information at larger sizes. Dynamic grouping features the card combination which facilitates idea generation and synthesization. Info glow highlights artifacts that have been recently accessed or changed, providing an enduring way to see modifications made by others. This paper explores usage of this environment and these techniques, focusing on their impacts on educational goals and objectives for K-8 and for undergraduates, to understand how students in pairs or triads work independently and together as they explore and create within information spaces. Results suggest that older students tend to engage more with each other, though students of all ages transition between quiet individual work and communicative group work. The techniques described in this paper inform ways that they can be configured to optimize idea exchange, sharing, and awareness, and ways that tools and classroom techniques can encourage a more dynamic and engaged learning environment.

Investigating Notifications and Awareness for Multi-user Multi-touch Tabletop Displays

Notifications seek to guide people’s attention toward timely, relevant, and important tasks and interactions. This work considers situations in which multiple people are sharing a single large display, with collaborative notifications targeted at increasing team awareness of the joint goals, activities, and interactions. Notifications in recent studies show promise in enhancing awareness of the actions of co-located collaborators, but lacking is critical knowledge to guide the evaluation of the benefits and costs of collaborative activities. This paper presents a framework for notifications in a multi-user multi-touch context. The framework is explored for a card-sorting task performed by two people (a participant and a scripted confederate) on a shared tabletop display. Notifications highlight actions performed by each participant to understand changes in social, action, and activity awareness. Our study investigates individual work, social norms and team performance as three co-located factors that are affected by incorporating notifications.

An Observational Study of Simultaneous and Sequential Interactions in Co-located Collaboration

Large-scale multi-touch displays provide highly interactive spaces for small group activities. These devices feature the ability to detect concurrent touch inputs, which enable multiple co-located collaborators to manipulate virtual spaces in myriad ways. This paper explores two types of interaction, simultaneous and sequential, with regard to how people engage in shared virtual space during a collaborative ideation task. Our findings suggest that the two types of interactions present different patterns in both temporal and spatial dimensions. Sequential interaction is the major interaction technique, while the simultaneous interaction is actively used for information exploration and manipulating objects in personal space. Observation of semantic actions suggests that some behaviors are preferably performed in turns, while others are used more in simultaneous manner. The relationship between the two interaction types with regard to different collaboration factors is explored. We share lessons learned from the study and suggest design implications for multi-user touch interfaces.

Non-contact human computer interaction system design and implementation

Tongue has been proved to be effective in substituting human sensory system. Tongue supported human-computer interaction can help people with motor disability to operate assistive devices and computers. Previous research and applications focus on implementing such an interface by using mechanical or electromagnetic hardware which raises safety issues and hygiene concerns. This paper presents the process of developing a tongue interface by using computer vision so that people with dexterity impairment can interact with assistive devices and computers without relying on mechanical or electromagnetic hardware. An evaluation of the system is conducted and suggests that the novel interaction technique is effective in pointing and text entry for people with dexterity impairment.

Teaching mobile application development through lectures, interactive tutorials, and Pair Programming

Research suggests that different teaching styles and multiple exposures of different styles to material can aid in the learning process. While there are guidelines for identifying the best teaching style for material, new and evolving areas can present unique challenges. The emerging area of mobile software development, which combines aspects of software, hardware, and interpersonal interaction, captures many such challenges; e.g., understanding how to develop for multiple screen sizes, designing for GPS time lag, dealing with unreliable sensor data. Teaching these challenging materials seemed well suited for multiple approaches that leveraged different learning styles. This paper examines three teaching approaches employed in ten teaching modules across two semesters of a mobile software development course. The approaches included lectures, interactive tutorials, and Pair Programming. Lectures were used to introduce topics and explore underlying theories of development. The lectures included time for questions from and for the students, but otherwise did not have an active learning component. Two active learning approaches used in the class were interactive tutorials and Pair Programming. Interactive tutorials presented applied development approaches, then explored their use in an individual-based hands-on demos. Pair Programming is an agile software development practice, used in both industry and education, which enforces a role-based approach to learning new programming concepts. Homeworks were used to assess learning, and surveys reflected student satisfaction. Results show areas of promise and of concern with regard to the learning styles. It seems that repetition of topics is important for mastery of the topics. Foundational theories seem well suited for lectures, while programming concepts work better in active learning situations. Additional learning took place through office hours, online question forums, and individual and group online exploration. The findings suggest specific approaches to teaching challenging and unique mobile software development topics as well as a general approach to identifying ways to distribute learning objectives across lectures, interactive tutorials, and Pair Programming sessions.