Jessica Merino

A Gesture Learning Interface for Simulated Robot Path Shaping With a Human Teacher

Recognition of human gestures is an active area of research integral for the development of intuitive human-machine interfaces for ubiquitous computing and assistive robotics. In particular, such systems are key to effective environmental designs that facilitate aging in place. Typically, gesture recognition takes the form of template matching in which the human participant is expected to emulate a choreographed motion as prescribed by the researchers. A corresponding robotic action is then a one-to-one mapping of the template classification to a library of distinct responses. In this paper, we explore a recognition scheme based on the growing neural gas (GNG) algorithm that places no initial constraints on the user to perform gestures in a specific way. Motion descriptors extracted from sequential skeletal depth data are clustered by GNG and mapped directly to a robotic response that is refined through reinforcement learning. A simple good/bad reward signal is provided by the user. This paper presents results that show that the topology-preserving quality of GNG allows generalization between gestured commands. Experimental results using an automated reward are presented that compare learning results involving single nodes versus results involving the influence of node neighborhoods. Although separability of input data influences the speed of learning convergence for a given neighborhood radius, it is shown that learning progresses toward emulation of an associative memory that maps input gesture to desired action.

Forward kinematic model for continuum robotic surfaces

In this paper, we consider the modeling of robotic continuous “continuum” two-dimensional surfaces. We discuss the fundamental differences between such robot surfaces and traditional rigid link and continuum robots. We then introduce new kinematic models for continuum robotic surfaces. We compare the kinematic models to physical continuum surfaces and validate their performance.

An assistive robotic table for older and post-stroke adults: results from participatory design and evaluation activities with clinical staff

An inevitable new frontier for the CHI community is the development of complex, larger-scale, cyber-physical artifacts where advancements in design, computing and robotics converge. Presented here is a design exemplar: the Assistive, Robotic Table (ART), the key component of our envisioned home suite of networked, robotic furnishings for hospitals and homes, promoting wellbeing and independent living. We begin with the motivations for ART, and present our iterative, five-phase, participatory design-and-evaluation process involving clinicians at a rehabilitation hospital, focusing here on the final usability study. From our wide-ranging design-research activities, which may be characterized as research through design, we found ART to be promising but also challenging. As a design exemplar, ART offers invaluable lessons to the CHI community as it comes to design larger-scale, cyber-physical artifacts cultivating interactions across people and their surroundings that define places of social, cultural and psychological significance.

A vision of the patient room as an architectural-robotic ecosystem

Healthcare is becoming more digital and technological, but healthcare environments have not yet become embedded with digital technologies to support the most productive (physical) interaction between medical patients, clinical staff and the physical artifacts that surround and envelop them. This shortcoming is an opportunity for the architecture and robotics communities to interface with each other and the everyday users of healthcare environments. Our extended lab focused ten weeks on sketching in hardware a robotic, patient-room ecosystem we call home+ with the help of clinicians at the Roger C. Peace Rehabilitation Hospital of the Greenville Hospital System University Medical Center [GHS]. This early prototyping effort represents our vision for the larger robotic patient room, and identifies opportunities for more focused work on an Assistive Robotic Table (ART).

Examination of How and Why Over-the-Bed Tables Are Used: Use Cases and Needs from Healthcare Providers

OBJECTIVE: This article presents the results of an exploratory study in which 14 clinical and staff subject matter experts (SMEs) at a regional rehabilitation hospital were interviewed in order to understand how and why over-the-bed tables are used. BACKGROUND: It is important to understand how and why a device or environment is used when designing it, and not just what features and functions are preferred. This knowledge is valuable both for contextualizing user feature and function preferences and for characterizing and prioritizing design challenges and opportunities. METHODS: Fourteen hospital clinical and support staff subject-matter experts participated in semi-structured interviews with scenario enactments in a medium-fidelity, full-scale mock-up of a typical patient room. During these interviews, they interacted with two personas played by actors and were asked to enact an example of a normal visit, from entering the room through treatment/service and then exiting. Data were analyzed via methodological triangulation including frequency analysis, content analysis, and affinity diagramming. RESULTS: The results include a use-case analysis with illustrative sketches, a list of needs statements, and final observations. CONCLUSIONS: Successfully using the over-the-bed table is dependent upon proper positioning, especially in bed during meals. There are fewer problems associated with over-the-bed table use while seated in a chair than when in the bed. The over-the-bed table is a key component in acute care, inpatient therapies. Clinicians are generally open to “smart” furniture in the patient room but question its cost-effectiveness, robustness, and flexibility.

Confirming, Classifying, and Prioritizing Needed Over-the-Bed Table Improvements via Methodological Triangulation.

OBJECTIVE: This article presents the results of a qualitative study that confirmed, classified, and prioritized user needs for the design of a more useful, usable, and actively assistive over-the-bed table. BACKGROUND: Manganelli et al. (2014) generated a list of 74 needs for use in developing an actively assistive over-the-bed table. This present study assesses the value and importance of those needs. METHODS: Fourteen healthcare subject matter experts and eight research and design subject matter experts engaged in a participatory and iterative research and design process. A mixed methods qualitative approach used methodological triangulation to confirm the value of the findings and ratings to establish importance. Open and closed card sorts and a Delphi study were used. Data analysis methods included frequency analysis, content analysis, and a modified Kano analysis. RESULTS: A table demonstrating the needs that are of high importance to both groups of subject matter experts and classification of the design challenges each represents was produced. Through this process, the list of 74 needs was refined to the 37 most important need statements for both groups. CONCLUSIONS: Designing a more useful, usable, and actively assistive over-the-bed table is primarily about the ability to position it optimally with respect to the user for any task, as well as improving ease of use and usability. It is also important to make explicit and discuss the differences in priorities and perspectives demonstrated between research and design teams and their clients.

A Method for Lifelong Gesture Learning Based on Growing Neural Gas

Gesture-based interfaces offer the possibility of an intuitive command language for assistive robotics and ubiquitous computing. As an individual’s health changes with age, their ability to consistently perform standard gestures may decrease, particularly towards the end of life. Thus, such interfaces will need to be capable of learning commands which are not choreographed ahead of time by the system designers. This circumstance illustrates the need for a system which engages in lifelong learning and is capable of discerning new gestures and the user’s desired response to them. This paper describes an innovative approach to lifelong learning based on clustered gesture representations identified through the Growing Neural Gas algorithm. The simulated approach utilizes a user-generated reward signal to progressively refine the response of an assistive robot toward a preferred goal configuration.

A Method for Neighborhood Gesture Learning Based on Resistance Distance

Multimodal forms of human-robot interaction (HRI) including non-verbal forms promise easily adopted and intuitive use models for assistive devices. The research described in this paper targets an assistive robotic appliance which learns a user’s gestures for activities performed in a healthcare or aging in place setting. The proposed approach uses the Growing Neural Gas (GNG) algorithm in combination with the Q-Learning paradigm of reinforcement learning to shape robotic motions over time. Neighborhoods of nodes in the GNG network are combined to collectively leverage past learning by the group. Connections between nodes are assigned weights based on frequency of use which can be viewed as measures of electrical resistance. In this way, the GNG network may be traversed based on distances computed in the same manner as resistance in an electrical circuit. It is shown that this distance metric provides faster convergence of the algorithm when compared to shortest path neighborhood learning.

The Design, Prototyping, and Formative Evaluation of an Assistive Robotic Table (ART) for Stroke Patients:

Objective: This article presents the results of an exploratory study in which 14 healthcare subject matter experts (H-SMEs) in addition to four research and design subject matter experts (RD-SMEs) at a regional rehabilitation hospital engaged in a series of complementary, participatory activities in order to design an assistive robotic table (ART). Background: As designers, human factor experts, and healthcare professionals continue to work to integrate assistive human–robot technologies in healthcare, it is imperative to understand how the technology affects patient care from clinicians’ perspectives. Method: Fourteen clinical H-SMEs rated a subset of conceptual ART design ideas; participated in the iterative design process of ART; and evaluated a final cardboard prototype, the rehabilitation hospital’s current over-the-bed table (OBT), an ART built with true materials, and two therapy surface prototypes. Four RD-SMEs conducted a heuristic evaluation on the ART built with true materials. Data were analyzed by frequency and content analysis. Results: The results include a design and prototype for the next generation ART and a pneumatically controlled therapy surface, a broadened list of specifications for the future design and implementation of assistive robotic furniture, and final observations. Conclusion: When compared to the rehabilitation hospital’s current OBT, the developed ART in this study was successful. Designing novel features is dependent upon ensuring patient safety. The inclusion of clinicians in the participatory iterative design and evaluation process and the use of personas provided a broadened list of specifications for the successful implementation of assistive robotic furniture.