Joe Manganelli

A Gesture Learning Interface for Simulated Robot Path Shaping With a Human Teacher

Recognition of human gestures is an active area of research integral for the development of intuitive human-machine interfaces for ubiquitous computing and assistive robotics. In particular, such systems are key to effective environmental designs that facilitate aging in place. Typically, gesture recognition takes the form of template matching in which the human participant is expected to emulate a choreographed motion as prescribed by the researchers. A corresponding robotic action is then a one-to-one mapping of the template classification to a library of distinct responses. In this paper, we explore a recognition scheme based on the growing neural gas (GNG) algorithm that places no initial constraints on the user to perform gestures in a specific way. Motion descriptors extracted from sequential skeletal depth data are clustered by GNG and mapped directly to a robotic response that is refined through reinforcement learning. A simple good/bad reward signal is provided by the user. This paper presents results that show that the topology-preserving quality of GNG allows generalization between gestured commands. Experimental results using an automated reward are presented that compare learning results involving single nodes versus results involving the influence of node neighborhoods. Although separability of input data influences the speed of learning convergence for a given neighborhood radius, it is shown that learning progresses toward emulation of an associative memory that maps input gesture to desired action.

Toward a "smart" nightstand prototype: an examination of nightstand table contents and preferences.

Objective: Two studies were conducted to obtain an understanding of the types of items seniors keep in their nightstands and to understand how users feel about the possibility of “smart” furniture. Background: To enable aging in place and universal design, it is vital to understand the needs of a broad range of aging individuals, especially since there is little research on nightstand usage and design. Methods: Study 1 allowed for the development of a structured inventory of nightstand use today in assisted living and rehabilitation facilities. Study 1 led to Study 2, demonstrating the need to conceptualize new ideas for smart nightstands. Feedback was obtained from intergenerational participants who could discuss their needs and preferences for a smart nightstand. Results: In Study 1, more than 150 items were recorded and categorized into 25 different groups. The authors found that participants utilized the top portion of their nightstand as opposed to the lower sections; most items were found on top of the nightstand or in the top drawer. In Study 2, the authors found that the vast majority of participants are willing to consider the use of a smart nightstand. Participants discussed key functions and design preferences, which included carefully designed storage, the ability to move the nightstand up and down, contemporary design, and interaction through voice activation. Conclusion: Existing nightstands do not meet the needs of current users. This research provides greater understanding of the existing limitations associated with nightstands. Study 2 confirmed that user-centered design and the use of technology can be used to enhance daily living. Smart furniture may play a role in promoting the health and independence of diverse user groups.

A vision of the patient room as an architectural-robotic ecosystem

Healthcare is becoming more digital and technological, but healthcare environments have not yet become embedded with digital technologies to support the most productive (physical) interaction between medical patients, clinical staff and the physical artifacts that surround and envelop them. This shortcoming is an opportunity for the architecture and robotics communities to interface with each other and the everyday users of healthcare environments. Our extended lab focused ten weeks on sketching in hardware a robotic, patient-room ecosystem we call home+ with the help of clinicians at the Roger C. Peace Rehabilitation Hospital of the Greenville Hospital System University Medical Center [GHS]. This early prototyping effort represents our vision for the larger robotic patient room, and identifies opportunities for more focused work on an Assistive Robotic Table (ART).

Group differences in preferences for a novel nightstand.

Objective: Multiple user groups (patients and employees at a rehabilitation facility, community-dwelling seniors, and university students) participated in a study that examined their preferences for the features and functions of three novel nightstand prototypes. Background: It is valuable to get input from different user groups in order to improve furniture usefulness and usability, especially furniture prevalent in clinical settings where users of all age groups are found. Methods: Feedback was obtained from different user groups in both clinical (rehabilitation facility) and nonclinical (university) settings. This was done using structured interviews to ask participants about multiple features of the novel nightstand designs. Results: There were several features that all groups preferred. There were also some distinctly opposing opinions between groups. In general, the patient group showed the most similarities to the other groups. Conclusion: This research explores differences and similarities in preferences for nightstand design across a variety of user groups. It yields ideas for improving the nightstand design to be useful for a wider group of people.

“Architectural Robotics”: An interdisciplinary course rethinking the machines we live in

We discuss disciplinary barriers which have traditionally prevented robotics from significantly impacting the built (architectural) environment we inhabit. Specifically, we describe the implementation of, and lessons learned from, a multidisciplinary graduate-level course in Architectural Robotics. The results from class interactions and projects provide insight into novel ways in which robotics expertise can be effectively leveraged in architecture. Conversely, our outcomes suggest ways in which the knowledge and perspective of architects could stimulate significant innovations in robotics.

An Exploration of the Nightstand and Over-the-Bed Table in an Inpatient Rehabilitation Hospital.

Objective: This study seeks to determine where patients in a rehabilitation hospital keep the greatest percentage of their belongings, that is, in/on the nightstand or on the over-the-bed table. Background: This study provides an inventory of patient items located on the over-the-bed table and in/on the nightstand. Understanding the functions of furnishings within the patient room is key for future preparation for designing a next-generation over-the-bed table or for redesigning a more useful nightstand. Methods: The contents on the top of the nightstand; the contents in the top, middle, and bottom drawers of the nightstand; items next to the nightstand; and the contents on the over-the-bed table within patient rooms were inventoried and placed into categories using similar, patient item categories as the Brooks et al. (2011) study, which examined the contents of the nightstand and the over-the-bed table in assisted living and skilled nursing facilities. Results: Overall, patients in a rehabilitation hospital had a greater percentage of their belongings on the top of the nightstand as compared to their belongings located in all three combined drawers of the nightstand. Overall, patients had a greater percentage of their belongings located on the over-the-bed table as compared to their belongings located on the nightstand. Conclusions: Tabletop surface area was used extensively in patient rooms at a rehabilitation hospital, but nightstand drawers were underutilized.

Examination of How and Why Over-the-Bed Tables Are Used: Use Cases and Needs from Healthcare Providers

OBJECTIVE: This article presents the results of an exploratory study in which 14 clinical and staff subject matter experts (SMEs) at a regional rehabilitation hospital were interviewed in order to understand how and why over-the-bed tables are used. BACKGROUND: It is important to understand how and why a device or environment is used when designing it, and not just what features and functions are preferred. This knowledge is valuable both for contextualizing user feature and function preferences and for characterizing and prioritizing design challenges and opportunities. METHODS: Fourteen hospital clinical and support staff subject-matter experts participated in semi-structured interviews with scenario enactments in a medium-fidelity, full-scale mock-up of a typical patient room. During these interviews, they interacted with two personas played by actors and were asked to enact an example of a normal visit, from entering the room through treatment/service and then exiting. Data were analyzed via methodological triangulation including frequency analysis, content analysis, and affinity diagramming. RESULTS: The results include a use-case analysis with illustrative sketches, a list of needs statements, and final observations. CONCLUSIONS: Successfully using the over-the-bed table is dependent upon proper positioning, especially in bed during meals. There are fewer problems associated with over-the-bed table use while seated in a chair than when in the bed. The over-the-bed table is a key component in acute care, inpatient therapies. Clinicians are generally open to “smart” furniture in the patient room but question its cost-effectiveness, robustness, and flexibility.

Confirming, Classifying, and Prioritizing Needed Over-the-Bed Table Improvements via Methodological Triangulation.

OBJECTIVE: This article presents the results of a qualitative study that confirmed, classified, and prioritized user needs for the design of a more useful, usable, and actively assistive over-the-bed table. BACKGROUND: Manganelli et al. (2014) generated a list of 74 needs for use in developing an actively assistive over-the-bed table. This present study assesses the value and importance of those needs. METHODS: Fourteen healthcare subject matter experts and eight research and design subject matter experts engaged in a participatory and iterative research and design process. A mixed methods qualitative approach used methodological triangulation to confirm the value of the findings and ratings to establish importance. Open and closed card sorts and a Delphi study were used. Data analysis methods included frequency analysis, content analysis, and a modified Kano analysis. RESULTS: A table demonstrating the needs that are of high importance to both groups of subject matter experts and classification of the design challenges each represents was produced. Through this process, the list of 74 needs was refined to the 37 most important need statements for both groups. CONCLUSIONS: Designing a more useful, usable, and actively assistive over-the-bed table is primarily about the ability to position it optimally with respect to the user for any task, as well as improving ease of use and usability. It is also important to make explicit and discuss the differences in priorities and perspectives demonstrated between research and design teams and their clients.

A Method for Lifelong Gesture Learning Based on Growing Neural Gas

Gesture-based interfaces offer the possibility of an intuitive command language for assistive robotics and ubiquitous computing. As an individual’s health changes with age, their ability to consistently perform standard gestures may decrease, particularly towards the end of life. Thus, such interfaces will need to be capable of learning commands which are not choreographed ahead of time by the system designers. This circumstance illustrates the need for a system which engages in lifelong learning and is capable of discerning new gestures and the user’s desired response to them. This paper describes an innovative approach to lifelong learning based on clustered gesture representations identified through the Growing Neural Gas algorithm. The simulated approach utilizes a user-generated reward signal to progressively refine the response of an assistive robot toward a preferred goal configuration.

A Method for Neighborhood Gesture Learning Based on Resistance Distance

Multimodal forms of human-robot interaction (HRI) including non-verbal forms promise easily adopted and intuitive use models for assistive devices. The research described in this paper targets an assistive robotic appliance which learns a user’s gestures for activities performed in a healthcare or aging in place setting. The proposed approach uses the Growing Neural Gas (GNG) algorithm in combination with the Q-Learning paradigm of reinforcement learning to shape robotic motions over time. Neighborhoods of nodes in the GNG network are combined to collectively leverage past learning by the group. Connections between nodes are assigned weights based on frequency of use which can be viewed as measures of electrical resistance. In this way, the GNG network may be traversed based on distances computed in the same manner as resistance in an electrical circuit. It is shown that this distance metric provides faster convergence of the algorithm when compared to shortest path neighborhood learning.