Nicole Lehrer

Exploring the bases for a mixed reality stroke rehabilitation system, Part I: A unified approach for representing action, quantitative evaluation, and interactive feedback

BackgroundAlthough principles based in motor learning, rehabilitation, and human-computer interfaces can guide the design of effective interactive systems for rehabilitation, a unified approach that connects these key principles into an integrated design, and can form a methodology that can be generalized to interactive stroke rehabilitation, is presently unavailable.ResultsThis paper integrates phenomenological approaches to interaction and embodied knowledge with rehabilitation practices and theories to achieve the basis for a methodology that can support effective adaptive, interactive rehabilitation. Our resulting methodology provides guidelines for the development of an action representation, quantification of action, and the design of interactive feedback. As Part I of a two-part series, this paper presents key principles of the unified approach. Part II then describes the application of this approach within the implementation of the Adaptive Mixed Reality Rehabilitation (AMRR) system for stroke rehabilitation.ConclusionsThe accompanying principles for composing novel mixed reality environments for stroke rehabilitation can advance the design and implementation of effective mixed reality systems for the clinical setting, and ultimately be adapted for home-based application. They furthermore can be applied to other rehabilitation needs beyond stroke.

Exploring the bases for a mixed reality stroke rehabilitation system, Part II: Design of Interactive Feedback for upper limb rehabilitation

BackgroundFew existing interactive rehabilitation systems can effectively communicate multiple aspects of movement performance simultaneously, in a manner that appropriately adapts across various training scenarios. In order to address the need for such systems within stroke rehabilitation training, a unified approach for designing interactive systems for upper limb rehabilitation of stroke survivors has been developed and applied for the implementation of an Adaptive Mixed Reality Rehabilitation (AMRR) System.ResultsThe AMRR system provides computational evaluation and multimedia feedback for the upper limb rehabilitation of stroke survivors. A participants movements are tracked by motion capture technology and evaluated by computational means. The resulting data are used to generate interactive media-based feedback that communicates to the participant detailed, intuitive evaluations of his performance. This article describes how the AMRR systems interactive feedback is designed to address specific movement challenges faced by stroke survivors. Multimedia examples are provided to illustrate each feedback component. Supportive data are provided for three participants of varying impairment levels to demonstrate the systems ability to train both targeted and integrated aspects of movement.ConclusionsThe AMRR system supports training of multiple movement aspects together or in isolation, within adaptable sequences, through cohesive feedback that is based on formalized compositional design principles. From preliminary analysis of the data, we infer that the systems ability to train multiple foci together or in isolation in adaptable sequences, utilizing appropriately designed feedback, can lead to functional improvement. The evaluation and feedback frameworks established within the AMRR system will be applied to the development of a novel home-based system to provide an engaging yet low-cost extension of training for longer periods of time.

Attractor-Shape for Dynamical Analysis of Human Movement: Applications in Stroke Rehabilitation and Action Recognition

In this paper, we propose a novel shape-theoretic framework for dynamical analysis of human movement from 3D data. The key idea we propose is the use of global descriptors of the shape of the dynamical attractor as a feature for modeling actions. We apply this approach to the novel application scenario of estimation of movement quality from a single-marker for future usage in home-based stroke rehabilitation. Using a dataset collected from 15 stroke survivors performing repetitive task therapy, we demonstrate that the proposed method outperforms traditional methods, such as kinematic analysis and use of chaotic invariants, in estimation of movement quality. In addition, we demonstrate that the proposed framework is sufficiently general for the application of action and gesture recognition as well. Our experimental results reflect improved action recognition results on two publicly available 3D human activity databases.

A Computational Framework for Quantitative Evaluation of Movement during Rehabilitation

This paper presents a novel generalized computational framework for quantitative kinematic evaluation of movement in a rehabilitation clinic setting. The framework integrates clinical knowledge and computational data‐driven analysis together in a systematic manner. The framework provides three key benefits to rehabilitation: (a) the resulting continuous normalized measure allows the clinician to monitor movement quality on a fine scale and easily compare impairments across participants, (b) the framework reveals the effect of individual movement components on the composite movement performance helping the clinician decide the training foci, and (c) the evaluation runs in real‐time, which allows the clinician to constantly track a patient’s progress and make appropriate adaptations to the therapy protocol. The creation of such an evaluation is difficult because of the sparse amount of recorded clinical observations, the high dimensionality of movement and high variations in subject’s performance. We addres...

Design of a home-based adaptive mixed reality rehabilitation system for stroke survivors

This paper presents the design of a home-based adaptive mixed reality system (HAMRR) for upper extremity stroke rehabilitation. The goal of HAMRR is to help restore motor function to chronic stroke survivors by providing an engaging long-term reaching task therapy at home. The system uses an intelligent adaptation scheme to create a continuously challenging and unique multi-year therapy experience. The therapy is overseen by a physical therapist, but day-to-day use of the system can be independently set up and completed by a stroke survivor. The HAMMR system tracks movement of the wrist and torso and provides real-time, post-trial, and post-set feedback to encourage the stroke survivor to self-assess his or her movement and engage in active learning of new movement strategies. The HAMRR system consists of a custom table, chair, and media center, and is designed to easily integrate into any home.

A Novel Adaptive Mixed Reality System for Stroke Rehabilitation: Principles, Proof of Concept, and Preliminary Application in 2 Patients

Abstract This article presents the principles of an adaptive mixed reality rehabilitation (AMRR) system, as well as the training process and results from 2 stroke survivors who received AMRR therapy, to illustrate how the system can be used in the clinic. The AMRR system integrates traditional rehabilitation practices with state-of-the-art computational and motion capture technologies to create an engaging environment to train reaching movements. The system provides real-time, intuitive, and integrated audio and visual feedback (based on detailed kinematic data) representative of goal accomplishment, activity performance, and body function during a reaching task. The AMRR system also provides a quantitative kinematic evaluation that measures the deviation of the stroke survivor’s movement from an idealized, unimpaired movement. The therapist, using the quantitative measure and knowledge and observations, can adapt the feedback and physical environment of the AMRR system throughout therapy to address each participant’s individual impairments and progress. Individualized training plans, kinematic improvements measured over the entire therapy period, and the changes in relevant clinical scales and kinematic movement attributes before and after the month-long therapy are presented for 2 participants. The substantial improvements made by both participants after AMRR therapy demonstrate that this system has the potential to considerably enhance the recovery of stroke survivors with varying impairments for both kinematic improvements and functional ability.

Adaptive mixed reality stroke rehabilitation: system architecture and evaluation metrics

This paper presents a novel system architecture and evaluation metrics for an Adaptive Mixed Reality Rehabilitation (AMRR) system for stroke patient. This system provides a purposeful, engaging, hybrid (visual, auditory and physical) scene that encourages patients to improve their performance of a reaching and grasping task and promotes learning of generalizable movement strategies. This system is adaptive in that it provides assistive adaptation tools to help the rehabilitation team customize the training strategy. Our key insight is to combine the patients, rehabilitation team, multimodal hybrid environments and adaptation tools together as an adaptive experiential mixed reality system. There are three major contributions in this paper: (a) developing a computational deficit index for evaluating the patients kinematic performance and a deficit-training-improvement (DTI) correlation for evaluating adaptive training strategy, (b) integrating assistive adaptation tools that help the rehabilitation team understand the relationship between the patients performance and training and customize the training strategy, and (c) combining the interactive multimedia environment and physical environment together to encourage patients to transfer movement knowledge from media space to physical space. Our system has been used by two stroke patients for one-month mediated therapy. They have significant improvement in their reaching and grasping performance (+48.84% and +39.29%) compared to other two stroke patients who experienced traditional therapy (-18.31% and -8.06%).

Interdisciplinary Concepts for Design and Implementation of Mixed Reality Interactive Neurorehabilitation Systems for Stroke

Interactive neurorehabilitation (INR) systems provide therapy that can evaluate and deliver feedback on a patients movement computationally. There are currently many approaches to INR design and implementation, without a clear indication of which methods to utilize best. This article presents key interactive computing, motor learning, and media arts concepts utilized by an interdisciplinary group to develop adaptive, mixed reality INR systems for upper extremity therapy of patients with stroke. Two INR systems are used as examples to show how the concepts can be applied within: (1) a small-scale INR clinical study that achieved integrated improvement of movement quality and functionality through continuously supervised therapy and (2) a pilot study that achieved improvement of clinical scores with minimal supervision. The notion is proposed that some of the successful approaches developed and tested within these systems can form the basis of a scalable design methodology for other INR systems. A coherent approach to INR design is needed to facilitate the use of the systems by physical therapists, increase the number of successful INR studies, and generate rich clinical data that can inform the development of best practices for use of INR in physical therapy.

Experiential Media and Digital Culture

Multidisciplinary value structures and a design approach focusing on combining efficiency, reflection, and quality of experience will foster the true hybrid physical- digital culture that is foundational to solving complex societal problems.

A home-based adaptive mixed reality rehabilitation system

This paper presents an interactive home-based adaptive mixed reality system (HAMRR) for upper extremity stroke rehabilitation. This home-based system is an extension of a previously designed and currently implemented clinical system. The goal of HAMRR is to restore motor function to chronic stroke survivors by providing an engaging long-term reaching task therapy at home. The HAMMR system tracks movement of the wrist and torso, and provides real-time, post-trial, and post-set multimodal feedback to encourage the stroke survivor to self-assess his or her movement and engage in active learning of new movement strategies. This experiential media system uses a computational adaptation scheme to create a continuously challenging and unique multi-year therapy experience through the use of multiple, integrated audio and visual feedback streams. Novel design features include creating an over-arching story for the participant, the ability of the system to adapt the feedback over multiple time scales, and the ability for this system to integrate into any home.