Yinpeng Chen

An Adaptive Mixed Reality Training System for Stroke Rehabilitation

This paper presents a novel mixed reality rehabilitation system used to help improve the reaching movements of people who have hemiparesis from stroke. The system provides real-time, multimodal, customizable, and adaptive feedback generated from the movement patterns of the subjects affected arm and torso during reaching to grasp. The feedback is provided via innovative visual and musical forms that present a stimulating, enriched environment in which to train the subjects and promote multimodal sensory-motor integration. A pilot study was conducted to test the system function, adaptation protocol and its feasibility for stroke rehabilitation. Three chronic stroke survivors underwent training using our system for six 75-min sessions over two weeks. After this relatively short time, all three subjects showed significant improvements in the movement parameters that were targeted during training. Improvements included faster and smoother reaches, increased joint coordination and reduced compensatory use of the torso and shoulder. The system was accepted by the subjects and shows promise as a useful tool for physical and occupational therapists to enhance stroke rehabilitation.

Exploring the bases for a mixed reality stroke rehabilitation system, Part II: Design of Interactive Feedback for upper limb rehabilitation

BackgroundFew existing interactive rehabilitation systems can effectively communicate multiple aspects of movement performance simultaneously, in a manner that appropriately adapts across various training scenarios. In order to address the need for such systems within stroke rehabilitation training, a unified approach for designing interactive systems for upper limb rehabilitation of stroke survivors has been developed and applied for the implementation of an Adaptive Mixed Reality Rehabilitation (AMRR) System.ResultsThe AMRR system provides computational evaluation and multimedia feedback for the upper limb rehabilitation of stroke survivors. A participants movements are tracked by motion capture technology and evaluated by computational means. The resulting data are used to generate interactive media-based feedback that communicates to the participant detailed, intuitive evaluations of his performance. This article describes how the AMRR systems interactive feedback is designed to address specific movement challenges faced by stroke survivors. Multimedia examples are provided to illustrate each feedback component. Supportive data are provided for three participants of varying impairment levels to demonstrate the systems ability to train both targeted and integrated aspects of movement.ConclusionsThe AMRR system supports training of multiple movement aspects together or in isolation, within adaptable sequences, through cohesive feedback that is based on formalized compositional design principles. From preliminary analysis of the data, we infer that the systems ability to train multiple foci together or in isolation in adaptable sequences, utilizing appropriately designed feedback, can lead to functional improvement. The evaluation and feedback frameworks established within the AMRR system will be applied to the development of a novel home-based system to provide an engaging yet low-cost extension of training for longer periods of time.

A dynamic decision network framework for online media adaptation in stroke rehabilitation

In this article, we present a media adaptation framework for an immersive biofeedback system for stroke patient rehabilitation. In our biofeedback system, media adaptation refers to changes in audio/visual feedback as well as changes in physical environment. Effective media adaptation frameworks help patients recover generative plans for arm movement with potential for significantly shortened therapeutic time. The media adaptation problem has significant challenges—(a) high dimensionality of adaptation parameter space; (b) variability in the patient performance across and within sessions; (c) the actual rehabilitation plan is typically a non-first-order Markov process, making the learning task hard. Our key insight is to understand media adaptation as a real-time feedback control problem. We use a mixture-of-experts based Dynamic Decision Network (DDN) for online media adaptation. We train DDN mixtures per patient, per session. The mixture models address two basic questions—(a) given a specific adaptation suggested by the domain experts, predict the patient performance, and (b) given the expected performance, determine the optimal adaptation decision. The questions are answered through an optimality criterion based search on DDN models trained in previous sessions. We have also developed new validation metrics and have very good results for both questions on actual stroke rehabilitation data.

A Computational Framework for Quantitative Evaluation of Movement during Rehabilitation

This paper presents a novel generalized computational framework for quantitative kinematic evaluation of movement in a rehabilitation clinic setting. The framework integrates clinical knowledge and computational data‐driven analysis together in a systematic manner. The framework provides three key benefits to rehabilitation: (a) the resulting continuous normalized measure allows the clinician to monitor movement quality on a fine scale and easily compare impairments across participants, (b) the framework reveals the effect of individual movement components on the composite movement performance helping the clinician decide the training foci, and (c) the evaluation runs in real‐time, which allows the clinician to constantly track a patient’s progress and make appropriate adaptations to the therapy protocol. The creation of such an evaluation is difficult because of the sparse amount of recorded clinical observations, the high dimensionality of movement and high variations in subject’s performance. We addres...

Design of a home-based adaptive mixed reality rehabilitation system for stroke survivors

This paper presents the design of a home-based adaptive mixed reality system (HAMRR) for upper extremity stroke rehabilitation. The goal of HAMRR is to help restore motor function to chronic stroke survivors by providing an engaging long-term reaching task therapy at home. The system uses an intelligent adaptation scheme to create a continuously challenging and unique multi-year therapy experience. The therapy is overseen by a physical therapist, but day-to-day use of the system can be independently set up and completed by a stroke survivor. The HAMMR system tracks movement of the wrist and torso and provides real-time, post-trial, and post-set feedback to encourage the stroke survivor to self-assess his or her movement and engage in active learning of new movement strategies. The HAMRR system consists of a custom table, chair, and media center, and is designed to easily integrate into any home.

Adaptive Mixed Reality Rehabilitation Improves Quality of Reaching Movements More Than Traditional Reaching Therapy Following Stroke

Background. Adaptive mixed reality rehabilitation (AMRR) is a novel integration of motion capture technology and high-level media computing that provides precise kinematic measurements and engaging multimodal feedback for self-assessment during a therapeutic task. Objective. We describe the first proof-of-concept study to compare outcomes of AMRR and traditional upper-extremity physical therapy. Methods. Two groups of participants with chronic stroke received either a month of AMRR therapy (n = 11) or matched dosing of traditional repetitive task therapy (n = 10). Participants were right handed, between 35 and 85 years old, and could independently reach to and at least partially grasp an object in front of them. Upper-extremity clinical scale scores and kinematic performances were measured before and after treatment. Results. Both groups showed increased function after therapy, demonstrated by statistically significant improvements in Wolf Motor Function Test and upper-extremity Fugl-Meyer Assessment (FMA) scores, with the traditional therapy group improving significantly more on the FMA. However, only participants who received AMRR therapy showed a consistent improvement in kinematic measurements, both for the trained task of reaching to grasp a cone and the untrained task of reaching to push a lighted button. Conclusions. AMRR may be useful in improving both functionality and the kinematics of reaching. Further study is needed to determine if AMRR therapy induces long-term changes in movement quality that foster better functional recovery.

A Novel Adaptive Mixed Reality System for Stroke Rehabilitation: Principles, Proof of Concept, and Preliminary Application in 2 Patients

Abstract This article presents the principles of an adaptive mixed reality rehabilitation (AMRR) system, as well as the training process and results from 2 stroke survivors who received AMRR therapy, to illustrate how the system can be used in the clinic. The AMRR system integrates traditional rehabilitation practices with state-of-the-art computational and motion capture technologies to create an engaging environment to train reaching movements. The system provides real-time, intuitive, and integrated audio and visual feedback (based on detailed kinematic data) representative of goal accomplishment, activity performance, and body function during a reaching task. The AMRR system also provides a quantitative kinematic evaluation that measures the deviation of the stroke survivor’s movement from an idealized, unimpaired movement. The therapist, using the quantitative measure and knowledge and observations, can adapt the feedback and physical environment of the AMRR system throughout therapy to address each participant’s individual impairments and progress. Individualized training plans, kinematic improvements measured over the entire therapy period, and the changes in relevant clinical scales and kinematic movement attributes before and after the month-long therapy are presented for 2 participants. The substantial improvements made by both participants after AMRR therapy demonstrate that this system has the potential to considerably enhance the recovery of stroke survivors with varying impairments for both kinematic improvements and functional ability.

Estimating Complexity of 2D Shapes

This paper deals with the problem of estimating 2D shape complexity. This has important applications in computer vision as well as in developing efficient shape classification algorithms. We define shape complexity using correlates of Kolmogorov complexity-entropy measures of global distance and local angle, and a measure of shape randomness. We tested our algorithm on synthetic and real world datasets with excellent results. We also conducted user studies that indicate that our measure is highly correlated with human perception. They also reveal an intuitive shape sensitivity curve-simple shapes are easily distinguished by small complexity variations, while complex shapes require significant complexity differences to be differentiated

A Generalized Approach to Linear Transform Approximations with Applications to the Discrete Cosine Transform

This paper aims to develop a generalized framework to systematically trade off computational complexity with output distortion in linear transforms such as the DCT, in an optimal manner. The problem is important in real-time systems where the computational resources available are time-dependent. Our approach is generic and applies to any linear transform and we use the DCT as a specific example. There are three key ideas: (a) a joint transform pruning and Haar basis projection-based approximation technique. The idea is to save computations by factoring the DCT transform into signal-independent and signal-dependent parts. The signal-dependent calculation is done in real-time and combined with the stored signal-independent part, saving calculations. (b) We propose the idea of the complexity-distortion framework and present an algorithm to efficiently estimate the complexity distortion function and search for optimal transform approximation using several approximation candidate sets. We also propose a measure to select the optimal approximation candidate set, and (c) an adaptive approximation framework in which the operating points on the C-D curve are embedded in the metadata. We also present a framework to perform adaptive approximation in real time for changing computational resources by using the embedded metadata. Our results validate our theoretical approach by showing that we can reduce transform computational complexity significantly while minimizing distortion.

Adaptive mixed reality stroke rehabilitation: system architecture and evaluation metrics

This paper presents a novel system architecture and evaluation metrics for an Adaptive Mixed Reality Rehabilitation (AMRR) system for stroke patient. This system provides a purposeful, engaging, hybrid (visual, auditory and physical) scene that encourages patients to improve their performance of a reaching and grasping task and promotes learning of generalizable movement strategies. This system is adaptive in that it provides assistive adaptation tools to help the rehabilitation team customize the training strategy. Our key insight is to combine the patients, rehabilitation team, multimodal hybrid environments and adaptation tools together as an adaptive experiential mixed reality system. There are three major contributions in this paper: (a) developing a computational deficit index for evaluating the patients kinematic performance and a deficit-training-improvement (DTI) correlation for evaluating adaptive training strategy, (b) integrating assistive adaptation tools that help the rehabilitation team understand the relationship between the patients performance and training and customize the training strategy, and (c) combining the interactive multimedia environment and physical environment together to encourage patients to transfer movement knowledge from media space to physical space. Our system has been used by two stroke patients for one-month mediated therapy. They have significant improvement in their reaching and grasping performance (+48.84% and +39.29%) compared to other two stroke patients who experienced traditional therapy (-18.31% and -8.06%).