Jonathan Feng-Shun Lin

Online Segmentation of Human Motion for Automated Rehabilitation Exercise Analysis

To enable automated analysis of rehabilitation movements, an approach for accurately identifying and segmenting movement repetitions is required. This paper proposes an approach for online, automated segmentation and identification of movement segments from continuous time-series data of human movement, obtained from body-mounted inertial measurement units or from motion capture data. The proposed approach uses a two-stage identification and recognition process, based on velocity features and stochastic modeling of each motion to be identified. In the first stage, motion segment candidates are identified based on a characteristic sequence of velocity features such as velocity peaks and zero velocity crossings. In the second stage, hidden Markov models are used to accurately identify segment locations from the identified candidates. The proposed approach is capable of online segmentation and identification, enabling interactive feedback in rehabilitation applications. The approach is validated on 20 healthy subjects and four rehabilitation patients performing rehabilitation movements, achieving segmentation accuracy of 87% with user specific templates and 79%-83% accuracy with user-independent templates.

Human pose recovery using wireless inertial measurement units

Many applications in rehabilitation and sports training require the assessment of the patients status based on observation of their movement. Small wireless sensors, such as accelerometers and gyroscopes, can be utilized to provide a quantitative measure of the human movement for assessment. In this paper, a kinematics-based approach is developed to estimate human leg posture and velocity from wearable sensors during the performance of typical physiotherapy and training exercises. The proposed approach uses an extended Kalman filter to estimate joint angles from accelerometer and gyroscopic data and is capable of recovering joint angles from arbitrary 3D motion. Additional joint limit constraints are implemented to reduce drift, and an automated approach is developed for estimating and adapting the process noise during online estimation. The approach is validated through a user study consisting of 20 subjects performing knee and hip rehabilitation exercises. When compared to motion capture, the approach achieves an average root-mean-square error of 4.27 cm for unconstrained motion, with an average joint error of 6.5°. The average root-mean-square error is 3.31 cm for sagittal planar motion, with an average joint error of 4.3°.

Movement Primitive Segmentation for Human Motion Modeling: A Framework for Analysis

Movement primitive segmentation enables long sequences of human movement observation data to be segmented into smaller components, termed movement primitives, to facilitate movement identification, modeling, and learning. It has been applied to exercise monitoring, gesture recognition, human-machine interaction, and robot imitation learning. This paper proposes a segmentation framework to categorize and compare different segmentation algorithms considering segment definitions, data sources, application-specific requirements, algorithm mechanics, and validation techniques. The framework is applied to human motion segmentation methods by grouping them into online, semionline, and offline approaches. Among the online approaches, distance-based methods provide the best performance, while stochastic dynamic models work best in the semionline and offline settings. However, most algorithms to date are tested with small datasets, and algorithm generalization across participants and to movement changes remains largely untested.

Human motion segmentation by data point classification

Contemporary physiotherapy and rehabilitation practice uses subjective measures for motion evaluation and requires time-consuming supervision. Algorithms that can accurately segment patient movement would provide valuable data for progress tracking and on-line patient feedback. In this paper, we propose a two-class classifier approach to label each data point in the patient movement data as either a segment point or a non-segment point. The proposed technique was applied to 20 healthy subjects performing lower body rehabilitation exercises, and achieves a segmentation accuracy of 82%.

Full-body multi-primitive segmentation using classifiers

During human-robot interaction, the robot observes a continuous stream of time-series data capturing the behaviour of the human and any changes in the environment. For applications such as imitation learning, intention and gesture recognition, the time-series data is typically segmented into action or motion primitives, requiring accurate and online temporal segmentation. This paper casts the time-series segmentation problem into a two-class classification problem, labelling each data point as either a segment edge or a within-segment point, and applies several common classifiers to a set of full body motion data. The support vector machine combined with principal component analysis dimensionality reduction were found to perform best, with a classification F1 score of 91% when applied to novel exemplars. The proposed approach can also generalize to motions unseen during training, achieving a classification F1 score of 83% when applied to novel motions.

IMU based single stride identification of humans

To facilitate human-robot interactions with the user, it is necessary for the robot to identify the interaction partner. We propose the use of a single wearable sensor worn at the center of the users belt to record the gait when the interaction partner approaches the robot. Based on the data of a single gait cycle recorded with a single inertial measurement unit (IMU), we identify a person by his/her walking style. For identification, we first detect individual strides. We introduce a simple feature that characterizes the individuals asymmetry of gait and classify the individual using a Bayes classifier. To evaluate our approach, we collect motion data from 20 persons; the classification accuracy based on the proposed asymmetry-based feature reaches 99.3%. We further investigate the robustness of our approach against slight variations in the sensor placement, variations in speed, and walking straight versus walking on a curved route.

Human pose recovery for rehabilitation using ambulatory sensors

In this paper, an approach for lower-leg pose recovery from ambulatory sensors is implemented and validated in a clinical setting. Inertial measurement units are attached to patients undergoing physiotherapy. The sensor data is combined with a kinematic model within an extended Kalman filter framework to perform joint angle estimation. Anthropometric joint limits and process noise adaptation are employed to improve the quality of the joint angle estimation. The proposed approach is tested on 7 patients following total hip or knee joint replacement surgery. The proposed approach achieves an average root-mean-square error of 0.12 radians at key poses.

Human motion segmentation using cost weights recovered from inverse optimal control

A common hypothesis in human motor control is that human movement is generated by optimizing with respect to a certain criterion and is task dependent. In this paper, a method to segment human movement by detecting changes to the optimization criterion being used via inverse optimal control is proposed. The control strategy employed by the motor system is hypothesized to be a weighted sum of basis cost functions, with the basis weights changing with changes to the motion objective(s). Continuous time series data of movement is processed using a sliding fixed width window, estimating the basis weights of each cost function for each window by minimizing the Karush-Kuhn-Tucker optimality conditions. The quality of the cost function recovery is verified by evaluating the residual. The successfully estimated basis weights are averaged together to create a set of time varying basis weights that describe the changing control strategy of the motion and can be used to segment the movement with simple thresholds. The proposed algorithm is first demonstrated on simulation data and then demonstrated on a dataset of human subjects performing a series of squatting tasks. The proposed approach reliably identifies the squatting movements, achieving a segmentation accuracy of 84%.

Segmentation of human upper body movement using multiple IMU sensors

This paper proposes an approach for the segmentation of human body movements measured by inertial measurement unit sensors. Using the angular velocity and linear acceleration measurements directly, without converting to joint angles, we perform segmentation by formulating the problem as a classification problem, and training a classifier to differentiate between motion end-point and within-motion points. The proposed approach is validated with experiments measuring the upper body movement during reaching tasks, demonstrating classification accuracy of over 85.8%.

Segmentation by Data Point Classification Applied to Forearm Surface EMG

Recent advances in wearable technologies have led to the development of new modalities for human-machine interaction such as gesture-based interaction via surface electromyograph (EMG). An important challenge when performing EMG gesture recognition is to temporally segment the individual gestures from continuously recorded time-series data. This paper proposes an approach for EMG data segmentation, by formulating the segmentation problem as a classification task, where a classifier is used to label each data point as either a segment point or a non-segment point. The proposed EMG segmentation approach is used to recognize 9 hand gestures from forearm EMG data of 10 participants and a balanced accuracy of 83 % is achieved.