Zhiyu Huo

Comparison of 3D Joint Angles Measured With the Kinect 2.0 Skeletal Tracker Versus a Marker-Based Motion Capture System

The Microsoft Kinect is becoming a widely used tool for inexpensive, portable measurement of human motion, with the potential to support clinical assessments of performance and function. In this study, the relative osteokinematic Cardan joint angles of the hip and knee were calculated using the Kinect 2.0 skeletal tracker. The pelvis segments of the default skeletal model were reoriented and 3-dimensional joint angles were compared with a marker-based system during a drop vertical jump and a hip abduction motion. Good agreement between the Kinect and marker-based system were found for knee (correlation coefficient = 0.96, cycle RMS error = 11°, peak flexion difference = 3°) and hip (correlation coefficient = 0.97, cycle RMS = 12°, peak flexion difference = 12°) flexion during the landing phase of the drop vertical jump and for hip abduction/adduction (correlation coefficient = 0.99, cycle RMS error = 7°, peak flexion difference = 8°) during isolated hip motion. Nonsagittal hip and knee angles did not correlate well for the drop vertical jump. When limited to activities in the optimal capture volume and with simple modifications to the skeletal model, the Kinect 2.0 skeletal tracker can provide limited 3-dimensional kinematic information of the lower limbs that may be useful for functional movement assessment.

Testing an assistive fetch robot with spatial language from older and younger adults

Methods and experimental results are presented for interpreting 3D spatial language descriptions used for human to robot communication in a fetch task. The work is based on human subject experiments in which spatial language descriptions were logged from younger and older adult participants. A spatial language model is proposed, and methods are presented for translating natural spatial language descriptions into robot commands that allow the robot to find the requested object. Robot command representation and robot behavior are also discussed. Experimental results compare path metrics of the robot system and human subjects in a common simulation environment. The overall success rate of the robot trials is 85%.

Natural Spatial Description Generation for Human-Robot Interaction in Indoor Environments

This paper proposes a spatial language generation system to communicate with a person about the location of an object in an indoor environment. It aims at finding a short, accurate and human-like description for building a natural and friendly interface between robots and humans using spatial language interaction. The system performs an inverse procedure to spatial language grounding which links natural commands to robot actions. The system works in two steps. It will first search for the best matching grounding model which describes the spatial relations between the target object and the references; then it will generate the natural language by mimicking a humans talking style. A corpus of 149 spatial language commands for an indoor environment fetch task is used to train the language generation model. An early-stage experiment is conducted and the results illustrate a potential for further development.

Spatial language experiments for a robot fetch task

This paper outlines a new study that investigates spatial language for use in human-robot communication. The scenario studied is a home setting in which the elderly resident has misplaced an object, such as eyeglasses, and the robot will help the resident find the object. We present results from phase I of the study in which we investigate spatial language generated to a human addressee or a robot addressee in a virtual environment.

Development and Validation of a Portable and Inexpensive Tool to Measure the Drop Vertical Jump Using the Microsoft Kinect V2

Background: Noncontact anterior cruciate ligament (ACL) injury in adolescent female athletes is an increasing problem. The knee-ankle separation ratio (KASR), calculated at initial contact (IC) and peak flexion (PF) during the drop vertical jump (DVJ), is a measure of dynamic knee valgus. The Microsoft Kinect V2 has shown promise as a reliable and valid marker-less motion capture device. Hypothesis: The Kinect V2 will demonstrate good to excellent correlation between KASR results at IC and PF during the DVJ, as compared with a “gold standard” Vicon motion analysis system. Study Design: Descriptive laboratory study. Level of Evidence: Level 2. Methods: Thirty-eight healthy volunteer subjects (20 male, 18 female) performed 5 DVJ trials, simultaneously measured by a Vicon MX-T40S system, 2 AMTI force platforms, and a Kinect V2 with customized software. A total of 190 jumps were completed. The KASR was calculated at IC and PF during the DVJ. The intraclass correlation coefficient (ICC) assessed the degree of KASR agreement between the Kinect and Vicon systems. Results: The ICCs of the Kinect V2 and Vicon KASR at IC and PF were 0.84 and 0.95, respectively, showing excellent agreement between the 2 measures. The Kinect V2 successfully identified the KASR at PF and IC frames in 182 of 190 trials, demonstrating 95.8% reliability. Conclusion: The Kinect V2 demonstrated excellent ICC of the KASR at IC and PF during the DVJ when compared with the Vicon system. A customized Kinect V2 software program demonstrated good reliability in identifying the KASR at IC and PF during the DVJ. Clinical Relevance: Reliable, valid, inexpensive, and efficient screening tools may improve the accessibility of motion analysis assessment of adolescent female athletes.

Using Spatial Language to Drive a Robot for an Indoor Environment Fetch Task

This paper proposes a system that allows the use of natural spatial language to control a robot performing a fetch task in an indoor environment. The system processes spatial referencing language and extracts a tree structure of language chunks. The spatial language system is then grounded to a robot navigation instruction in the form of a sequence of actions based on spatial references to furniture and room structure; the best navigation instruction is selected by scoring. In addition, the Reference-Direction-Target (RDT) model is proposed to represent indoor robot actions. To control the robot for the fetch task, a behavior model is designed based on the RDT model. An assistive robot has been designed and programmed based on this system. The proposed spatial language grounding model and robot behavior model are tested experimentally in three sets of experiments. Results show that the system enables a robot to follow spatial language commands in a physical indoor environment even if the referenced furniture items are re-positioned.

Mass Screening of Youth Athletes for High Risk Landing Patterns using a Portable and Inexpensive Motion Sensor Device

Objectives: Biomechanical factors such as dynamic knee valgus predispose young athletes to lower extremity injury including tears of the anterior cruciate ligament (ACL). Identifying these risk factors may allow for targeted injury prevention strategies. Our prior work has validated the Microsoft Kinect vs. Vicon to detect knee-ankle separation ratio (KASR) during the drop vertical jump test (DVJ). Our hypothesis is that screening with the Microsoft Kinect will be safe, efficient, and provide information to help detect injury risk in youth athletes. Methods: A total of 180 healthy high school athletes, ages 14-18 (80 males and 100 females, age of 16.9 ± 1.31 and BMI of 22.8 ± 3.7) participated in this study. Each subject performed three successful DVJ (Fig. 1). We used an inexpensive, portable motion sensor device to measure the KASR, which captures the ratio of the horizontal distance between knees to the horizontal distance between ankles. From previous studies, a KASR value <1 indicates dynamic valgus, while being below 0.6 is considered severe risk. Demographic information and measurements for KASR on initial contact and peak flexion were analyzed statistically. Results: Using two motion sensor device stations, it took 1.5 minutes to screen and test each subject. There were no injuries that occurred during the screening process. Our results showed a statistically significant difference between male and female KASR for both initial contact (p<0.001) and peak flexion (p<0.001). Sixty out of the 100 female subjects (60%) tested positive for valgus (9 initial contact KASR, 1 peak flexion KASR, 50 both initial contact and peak flexion KASR) with two subjects screening for severe risk (KASR<0.6). The average KASR for females was 1.01 (peak flexion) and 0.967 (initial contact). Eighteen out of the 80 male subjects (22.5%) tested positive for valgus (6 initial contact KASR, 2 peak flexion KASR, 10 both initial contact and peak flexion KASR) with no male subjects screening positive for severe risk. The average KASR for males was 1.26 (peak flexion) and 1.13 (initial contact). Comparing males and females that screened positive for KASR, there was a significant gender difference between the KASR at initial contact with females exhibiting more valgus than males (p<0.001) (Table 1). Conclusion: Our findings suggest that a portable and inexpensive motion analysis device can detect dynamic valgus during the DVJ in youth athletes. Large scale screening for dynamic valgus was safe and efficient. Known gender disparities between male and females for neuromuscular imbalances were identified. Potential use of this information for targeted injury prevention is appealing and requires further investigation.

Examining the feasibility of a Microsoft Kinect ™ based game intervention for individuals with anterior cruciate ligament injury risk

We describe a feasibility study in which the Microsoft Kinect is used for a game-based exercise to strengthen posterior chain muscles which are often weak in those at high risk of anterior cruciate ligament (ACL) injury. In the game, subjects perform a single posterior chain strengthening exercise. The game uses a side-scrolling video display driven by a hip abduction exercise while a player lies down on the floor. Leg lifts beyond a predetermined angle trigger the jumping action of an animated tiger. We describe the scene and game control, which uses depth images from the Kinect. Although Kinect-based skeletal data are used for many games, the skeletal model does not yield good estimates for positions on the floor. Our proposed system uses multiple leg angle estimators for different angle regions to recognize the player lying down and capture the angle between two legs. We conducted an experiment that validates our system with marker-based Vicon ground truth data. We also present results of an end-to-end test using the game, showing feasibility.