Akihiko Murai

Musculoskeletal-see-through mirror: Computational modeling and algorithm for whole-body muscle activity visualization in real time

In this paper, we present a system that estimates and visualizes muscle tensions in real time using optical motion capture and electromyography (EMG). The system overlays rendered musculoskeletal human model on top of a live video image of the subject. The subject therefore has an impression that he/she sees the muscles with tension information through the cloth and skin. The main technical challenge lies in real-time estimation of muscle tension. Since existing algorithms using mathematical optimization to distribute joint torques to muscle tensions are too slow for our purpose, we develop a new algorithm that computes a reasonable approximation of muscle tensions based on the internal connections between muscles known as neuronal binding. The algorithm can estimate the tensions of 274 muscles in only 16 ms, and the whole visualization system runs at about 15 fps. The developed system is applied to assisting sport training, and the user case studies show its usefulness. Possible applications include interfaces for assisting rehabilitation.

Computationally fast estimation of muscle tension for realtime Bio-feedback

In this paper, we propose a method for realtime estimation of whole-body muscle tensions. The main problem of muscle tension estimation is that there are infinite number of solutions to realize a particular joint torque due to the actuation redundancy. Numerical optimization techniques, e.g. quadratic programming, are often employed to obtain a unique solution, but they are usually computationally expensive. For example, our implementation of quadratic programming takes about 0.17 sec per frame on the musculoskeletal model with 274 elements, which is far from realtime computation. Here, we propose to reduce the computational cost by using EMG data and by reducing the number of unknowns in the optimization. First, we compute the tensions of muscles with surface EMG data based on a biological muscle data, which is a very efficient process. We also assume that their synergists have the same activity levels and compute their tensions with the same model. Tensions of the remaining muscles are then computed using quadratic programming, but the number of unknowns is significantly reduced by assuming that the muscles in the same heteronymous group have the same activity level. The proposed method realizes realtime estimation and visualization of the whole-body muscle tensions that can be applied to sports training and rehabilitation.

Modeling and identification of human neuromusculoskeletal network based on biomechanical property of muscle

In this paper, we build a whole-body neuromusculoskeletal network model including somatic reflex, and identify its parameters through non-invasive measurements and statistical analysis. Such models are crucial for analyzing and estimating signals in the nervous system. Our neuromuscular model consists of two parts. The first part models the neuromuscular network that represents the relationships between the spinal nerve signals and muscle activities, which are then converted to muscle tensions using a physiological muscle dynamics model. The second part includes the feedback loops from muscle spindles and Golgi tendon organs to the spinal nerve that represent the somatic reflex using muscle length, velocity, and tension information. We demonstrate the consistency of the model by showing that a forward dynamics simulation of somatic reflex yields a motion similar to actual human response.

Modeling and Identifying the Somatic Reflex Network of the Human Neuromuscular System

In this paper, we build a mathematical model of the whole-body neuromuscular network and identify its parameters by optical motion capture, inverse kinematics, inverse dynamics computation, and statistical analysis. The model includes a skeleton, a musculotendon network, and a neuromuscular network. The skeleton is composed of 155 joints representing the inertial property and mobility of the human body. The musculotendon network includes more than 1000 muscles, tendons, and ligaments modeled as ideal wires with any number of via points. We also develop an inverse dynamics algorithm to estimate the muscle tensions required to perform a given motion sequence. Finally, we model the somatic reflex network based on the relationship between the spinal nerves and the muscle tensions by a neural network. The resulting parameters match well with the agonist-antagonist relationship of the muscles. We also demonstrate that the model inherently includes low-level somatic reflexes such as the patellar tendon reflex using the neuromuscular model. This is the attempt to build and identify the neuromuscular network based only on noninvasive motion measurements, and the result shows that the whole-body muscles can be controlled by the command signals as few as the number of spinal nerve rami.

Effects of nerve signal transmission delay in somatosensory reflex modeling based on inverse dynamics and optimization

Human motion coordination is a long-standing research issue in biomechanics, and it should also have some implications for humanoid robot control.We have built a whole-body somatosensory reflex model based on our neuromusculoskeletal model and identified its parameters through non-invasive measurements and statistical analysis. Such models are crucial for analyzing and estimating signals in the nervous system. In this paper, we focus on signal transmission delay of the somatosensory reflex loop and investigate its relationship with the generalization capability of the reflex model. We obtain some sets of model parameters assuming different time delays using the data obtained from a stepping motion, and perform cross validations against stepping motions with different cycles as well as entirely different behaviors such as squat and jump. Interestingly, time delays close to the value expected from physiological properties show better cross-validation results than others. This result suggests that relatively simple reflex control can be generalized to multiple behaviors if the parameters are appropriate, and that robust control is possible even with large feedback delay.

Characterization of motor skill based on musculoskeletal model

In this paper, we propose two methods to quantitatively analyze the motor skill in sports. The first method is the dimensionality reduction using the principal component analysis (PCA). The motion data, e.g. the joint angles (143-dimensional vector) or the muscle tensions (989-dimensional vector), are projected to a lower dimensional space that well represents the characteristics of original data. The similarities and differences become clear by observing the data in the low-dimensional space. The second method utilizes the joint stiffness obtained from joint kinematics and a biological muscle model. Though muscle tension data contain richer information than joint angle data, the dimension is so high that simply applying PCA does not give useful insights. Here we calculate the joint stiffness using the muscle tension data and a biological muscle model. This information represents the muscle usage skill which can not be observed only from motion data, and reflects the redundancy of the muscle tensions. We demonstrate the two methods by analyzing skilled performers’ motions.

Muscle tension database for contact-free estimation of human somatosensory information

Contact-free estimation of the human somatosensory information is an essential skill for robots working in daily environments. The main objective of this paper is to develop a method for estimating muscle tensions without any sensors attached to the body. Muscle tension is an important information for evaluating physical load during motions. Existing approaches utilizing optimization techniques and/or electromyography (EMG) signals are not appropriate due to lack of physiological validity or usage of electrodes. In this paper, we propose to use a database of muscle tension distribution for obtaining physiologically realistic muscle tensions only from motion data. Using such database instead of direct EMG measurement is justified by the fact that muscle tension distribution is relatively highly correlated even among different subjects. For each new motion frame, we search for a similar entry in the database and use the corresponding muscle tension distribution to estimate the current muscle tensions. We demonstrate that the muscle tensions obtained by this approach is much closer to the result using the EMG data than that using pure numerical optimization, even when the database is constructed from other persons data.

Musculoskeletal modeling and physiological validation

Digital human models are applied in human motion analysis and simulation. They can be applied in rehabilitation, sports science, biomedicine, and so on. The difficulty of validation of models and analysis algorithms constricts their practical usage. In this paper, we show our work on human motion analysis using a whole-body musculoskeletal model and validate the analysis results in terms of physiology. First, we build the detailed musculoskeletal model that represents the kinematics and dynamics characteristics of human body. Optical motion capture, force plates and electromyography(EMG) are used for human motion capture. We estimate the muscle tensions required to generate the captured motion sequence based on an inverse kinematics and dynamics computation and a mathematical optimization. The estimated muscle tensions for locomotion cycles are compared with the tensions computed from the simultaneously measured EMG data and a physiological muscle model. The model and the analysis algorithm are also applied to a neurophysiological phenomenon, the nontrivial preferred direction that is a result of the cosine tuning. Our model and analysis algorithm achieves results that correspond with the experimental physiological data well. The possible applications of our model and algorithm involve rehabilitation, sports science, biomedicine, and robotics, e.g. a controller of an exoskeletal robot for human support.

Modeling and identification of the human arm stretch reflex using a realistic spiking neural network and musculoskeletal model

This study proposes a model that combines a realistically scaled neural network made up of pools of spiking neurons, with a musculoskeletal model of the human arm. We used evidence from literature to design topological pools of spinal neurons and their synaptic connections. The spiking output of the motor neuron pools were used as the command signals that generated motor unit forces, and drove joint motion. Feedback information from muscle spindles was relayed to the neural network via monosynaptic and disynaptic pathways. Participant-specific parameters of the combined neuromusculoskeletal (NMS) system were then identified from recorded experimental data. The identified NMS model was used to simulate the arm stretch reflex and the results were validated by comparison to an independent recorded dataset. The models and methodology proposed in this study show that large and complex neural systems can be identified in conjunction with the musculoskeletal systems that they control. This additional layer of detail in NMS models has important relevance to the research communities related to rehabilitation robotics and human movement analysis.

Constraints and deformations analysis for machining accuracy assessment of closed kinematic chains

This paper discusses the design issue of general closed kinematic chains focusing on analysis of constraints and elastic deformations. Closed kinematic chains are considered more advantageous in rigidity, power-output, and accuracy than open kinematic chains. However, it is not much stressed that closed kinematic chains are so sensitive to machining errors that a tenth of a millimeter of error might result in jamming and immobility. To avoid this critical problem, closed kinematic chains tend to be designed with play that reduces their native advantages. If 3D-CAD systems are equipped with a mathematical tool that evaluates machining accuracy, elasticity, and constraints, it will significantly assist the skill of designers and extend the field of applications of closed kinematic chains. In this paper, we clarify machining errors that are absorbable as errors in unactuated joints. These kind of errors are permissible. Mobility analysis in the presence of unabsorbable machining errors is discussed taking account of elastic deformations and strain energy. The mechanism can move smoothly even with machining errors if the strain energy remains less fluctuate along its motion locus. The established mathematical method of mobility analysis is applied to the design of a closed kinematic chain and used in practice to fabricate a medical robot system.