Anith Mathai

Towards a new generation of virtual humans

This paper presents work from an ongoing project towards developing a new generation of virtual human models that are highly realistic in appearance, movement, and feedback. Santos™, an anatomically correct human model with more than 100 degrees of freedom, is an avatar that exhibits extensive modelling and simulation capabilities, resides in a virtual environment, and conducts human-factors analysis. The paper presents an optimisation-based approach to posture and motion prediction that allows the avatar to operate with autonomy rather than depending on stored animations and data or being restricted by inverse kinematics. It also presents approaches to determining reach envelopes and workspace zone differentiation, and discusses methods for evaluating the physiological status of the virtual human as it completes tasks. Muscle modelling including muscle wrapping, muscle force and stress determination is also discussed. Finally, the process of building a 25-DOF hand model is described. The result is an exciting step towards a virtual human that is more extensive and complete than any other.

Development of the virtual-human Santos®

This paper presents the background and history of the virtual human Santos™ developed by the Virtual Soldier Research (VSR) Program at The University of Iowa. The early virtual human environment was called Mira™. This 15-degree-of-freedom (DOF) upper-body model with posture and motion prediction was funded by John Deere Inc. and US Army TACOM Automotive Research Center. In 2003 US Army TACOM began funding VSR to develop a new generation of virtual humans called Santos (109 DOFs), which was to be another generation of Mira. Later on, Caterpillar Inc., Honda R&D North Americas, Natick Soldier System Center, and USCAR (GM, Ford, and Chrysler) joined the VSR partnership. The objective is to develop a new generation of digital humans comprising realistic human models including anatomy, biomechanics, physiology, and intelligence in real time, and to test digital mockups of products and systems before they are built, thus reducing the significant costs and time associated with making prototypes. The philosophy is based on a novel optimization-based approach for empowering these digital humans to perform, un-aided, in a physics-based world. The research thrusts include the following areas: (1) predictive dynamics, (2) modeling of cloth, (3) hand model, (4) intuitive interface, (5) motion capture, (6) muscle and physiology modeling, (7) posture and motion prediction, (8) spine modeling, and (9) real-time simulation and virtual reality (VR). Currently, the capabilities of Santos include whole-body posture prediction, advanced inverse kinematics, reach envelope analysis, workspace zone differentiation, muscle force and stress analysis, muscle fatigue prediction, simulation of walking and running, dynamic motion prediction, physiologic assessment, a user-friendly interface, a hand model and grasping capability, clothing modeling, thermo discomfort assessment, muscle wrapping and sliding, whole-body vibration analysis, and collision avoidance.

A Robust Formulation for Prediction of Human Running

Abstract : A method to simulate digital human running using an optimization-based approach is presented. The digital human is considered as a mechanical system that includes link lengths, mass moments of inertia, joint torques, and external forces. The problem is formulated as an optimization problem to determine the joint angle profiles. The kinematics analysis of the model is carried out using the Denavit-Hartenberg method. The B-spline approximation is used for discretization of the joint angle profiles, and the recursive formulation is used for the dynamic equilibrium analysis. The equations of motion thus obtained are treated as equality constraints in the optimization process. With this formulation, a method for the integration of constrained equations of motion is not required. This is a unique feature of the present formulation and has advantages for the numerical solution process. The formulation also offers considerable flexibility for simulating different running conditions quite routinely. The zero moment point (ZMP) constraint during the foot support phase is imposed in the optimization problem. The proposed approach works quite well, and several realistic simulations of human running are generated.

A physics-based digital human model

This paper presents a comprehensive human modelling and simulation environment. This environment, called Santos™, is a new generation of digital human simulation systems that allows a user to interact with a digital character with full and accurate biomechanics and a complete muscular system, subject to the laws of physics. Major results in the areas of dynamic motion prediction, advanced posture prediction and comfort level assessment, physiology model, modelling of clothing and muscle wrapping and force assessment will be presented. This paper will feature the various modules that comprise the Santos environment.

Santos™: A New Generation of Virtual Humans

Abstract : Presented in this paper is an on-going project to develop a new generation of virtual human models that are highly realistic in terms of appearance, movement, and feedback (evaluation of the human body during task execution). Santos(Trademark) is an avatar that exhibits extensive modeling and simulation capabilities. It is an anatomically correct human model with more than 100 degrees of freedom. Santos(Trademark) resides in a virtual environment and can conduct human-factors analysis. This analysis entails, among other things, posture prediction, motion prediction, gain analysis, reach envelope analysis, and ergonomics studies. There are essentially three stages to developing virtual humans: (1) basic human modeling (representing how a human functions independently); (2) input functionality (awareness and analysis of the humans environment); and (3) intelligent reaction to input (memory, reasoning, etc.). This paper addresses the first stage. Specifically, we discuss a new human model in terms of mechanics and appearance. We present an optimization-based approach to kinematic and dynamic motion analysis. This approach allows the avatar to operate with complete autonomy rather than with dependence on stored animations and data, or restrictions associated with inverse kinematics. With dynamic analysis, it is not necessary to solve equations of motion. A novel approach for determining reach envelopes is also presented, and this approach provides a unique tool for ergonomic studies. Methods for evaluating the physiological status of the virtual human as tasks are completed are discussed. Finally, additional on-going research is summarized. The result is an exciting step towards a virtual human that is more extensive and more complete than any other.

Santos: A Physics-Based Digital Human Simulation Environment:

This paper presents a comprehensive human modeling and simulation environment under development by the University of Iowa Virtual Soldier Research (VSR) program. This environment, called SantosTM, is a new generation of digital human simulation systems that allows for a user to interact with a digital character with full and accurate biomechanics and a complete muscular system, subject to the laws of physics. Highlighting major results in the areas of dynamic motion prediction, modeling of clothing, modeling of muscle activation and loading, and the Santos intuitive interface will be presented. This paper will feature the various modules that comprise the Santos environment.

Dynamic optimization of human stair-climbing motion

Abstract : The objective of this paper is to present our method of predicting and simulating visually realistic and dynamically consistent human stair-climbing motion. The digital human is modeled as a 55-degrees of freedom branched mechanical system with associated human anthropometry-based link lengths, mass moments of inertia, and centers of gravity. The joint angle profiles are determined using a B-spline-based parametric optimization technique subject to different physics-based, task-based, and environment-based constraints. The formulation offers the ability to study effects of the magnitude and location of external forces on the resulting joint angle profiles and joint torque profiles. Several virtual experiments are conducted using this optimization-based approach and results are presented.

General Biped Motion and Balance of a Human Model

ABSTRACT We propose an algorithm of predicting dynamic biped motions of Santos TM human model. An alternative and efficient formulation of the Zero-Moment Point (ZMP) for dynamic balance and the approximated ground reaction forces/moments are derived from the resultant reaction loads, which includes the gravity, the externally applied loads, and the inertia. The optimization problem is formulated to address the redundancy of the human task, where the general biped and the task-specific constraints are imposed depending on the task requirements. The proposed method is fully predictive and generates physically feasible human-like motions from scratch without any input reference from motion capture or animation. The resulting generated motions demonstrate how a human reacts effectively to different external load conditions in performing a given task by showing realistic features of cause and effect. Key words : human motion generation, Lagrangian dynamics, optimization, Zero-Moment Point.

Multiple User Defined End-Effectors with Shared Memory Communication for Posture Prediction

Inverse Kinematics on a human model combined with optimization provides a powerful tool to predict realistic human postures. A human posture prediction tool brings up the need for greater flexibility for the user, as well as efficient computation performance. This paper demonstrates new methods that were developed for the application of digital human simulation as a software package by allowing for any number of user specified end-effectors and increasing communication efficiency for posture prediction. The posture prediction package for the digital human, Santos, uses optimization constrained by end-effectors on the body with targets in the environment, along with variable cost functions that are minimized, to solve for all joint angles in a human body. This results in realistic human postures which can be used to create optimal designs for things that humans can physically interact with. Previously the end-effectors could only be specified in relation to the left and right wrist and ankle joints. Since the tool was still in developmental phases, communication between the software used to visualize the digital human and environment was done through file I/O. A new optimization method has been developed and implemented to allow for any number of user specified end-effectors, which can be in relation to any joint in the body. Each end-effector can be constrained to any individual target in the environment, which allows for much more flexible interface for a user to define the boundaries of predicting human posture. Communication speeds were increased on average by almost six times through the use of creating a shared memory block, which can be accessed by posture prediction code and the application to visualize the resulting postures at the same time. The combined results of these additional features for posture prediction allow for dynamically updating and visualizing of posture prediction results as new targets for any part of the human body are created or changed in the environment. This in turn provides a new intuitive method for creating a posture prediction simulation which is more interactive with the user. INTRODUCTION A key element with human modeling and thus with product design is posture prediction. Posture Prediction gives the ability to predict human posture accurately and quickly and then provides posture-related feedback to the user. In response to this need, The Virtual Soldier Research (VSR) Program has developed an optimization-based approach to posture prediction that operates in real time and includes a variety of features. This paper presents two new advances with optimization-based posture prediction. First, a method for specifying end-effectors anywhere on the body was developed. Secondly, a technique was utilized for leveraging shared memory as a method of communication between the posture prediction code and the user interface, which substantially reduces the total communication time. Combining the two advancements allows a user to manipulate any point on an avatar in real time, while all other components of the avatar are simultaneously governed by posture prediction. The result is a powerful new product for human-centric design. Predicting human posture is often limited by the location of end-effectors. End-effectors are points of interest on the avatar and are typically constrained to specified locations in the avatar’s environment. When endeffectors are only related to the end of link segments, such as the wrists and ankles, the user is limited to conducting posture prediction studies with which the avatar touches targets with only the hands or feet. As computer aided product design become more popular in an effort to reduce physical prototypes, and thus save money and time, there is a growing need for more advanced posture prediction simulations that go beyond using points on the hands and feet as end-effectors. Since posture prediction is the key feature that the application described in this paper advances, a brief overview of different posture prediction approaches and advances is provided first. Currently, there are two fundamental approaches to predicting postures, on the research side as well with commercially available tools. One method predicts what human posture looks like based on prerecorded motion capture, anthropometric data, and functional regression models (Beck and Chaffin, 1992; Zhang and Chaffin, 1996; Faraway, 1997; Das and Behara, 1998; Faraway et al, 1999; Chaffin, 2002). The second method of predicting posture is done with traditional inverse kinematics, which does not use any observed data. A common approach to inverse kinematics is called the pseudo-inverse method. With such methods, the motion of each link segment is modeled to formulate a set of governing joint equations (Jung et al, 1995; Jung and Choe, 1996; Wang, 1999; Tolani and Badler, 2000). With this method however, as the model’s degrees of freedom increase, the systems of equations become increasingly challenging to solve. A different inverse kinematics method involves optimization. Optimization is used to find a set of joint angle values (each subject to their own constraints), that are used to minimize certain human performance measures, such as discomfort. The constraint in the optimization problem is restricting an end-effector to reach a target point (Abdel-Malek et al, 2001; Mi et al, 2002). This approach does not require any prerecorded data and can be computed efficiently (Farrell and Marler, 2004). Most recent advancements concerning the number of end-effectors in the posture prediction problem have been creating a dual-arm posture prediction, where end-effectors are placed at the end of each arm, each having their own individual constraint (Farrell and Marler, 2005; Yang et al 2004, 2006, 2007). When legs are included in the model, end-effector can also be placed at the end of each leg, since it is also the end of each link segment. The development of a multiple end-effectors feature described in this paper extends the current state of the art and allows a user to place any number of end-effectors on any part of the body. This is a new capability with respect both to research efforts and currently available human-modeling products. In addition to incorporating multiple endeffectors, shared memory is leveraged to increase speed. Shared memory allows multiple programs to access the same memory location, so when one program changes a variable’s value, all other programs that are allowed immediately have access to that change. This makes it an efficient method of passing data. If posture prediction is to provide an efficient design tool, computational efficiency is critical. Trade-off analysis of various design alterations necessitates the use of real-time simulations, and shared memory can enable this. Shared memory is typically used as a method of inter-process communication, which is a way of exchanging data between multiple programs running at the same time. When applications have a need for more efficient communications and higher performance, parallel processing is considered, because multiple processes can run simultaneously. Using shared memory for this function allows one to avoid expensive overheads associated with other parallel methods (Wang et al, 1999). However, shared memory has never been applied to posture prediction as a way of increasing speed. To increase the speed of posture prediction to the point where a user can drag any point on the body and the simulation can concurrently predict the consequent joint angles in real time provides a new tool that saves time and money during ergonomic design studies. The following section in this paper provides an overview of optimization-based posture prediction and how it is integrated with 3D visualization. We then discuss the advantages of, and a method for allowing the user to specify multiple end-effectors anywhere on the body. After that, shared memory is addressed and how it is implemented with posture prediction. Finally, we show how combing multiple end-effectors and shared memory allows for a new intuitive tool for human modeling product design, which allows the user to drag any part of the body with realistic human postures being predicted and visualized in real time. OPTIMIZATION-BASED POSTURE PREDICTION This section discusses the fundamental of optimizationbased posture prediction, as implemented in Santos, a new kind of virtual human developed at VSR (AbdelMalek et al, 2006). Marler et al (2005) explain how the most basic posture prediction problem entails having an avatar use a natural posture to contact a specified target point with an end-effector on a kinematic system such as a human arm. With an optimization-based approach, the joint angles for all of the degrees-of-freedom (DOFs) in the human model provide the design variables and are determined by optimizing an objective function that represents a human performance measure. These performance measures can include discomfort, joint displacement, potential energy, effort, visual acuity, etc. The interface and all interaction with the user are done through the Virtools 3D development engine. To date, this interaction has been conducted as follows. The user can select through posture prediction options, such as the performance measures mentioned above, toggle vision on or off, and toggle collision avoidance on or off. There are end-effectors in relation to the end of each link series (left and right wrists and ankles), which the user can specify targets in the world for each (Farrell et al, 2005). Every time a new target is selected for one of these end-effectors, Virtools writes out the information posture prediction needs to input files and runs posture prediction as an executable. When posture prediction is run, it reads the input files, calculates the optimal solution of joint angles, and then writes out that solution to a separate output file. Virtools, which ha