Elizabeth A. Crane

Effort-Shape and kinematic assessment of bodily expression of emotion during gait

The purpose of this study was to identify the movement characteristics associated with positive and negative emotions experienced during walking. Joy, contentment, anger, sadness, and neutral were elicited in 16 individuals, and motion capture data were collected as they walked while experiencing the emotions. Observers decoded the target emotions from side and front view videos of the walking trials; other observers viewed the same videos to rate the qualitative movement features using an Effort-Shape analysis. Kinematic analysis was used to quantify body posture and limb movements during walking with the different emotions. View did not affect decoding accuracy except for contentment, which was slightly enhanced with the front view. Walking speed was fastest for joy and anger, and slowest for sadness. Although walking speed may have accounted for increased amplitude of hip, shoulder, elbow, pelvis and trunk motion for anger and joy compared to sadness, neck and thoracic flexion with sadness, and trunk extension and shoulder depression with joy were independent of gait speed. More differences among emotions occurred with the Effort-Shape rather than the kinematic analysis, suggesting that observer judgments of Effort-Shape characteristics were more sensitive than the kinematic outcomes to differences among emotions.

Effect of registration on cyclical kinematic data.

Given growing interest in functional data analysis (FDA) as a useful method for analyzing human movement data, it is critical to understand the effects of standard FDA procedures, including registration, on biomechanical analyses. Registration is used to reduce phase variability between curves while preserving the individual curves shape and amplitude. The application of three methods available to assess registration could benefit those in the biomechanics community using FDA techniques: comparison of mean curves, comparison of average RMS values, and assessment of time-warping functions. Therefore, the present study has two purposes. First, the necessity of registration applied to cyclical data after time normalization is assessed. Second, we illustrate the three methods for evaluating registration effects. Masticatory jaw movements of 22 healthy adults (2 males, 21 females) were tracked while subjects chewed a gum-based pellet for 20s. Motion data were captured at 60 Hz with two gen-locked video cameras. Individual chewing cycles were time normalized and then transformed into functional observations. Registration did not affect mean curves and warping functions were linear. Although registration decreased the RMS, indicating a decrease in inter-subject variability, the difference was not statistically significant. Together these results indicate that registration may not always be necessary for cyclical chewing data. An important contribution of this paper is the illustration of three methods for evaluating registration that are easy to apply and useful for judging whether the extra data manipulation is necessary.

Let's get emotional: emotion research in human computer interaction

Emotion is a topic of growing interest in the HCI community. Studying emotion within the HCI discipline is an exciting interdisciplinary task. This can be facilitated by the exchange of thoughts and ideas with others working on related projects. The aim of this SIG is to bring together an interdisciplinary group of researchers and practitioners actively working on projects where emotion is an essential component. The goals of the SIG are to identify current themes related to emotion specific HCI work and discuss strategies for moving forward.

Methods for Quantifying Emotion-Related Gait Kinematics

Quantitative models of whole body expressive movement can be developed by combining methods form biomechanics, psychology, and statistics. The purpose of this paper was to use motion capture data to assess emotion-related gait kinematics of hip and shoulder sagittal plane movement to evaluate the feasibility of using functional data analysis (FDA) for developing quantitative models. Overall, FDA was an effective method for comparing gait waveforms and emotion-related kinematics were associated with emotion arousal level.

Functional Data Analysis for Biomechanics

The application of nonlinear tools and advanced statistical methods is becoming more prevalent in biomechanical analyses. In a traditional biomechanics laboratory with motion analysis equipment, large amounts of kinematic data can be collected relatively easily. However, a significant gap exists between all the data that are collected and the data that are actually analyzed. Because movements occur over a period of time, whether seconds or minutes, each movement is represented by a continuous series of kinematic data (e.g., 60 or 120 observations per second). Using standard analytic methods, the continuous data associated with each movement are often reduced to a single discrete number. This reduction to a single summary value, such as a peak flexion, extension, or range of motion, excludes potentially valuable information. Reducing a curve representing hip motion during gait to a single range of motion value, for example, precludes the analysis of the entire movement pattern or the timing of the movement. A handful of investigations have recognized this limitation and have begun using functions to maintain the shape and timing of the movement in the analysis. The primary purpose of this chapter is to introduce an emerging collection of statistical methods called Functional Data Analysis (FDA). FDA is distinct from traditional analytic methods because how data changes continuously over time can be assessed. Therefore, information in continuous signals can be retained, such as changes in joint angles or in landmark positions during a movement task. FDA can be used for both exploratory and hypothesis driven analyses with traditional multivariate statistical methods that have been modified for functional predictor and response variables. Although representing motion data as a set of functions is not new to biomechanics analyses (Chester & Wrigley, 2008; Deluzio & Astephen, 2007; Landry et al., 2007; Lee et al., 2009; Sadeghi et al., 2002; 2000), statistical methods developed specifically for analyzing these functions have not been available. More recently, FDAmethods have been usedwithin biomechanics to studymastication (Crane et al., 2010), back pain (Page et al., 2006), as well as age, gender, and speed effects on walking (Roislien et al., 2009). Given the interest in and need for treating motion data as functions, it is important that methods for analyzing a set a functions using emerging statistical methods are brought to the attention of those in the biomechanics community. Although several excellent references exist for Functional Data Analysis (Ramsay, 2000; Ramsay et al., 2009; Ramsay & Silverman, 2002; 2005) there are important issues for biomechanists to be aware of when implementing this set of statistical tools. Therefore, the aims of this chapter are to provide an overview of the steps associated with FDA, to focus on 4

Mammalian Oral Rhythms and Motor Control

Mastication is a derived mammalian trait, characterized by rhythmic jaw movements associated with intra-oral food handling, reduction and bolus formation. Hiiemae defined it as “a key feature of mammalian feeding that involves the coordination of complex movements and precise dental occlusion during a distinct power stroke of the chewing cycle (Hiiemae, 2000). Lund and Kolta refer to mastication as the time “during which the food is mechanically broken down and mixed with saliva to create a slurry of small particles or bolus that can be easily swallowed” (Lund & Kolta, 2006). There is a debate as to whether to define mastication in general or precise terms. The debate focuses on whether to include in its definition the requirements of precise post-canine occlusion, unilateral food bolus placement, and transverse motion of the mandible during the power stroke. Given that feeding in most mammalian and non-mammalian species has yet to be studied and characterized, we opt to use fewer qualifiers and to rely on a more general definition of mastication or chewing in this chapter. The variety of masticatory kinematics and dentoskeletal morphologies (Ungar, 2010) across mammals is almost as striking as plumage variation is among birds. The increased efficiency afforded by masticatory forms and functions may have been necessary to keep pace with another mammalian synapomorphy, the increased energy demands of endothermy. Alternatively, given that erupted enamel cannot be replaced, and that healthy teeth are requisite for longevity and fecundity, efficiency may be required to maximize the life of teeth. Whatever the case, mastication is only one of several distinct oral motor behaviors, which also include (a) suckling, a mammalian-specific trait involved in milk ingestion, (b) lapping or sucking which are used to ingest liquids, fruit juices or insects, (c) rumination or chewing of cud, (d) gnawing of bones or tough food items, (e) tongue rasping used by cats as a food softening behavior, (f) incising, chopping or cutting food, (g) tooth sharpening or thegosis, (h) speech, whistling and communication, (i) facial expressions such as smiling or gritting teeth aggressively, (j) protective behaviors such as sneezing, coughing, gagging or vomiting, (k) tool use such as blowing on, holding or catching objects, (l) respiratory behaviors such as breathing and panting (m) sensory pleasures such as tasting or kissing.

Expression of emotion in body and face

Intelligent interaction with an environment, other IVAs, and human users requires a system that identifies subtle expressive cues and behaves naturally using modalities such as body, face, and voice to communicate. Although research on individual affective channels has increased, little is known about expressive qualities in whole body movement. This study has three goals: (1) to determine rates of observer recognition of emotion in walking, (2) to use kinematic analysis to quantify how emotions change gait patterns in characteristic ways, and (3) to describe the concurrence of facial and bodily expression of emotion. Twenty-six undergraduate students recalled an experience from their own lives in which they felt angry, sad, content, joy, or no emotion at all (neutral). After recalling a target emotion, participants walked across the lab. Whole body motion capture data were acquired using a video-based, 6-camera system. Side view video was also recorded. Ten participants wore a special head mounted camera designed to record video of facial expression. After each trial, participants rated the intensity of eight emotions (4 target and 4 non-target). After blurring the faces in the side view video so that facial expressions were not observable, randomized composite videos were shown to untrained observers. After viewing each video clip, observers selected one of ten responses corresponding to the emotion that they thought the walker felt during the trial. FACS coding was used to evaluate the face video for evidence of emotion and timing of facial expressions with respect to the gait cycle. Self-report data indicated that the walkers felt the target emotions at levels corresponding to “moderately” or above in all trials. Validation data were collected from five observers on gait trials from a subset of subjects (n=16). Recognition rates for sad, anger, neutral and content were 45%, 25%, 20% and 16%, respectively. Joy was recognized at chance levels (10%). Normalized velocity, normalized stride length, cycle duration and velocity were significantly affected by emotion. This study is unique in describing the effects of specific emotions on gait. The preliminary results indicate that gait kinematics change with emotion. Although temporal-spatial kinematics were related to arousal levels, angular kinematics are needed to distinguish emotions with similar levels of arousal.