Björn Krüger

Motion reconstruction using sparse accelerometer data

The development of methods and tools for the generation of visually appealing motion sequences using prerecorded motion capture data has become an important research area in computer animation. In particular, data-driven approaches have been used for reconstructing high-dimensional motion sequences from low-dimensional control signals. In this article, we contribute to this strand of research by introducing a novel framework for generating full-body animations controlled by only four 3D accelerometers that are attached to the extremities of a human actor. Our approach relies on a knowledge base that consists of a large number of motion clips obtained from marker-based motion capturing. Based on the sparse accelerometer input a cross-domain retrieval procedure is applied to build up a lazy neighborhood graph in an online fashion. This graph structure points to suitable motion fragments in the knowledge base, which are then used in the reconstruction step. Supported by a kd-tree index structure, our procedure scales to even large datasets consisting of millions of frames. Our combined approach allows for reconstructing visually plausible continuous motion streams, even in the presence of moderate tempo variations which may not be directly reflected by the given knowledge base.

A Dual-Source Approach for 3D Pose Estimation from a Single Image

One major challenge for 3D pose estimation from a single RGB image is the acquisition of sufficient training data. In particular, collecting large amounts of training data that contain unconstrained images and are annotated with accurate 3D poses is infeasible. We therefore propose to use two independent training sources. The first source consists of images with annotated 2D poses and the second source consists of accurate 3D motion capture data. To integrate both sources, we propose a dual-source approach that combines 2D pose estimation with efficient and robust 3D pose retrieval. In our experiments, we show that our approach achieves state-of-the-art results and is even competitive when the skeleton structure of the two sources differ substantially.

MotionExplorer: Exploratory Search in Human Motion Capture Data Based on Hierarchical Aggregation

We present MotionExplorer, an exploratory search and analysis system for sequences of human motion in large motion capture data collections. This special type of multivariate time series data is relevant in many research fields including medicine, sports and animation. Key tasks in working with motion data include analysis of motion states and transitions, and synthesis of motion vectors by interpolation and combination. In the practice of research and application of human motion data, challenges exist in providing visual summaries and drill-down functionality for handling large motion data collections. We find that this domain can benefit from appropriate visual retrieval and analysis support to handle these tasks in presence of large motion data. To address this need, we developed MotionExplorer together with domain experts as an exploratory search system based on interactive aggregation and visualization of motion states as a basis for data navigation, exploration, and search. Based on an overview-first type visualization, users are able to search for interesting sub-sequences of motion based on a query-by-example metaphor, and explore search results by details on demand. We developed MotionExplorer in close collaboration with the targeted users who are researchers working on human motion synthesis and analysis, including a summative field study. Additionally, we conducted a laboratory design study to substantially improve MotionExplorer towards an intuitive, usable and robust design. MotionExplorer enables the search in human motion capture data with only a few mouse clicks. The researchers unanimously confirm that the system can efficiently support their work.

Data-Driven Completion of Motion Capture Data

We present a data-driven method for completion of corrupted marker-based motion capture data. Our novel approach is especially suitable for challenging cases, e.g. if complete marker sets of multiple body parts are missing over a long period of time. Without the need for extensive preprocessing we are able to fix missing markers across different actors and motion styles. Our approach can be used for incrementally increasing prior-databases, as the underlying search technique for similar motions scales well to huge databases.

Efficient Unsupervised Temporal Segmentation of Motion Data

We introduce a method for automated temporal segmentation of human motion data into distinct actions and compositing motion primitives based on self-similar structures in the motion sequence. We use neighborhood graphs for the partitioning and the similarity information in the graph is further exploited to cluster the motion primitives into larger entities of semantic significance. The method requires no assumptions about the motion sequences at hand and no user interaction is required for the segmentation or clustering. In addition, we introduce a feature bundling preprocessing technique to make the segmentation more robust to noise, as well as a notion of motion symmetry for more refined primitive detection. We test our method on several sensor modalities, including markered and markerless motion capture as well as on electromyograph and accelerometer recordings. The results highlight our systems capabilities for both segmentation and for analysis of the finer structures of motion data, all in a completely unsupervised manner.

Motion reconstruction using very few accelerometers and ground contacts

Due to the rapid development in sensor technology, the recording of human motion sequences is making its way out of controlled studio environments. Accelerometers are available in a broad range of devices that can be used practically everywhere. In order to show that the reconstruction of full body motions is possible with standard wearable devices, the work at hand introduces a prototype that is capable of doing so on the basis of a very sparse sensor setup: we make use of accelerometers placed on both wrists and lower trunk, and ground contact information only. In this setting human motion reconstruction is a difficult and challenging task due to sparse spatial distribution of sensors and the noisy nature of input data. That is why, we first identify ground contacts from the lower trunk sensor signals and then in a second step combine these results with a fast database look-up that allows a data-driven motion reconstruction. We show the effectiveness of our approach in an extensive set of experiments on both simulated and real data. Our results show appealing reconstructed motions in a variety of human motion scenarios. Display Omitted

One Small Step for a Man: Estimation of Gender, Age and Height from Recordings of One Step by a Single Inertial Sensor

A number of previous works have shown that information about a subject is encoded in sparse kinematic information, such as the one revealed by so-called point light walkers. With the work at hand, we extend these results to classifications of soft biometrics from inertial sensor recordings at a single body location from a single step. We recorded accelerations and angular velocities of 26 subjects using integrated measurement units (IMUs) attached at four locations (chest, lower back, right wrist and left ankle) when performing standardized gait tasks. The collected data were segmented into individual walking steps. We trained random forest classifiers in order to estimate soft biometrics (gender, age and height). We applied two different validation methods to the process, 10-fold cross-validation and subject-wise cross-validation. For all three classification tasks, we achieve high accuracy values for all four sensor locations. From these results, we can conclude that the data of a single walking step (6D: accelerations and angular velocities) allow for a robust estimation of the gender, height and age of a person.

Interactive steering of mesh animations

Creating geometrically detailed mesh animations is an involved and resource-intensive process in digital content creation. In this work we present a method to rapidly combine available sparse motion capture data with existing mesh sequences to produce a large variety of new animations. The key idea is to model shape changes correlated to the pose of the animated object via a part-based statistical shape model. We observe that compact linear models suffice for a segmentation into nearly rigid parts. The same segmentation further guides the parameterization of the pose which is learned in conjunction with the marker movement. Besides the inherent high geometric detail, further benefits of the presented method arise from its robustness against errors in segmentation and pose parameterization. Due to efficiency of both learning and synthesis phase, our model allows to interactively steer virtual avatars based on few markers extracted from video data or input devices like the Kinect sensor.

Model based full body human motion reconstruction from video data

This paper introduces a novel framework for full body human motion reconstruction from 2D video data using a motion capture database as knowledge base containing information on how people move. By extracting suitable two-dimensional features from both, the input video sequence and the motion capture database, we are able to employ an efficient retrieval technique to run a data-driven optimization. Only little preprocessing is needed by our method, the reconstruction process runs close to real time. We evaluate the proposed techniques on synthetic two-dimensional input data obtained from motion capture data and on real video data.

From maturity to old age: tasks of daily life require a different muscle use in horses

In vertebrates ageing is characterized by reduced viscoelasticity of the ligamentous and tendineous structures and fibre changes in muscle. Also, some vertebral joint degeneration develops with ageing. The aim of this study was to apply dynamic time warping to compare the temporal characteristics of the surface electromyography (sEMG) data and to illustrate the differences in the pattern of muscle use during tasks of daily life in old and mature horses. In vivo kinematics (24 skin markers) and sEMG measurements of neck extensors and flexors were taken in five mature horses (age 10 ± 2 years, half of mean life expectancy) and five old horses (age 25 ± 5 years, older than the mean life expectancy). All horses had the same level of activity in the 12 months prior to the measurement. Tasks measured were neck flexion and neck extension as well as neutral neck position. Muscle activation, minimum and maximum muscle activation were collected. Quartiles of muscle activity based on the maximum observed activity of each muscle were calculated to document the relative increase of activity level during the task. Kinematics as well as overall muscle activity patterns were similar across horses and age groups. However, in the neutral position old horses showed increased extensor activity compared to mature horses, indicating that old equine muscle requires more activity to counteract gravity. Dynamic time warping specified optimal temporal alignments of time series, and different temporal performances were identified. The age groups differed during the flexion task, while extension and neutral were more similar. The results of this study show that even in the second half of life and in the absence of muscle disuse the muscular strategy employed by horses continues to be adapted.