Thomas Stiefmeier

Wearable Activity Tracking in Car Manufacturing

A context-aware wearable computing system could support a production or maintenance worker by recognizing the workers actions and delivering just-in-time information about activities to be performed.

Activity recognition from on-body sensors: accuracy-power trade-off by dynamic sensor selection

Activity recognition from an on-body sensor network enables context-aware applications in wearable computing. A guaranteed classification accuracy is desirable while optimizing power consumption to ensure the systems wearability. In this paper, we investigate the benefits of dynamic sensor selection in order to use efficiently available energy while achieving a desired activity recognition accuracy. For this purpose we introduce and characterize an activity recognition method with an underlying run-time sensor selection scheme. The system relies on a meta-classifier that fuses the information of classifiers operating on individual sensors. Sensors are selected according to their contribution to classification accuracy as assessed during system training. We test this system by recognizing manipulative activities of assembly-line workers in a car production environment. Results show that the systems lifetime can be significantly extended while keeping high recognition accuracies. We discuss how this approach can be implemented in a dynamic sensor network by using the context-recognition framework Titan that we are developing for dynamic and heterogeneous sensor networks.

Activity recognition from on-body sensors by classifier fusion: sensor scalability and robustness

Activity recognition from on-body sensors is affected by sensor degradation, interconnections failures, and jitter in sensor placement and orientation. We investigate how this may be balanced by exploiting redundant sensors distributed on the body. We recognize activities by a meta-classifier that fuses the information of simple classifiers operating on individual sensors. We investigate the robustness to faults and sensor scalability which follows from classifier fusion. We compare a reference majority voting and a naive Bayesian fusion scheme. We validate this approach by recognizing a set of 10 activities carried out by workers in the quality assurance checkpoint of a car assembly line. Results show that classification accuracy greatly increases with additional sensors (50% with 1 sensor, 80% and 98% with 3 and 57 sensors), and that sensor fusion implicitly allows to compensate for typical faults up to high fault rates. These results highlight the benefit of large on- body sensor network rather than a minimum set of sensors for activity recognition and prompts further investigation.

Using a complex multi-modal on-body sensor system for activity spotting

This paper describes an approach to real-life task tracking using a multi-modal, on-body sensor system. The specific example that we study is quality inspection in car production. This task is composed of up to 20 activity classes such as checking gaps between parts of the chassis, opening and closing the hood and trunk, moving the drivers seat, and turning the steering wheel. Most of these involve subtle and short movements and have a high degree of variability in the way they are performed. To nonetheless spot those actions in a continuous data stream we use a wearable system composed of 7 motion sensors, 16 force sensing resistors (FSR) for lower arm muscle monitoring and 4 ultra-wide band (UWB) tags for tracking user position. We propose a recognition approach that deals separately with each activity class and then merges the results in a final reasoning step. This allows us to fine-tune the system parameters separately for each activity. It also means that the system can be easily extended to accommodate further activities. To demonstrate the feasibility of our approach we present the results of a study with 8 participants and a total of 2394 activities.

Recognition of user activity sequences using distributed event detection

We describe and evaluate a distributed architecture for the online recognition of user activity sequences. In a lower layer, simple heterogeneous atomic activities were recognised on multiple on-body and environmental sensor-detector nodes. The atomic activities were grouped in detection events, depending on the detector location. In a second layer, the recognition of composite activities was performed by an integrator. The approach minimises network communication by local activity aggregation at the detector nodes and transforms the temporal activity sequence into a spatial representation for simplified composite recognition. Metrics for a general description of the architecture are presented. We evaluated the architecture in a worker assembly scenario using 12 sensor-detector nodes. An overall recall and precision of 77% and 79% was achieved for 11 different composite activities. The architecture can be scaled in the number of sensor-detectors, activity events and sequences while being adequately quantified by the presented metrics.

Recognition of hearing needs from body and eye movements to improve hearing instruments

Hearing instruments (HIs) have emerged as true pervasive computers as they continuously adapt the hearing program to the users context. However, current HIs are not able to distinguish different hearing needs in the same acoustic environment. In this work, we explore how information derived from body and eye movements can be used to improve the recognition of such hearing needs. We conduct an experiment to provoke an acoustic environment in which different hearing needs arise: active conversation and working while colleagues are having a conversation in a noisy office environment. We record body movements on nine body locations, eye movements using electrooculography (EOG), and sound using commercial HIs for eleven participants. Using a support vector machine (SVM) classifier and person-independent training we improve the accuracy of 77% based on sound to an accuracy of 92% using body movements. With a view to a future implementation into a HI we then perform a detailed analysis of the sensors attached to the head. We achieve the best accuracy of 86% using eye movements compared to 84% for head movements. Our work demonstrates the potential of additional sensor modalities for future HIs and motivates to investigate the wider applicability of this approach on further hearing situations and needs.

Design of a reconfigurable AES encryption/decryption engine for mobile terminals

This work presents the hardware design of a reconfigurable encryption/decryption engine for the Advanced Encryption Standard (AES) supporting all key lengths. The reconfigurable crypto-engine is integrated as a function unit in a 32 bit RISC processor and can operate in parallel with the standard ALU. Neither the pipeline structure nor the control logic for register forwarding and hazard detection are affected, allowing an easy integration into different RISC architectures. Reconfiguration can be done during runtime, allowing the processor to utilize the arithmetic components and memory elements of the crypto-unit for additional tasks like multiplication in the Galois Field GF(2/sup 8/) required for Reed-Solomon code generation. The RISC processor with the crypto-engine was synthesized using a 0.25 /spl mu/m CMOS technology.

Fusion of String-Matched Templates for Continuous Activity Recognition

This paper describes a new method for continuous activity recognition based on fusion of string-matched activity templates. The underlying segmentation and spotting approach is carried out on several symbol streams in parallel. These streams represent motion trajectories of body limbs in Cartesian space, acquired from body-worn inertial sensors. First results of our method in a highly complex real-world application are presented. 8 subjects performed 3800 activity instances of a checking procedure in car assembly adding up to 480 minutes of recordings. Selecting 6 activity classes with 468 occurrences for first investigations, we obtained an accuracy of up to 87% for the user-dependent case.

Self-powered water meter for direct feedback

Hot water usage accounts for 16% of household demand for energy, much more than lighting and cooking (5% each) and is comparable to electricity usage for appliances (21%). As a means of helping consumers to save hot water, we present a novel self-powered water consumption sensor that enables direct consumption feedback. We equipped 91 Swiss households with the sensors and recorded 3,164 individual showers during the period of three months. The presence of feedback during a shower resulted in the reduction of average shower water consumption from 79 l to 61 l (-22.2%) per day and household. In addition to savings attributable to already installed flow restrictors, an average household could conserve 6,400 l of drinking water and 210 kWh of heat energy (projected to one year). Furthermore, we show that the effects of direct feedback on water consumption did not decline over the course of the study.

Ear-worn reference data collection and annotation for multimodal context-aware hearing instruments

In this work we present a newly developed ear-worn sensing and annotation device to unobtrusively capture head movements in real life situations. It has been designed in the context of developing multimodal hearing instruments (HIs), but is not limited to this application domain. The ear-worn device captures triaxial acceleration, rate of turn and magnetic field and features a one-button-approach for real-time data annotation through the user. The system runtime is over 5 hours at a sampling rate of 128 Hz. In a user study with 21 participants the device was perceived as comfortable and showed a robust hold at the ear. On the example of head acceleration data we perform unsupervised clustering to demonstrate the benefit of head movements for multimodal HIs. We believe the novel technology will help to push the boundaries of HI technology.