Sebastian Feese

AmbientSense: A real-time ambient sound recognition system for smartphones

This paper presents design, implementation, and evaluation of AmbientSense, a real-time ambient sound recognition system on a smartphone. AmbientSense continuously recognizes user context by analyzing ambient sounds sampled from a smartphones microphone. The phone provides a user with realtime feedback on recognised context. AmbientSense is implemented as an Android app and works in two modes: in autonomous mode processing is performed on the smartphone only. In server mode recognition is done by transmitting audio features to a server and receiving classification results back. We evaluated both modes in a set of 23 daily life ambient sound classes and describe recognition performance, phone CPU load, and recognition delay. The application runs with a fully charged battery up to 13.75 h on a Samsung Galaxy SII smartphone and up to 12.87 h on a Google Nexus One phone. Runtime and CPU load were similar for autonomous and server modes.

Sensing spatial and temporal coordination in teams using the smartphone

Teams are at the heart of today’s organizations and their performance is crucial for organizational success. It is therefore important to understand and monitor team processes. Traditional approaches employ questionnaires, which have low temporal resolution or manual behavior observation, which is labor intensive and thus costly. In this work, we propose to apply mobile behavior sensing to capture team coordination processes in an automatic manner, thereby enabling cost-effective and real-time monitoring of teams. In particular, we use the built-in sensors of smartphones to sense interpersonal body movement alignment and to detect moving sub-groups. We aggregate the data on team level in form of networks that capture a) how long team members are together in a sub-group and b) how synchronized team members move. Density and centralization metrics extract team coordination indicators from the team networks. We demonstrate the validity of our approach in firefighting teams performing a realistic training scenario and investigate the link between the coordination indicators and team performance as well as experienced team coordination. Our method enables researchers and practitioners alike to capture temporal and spatial team coordination automatically and objectively in real-time.

CoenoFire: monitoring performance indicators of firefighters in real-world missions using smartphones

Firefighting is a dangerous task and many research projects have aimed at supporting firefighters during missions by developing new and often costly equipment. In contrast to previous approaches, we use the smartphone to monitor firefighters during real-world missions in order to provide objective data that can be used in post-incident briefings and trainings. In this paper, we present CoenoFire, a smartphone based sensing system aimed at monitoring temporal and behavioral performance indicators of firefighting missions. We validate the performance metrics showing that they can indicate why certain teams performed faster than others in a training scenario conducted by 16 firefighting teams. Furthermore, we deployed CoenoFire over a period of six weeks in a professional fire brigade. In total, 71 firefighters participated in our study and the collected data includes 76 real-world missions totaling to over 148 hours of mission data. Additionally, we visualize real-world mission data and show how mission feedback is supported by the data.

Towards an online detection of pedestrian flocks in urban canyons by smoothed spatio-temporal clustering of GPS trajectories

Detecting pedestrians moving together through public spaces can provide relevant information for many location-based social applications. In this work we present an online method to detect such pedestrian flocks by spatio-temporal clustering of location trajectories. Compared to prior work, our method provides increased robustness against the influence of noisy and missing GPS data often encountered in urban environments. To assess the performance of the method, we record GPS trajectories from ten subjects walking through a city. The data set contains various flock formations and corresponding ground truth information is available. With this data set, we can evaluate the accuracy of our method to detect flocks. Results show that we can detect flocks and their members with an accuracy of 91.3%. We evaluate the influence of noisy and missing location data on the detection accuracy and show that the introduced filtering heuristics provides increased detection accuracy in such realistic situations.

Sensing group proximity dynamics of firefighting teams using smartphones

Firefighters work in dangerous and unfamiliar situations under a high degree of time pressure and thus team work is of utmost importance. Relying on trained automatisms, firefighters coordinate their actions implicitly by observing the actions of their team members. To support training instructors with objective mission data, we aim to automatically detect when a firefighter is in-sight with other firefighters and to visualize the proximity dynamics of firefighting missions. In our approach, we equip firefighters with smartphones and use the built-in ANT protocol, a low-power communication radio, to measure proximity to other firefighters. In a second step, we cluster the proximity data to detect moving sub-groups. To evaluate our method, we recorded proximity data of 16 professional firefighting teams performing a real-life training scenario. We manually labeled six training sessions, involving 51 firefighters, to obtain 79 minutes of ground truth data. On average, our algorithm assigns each group member to the correct ground truth cluster with 80% accuracy. Considering height information derived from atmospheric pressure signals increases group assignment accuracy to 95%.

Quantifying Behavioral Mimicry by Automatic Detection of Nonverbal Cues from Body Motion

Effective leadership can increase team performance, however the underlying micro-level behaviors that support team performance are still unclear. At the same time, traditional behavioral observation methods rely on manual video annotation which is a time consuming and costly process. In this work, we employ wearable motion sensors to automatically extract nonverbal cues from body motion. We utilize activity recognition methods to detect relevant nonverbal cues such as head nodding, gesticulating and posture changes. Further, we combine the detected individual cues to quantify behavioral mimicry between interaction partners. We evaluate our methods on data that was acquired during a psychological experiment in which 55 groups of three persons worked on a decision-making task. Group leaders were instructed to either lead with individual consideration orin an authoritarian way. We demonstrate that nonverbal cues can be detected with a F1-measure between 56% and 100%. Moreover, we show how our methods can highlight nonverbal behavioral differences of the two leadership styles. Our findings suggest that individually considerate leaders mimic head nods of their followers twice as often and that their face touches are mimicked three times as often by their followers when compared with authoritarian leaders.

Detecting Posture Mirroring in Social Interactions with Wearable Sensors

We envision wearable social-behavioral assistants which measure the nonverbal behavior of their users during social interaction. Research in psychology has linked posture mirroring, a key element of nonverbal behavior, to rapport and empathy and has been found to support communication. In this paper, we present a method to measure posture mirroring in social interaction with body-worn motion sensors. Our method is based on the detection of basic posture classes and the comparison of displayed postures across group members. We apply our method in a group discussion scenario involving 42 groups consisting of three subjects each in which group leaders express different leaderships styles. Our results show that we can measure differences in posture mirroring across groups of different leadership styles.

Discriminating Individually Considerate and Authoritarian Leaders by Speech Activity Cues

Effective leadership can increase team performance, however up to now the influence of specific micro-level behavioral patterns on team performance is unclear. At the same time, current behavior observation methods in social psychology mostly rely on manual video annotations that impede research. In our work, we follow a sensor-based approach to automatically extract speech activity cues to discriminate individualized considerate from authoritarian leadership. On a subset of 35 selected group discussions lead by leaders of different styles, we predict leadership style with75.5\% accuracy using logistic regression. We find that leadership style predictability is dependent on the relative discussion time and is highest for the middle parts of the discussions. Analysis of regression coefficients suggests that individually considerate leaders start speaking more often while others speak, use short utterances more often, change their speech loudness more and speak less than authoritarian leaders.

What good leaders actually do: micro-level leadership behaviour, leader evaluations, and team decision quality

We supplement broad definitions of leadership behaviour with the concept of micro-level leadership behaviour, leaders’ verbal and non-verbal visible conduct and interaction. For the context of team decision-making, we identify two potentially beneficial micro-level leadership behaviours, question asking and behavioural mimicry. Specifically, we propose that under conditions of informational complexity and unshared information, participative leadership is most appropriate for team decision-making, that its effects are mediated by inquiring and empathy, and that question asking and mimicry are the behavioural micro-level manifestations of inquiring and empathy. We thus hypothesize that the effect of participative leadership on team decision quality and leader evaluation is mediated by question asking and mimicry. We conduct a laboratory experiment with student teams working on a hidden profile decision-making task and measure question asking through behavioural coding and mimicry with motion sensors. Results show that the effect of participative leadership on decision quality is mediated by question asking, and that the effect of participative leadership on leader evaluation as transformational is mediated by leaders’ behavioural mimicry and question asking. Under control of these micro-level behaviours, team decision quality and leader evaluations were unrelated.

Towards monitoring firefighting teams with the smartphone

Two important aspects for efficient and safe firefighting operations are team communication behavior and physical activity coordination. In close cooperation with a firefighting brigade we investigate the potential of modern smartphones to acquire objective data on team communication and physical activity in an automatic way. We envision that such a monitoring is helpful for improving post incident feedback to enhance the efficiency and safety of firefighting operations. In this contribution we present our findings of a feasibility study in which two firefighting teams had to extinguish a kitchen fire. We present the obtained measures of speech and physical activity levels and show how the difference in performance between the two teams can be explained by the smartphone measures.