Mirco Rossi

Collecting complex activity datasets in highly rich networked sensor environments

We deployed 72 sensors of 10 modalities in 15 wireless and wired networked sensor systems in the environment, in objects, and on the body to create a sensor-rich environment for the machine recognition of human activities. We acquired data from 12 subjects performing morning activities, yielding over 25 hours of sensor data. We report the number of activity occurrences observed during post-processing, and estimate that over 13000 and 14000 object and environment interactions occurred. We describe the networked sensor setup and the methodology for data acquisition, synchronization and curation. We report on the challenges and outline lessons learned and best practice for similar large scale deployments of heterogeneous networked sensor systems. We evaluate data acquisition quality for on-body and object integrated wireless sensors; there is less than 2.5% packet loss after tuning. We outline our use of the dataset to develop new sensor network self-organization principles and machine learning techniques for activity recognition in opportunistic sensor configurations. Eventually this dataset will be made public.

AmbientSense: A real-time ambient sound recognition system for smartphones

This paper presents design, implementation, and evaluation of AmbientSense, a real-time ambient sound recognition system on a smartphone. AmbientSense continuously recognizes user context by analyzing ambient sounds sampled from a smartphones microphone. The phone provides a user with realtime feedback on recognised context. AmbientSense is implemented as an Android app and works in two modes: in autonomous mode processing is performed on the smartphone only. In server mode recognition is done by transmitting audio features to a server and receiving classification results back. We evaluated both modes in a set of 23 daily life ambient sound classes and describe recognition performance, phone CPU load, and recognition delay. The application runs with a fully charged battery up to 13.75 h on a Samsung Galaxy SII smartphone and up to 12.87 h on a Google Nexus One phone. Runtime and CPU load were similar for autonomous and server modes.

RoomSense: an indoor positioning system for smartphones using active sound probing

We present RoomSense, a new method for indoor positioning using smartphones on two resolution levels: rooms and within-rooms positions. Our technique is based on active sound fingerprinting and needs no infrastructure. Rooms and within-rooms positions are characterized by impulse response measurements. Using acoustic features of the impulse response and pattern classification, an estimation of the position is performed. An evaluation study was conducted to analyse the localization performance of RoomSense. Impulse responses of 67 within-rooms positions from 20 rooms were recorded with the hardware of a smartphone. In total 5360 impulse response measurements were collected. Our evaluation study showed that RoomSense achieves a room-level accuracy of > 98% and a within-rooms positions accuracy of > 96%. Additionally, the implementation of RoomSense as an Android App is presented in detail. The RoomSense App enables to identify an indoor location within one second.

Recognizing Daily Life Context Using Web-Collected Audio Data

This work presents an approach to model daily life contexts from web-collected audio data. Being available in vast quantities from many different sources, audio data from the web provides heterogeneous training data to construct recognition systems. Crowd-sourced textual descriptions (tags) related to individual sound samples were used in a configurable recognition system to model 23 sound context categories. We analysed our approach using different outlier filtering techniques with dedicated recordings of all 23 categories and in a study with 230 hours of full-day recordings of 10 participants using smart phones. Depending on the outlier technique, our system achieved recognition accuracies between 51% and 80%.

Discovery of activity composites using topic models

In this work we investigate unsupervised activity discovery approaches using three topic modelź(TM) approaches, based on Latent Dirichlet Allocationź(LDA), n -gram TMź(NTM), and correlated TMź(CTM). While LDA structures activity primitives, NTM adds primitive sequence information, and CTM exploits co-occurring topics. We use an activity composite/primitive abstraction and analyze three public datasets with different properties that affect the discovery, including primitive rate, activity composite specificity, primitive sequence similarity, and composite-instance ratio. We compare the activity composite discovery performance among the TM approaches and against a baseline using k -means clustering. We provide guidelines for method and optimal TM parameter selection, depending on data properties and activity primitive noise. Results indicate that TMs can outperform k -means clustering up to 17%, when composite specificity is low. LDA-based TMs showed higher robustness against noise compared to other TMs and k -means.

Collaborative real-time speaker identification for wearable systems

We present an unsupervised speaker identification system for personal annotations of conversations and meetings. The system dynamically learns new speakers and recognizes already known speakers using one audio channel and speech-independent modeling. Multiple personal systems could collaborate in robust unsupervised speaker identification and online learning. The system was optimized for real-time operation on a DSP system that can be worn during daily activities. The system was evaluated on the freely available 24-speaker Augmented Multiparty Interaction dataset. For 5 s recognition time, the system achieves 81% recognition rate. Collaboration between four identification systems resulted in a performance increase of up to 17%, however even two collaborating systems yield an performance improvement. A prototypical wearable DSP implementation could continuously operate for more than 8 hours from a 4.1 Ah battery.

A Data-Driven Approach to Kinematic Analysis in Running Using Wearable Technology

Millions of people run. Movement scientists investigate the relationship of running kinematics to fatigue, injury, or running economy mainly using optical motion capture. It was found that running kinematics are highly individual and often cannot be summarized by single variables. We thus present a data-driven analysis of running technique using wearable technology, combining statistical features and machine learning techniques, which allows to identify non-linear, complex relationships. Wearable technology enables running kinematic analysis to a broad mass in unconstrained environments. 20 runners wore 12 sensor units during two experiments: an all out test and a fatiguing run. We used a Support Vector Machine (SVM) to distinguish skill level groups and achieved an accuracy of 76.92% with an acceleration sensor on the upper body. Sensor positions were ranked according to the movement change with fatigue using a feature selection. This ranking was consistent with visual annotations of a movement scientist. We propose a quantitative measure of movement change using a principal component analysis (PCA) and found an average correlation of 0.8369 for all runners with their perceived rating of fatigue.

Towards monitoring firefighting teams with the smartphone

Two important aspects for efficient and safe firefighting operations are team communication behavior and physical activity coordination. In close cooperation with a firefighting brigade we investigate the potential of modern smartphones to acquire objective data on team communication and physical activity in an automatic way. We envision that such a monitoring is helpful for improving post incident feedback to enhance the efficiency and safety of firefighting operations. In this contribution we present our findings of a feasibility study in which two firefighting teams had to extinguish a kitchen fire. We present the obtained measures of speech and physical activity levels and show how the difference in performance between the two teams can be explained by the smartphone measures.

Collaborative personal speaker identification: A generalized approach

This paper introduces a collaborative personal speaker identification system to annotate conversations and meetings using speech-independent speaker modeling and one audio channel. This system can operate in standalone and collaborative modes, and learn about speakers online that were detected as unknown. In collaborative mode, the system exchanges current speaker information with personal systems of others to improve identification performance. Our collaboration concept is based on distributed personal systems only, hence it does not require a specific infrastructure to operate. We present a generalized description of collaboration situations and derive three use scenarios in which the system was subsequently evaluated. Compared to standalone operation, collaboration among four personal identification systems increased system performance by up to 9% for 4 relevant speakers and up to 21% for 24 relevant speakers. Allowing unknown speakers in a conversation did not impede performance gains of a collaboration. In a scenario where individual systems had nonidentical speaker sets, collaboration gains were 16% for 24 relevant speakers.

MyConverse: recognising and visualising personal conversations using smartphones

MyConverse is a personal conversation recogniser and visualiser for smartphones. MyConverse uses the smartphones microphone to continuously recognise the users conversations during daily life. While it recognises pre-trained speakers, unknown speakers are detected and subsequently trained for future identification. Based on the recognition, MyConverse visualises users social interactions on the smartphone. An extensive system parameter evaluation has been done based on a freely available dataset. Additionally, MyConverse was tested in different real-life environments and in a full-day evaluation study. The speaker recognition system reached an identification accuracy of 75% for 24 speakers in meeting room conditions. In other daily life situations MyConverse reached accuracies from 60% to 84%.