Hans Scholten

A survey of online activity recognition using mobile phones

Physical activity recognition using embedded sensors has enabled many context-aware applications in different areas, such as healthcare. Initially, one or more dedicated wearable sensors were used for such applications. However, recently, many researchers started using mobile phones for this purpose, since these ubiquitous devices are equipped with various sensors, ranging from accelerometers to magnetic field sensors. In most of the current studies, sensor data collected for activity recognition are analyzed offline using machine learning tools. However, there is now a trend towards implementing activity recognition systems on these devices in an online manner, since modern mobile phones have become more powerful in terms of available resources, such as CPU, memory and battery. The research on offline activity recognition has been reviewed in several earlier studies in detail. However, work done on online activity recognition is still in its infancy and is yet to be reviewed. In this paper, we review the studies done so far that implement activity recognition systems on mobile phones and use only their on-board sensors. We discuss various aspects of these studies. Moreover, we discuss their limitations and present various recommendations for future research.

Fusion of Smartphone Motion Sensors for Physical Activity Recognition

For physical activity recognition, smartphone sensors, such as an accelerometer and a gyroscope, are being utilized in many research studies. So far, particularly, the accelerometer has been extensively studied. In a few recent studies, a combination of a gyroscope, a magnetometer (in a supporting role) and an accelerometer (in a lead role) has been used with the aim to improve the recognition performance. How and when are various motion sensors, which are available on a smartphone, best used for better recognition performance, either individually or in combination? This is yet to be explored. In order to investigate this question, in this paper, we explore how these various motion sensors behave in different situations in the activity recognition process. For this purpose, we designed a data collection experiment where ten participants performed seven different activities carrying smart phones at different positions. Based on the analysis of this data set, we show that these sensors, except the magnetometer, are each capable of taking the lead roles individually, depending on the type of activity being recognized, the body position, the used data features and the classification method employed (personalized or generalized). We also show that their combination only improves the overall recognition performance when their individual performances are not very high, so that there is room for performance improvement. We have made our data set and our data collection application publicly available, thereby making our experiments reproducible.

A directional data dissemination protocol for vehicular environments

This paper presents a simple and robust dissemination protocol that efficiently deals with data dissemination in both dense and sparse vehicular networks. Our goal is to address highway scenarios where vehicles equipped with sensors detect an event, e.g., a hazard and broadcast an event message to a specific direction of interest. In order to deal with broadcast communication under diverse network densities, we design a dissemination protocol in such a way that: (i) it prevents the so-called broadcast storm problem in dense networks by employing an optimized broadcast suppression technique; and (ii) it efficiently deals with disconnected networks by relying on the store-carry-forward communication model. The novelty of the protocol lies in its simplicity and robustness. Simplicity is achieved by only considering two states (i.e., cluster tail and non-tail) for vehicles. Furthermore, vehicles in both directions help disseminating messages in a seamlessly manner, without resorting to different operation modes for each direction. Robustness is achieved by assigning message delivery responsibility to multiple vehicles in sparse networks. Our simulation results show that our protocol achieves higher delivery ratio and higher robustness when compared with DV-CAST under diverse road scenarios.

Towards Physical Activity Recognition Using Smartphone Sensors

In recent years, the use of a smartphone accelerometer in physical activity recognition has been well studied. However, the role of a gyroscope and a magnetometer is yet to be explored, both when used alone as well as in combination with an accelerometer. For this purpose, we investigate the role of these three smartphone sensors in activity recognition. We evaluate their roles on four body positions using seven classifiers while recognizing six physical activities. We show that in general an accelerometer and a gyroscope complement each other, thereby making the recognition process more reliable. Moreover, in most cases, a gyroscope does not only improve the recognition accuracy in combination with an accelerometer, but it also achieves a reasonable performance when used alone. The results for a magnetometer are not encouraging because it causes over-fitting in training classifiers due to its dependence on directions. Based on our evaluations, we show that it is difficult to make an exact general statement about which sensor performs better than the others in all situations because their recognition performance depends on the smartphones position, the selected classifier, and the activity being recognized. However, statements about their roles in specific situations can be made. We report our observations and results in detail in this paper, while our data-set and data-collection app is publicly available, thereby making our experiments reproducible.

Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors.

The position of on-body motion sensors plays an important role in human activity recognition. Most often, mobile phone sensors at the trouser pocket or an equivalent position are used for this purpose. However, this position is not suitable for recognizing activities that involve hand gestures, such as smoking, eating, drinking coffee and giving a talk. To recognize such activities, wrist-worn motion sensors are used. However, these two positions are mainly used in isolation. To use richer context information, we evaluate three motion sensors (accelerometer, gyroscope and linear acceleration sensor) at both wrist and pocket positions. Using three classifiers, we show that the combination of these two positions outperforms the wrist position alone, mainly at smaller segmentation windows. Another problem is that less-repetitive activities, such as smoking, eating, giving a talk and drinking coffee, cannot be recognized easily at smaller segmentation windows unlike repetitive activities, like walking, jogging and biking. For this purpose, we evaluate the effect of seven window sizes (2–30 s) on thirteen activities and show how increasing window size affects these various activities in different ways. We also propose various optimizations to further improve the recognition of these activities. For reproducibility, we make our dataset publicly available.

Movement-based group awareness with wireless sensor networks

We propose a method through which dynamic sensor nodes determine that they move together, by communicating and correlating their movement information. We describe two possible solutions, one using inexpensive tilt switches, and another one using low-cost MEMS accelerometers. We implement a fast, incremental correlation algorithm, with an execution time of 6ms, which can run on resource constrained devices. The tests with the implementation on real sensor nodes show that the method is reliable and distinguishes between joint and separate movements. In addition, we analyze the scalability from four different perspectives: communication, energy, memory and execution speed. The solution using tilt switches proves to be simpler, cheaper and more energy efficient, while the accelerometer-based solution is more reliable, more robust to sensor alignment problems and, potentially, more accurate by using extended features, such as speed and distance.

A scalable data dissemination protocol for both highway and urban vehicular environments

Vehicular ad hoc networks (VANETs) enable the timely broadcast dissemination of event-driven messages to interested vehicles. Especially when dealing with broadcast communication, data dissemination protocols must achieve a high degree of scalability due to frequent deviations in the network density. In dense networks, suppression techniques are designed to prevent the so-called broadcast storm problem. In sparse networks, protocols incorporate store-carry-forward mechanisms to take advantage of the mobility of vehicles to store and relay messages until a new opportunity for dissemination emerges. Despite numerous efforts, most related works focus on either highway or urban scenarios, but not both. Highways are mostly addressed with a single directional dissemination. For urban scenarios, protocols mostly concentrate on either using infrastructure or developing methods for selecting vehicles to perform the store-carry-forward task. In both cases, dense networks are dealt with suppression techniques that are not optimal for multi-directional dissemination. To fill this gap, we present an infrastructure-less protocol that combines a generalized time slot scheme based on directional sectors and a store-carry-forward algorithm to support multi-directional data dissemination. By means of simulations, we show that our protocol scales properly in various network densities in both realistic highway and urban scenarios. Most importantly, it outperforms state-of-the-art protocols in terms of delivery ratio, end-to-end delay, and number of transmissions. Compared to these solutions, our protocol presents up to seven times lower number of transmissions in dense highway scenarios.

Towards detection of bad habits by fusing smartphone and smartwatch sensors

Recently, there has been a growing interest in the research community about using wrist-worn devices, such as smartwatches for human activity recognition, since these devices are equipped with various sensors such as an accelerometer and a gyroscope. Similarly, smartphones are already being used for activity recognition. In this paper, we study the fusion of a wrist-worn device (smartwatch) and a smartphone for human activity recognition. We evaluate these two devices for their strengths and weaknesses in recognizing various daily physical activities. We use three classifiers to recognize 13 different activities, such as smoking, eating, typing, writing, drinking coffee, giving a talk, walking, jogging, biking, walking upstairs, walking downstairs, sitting, and standing. Some complex activities, such as smoking, eating, drinking coffee, giving a talk, writing, and typing cannot be recognized with a smartphone in the pocket position alone. We show that the combination of a smartwatch and a smartphone recognizes such activities with a reasonable accuracy. The recognition of such complex activities can enable well-being applications for detecting bad habits, such as smoking, missing a meal, and drinking too much coffee. We also show how to fuse information from these devices in an energy-efficient way by using low sampling rates. We make our dataset publicly available in order to make our work reproducible.

Service discovery at home

Service discovery is a fairly new field that kicked off since the advent of ubiquitous computing and has been found essential in the making of intelligent networks by implementing automated discovery and remote control between devices. This paper provides an overview and comparison of several prominent service discovery mechanisms currently available. It also introduces the at home anywhere service discovery protocol (SDP@HA) design which improves on the current state of the art by accommodating resource lean devices, implementing a dynamic leader election for a central cataloguing device and embedding robustness to the service discovery architecture as an important criterion.

Energy-Efficient Cluster-Based Service Discovery in Wireless Sensor Networks

We propose an energy-efficient service discovery protocol for wireless sensor networks. Our solution exploits a cluster overlay, where the clusterhead nodes form a distributed service registry. A service lookup results in visiting only the clusterhead nodes. We aim for minimizing the communication costs during discovery of services and maintenance of a functional distributed service registry. We compare theoretically and by simulation the impact of the chosen clustering algorithm on the service discovery protocol