Marian Haescher

Sensor requirements for activity recognition on smart watches

The new generation of watches is smart. Smart watches are connected to the internet and provide sensor functionality that allows an enhanced human-computer-interaction. Smart watches provide a gesture interaction and a permanent monitoring of physical activities. In comparison to other electronic home consumer devices with integrated sensors, Smart watches provide monitoring data for 24h per day, many watches are water resistant and can be worn constantly. The integrated sensors are varying in performance and are not intended to distinguish between different states of activity and inactivity. This paper reports on identified requirements on sensors of smart watches for detection of activity, inactivity as well as sleep detection. Hereby a new measurement quantity is introduced and applications of heart beat detection or wearing situation are presented.

A study on measuring heart- and respiration-rate via wrist-worn accelerometer-based seismocardiography (SCG) in comparison to commonly applied technologies

Since the human body is a living organism, it emits various life signs which can be traced with an action potential sensitive electromyography, but also with motion sensitive sensors such as typical inertial sensors. In this paper, we present a possibility to recognize the heart rate (HR), respiration rate (RR), and the muscular microvibrations (MV) by an accelerometer worn on the wrist. We compare our seismocardiography (SCG) / ballistocardiography (BCG) approach to commonly used measuring methods. In conclusion, our study confirmed that SCG/BCD with a wrist-worn accelerometer also provides accurate vital parameters. While the recognized RR deviated slightly from the ground truth (SD=16.61%), the detection of HR is non-significantly different (SD=1.63%) to the gold standard.

SeismoTracker: Upgrade any Smart Wearable to enable a Sensing of Heart Rate, Respiration Rate, and Microvibrations

In this paper we present a method to enable any smart Wearable to sense vital data in resting states. These resting states (e.g. sleeping, sitting calmly, etc.) imply the presence of low-amplitude body-motions. Our approach relies on seismocardiography (SCG), which only requires a built-in accelerometer. Compared to commonly applied technologies, such as photoplethysmography (PPG), our approach is not only tracking heart rate (HR), but also respiration rate (RR), and microvibrations (MV) of the muscles, while being also computational inexpensive. In addition, we can calculate several other parameters, such as HR variability and RR variability. Our extracted vital parameters match with the vital data gathered from clinical state-of-the art technology. These data allow us to gain an impression on the users activity, quality of sleep, arousal and stress level over the whole day, week, month, or year. Moreover, we can detect whether a device is actually worn or doffed, which is crucial when connecting such data with health services. We implemented our method on two current smartwatches: a Simvalley AW420 RX as well as on a LG G Watch R and recorded user data for several months. A web platform enables to keep track of ones data.

CapWalk: a capacitive recognition of walking-based activities as a wearable assistive technology

In this research project, we present an alternative approach to recognize various walking-based activities based on the technology of capacitive sensing. While accelerometry-based walking detections suffer from reduced accuracy at low speeds, the technology of capacitive sensing uses physical distance parameters, which makes it invariant to the duration of step performance. Determining accurate levels of walking activity is a crucial factor for people who perform walking with tiny step lengths such as elderlies or patients with pathologic conditions. In contrast to other gait analysis solutions, CapWalk is mobile and less affected by external influences such as bad lighting conditions, while it is also invariant to external acceleration artifacts. Our approach enables a reliable recognition of very slow walking speeds, in which accelerometer-based implementations can fail or provide high deviations. In CapWalk we present three different capacitive sensing prototypes (Leg Band, Chest Band, Insole) in the setup of loading mode to demonstrate recognition of sneaking, normal walking, fast walking, jogging, and walking while carrying weight. Our designs are wearable and could easily be integrated into wearable objects, such as shoes, pants or jackets. We envision such gathered information to be used to assist certain user groups such as diabetics, whose optimal insulin dose is depending on bread units and physical activity or elderlies whose personalized dosage of medication can be better determined based on their physical activity.

aHead: Considering the Head Position in a Multi-sensory Setup of Wearables to Recognize Everyday Activities with Intelligent Sensor Fusions

In this paper we examine the feasibility of Human Activity Recognition (HAR) based on head mounted sensors, both as stand-alone sensors and as part of a wearable multi-sensory network. To prove the feasibility of such setting, an interactive online HAR-system has been implemented to enable for multi-sensory activity recognition while making use of a hierarchical sensor fusion. Our system incorporates 3 sensor positions distributed over the body, which are head (smart glasses), wrist (smartwatch), and hip (smartphone). We are able to reliably distinguish 7 daily activities, which are: resting, being active, walking, running, jumping, cycling and office work. The results of our field study with 14 participants clearly indicate that the head position is applicable for HAR. Moreover, we demonstrate an intelligent multi-sensory fusion concept that increases the recognition performance up to 86.13 % (recall). Furthermore, we found the head to possess very distinctive movement patterns regarding activities of daily living.

Anomaly Detection with Smartwatches as an Opportunity for Implicit Interaction

In this paper we introduce application scenarios for implicit interaction with Smartwatches for the purpose of user assistance, to create awareness, and to enhance as well as simplify the interaction with Wearables. We envision three scenarios (1) the detection of sleep apnea, (2) the detection of epileptic seizures, and (3) a detection of accidents such as falling, car crashes etc., which are presented and discussed. Therefore, the recognition of all incidents described will be discussed under the meta-topic of anomaly detection.

SmartMove: a smartwatch algorithm to distinguish between high- and low-amplitude motions as well as doffed-states by utilizing noise and sleep

In this paper, we describe a self adapting algorithm for smart watches to define individual transitions between motion intensities. The algorithm enables for a distinction between high-amplitude motions (e.g. walking, running, or simply moving extremities) low-amplitude motions (e.g. human microvibrations, and heart rate) as well as a general doffed-state. A prototypical implementation for detecting all three motion types was tested with a wrist-worn acceleration sensor. Since the aforementioned motion types are user-specific, SmartMove incorporates a training module based on a novel actigraphy-based sleep detection algorithm, in order to learn the specific motion types. In addition, our proposed sleep algorithm enables for reduced power consumption since it samples at a very low rate. Furthermore, the algorithm can identify suitable timeframes for an inertial sensor-based detection of vital-signs (e.g. seismocardiography or ballistocardiography).

Recognition of Low Amplitude Body Vibrations via Inertial Sensors for Wearable Computing

Pathological shaking of the body or extremities is widely known and might occur at chronic diseases e.g. Parkinson. The rhythmical shaking, also known as tremor, can be such intense that extremities are flapping. Under certain circumstances, healthy people also show a shivering and shaking of their body. For example, humans start to shiver whenever it is too cold or if feelings such as stress or fear become dominant. Some wearable devices that are in direct contact to the body, such as smartwatches or smartglasses, provide a sensing functionality of acceleration force that is sufficient to detect the tremor of the wearer. The tremor varies in frequency and intensity and can be identified, by applying detection algorithms and signal filtering. Former works figured that all endotherms show muscle vibrations. These vibrations occur in the condition of sleeping as well as when being awake, or in unconsciousness. Furthermore, the vibrations are also present when subjects are physically active, emotionally stressed, or absolutely relaxed. The vibration itself varies in structure, amplitude, and frequency. This paper shows that these muscle vibrations are measureable by acceleration sensors attached to the user, and provides an outlook to new applications in the future. It also proves that custom mobile devices are able to detect body and muscle vibration and should motivate designers to develop new applications and treatment opportunities.

Properties of a Peripheral Head-Mounted Display (PHMD)

In this paper we propose a definition for Peripheral Head-Mounted Display (PHMD) for Near Field Displays. This paper introduces a taxonomy for head-mounted displays that is based on the property of its functionality and the ability of our human eye to perceive peripheral information, instead of being technology-dependent. The aim of this paper is to help designers to understand the perception of the human eye, as well as to discuss the factors one needs to take into consideration when designing visual interfaces for PHMDs. We envision this term to help classifying devices such as Google Glass, which are often misclassified as a Head-Up Display (HUD) following NASA’s definition.

SeismoPen: Pulse Recognition via a Smart Pen

We propose SeismoPen, an enhanced ballpoint pen, which is capable of calculating the patients heart rate. This is enabled when being pressed the pen towards the patients throat so it can sense and analyze the seismographic micro-eruption caused by the pulsing blood. We developed a suitable algorithm and tested three sensor setups in which we attached (1) a force-sensing resistor (FSR), (2) an accelerometer, and (3) a piezoelectric transducer to the pens head. We also conducted a user study, which resulted in suggesting SeismoPen to be potentially more accepted by users, since it is less obtrusive than alternative measurement methods. In contrast to medical devices, this simple pen looks less perilous and potentially reduces the risk of triggering symptoms of a white coat hypertension.