Sara Khalifa

Pervasive self-powered human activity recognition without the accelerometer

Conventional human activity recognition (HAR) relies on accelerometers to frequently sample human motion (acceleration). Unfortunately, power consumption of accelerometers becomes a bottleneck for realising pervasive self-powering HAR as the amount of power that can be practically harvested from the environment is very small. Instead of using accelerometer, this paper advocates the use of energy harvesting power signal as the source of HAR when motion (kinetic) energy is being harvested to power the device. The proposed use of harvested power for classifying human activities is motivated by the fact that different activities produce kinetic energy in a different way leaving their signatures in the harvested power signal. Using information theoretic analysis of experimental data, we show that many standard statistical features provide significant information gain when the kinetic power signal is used for discriminating between different activities, confirming its potential use for HAR. We have evaluated activity recognition accuracy for kinetic power signal based HAR using 14 different sets of common activities each containing between 2-10 different activities to be classified. HAR accuracies varied between 68% to 100% depending on the set of activities. The average accuracy over all activity sets is 83%, which is within 13% of what could be achieved with an accelerometer without any power constraints.

Adaptive pedestrian activity classification for indoor dead reckoning systems

A pedestrian activity classification (PAC) system classifies pedestrian motion data into activities related to the usage of specific building facilities, such as going up on an escalator or descending a staircase. Recent studies confirm that use of PAC significantly reduces indoor localization errors of a pedestrian dead reckoning (PDR) system as exact facility locations in the building can be retrieved from the floor map. However, classification complexity may become an issue for resource constraint mobile devices. We propose a novel PAC system that, instead of using a single complex classifier based on a large set of features, employs multiple simple classifiers each trained to classify only a subset of the activities using a small number of features. As the pedestrian moves around inside a building, the proposed adaptive-PAC dynamically switches to the right (simple) classifier based on the facilities that exist within the immediate proximity. By always using a simple classifier, adaptive-PAC has the potential to drastically reduce the average classification complexity for PAC-aided PDR systems. Using experimental data, we quantify and compare the performance of the proposed adaptive-PAC against the conventional PAC. We find that for typical shopping centers, adaptive-PAC reduces classification complexity by 91-97% without any degradation in classification accuracy rates.

Evaluating mismatch probability of activity-based map matching in indoor positioning

If users are known to perform specific activities at specific locations within a building, then indoor positioning could be achieved by monitoring user activities and matching them to specific locations in a preloaded floor map. This is the fundamental idea behind activity-based map matching (AMM). For example, the users smartphone could use the accelerometer readings to detect whether a user is using an escalator, and then match the current location of the user to the nearest escalator. AMM therefore could be used for frequently recalibrating location estimators to ground-truth values. This is especially useful for recalibrating pedestrian dead reckoning (PDR), which can estimate indoor position if started from a known location, but error grows unboundedly with time or distance traveled. However, AMM is not perfect and could potentially cause mismatches by matching the current location of the user to a wrong location. In this paper we propose a methodology and derive a closed-form expression for mismatch probability as a function of PDR sensor error and proximity between two facilities. By applying our methodology to a practical indoor complex (Sydney airport) we find several interesting results: (1) that mismatch probability is spatially non-uniform, i.e., it can be different in different parts of the floor, (2) for some specific facilities, mismatch probability can be very high (up to 80%), and (3) if escalators could be distinguished from lifts with high accuracy, we could reduce mismatch probability significantly (by up to 68%).

Transportation mode detection using kinetic energy harvesting wearables

Detecting the transportation mode of an individuals everyday travel provides useful information in urban design, real-time journey planning, and activity monitoring. In existing systems, accelerometer and GPS are the dominantly used signal sources which quickly drain the limited battery life of the wearable devices. In this paper, we investigate the feasibility of using the output voltage from the kinetic energy harvesting device as the signal source to achieve transportation mode detection. The proposed idea is based on the intuition that the vibrations experienced by the passenger during motoring of different transportation modes are different. Thus, voltage generated by the energy harvesting devices should contain distinctive features to distinguish different transportation modes. Using the dataset collected from a real energy harvesting device, we present the initial demonstration of the proposed method. We can achieve 98.84% of accuracy in determining whether the user is traveling by pedestrian or motorized modes, and in a fine-grained classification of three different motorized modes (car, bus, and train), an overall accuracy over 85% is achieved.

Estimating calorie expenditure from output voltage of piezoelectric energy harvester: an experimental feasibility study

There is a growing interest in developing energy harvesting solutions for wearable devices so they can self-power themselves without relying on batteries. Piezoelectric energy harvesters (PEHs) can convert kinetic energy released from human activities into usable electrical energy for powering various electronic circuits inside the wearable device. Intuitively, the kinetic energy is produced because the user expends some calories during the physical activities. We therefore postulate that the voltage output of a PEH in a wearable device should contain information that can be used to estimate the amount of calorie expended. If this is true, then the PEH can be used as a new source for calorie estimation. Unlike conventional sensors, such as accelerometers, a PEH does not consume any power, which would make this new source very attractive. In this paper, using real PEH hardware and the data collected from ten real subjects, we conduct an experimental study to assess the suitability of PEH voltage in estimating calorie expenditure for two different activities, walking and running. We find that, for most subjects, the calorie estimations obtained from the output voltage of PEH is very close to those obtained from a 3-axial accelerometer.

Feature selection for floor-changing activity recognition in multi-floor pedestrian navigation

In large shopping malls and airports, pedestrians often change floors using conveniently located lifts and escalators. Floor changing activity recognition (FCAR) therefore can be a vital aid to multi-floor pedestrian navigation systems. The focus of this paper is to achieve accurate FCAR with the minimal number of features. Using experimental data, we compare the performance of various feature selection methods and classifiers trained to detect whether the user is using an escalator or a lift. The results show that an accelerometer embedded in a smartphone can achieve 94% recognition accuracy using only 5 features.

Step Detection from Power Generation Pattern in Energy-Harvesting Wearable Devices

Energy-harvesting wearable devices generate power by converting natural phenomena such as human motion into usable electricity. We conduct an experimental study to validate the feasibility of detecting steps from the power generation patterns of a wearable piezoelectric energy harvester (PEH). Four healthy adults took part in the study, which includes walking along straight and turning walkways as well as descending and ascending stairs. We find that power generation exhibits distinctive peaks for each step, making it possible to accurately detect steps using widely used peak detection algorithms. Using our PEH prototype, we successfully detected 550 steps out of 570, achieving a step detection accuracy of 96%.

Feasibility and accuracy of hotword detection using vibration energy harvester

Vibration energy harvesting (VEH) is a promising source of renewable energy that can be used to extend battery life of next generation mobile devices. In this paper, we study the feasibility and accuracy of VEH for detecting hotwords, such as “OK Google”, used by popular voice control applications to distinguish user commands from other conversations. The idea of using power signals of VEH to detect hotwords is based on the fact that human voice creates vibrations in the air, which could be potentially picked up by the VEH hardware inside a mobile device. Using off-the-shelf VEH product, we conduct a comprehensive experimental study involving 8 subjects. We analyse two possible usage scenarios for the VEH hardware. In the first scenario, the user is not required to talk directly to the device (indirect), but the VEH is expected to pick up the ambient vibrations caused by user-generated sound waves. In the second, the user is expected to direct his voice to the VEH (direct) and talk to it from a close distance. For both usage scenarios, we evaluate two types of hotword detection, speaker-independent and speaker-dependent. We find that VEH can detect hotwords more accurately in the direct scenario compared to the indirect. For the direct scenario, our results show that a simple Decision Tree classifier can detect hotwords from VEH signals with accuracies of 73% and 85%, respectively, for speaker-independent and speaker-dependent detections. Finally, we show that these accuracies are comparable to what could be achieved with an accelerometer sampled at 200 Hz.

HARKE: Human Activity Recognition from Kinetic Energy Harvesting Data in Wearable Devices

Kinetic energy harvesting (KEH) may help combat battery issues in wearable devices. While the primary objective of KEH is to generate energy from human activities, the harvested energy itself contains information about human activities that most wearable devices try to detect using motion sensors. In principle, it is therefore possible to use KEH both as a power generator and a sensor for human activity recognition (HAR), saving sensor-related power consumption. Our aim is to quantify the potential of human activity recognition from kinetic energy harvesting (HARKE). We evaluate the performance of HARKE using two independent datasets: (i) a public accelerometer dataset converted into KEH data through theoretical modeling; and (ii) a real KEH dataset collected from volunteers performing activities of daily living while wearing a data-logger that we built of a piezoelectric energy harvester. Our results show that HARKE achieves an accuracy of 80 to 95 percent, depending on the dataset and the placement of the device on the human body. We conduct detailed power consumption measurements to understand and quantify the power saving opportunity of HARKE. The results demonstrate that HARKE can save 79 percent of the overall system power consumption of conventional accelerometer-based HAR.

A Bayesian framework for energy-neutral activity monitoring with self-powered wearable sensors

Achieving energy-efficiency is a challenging task in human activity monitoring. The continuous activity sensing using accelerometer and the burdensome on-node classification rapidly deplete the limited battery resource of the wearable nodes. To reduce the energy overhead and achieve the system energy-neutrality, we present a novel Bayesian framework for human activity monitoring using the energy-harvesting wearable sensors. The proposed framework utilizes a capacitor to store the harvested kinetic energy and uses all the stored energy to transmit an unmodulated signal, called an activity pulse. Our framework can infer the human activity directly from the received signal strength of the activity pulse at a remote server. Neither accelerometer nor classifier is required on the wearable devices, and therefore, our framework guarantees the system energy-neutrality. Using a real dataset collected from a kinetic energy harvester coupled with a Bluetooth prototype, an overall accuracy of 91% is achieved when the distance between the transmitter and the receiver is set to 30 cm.