Abdullah Al Hafiz Khan

Active learning enabled activity recognition

Activity recognition in smart environment has been investigated rigorously in recent years. Researchers are enhancing the underlying activity discovery and recognition process by adding various dimensions and functionalities. But one significant barrier still persists which is collecting the ground truth information. Ground truth is very important to initialize a supervised learning of activities. Due to a large variety in number of Activities of Daily Living (ADLs), acknowledging them in a supervised way is a non-trivial research problem. Most of the previous researches have referenced a subset of ADLs and to initialize their model, they acquire a vast amount of informative labeled training data. On the other hand to collect ground truth and differentiate ADLs, human intervention is indispensable. As a result it takes an immense effort and raises privacy concerns to collect a reasonable amount of labeled data. In this paper, we propose to use active learning to alleviate the labeling effort and ground truth data collection in activity recognition pipeline. We investigate and analyze different active learning strategies to scale activity recognition and propose a dynamic k-means clustering based active learning approach. Experimental results on real data traces from a retirement community-(IRB #HP-00064387) help validate the early promise of our approach.

Infrastructure-less Occupancy Detection and Semantic Localization in Smart Environments

Accurate estimation of localized occupancy related informa- tion in real time enables a broad range of intelligent smart environment applications. A large number of studies using heterogeneous sensor arrays reflect the myriad requirements of various emerging pervasive, ubiquitous and participatory sensing applications. In this paper, we introduce a zero- configuration and infrastructure-less smartphone based lo- cation specific occupancy estimation model. We opportunis- tically exploit smartphone’s acoustic sensors in a conversing environment and motion sensors in absence of any conver- sational data. We demonstrate a novel speaker estimation algorithm based on unsupervised clustering of overlapped and non-overlapped conversational data and a change point detection algorithm for locomotive motion of the users to infer the occupancy. We augment our occupancy detection model with a fingerprinting based methodology using smart- phone’s magnetometer sensor to accurately assimilate loca- tion information of any gathering. We postulate a novel crowdsourcing-based approach to annotate the semantic lo- cation of the occupancy. We evaluate our algorithms in dif- ferent contexts; conversational, silence and mixed in pres- ence of 10 domestic users. Our experimental results on real-life data traces in natural settings show that using this hybrid approach, we can achieve approximately 0.76 error count distance for occupancy detection accuracy on aver- age.

Sleep Well: A Sound Sleep Monitoring Framework for Community Scaling

Following healthy lifestyle is a key for active living. Regular exercise, controlled diet and sound sleep play an invisible role on the well being and independent living of the people. Sleep being the most durative activities of daily living (ADL) has a major synergistic influence on peoples mental, physical and cognitive health. Understanding the sleep behavior longitudinally and its underpinning clausal relationships with physiological signals and contexts (such as eye or body movement etc.) horizontally responsible for a sound or disruptive sleep pattern help provide meaningful information for promoting healthy lifestyle and designing appropriate intervention strategy. In this paper we propose to detect the microscopic states of the sleep which fundamentally constitute the components of a good or bad sleeping behavior and help shape the formative assessment of sleep quality. We initially investigate several classification techniques to identify and correlate the relationship of microscopic sleep states with the overall sleep behavior. Subsequently we propose an online algorithm based on change point detection to better process and classify the microscopic sleep states and then test a lightweight version of this algorithm for real time sleep monitoring activity recognition and assessment at scale. For a larger deployment of our proposed model across a community of individuals we propose an active learning based methodology by reducing the effort of ground truth data collection. We evaluate the performance of our proposed algorithms on real data traces, and demonstrate the efficacy of our models for detecting and assessing fine-grained sleep states beyond an individual.

Demo abstract: A microphone sensor based system for green building applications

Acoustic sensing has influenced many applications in green building energy management, such as designing multi-modal energy disaggregation algorithms through fine-grained appliance state identifications or efficiently controlling the HVAC system based on the occupancy of the environment. In this demo paper we build a low-cost system prototype using off-the-shelf commercially available hardware (Raspberry Pi and super high gain microphone) to handle both acoustic sensing and its processing that is portable and easily deployable in any indoor environment. Our system is useful in detecting appliance noise for fine-grained energy metering and human voice for managing building energy footprint. We use the decibel strength of the sound to determine if it should be filtered out as a silence or stored in as an audio of interest. A fast fourier transform that quickly converts the sinusoidal input of the audio signals into its associated frequencies is implemented along with the Mel-Frequency Cepstral Coefficients (MFCCs) feature to distinguish between a human voice and an appliance noise. We also implement all the computations on-chip to quantify the energy-delay benefits.

SoccerMate: A personal soccer attribute profiler using wearables

The use of smartphone and wearable devices in various sporting events is an optimistic opportunity to profile players physical fitness and physiological health conditioning attributes. Recently a variety of commercial wearables with respect to different sports are available in the market. As these wearables differ for distinctive sports, it becomes a hassle to effectively profile them for multiple sports sessions in day to day practice events. Wrist worn devices like smartwatches are becoming a trend in sports analytics recently and researchers are leveraging them to infer various contexts of the players to improve the quality, tactics, strategy of playing matches against the opponents. Visual observation is the most popular way to track a players abilities in soccer, but as a player it is not always possible to self-assess your own strengths and weaknesses in a field. In this paper, we propose to exploit the wrist worn devices with built in accelerometer to help represent attributes of technical judgement, tactical awareness and physical aspects of a soccer player. We propose to use deep learning to build our classification model which analyzes different soccer events like in-possession, pass, kick, sprint, run and dribbling. Based on these soccer events, we evaluate the overall ability of a soccer player. Our experiments show that, these wearable technology guided attributes profiling can help a coach or scout to better understand the competence of a player in addition to traditional visual observation.

SenseBox: A low-cost smart home system

Smart home technologies are getting acclaimed for providing a wide variety of functionalities - security, appliance control, HVAC control and remote monitoring. Installation cost and interoperability issues have restricted the adaptability of these technologies. In this demo paper, we demonstrate the design of an interoperable prototype of our smart home system, SenseBox using low-cost embedded device as part of an NSF I-Corps project. We integrated an off-the-shelf commercially available inexpensive (

TransAct: Transfer learning enabled activity recognition

12) hardware (PogoPlug). We paired commercially available PIR sensors, door sensors and moisture sensors with our smart home system prototype to monitor human activities. We deployed our prototype and collected real-time activity data traces from a retirement community center. We articulated the SenseBox development, integration and testing challenges and insights along with real deployment experience.

RAM: Radar-based activity monitor

Activity recognition using smartphone has great potential in many applications like healthcare, obesity management, abnormal behavior detection, public safety and security etc. Typical activity detection systems are built on to recognize a limited set of activities that are present in the training and testing environments. However, these systems require similar data distributions, activity sets and sufficient labeled training data in both training and testing phases. Therefore, inferring new activities is challenging in practical scenarios where training and testing environments are volatile, data distributions are diverge and testing environment has new set of activities with limited training samples. The shortage of labeled training data samples also degrades the activity recognition performance. In this work, we address these challenges by augmenting the Instance based Transfer Boost algorithm with k-means clustering. We evaluated our TransAct model with three public datasets - HAR, MHealth and DailyAndSports and demonstrated that our TransAct model outperforms traditional activity recognition approaches. Our experimental results show that our TransAct model achieves ≈ 81% activity detection accuracy on average.

Wearable Sensor-Based Location-Specific Occupancy Detection in Smart Environments

Activity recognition has applications in a variety of human-in-the-loop settings such as smart home health monitoring, green building energy and occupancy management, intelligent transportation, and participatory sensing. While fine-grained activity recognition systems and approaches help enable a multitude of novel applications, discovering them with non-intrusive ambient sensor systems pose challenging design, as well as data processing, mining, and activity recognition issues. In this paper, we develop a low-cost heterogeneous Radar based Activity Monitoring (RAM) system for recognizing fine-grained activities. We exploit the feasibility of using an array of heterogeneous micro-doppler radars to recognize low-level activities. We prototype a short-range and a long-range radar system and evaluate the feasibility of using the system for fine-grained activity recognition. In our evaluation, using real data traces, we show that our system can detect fine-grained user activities with 92.84% accuracy.

An Active Sleep Monitoring Framework Using Wearables

Occupancy detection helps enable various emerging smart environment applications ranging from opportunistic HVAC (heating, ventilation, and air-conditioning) control, effective meeting management, healthy social gathering, and public event planning and organization. Ubiquitous availability of smartphones and wearable sensors with the users for almost 24 hours helps revitalize a multitude of novel applications. The inbuilt microphone sensor in smartphones plays as an inevitable enabler to help detect the number of people conversing with each other in an event or gathering. A large number of other sensors such as accelerometer and gyroscope help count the number of people based on other signals such as locomotive motion. In this work, we propose multimodal data fusion and deep learning approach relying on the smartphone’s microphone and accelerometer sensors to estimate occupancy. We first demonstrate a novel speaker estimation algorithm for people counting and extend the proposed model using deep nets for handling large-scale fluid scenarios with unlabeled acoustic signals. We augment our occupancy detection model with a magnetometer-dependent fingerprinting-based localization scheme to assimilate the volume of location-specific gathering. We also propose crowdsourcing techniques to annotate the semantic location of the occupant. We evaluate our approach in different contexts: conversational, silence, and mixed scenarios in the presence of 10 people. Our experimental results on real-life data traces in natural settings show that our cross-modal approach can achieve approximately 0.53 error count distance for occupancy detection accuracy on average.