H M Sajjad Hossain

Active learning enabled activity recognition

Activity recognition in smart environment has been investigated rigorously in recent years. Researchers are enhancing the underlying activity discovery and recognition process by adding various dimensions and functionalities. But one significant barrier still persists which is collecting the ground truth information. Ground truth is very important to initialize a supervised learning of activities. Due to a large variety in number of Activities of Daily Living (ADLs), acknowledging them in a supervised way is a non-trivial research problem. Most of the previous researches have referenced a subset of ADLs and to initialize their model, they acquire a vast amount of informative labeled training data. On the other hand to collect ground truth and differentiate ADLs, human intervention is indispensable. As a result it takes an immense effort and raises privacy concerns to collect a reasonable amount of labeled data. In this paper, we propose to use active learning to alleviate the labeling effort and ground truth data collection in activity recognition pipeline. We investigate and analyze different active learning strategies to scale activity recognition and propose a dynamic k-means clustering based active learning approach. Experimental results on real data traces from a retirement community-(IRB #HP-00064387) help validate the early promise of our approach.

Infrastructure-less Occupancy Detection and Semantic Localization in Smart Environments

Accurate estimation of localized occupancy related informa- tion in real time enables a broad range of intelligent smart environment applications. A large number of studies using heterogeneous sensor arrays reflect the myriad requirements of various emerging pervasive, ubiquitous and participatory sensing applications. In this paper, we introduce a zero- configuration and infrastructure-less smartphone based lo- cation specific occupancy estimation model. We opportunis- tically exploit smartphone’s acoustic sensors in a conversing environment and motion sensors in absence of any conver- sational data. We demonstrate a novel speaker estimation algorithm based on unsupervised clustering of overlapped and non-overlapped conversational data and a change point detection algorithm for locomotive motion of the users to infer the occupancy. We augment our occupancy detection model with a fingerprinting based methodology using smart- phone’s magnetometer sensor to accurately assimilate loca- tion information of any gathering. We postulate a novel crowdsourcing-based approach to annotate the semantic lo- cation of the occupancy. We evaluate our algorithms in dif- ferent contexts; conversational, silence and mixed in pres- ence of 10 domestic users. Our experimental results on real-life data traces in natural settings show that using this hybrid approach, we can achieve approximately 0.76 error count distance for occupancy detection accuracy on aver- age.

Sleep Well: A Sound Sleep Monitoring Framework for Community Scaling

Following healthy lifestyle is a key for active living. Regular exercise, controlled diet and sound sleep play an invisible role on the well being and independent living of the people. Sleep being the most durative activities of daily living (ADL) has a major synergistic influence on peoples mental, physical and cognitive health. Understanding the sleep behavior longitudinally and its underpinning clausal relationships with physiological signals and contexts (such as eye or body movement etc.) horizontally responsible for a sound or disruptive sleep pattern help provide meaningful information for promoting healthy lifestyle and designing appropriate intervention strategy. In this paper we propose to detect the microscopic states of the sleep which fundamentally constitute the components of a good or bad sleeping behavior and help shape the formative assessment of sleep quality. We initially investigate several classification techniques to identify and correlate the relationship of microscopic sleep states with the overall sleep behavior. Subsequently we propose an online algorithm based on change point detection to better process and classify the microscopic sleep states and then test a lightweight version of this algorithm for real time sleep monitoring activity recognition and assessment at scale. For a larger deployment of our proposed model across a community of individuals we propose an active learning based methodology by reducing the effort of ground truth data collection. We evaluate the performance of our proposed algorithms on real data traces, and demonstrate the efficacy of our models for detecting and assessing fine-grained sleep states beyond an individual.

SoccerMate: A personal soccer attribute profiler using wearables

The use of smartphone and wearable devices in various sporting events is an optimistic opportunity to profile players physical fitness and physiological health conditioning attributes. Recently a variety of commercial wearables with respect to different sports are available in the market. As these wearables differ for distinctive sports, it becomes a hassle to effectively profile them for multiple sports sessions in day to day practice events. Wrist worn devices like smartwatches are becoming a trend in sports analytics recently and researchers are leveraging them to infer various contexts of the players to improve the quality, tactics, strategy of playing matches against the opponents. Visual observation is the most popular way to track a players abilities in soccer, but as a player it is not always possible to self-assess your own strengths and weaknesses in a field. In this paper, we propose to exploit the wrist worn devices with built in accelerometer to help represent attributes of technical judgement, tactical awareness and physical aspects of a soccer player. We propose to use deep learning to build our classification model which analyzes different soccer events like in-possession, pass, kick, sprint, run and dribbling. Based on these soccer events, we evaluate the overall ability of a soccer player. Our experiments show that, these wearable technology guided attributes profiling can help a coach or scout to better understand the competence of a player in addition to traditional visual observation.

SenseBox: A low-cost smart home system

Smart home technologies are getting acclaimed for providing a wide variety of functionalities - security, appliance control, HVAC control and remote monitoring. Installation cost and interoperability issues have restricted the adaptability of these technologies. In this demo paper, we demonstrate the design of an interoperable prototype of our smart home system, SenseBox using low-cost embedded device as part of an NSF I-Corps project. We integrated an off-the-shelf commercially available inexpensive (

DeActive: Scaling Activity Recognition with Active Deep Learning

12) hardware (PogoPlug). We paired commercially available PIR sensors, door sensors and moisture sensors with our smart home system prototype to monitor human activities. We deployed our prototype and collected real-time activity data traces from a retirement community center. We articulated the SenseBox development, integration and testing challenges and insights along with real deployment experience.

Wearable Sensor-Based Location-Specific Occupancy Detection in Smart Environments

Deep learning architectures have been applied increasingly in multi-modal problems which has empowered a large number of application domains needing much less human supervision in the process. As unlabeled data are abundant in most of the application domains, deep architectures are getting increasingly popular to extract meaningful information out of these large volume of data. One of the major caveat of these architectures is that the training phase demands both computational time and system resources much higher than shallow learning algorithms and it is posing a difficult challenge for the researchers to implement the architectures in low-power resource constrained devices. In this paper, we propose a deep and active learning enabled activity recognition model, DeActive, which is optimized according to our problem domain and reduce the resource requirements. We incorporate active learning in the process to minimize the human supervision along with the effort needed for compiling ground truth. The DeActive model has been validated using real data traces from a retirement community center (IRB #HP-00064387) and 4 public datasets. Our experimental results show that our model can contribute better accuracy while ensuring less amount of resource usages in reduced time compared to other traditional deep learning approaches in activity recognition.

An Active Sleep Monitoring Framework Using Wearables

Occupancy detection helps enable various emerging smart environment applications ranging from opportunistic HVAC (heating, ventilation, and air-conditioning) control, effective meeting management, healthy social gathering, and public event planning and organization. Ubiquitous availability of smartphones and wearable sensors with the users for almost 24 hours helps revitalize a multitude of novel applications. The inbuilt microphone sensor in smartphones plays as an inevitable enabler to help detect the number of people conversing with each other in an event or gathering. A large number of other sensors such as accelerometer and gyroscope help count the number of people based on other signals such as locomotive motion. In this work, we propose multimodal data fusion and deep learning approach relying on the smartphone’s microphone and accelerometer sensors to estimate occupancy. We first demonstrate a novel speaker estimation algorithm for people counting and extend the proposed model using deep nets for handling large-scale fluid scenarios with unlabeled acoustic signals. We augment our occupancy detection model with a magnetometer-dependent fingerprinting-based localization scheme to assimilate the volume of location-specific gathering. We also propose crowdsourcing techniques to annotate the semantic location of the occupant. We evaluate our approach in different contexts: conversational, silence, and mixed scenarios in the presence of 10 people. Our experimental results on real-life data traces in natural settings show that our cross-modal approach can achieve approximately 0.53 error count distance for occupancy detection accuracy on average.

SensePresence: Infrastructure-Less Occupancy Detection for Opportunistic Sensing Applications

Sleep is the most important aspect of healthy and active living. The right amount of sleep at the right time helps an individual to protect his or her physical, mental, and cognitive health and maintain his or her quality of life. The most durative of the Activities of Daily Living (ADL), sleep has a major synergic influence on a person’s fuctional, behavioral, and cognitive health. A deep understanding of sleep behavior and its relationship with its physiological signals, and contexts (such as eye or body movements), is necessary to design and develop a robust intelligent sleep monitoring system. In this article, we propose an intelligent algorithm to detect the microscopic states of sleep that fundamentally constitute the components of good and bad sleeping behaviors and thus help shape the formative assessment of sleep quality. Our initial analysis includes the investigation of several classification techniques to identify and correlate the relationship of microscopic sleep states with overall sleep behavior. Subsequently, we also propose an online algorithm based on change point detection to process and classify the microscopic sleep states. We also develop a lightweight version of the proposed algorithm for real-time sleep monitoring, recognition, and assessment at scale. For a larger deployment of our proposed model across a community of individuals, we propose an active-learning-based methodology to reduce the effort of ground-truth data collection and labeling. Finally, we evaluate the performance of our proposed algorithms on real data traces and demonstrate the efficacy of our models for detecting and assessing the fine-grained sleep states beyond an individual.