Muhammad Asif Razzaq

Human Behavior Analysis by Means of Multimodal Context Mining

There is sufficient evidence proving the impact that negative lifestyle choices have on people’s health and wellness. Changing unhealthy behaviours requires raising people’s self-awareness and also providing healthcare experts with a thorough and continuous description of the user’s conduct. Several monitoring techniques have been proposed in the past to track users’ behaviour; however, these approaches are either subjective and prone to misreporting, such as questionnaires, or only focus on a specific component of context, such as activity counters. This work presents an innovative multimodal context mining framework to inspect and infer human behaviour in a more holistic fashion. The proposed approach extends beyond the state-of-the-art, since it not only explores a sole type of context, but also combines diverse levels of context in an integral manner. Namely, low-level contexts, including activities, emotions and locations, are identified from heterogeneous sensory data through machine learning techniques. Low-level contexts are combined using ontological mechanisms to derive a more abstract representation of the user’s context, here referred to as high-level context. An initial implementation of the proposed framework supporting real-time context identification is also presented. The developed system is evaluated for various realistic scenarios making use of a novel multimodal context open dataset and data on-the-go, demonstrating prominent context-aware capabilities at both low and high levels.

Ontology-Based High-Level Context Inference for Human Behavior Identification

Recent years have witnessed a huge progress in the automatic identification of individual primitives of human behavior, such as activities or locations. However, the complex nature of human behavior demands more abstract contextual information for its analysis. This work presents an ontology-based method that combines low-level primitives of behavior, namely activity, locations and emotions, unprecedented to date, to intelligently derive more meaningful high-level context information. The paper contributes with a new open ontology describing both low-level and high-level context information, as well as their relationships. Furthermore, a framework building on the developed ontology and reasoning models is presented and evaluated. The proposed method proves to be robust while identifying high-level contexts even in the event of erroneously-detected low-level contexts. Despite reasonable inference times being obtained for a relevant set of users and instances, additional work is required to scale to long-term scenarios with a large number of users.

High-Level Context Inference for Human Behavior Identification

This work presents the Mining Minds Context Ontology, an ontology for the identification of human behavior. This ontology comprehensively models high-level context based on low-level information, including the user activities, locations, and emotions. The Mining Minds Context Ontology is the means to infer high-level context from the low-level information. High-level contexts can be inferred from unclassified contexts by reasoning on the Mining Minds Context Ontology. The Mining Minds Context Ontology is shown to be flexible enough to operate in real life scenarios in which emotion recognition systems may not always be available. Furthermore, it is demonstrated that the activity and the location might not be enough to detect some of the high-level contexts, and that the emotion enables a more accurate high-level context identification. This work paves the path for the future implementation of the high-level context recognition system in the Mining Minds project.

Context-Based Lifelog Monitoring for Just-in-Time Wellness Intervention.

These days adoption of healthy behavior can be quantified through Ubiquitous computing and smart gadgets. This digital technology has revolutionized the self-quantification to monitor activities for improving lifestyle. Context based lifelog monitoring is among the processes of tracking individual’s lifestyle in an effective manner. We have proposed a methodology for context-based monitoring of an individual’s prolonged sedentary physical activity and unhealthy dietary behavior in the domain of wellness and give just-in-time intervention to adapt healthy behavior. It detects multiple unhealthy activities of its users and verifies the context for intervention generation. The results depict that the average response of the context-based just-in-time interventions is about 75%.

Transfer Learning and Data Fusion Approach to Recognize Activities of Daily Life

Activity recognition is a core domain within intelligent systems that utilizes the sensing devices available in an environment to identify human activity. Conventional solutions rely on machine-learning approaches and the assumption that the target scenario will Rit the algorithm training conditions, which raises the cost and effort of labelling data, as daily living environments are dynamic, unpredictable, and exposed to new activities. Hence, we take advantage of the ubiquitous presence of personal gadgets such as smart-watches combined with data fusion approaches to dynamically transfer learned knowledge across devices in a natural environment while performing daily living activities. In this paper, we focus on recognizing walking as an activity, which might enable carers or medical practitioners to monitor the risk of falling or suffering from a chronic disease whose progression is linked to a reduction in movement and mobility. Preliminary results show a 2% increase in activity recognition accuracy on the wearable approach, and a 10% improvement in accuracy when combining features from both wearable and environmental domains.

Intent-Context Fusioning in Healthcare Dialogue-Based Systems Using JDL Model

A revolutionized wave of intelligent assistants has emerged in daily life of human over the recent years, therefore huge progress has been witnessed for development of healthcare assistants having the capability to communicate with users. However, the conversational complexities demand building more personalized and user-oriented dialogue process systems. To support human-computer dialogue process many models have been proposed. Considering personalization aspect, this research work presents novel Context-aware Dialogue Manager (CADM) model with its foundation based on well-known JDL fusion model. The proposed model addresses modern techniques for multi-turn dialogue process, by identifying dialogue intents, contexts and fusing personalized contexts over them. The model also maintains the dialogue context for progressing complex and multi-turn dialogue. It also helps using intent-context relationship in identifying optimized knowledge source for accurate dialogue expansion and its coherence. CADM functionality is discussed using support of Intelligent Medical Assistant in healthcare domain, which has the speech-based capability to communicate with users.

An ontology-based hybrid approach for accurate context reasoning

The combination of ontology based context-awareness and machine learning context classification is an interesting research area. The determined contexts are obtained using semantic reasoning based on context ontology developed by expert using domain specific rules. This reasoning suffer challenges of soundness and completeness in real-time deployment. This paper addresses the aforementioned challenges from semantic reasoning by embracing machine learning modeling and classification benefits. Machine learning relies on data, for this we developed training and deployment phase for ontological ABox assertions. Approximately 99.99% precision through machine learning approach was achieved over 91.5% accuracy with semantic reasoning. The statistical evaluation proves the improvement in terms of accuracy for context prediction and overall performance.

mlCAF: Multi-Level Cross-Domain Semantic Context Fusioning for Behavior Identification

The emerging research on automatic identification of user’s contexts from the cross-domain environment in ubiquitous and pervasive computing systems has proved to be successful. Monitoring the diversified user’s contexts and behaviors can help in controlling lifestyle associated to chronic diseases using context-aware applications. However, availability of cross-domain heterogeneous contexts provides a challenging opportunity for their fusion to obtain abstract information for further analysis. This work demonstrates extension of our previous work from a single domain (i.e., physical activity) to multiple domains (physical activity, nutrition and clinical) for context-awareness. We propose multi-level Context-aware Framework (mlCAF), which fuses the multi-level cross-domain contexts in order to arbitrate richer behavioral contexts. This work explicitly focuses on key challenges linked to multi-level context modeling, reasoning and fusioning based on the mlCAF open-source ontology. More specifically, it addresses the interpretation of contexts from three different domains, their fusioning conforming to richer contextual information. This paper contributes in terms of ontology evolution with additional domains, context definitions, rules and inclusion of semantic queries. For the framework evaluation, multi-level cross-domain contexts collected from 20 users were used to ascertain abstract contexts, which served as basis for behavior modeling and lifestyle identification. The experimental results indicate a context recognition average accuracy of around 92.65% for the collected cross-domain contexts.