Manolo Dulva Hina

A Multi-Agent based Multimodal System Adaptive to the User’s Interaction Context

Communication is an important aspect of human life; it is with communication that helps human beings connect with each other as individuals and as independent groups. In informatics, the very purpose of the existence of computer is information dissemination – to be able to send and receive information. Humans are quite successful in conveying ideas with one another and reacting appropriately because we share the richness of our language, have a common understanding of how things work and have an implicit understanding of everyday situations. When human communicate with human, they comprehend the information that is apparent to the current situation, or context, hence increasing the conversational bandwidth. This ability to convey ideas, however, does not transfer when human interacts with computer. On its own, computers do not understand our language, do not understand how the world works and cannot sense information about the current situation. In a typical impoverished computing set-up where providing computer with information is through the use of mouse, keyboard and screen, the result is we explicitly provide information to computers, producing an effect that is contrary to the promise of transparency and calm technology in Marc Weiser’s vision of ubiquitous computing (Weiser 1991; Weiser 1993; Weiser and Brown 1996). To reverse this, it is imperative that methodologies are developed that will enable computers to have access to context. It is through context-awareness that we can increase the richness of communication in humancomputer interaction, through which we can reap the most likely benefit of more useful computational services. Context (Dey and Abowd 1999; Gwizdka 2000; Dey 2001; Coutaz, Crowley et al. 2005) is a subjective idea and its interpretation is personal. Context evolves and the acquisition of contextual information is essential. However, we believe that the one with the final word on whether the envisioned context is correctly captured/acquired or not is the end user. Current research works indicate that some contextual information are already predefined by their systems from the very beginning – this is correct if the application domain is fixed but is incorrect if we infer that a typical user does different computing tasks in different occasions. With the aim of coming up with more conclusive and inclusive design, we conjure that the contextual information that is important to the user should be left to the judgment of the end

Multimodal Fusion, Fission and Virtual Reality Simulation for an Ambient Robotic Intelligence☆

Abstract In this paper, we present an architecture that demonstrates multimodal fusion and fission using semantic agents and web services that interacts with worldwide-web computers, services and users. This solution extracts the meaning of a situation using the semantic memory of agents to manage the interaction process involved. The fusion of values from different sensors produces an event that needs implementation. The fission process suggests a detailed set of actions that are for implementation. Before such actions are implemented by actuators, these actions are first evaluated in a virtual environment which mimics the real-world environment. If no danger arises from such virtual evaluation, then implementation is feasible. Otherwise, there might be a need to add one or more smaller actions to render the action safe and free from danger. Our work presents the following contributions: (i) a design of agent memory and a model of the world environment using a knowledge representation language that is compatible with the current standards, (ii) creation of a pervasive architecture with several scenarios of composition and adaptation, (iii) presentation of how agents and services interact to provide support in a real-world, and (iv) simulation of an event in a virtual environment to assess the feasibility of the events implementation.

Autonomic Communication in Pervasive Multimodal Multimedia Computing System

Autonomic communication in a computing system analyzes the individual system element as it is affected by and affects other elements. In the human-machine communication aspect, the autonomic system performs its services autonomously, adjusting the behavior of its services to suit what the user might request implicitly or explicitly. The pervasive multimodal multimedia computing system aims at realizing anytime, anywhere computing. Its autonomic communication includes the protocols that selects, on behalf of the user, the modalities and media devices that are appropriate to the given interaction context. The modalities are the modes of interaction (i.e. for data input and output) between the user and the computer while the media devices are the physical devices that are used to support the chosen modalities. The interaction context itself is the combined user, environment, and system contexts. In this paper, we present the autonomic communication protocols involved in the detection of interaction context and the multimodal computing system’s corresponding adaptation. The heart of this paradigm’s design is the machine learning’s knowledge acquisition and the use of the layered virtual machine for definition and detection of interaction context.

Patterns architecture for fusion engines

This paper presents a design of an architecture that facilitates the work of a fusion engine. The logical combination or merging of input streams invoked by the fusion engine is based upon the definition of a set of patterns and its similarity with previously collected data from various modalities. Previous fusion engine designs had many weaknesses, among them their being specialized on a specific domain of application. The proposed architecture addresses such weakness and provides additional features, namely its ability to handle large number of modalities and due to its using knowledge base and standardization characteristics; it becomes suitable to various types of multimodal systems. The techniques used to achieve these features are discussed in this paper.

Ontological and Machine Learning Approaches for Managing Driving Context in Intelligent Transportation.

In this paper, a novel approach of managing driving context information in smart transportation is presented. The driving context refers to the ensemble of parameters that make up the contexts of the environment, the vehicle and the driver. To manage this rich information, knowledge representation using ontology is used and through it, such information becomes a source of knowledge. When this context information (i.e. basically a template or model) is instantiated with actual instances of objects, we can describe any kind of driving situation. Furthermore, through ontological knowledge management, we can find the answers related to various queries of the given driving situation. A smart vehicle is equipped with machine learning functionalities that are capable of classifying any driving situation, and accord assistance to the driver or the vehicle or both to avoid accident, when necessary. This work is a contribution to the ongoing research in safe driving, and a specific application of using data from the internet of things.

CASA: Safe and Green Driving Assistance System for Real-Time Driving Events

The goal of Project CASA (Car Safety Apps) is to build an Android app, as an alternative to the traditional ADAS (Advanced Driving Assistance System), to promote secured and environmental-friendly driving. Our team is a composed of industrial and educational entities in France. In this paper, an innovative way of capturing real-time signals, combining them to determine the current driving situation, associate the required action and generate the corresponding message towards the driver and the vehicle will be presented. Every message is suitable to the real-time driving context and event. The novelties of this work are: (i) given the actual driving context, generate message to notify of incoming traffic event and inform of the danger present when driver incurs traffic rule infraction. Such messages are intended to promote secured and environmental-friendly driving, (ii) dynamic and real-time consideration of driver status, vehicle status, and of the environment (iii) providing priority level to traffic messages based on the danger involved in the scenario.

Techniques for cognition of driving context for safe driving application

In this work, given the context of the driver, of the vehicle and of the environment, our objective is to correctly recognize the traffic situation and provide the driver with the corresponding assistance by providing notification or alert about the situation or the infraction that was committed, or acting directly on the vehicle. To do so, we need to consider the signal processing related to these context parameters. We built knowledge representation using ontology, built rules related to the fusion of context parameters and the deduction corresponding to the traffic situation using Semantic Web Rule Language. We built fission component that deals with traffic situation and the corresponding action directed towards the driver or the vehicle. Ontology is used to represent driving model and road environment. This work is our contribution in the ongoing research for the prevention of vehicular traffic accident.

Serious Gaming: Autonomy and Better Health for the Elderly

Serious Gaming for the Elderly is an undertaking that is intended to contribute to the promotion of autonomy and good health for the elderly. Its principal users are the old people, normally in their retirement age. They are given physical activity challenges as well as mental challenge and an option to participate in events with their peers. The secondary users are medical practitioners who provide exercises suitable to the health status of the participants. Results of the activities are also evaluated by medical personnel. New challenges are designed based on results of previous ones. The beta version of the system is initially tested with five specimen participants. The results of preliminary evaluation based on one-month test with participants indicated that their autonomy and health conditions improve with participation in serious gaming.

Towards Self-Optimization of a Pervasive Computing Task

Abstract In pervasive computing, our aim for the end user is continuous computing anytime, anywhere whenever possible, in a way that the computing system adapts to the users context. An integral component of pervasive computing is pervasive task, a computing task that follows wherever the user is, adapting to the computing needs of the user and self-optimizing on users behalf. In this paper, a task is represented by a set of computing files which then are mapped to their corresponding applications and suppliers, and optimal quality of service dimensions for each application. Here, we present how pervasive task can be realized, implementing selfconfiguration and self-optimization to adapt to the users preferences. This work is a contribution in the ongoing research towards adaptation of computing system to suit the users context and preferences in a pervasive computing system