Yacine Bellik

Multimodal output specification / simulation platform

The design of an output multimodal system is a complex task due to the richness of today interaction contexts. The diversity of environments, systems and user profiles requires a new generation of software tools to specify complete and valid output interactions. In this paper, we present a multimodal output specification and simulation platform. After introducing the design process which inspired this platform, we describe the two main platforms tools which respectively allow the outputs specification and the outputs simulation of a multimodal system. Finally, an application of the platform is illustrated through the outputs design on a mobile phone application.

WWHT: un modèle conceptuel pour la présentation multimodale d'information

Multimodal presentation of information aims at using several modalities of communication to produce relevant outputs to the user. This presentation involves different concepts regarding to information, interaction components (modes, modalities, media) and interaction context. An incremental process is required to create an adapted presentation. This paper presents a conceptual model for the multimodal presentation of information called WWHT and based on four concepts: What, Which, How, Then. These concepts describe the life cycle of a multimodal presentation from its birth to its disappearance including its evolutions. The application of the model is introduced through the outputs specification of a mobile phone application.

A Fault Detection and Diagnosis Framework for Ambient Intelligent Systems

Ambient intelligence (AmI) systems are smart interactive systems that perceive their surroundings using sensors and act upon them using actuators. One of the most common applications of such systems is Smart Homes. In this context, the ambient system can offer a great level of dependability if it is able to exploit available sensor data in order to autonomously perform diagnosis. However, ambient environments are dynamic in a sense that components, in general, and actuators and sensors, in particular, can be added or removed from the system at run-time. This dynamicity raises new challenges not addressed in the state of the art of fault detection and diagnosis techniques. Unlike classical control theory methods, control-loops between ambient system components cannot be pre-determined at design time. In this paper we propose a new approach based on the modeling of physical phenomena, allowing one to use available resources to predict the values that are supposed to be read by sensors. Comparing the predictions and the real readings allows us to detect potential faults. Fault detection may be followed by fault isolation, which tries to identify the faulty component precisely.

A paradigm of a pervasive multimodal multimedia computing system for the visually-impaired users

Incorporating multimodality in a computing system makes computing more accessible to a wide range of users, including those with impairments. This work presents a paradigm of a multimodal multimedia computing system to make informatics accessible to visually-impaired users. The systems infrastructure determines the suitable applications to be used. The users context and user data type are considered in determining the types of applications, media and modalities that are appropriate to use. The system design is pervasive, fault-tolerant and capable of self-adaptation under varying conditions (e.g. missing or defective components). It uses machine learning so that the system would behave in a pre-defined manner given a pre-conceived scenario. Incremental learning is adapted for added machine knowledge acquisition. A simulation of systems behaviour, using a test case scenario, is presented in this paper. This work is our original contribution to an ongoing research to make informatics more accessible to handicapped users.

An Adaptive Multimodal Multimedia Computing System for Presentation of Mathematical Expressions to Visually-Impaired Users

Presenting mathematical expressions to visually-impaired users is a challenging task because, unlike text, a mathematical expression is bi-dimensional and has some distinguishing characteristics. At present, the state-of-the-art solutions present mathematical expressions to blind users in only one presentation format and with no consideration of user’s context. In this work, we present mathematical expressions in more than one presentation format and consider the context of the user, his environment and this computing system as well as the nature of the expression itself and of the user’s preferences. The solution presented in this work is most efficient when users are familiar with many presentation formats. Unlike the current state-of-the-art solutions, ours takes into account the user’s situation and present a solution that is suitable to his context and capacity. This work is our contribution to an ongoing research to make informatics more accessible to handicapped users.

Task migration in a pervasive multimodal multimedia computing system for visually-impaired users

In a pervasive multimodal multimedia computing system, the user can continue working on a computing task anytime and anywhere using forms of modality that suit his context. Similarly, the media supporting the chosen modality are selected based on their availability and users context. In this paper, we present the infrastructure supporting the migration of a visually-impaired users task in a pervasive multimodal multimedia computing environment. Using users preferences which quantify users satisfaction, we derive the users task feasible configuration. The heart of this work is the machine learning-derived training to acquire knowledge leading to configuration optimization. Data validation is presented through scenario simulations and design specification. This work is our continuing contribution to advance research on making informatics more accessible to handicapped users.

Tactile interface to steer electric wheelchairs

In the framework of improving assistive technologies for people with disabilities, this project aims to design, develop and evaluate a tactile interface to control an electric wheelchair. It targets users who have a difficulty using a joystick, the most common control device for electric wheelchairs. We propose a new steering system with a tactile tablet or smartphone (Android). We have developed a prototype that we tested with a group of disabled users. The results were encouraging. The feedback we collected then lead to developing a second prototype which will soon be in the testing phase.

A system for user task monitoring and assistance in ambient intelligent settings

Existing task models are generally static (not used at runtime) and are used for the design or predictive evaluation of interactive systems. We propose to use the task model at runtime, in order to monitor user actions, check that they have not made any mistakes and give help when needed. We present a task model suitable for ambient environments that dynamically assigns states to tasks at the runtime. We also describe a monitoring and assistance system that uses our dynamic task model. Finally, we present a validation of our system through a simulation that shows how the interactions with the task model at runtime results in a dynamic system capable of providing assistance to users while they are carrying out their daily tasks.

KUP, un modèle pour la présentation multimodale et opportuniste d'informations en situation de mobilité

This paper deals with the design of a multimodal information platform, in which in- formation items are opportunistically provided and presented to mobile users. It relies on KUP, an interaction model which is radically different from traditional human-computer interaction models. Interactions are proximity-based, so we give formal definitions of related notions, such as perceptive and radiance spaces. Then, we give a detailed description of the algorithms used to choose an interaction modality between a user and a given interaction device.

Comparative study of collaborative methods to control rotation of shared objects

Although collaborative interaction techniques play a key role in the manipulation of objects in virtual collaborative environments, very few studies investigated their effects on the working efficiency. This paper studies different collaborative manipulation techniques which are inspired by different mechanical joints. Each mechanical joint introduces a different way to control the rotation of the shared object. An experimental study was carried out, with tasks presenting different types of constraints, in order to highlight the features of each method. The results show that manipulation based on ball and socket joins is the most natural and the fastest method to collaboratively manipulate an object. However, the manipulation method based on hinge joint constraint is more accurate in situation requiring translation and rotation simultaneously.