José Rouillard

Contextual QR Codes

QR codes (2D barcodes) are used to encode and decode data at a rapid rate. Using camera phones to read two dimensional barcodes for various purposes is currently a popular topic in both research and in practical applications. But until now, the information provided by QR codes was solely static. What you see is exactly what was encoded. This paper proposes the notion of contextual QR codes that merge a public QR code and private information, in order to provide data related to a particular context. For example, a public tag <Hello> will be decoded and translated into ldquoGood evening John Smithrdquo or ldquoBonjour Jean Dupondrdquo, if the system is able to obtain data such as the name of the user, the language used on the machine (English or French for instance) and the moment when the interaction takes place.

Web services and speech-based applications around VoiceXML

VoiceXML applications are described by context free grammars. Then, recognized vocabulary is limited. We show in this paper one possible approach in order to use VoiceXML applications and speech web services, together. The idea is to use capabilities of speech-dictation systems when the input value is not recognized in an active grammar. A prototype of VoiceXML application using speech web services is presented. A user can speak a free sentence, in English for instance, and receive a French translation, on the same modality (phone) or via another one (PC screen, for example).

Traces using aspect oriented programming and interactive agent-based architecture for early usability evaluation: basic principles and comparison

Early evaluation of interactive systems is currently the subject of numerous researches. Some of them aim at explicitly coupling design and evaluation by various software mechanisms. We describe in this paper two approaches of early evaluation exploiting new technologies and paradigms. The first approach is based on aspect oriented programming; the second one proposes an explicit coupling between agent-oriented architecture and evaluation agents. These two approaches are globally compared in this paper.

PerZoovasive: contextual pervasive QR codes as tool to provide an adaptive learning support

This paper presents an adaptive pervasive learning environment, based on contextual QR Codes, where information is presented to learner at the appropriate time and place, and according to a particular task. This Learning environment is called PerZoovasive, where learning activities take place in a zoo and are meant to enhance classroom activities. This paper discusses adaptivity and context awareness system as strategies to provide support for learners in mobile pervasive situations. How to adapt the information while taking into consideration the context of the student or the group? Do context-awareness applications affect information proposed to users? Which role can be played by QR Codes in a pervasive environment? Can this experimentation be used in other learning situations?

Towards multimodal user interfaces composition based on UsiXML and MBD principles

In software design, the reuse issue brings the increasing of web services, components and others techniques. These techniques allow reusing code associated to technical aspect (as software component). With the development of business components which can integrate technical aspect with HCI, the composition issue has appeared. Our previous work concerned the GUI composition based on an UIDL as UsiXML. With the generalization of Multimodal User Interfaces (MUI), MUI composition principles have to be studied. This paper aims at extend existing basic composition principles in order to treat multimodal interfaces. The same principle as in the previous work, based on the tree algebra, can be used in another level (AUI) of the UsiXML framework to support the Multimodal User Interfaces composing. This paper presents a case study on the food ordering system based on multimodal (coupling GUI and MUI). A conclusion and the future works in the HCI domain are presented.

Some issues for the modelling of interactive e-services from the customer multi-channel interaction perspectives

We are involved in research on new interactive systems for e-commerce, especially in the B-to-C perspective. These works are in the crossroads of HCI and e-services area. We have studied existing researches about Web services, which appear insufficient for our needs. We explain the complexity of the interactive systems for direct marketing. Then we discuss about the design of generic architecture for rich interactions in B-to-C and we give a definition of our interpretation of e-service. Finally, we propose an analysis of the models and protocols for the e-services.

Multimodal interaction: a survey from model driven engineering and mobile perspectives

The multimodal interaction is becoming richer in last years thanks to the increasing evolution of mobile devices (smartphones/tablets) and their embedded sensors including accelerometer, gyroscope, global positioning system, near field communication and proximity sensors. Using such sensors, either sequentially or simultaneously, to interact with applications ensures an intuitive interaction and the user acceptance. Today, the development of multimodal mobile systems incorporating input and output modalities through sensors is a long and difficult task. Despite the facts that numerous model-based approaches have emerged and are supposed to simplify the multimodal mobile applications engineering, the design and implementation of these applications are still generally in an ad hoc way. In order to explain this situation, the present paper reviews, discusses, and analyses different model-based approaches proposed to develop multimodal mobile applications. The analysis considers not only the modelling and generation of mobile multimodality features, but also the inclusion of model-driven engineering features such as guidance and model reuse that allows the appropriate use of models and benefits from them. Our aim is to identify the current gaps that hinder the facility and acceleration of the multimodal mobile applications development using model-based approaches.

Facilitating the Design of Multi-channel Interfaces for Ambient Computing

Ambient computing is one of the more significant recent advances in computer-human interactions. With the ambient intelligence paradigm, computers become embedded in our natural surroundings. As they are context sensitive and adaptable, they better provide smart services to humans. But ambient computing requires communication between several heterogeneous components that are not supposed to communicate each other. This paper describes how we use a workflow to facilitate the design of multichannel interfaces for ambient computing. Our results show that different devices (such as Wiimote, multi-touch screen, telephone, etc.) can be managed in order to activate real things (such as lamp, fan, robot, webcam, etc.). A smart digital home case study illustrates a possible implementation of our approach and shows how it allows redesigning easily some parts of the ambient system just by modifying the workflow.

MIMIC: leveraging sensor-based interactions in multimodal mobile applications

In recent years, there has been an increasing interest in the presence of sensors in mobile devices. Emergence of multiple modalities based on these sensors greatly enriches the human-mobile interaction. However, mobile applications slightly involve sensors and rarely combine them simultaneously. In this paper, we seek to remedy this problem by detailing the key challenges that face developers who want to integrate several sensor-based modalities and combine them. We then present our model-based approach solution. We introduce M4L modeling language and MIMIC framework that aim to produce easily sensor-based multimodal mobile applications by generating up to 100% of their interfaces.

A Model-based Approach for Engineering Multimodal Mobile Interactions

Mobile phones have benefited from many enhancements since several years, including an increasing number of their embedded sensors (accelerometer, light sensor...). Created for giving context-aware abilities, these sensors also allow new types of interaction and pave the way to a great expansion of multimodal mobile interactions. Unfortunately, the current context of mobile software development makes difficult the development of multimodal applications. We propose our model-based solution that aims to facilitate the development of multimodal mobile interfaces. We adopt the principles of the Model-Driven Engineering (MDE) which is particularly fitted for such context. Our proposition includes M4L modeling language, library of high-level sensor-based modalities and MIMIC framework. Our approach respects the main criteria of the MDE in order to define an efficient model-based approach. Results of the first evaluation suggest that using our approach facilitates the development of sensor-based multimodal mobile interfaces.