Xavier Le Pallec

Multimodal interaction: a survey from model driven engineering and mobile perspectives

The multimodal interaction is becoming richer in last years thanks to the increasing evolution of mobile devices (smartphones/tablets) and their embedded sensors including accelerometer, gyroscope, global positioning system, near field communication and proximity sensors. Using such sensors, either sequentially or simultaneously, to interact with applications ensures an intuitive interaction and the user acceptance. Today, the development of multimodal mobile systems incorporating input and output modalities through sensors is a long and difficult task. Despite the facts that numerous model-based approaches have emerged and are supposed to simplify the multimodal mobile applications engineering, the design and implementation of these applications are still generally in an ad hoc way. In order to explain this situation, the present paper reviews, discusses, and analyses different model-based approaches proposed to develop multimodal mobile applications. The analysis considers not only the modelling and generation of mobile multimodality features, but also the inclusion of model-driven engineering features such as guidance and model reuse that allows the appropriate use of models and benefits from them. Our aim is to identify the current gaps that hinder the facility and acceleration of the multimodal mobile applications development using model-based approaches.

Support for quality metrics in metamodelling

The maturity of Model Driven Engineering facilitates the development of domain specific languages. Their creation relies on the definition of metamodels, but also on their corresponding visual notations. One can wonder about the quality of any new language, which can result in inunderstandable diagrams with inappropriate notations. Then our goal is to provide indicators about the quality of notations thanks to metrics. In this paper, we present functions that are necesary to calculate these metrics in a metamodeling environment. Then we introduce how metrics are integrated in a modeling environment named ModX.

Facilitating the Design of Multi-channel Interfaces for Ambient Computing

Ambient computing is one of the more significant recent advances in computer-human interactions. With the ambient intelligence paradigm, computers become embedded in our natural surroundings. As they are context sensitive and adaptable, they better provide smart services to humans. But ambient computing requires communication between several heterogeneous components that are not supposed to communicate each other. This paper describes how we use a workflow to facilitate the design of multichannel interfaces for ambient computing. Our results show that different devices (such as Wiimote, multi-touch screen, telephone, etc.) can be managed in order to activate real things (such as lamp, fan, robot, webcam, etc.). A smart digital home case study illustrates a possible implementation of our approach and shows how it allows redesigning easily some parts of the ambient system just by modifying the workflow.

Applying Model Driven Engineering Techniques and Tools to the Planets Game Learning Scenario

CPM (Cooperative Problem-Based learning Metamodel) is a visual language for the instructional design of Problem-Based Learning (PBL) situations. This language is a UML profile implemented on top of the Objecteering UML Case tool. In this article, we first present the way we used CPM language to bring about the pedagogical transposition of the planets game learning scenario. Then, we propose some related works conducted to improve CPM usability: on the one hand, we outline a MOF solution and an Eclipse GMF solution instead of the UML profile approach. On the other hand, we propose some explanations about transforming CPM models into LMS compliant data, and tool functionality.

La contextualisation de modèles, une étape indispensable à un développement dirigé par les modèles?

The MDA approach may be broadly reduced to the following lifecycle: defining platform independent business models and transforming them into platform dependent models for software implementation concerns. The NOCE team works in the Technology Enhanced Learning (TEL) area. One of its works aims to apply the MDA approach in TEL domain. This application presents two particularities: the versatility of the user who defines models, refines them and drives the corresponding software construction; the final automatic construction which has to contextualise the modelled dispositif within a platform already active and populated. The conducted experience has underlined several difficulties which are related to such approach. We bring out here problems which result from transformations between distinct metamodels and we propose some perspectives to reduce distance between business models and platforms.

MIMIC: leveraging sensor-based interactions in multimodal mobile applications

In recent years, there has been an increasing interest in the presence of sensors in mobile devices. Emergence of multiple modalities based on these sensors greatly enriches the human-mobile interaction. However, mobile applications slightly involve sensors and rarely combine them simultaneously. In this paper, we seek to remedy this problem by detailing the key challenges that face developers who want to integrate several sensor-based modalities and combine them. We then present our model-based approach solution. We introduce M4L modeling language and MIMIC framework that aim to produce easily sensor-based multimodal mobile applications by generating up to 100% of their interfaces.

A Model-based Approach for Engineering Multimodal Mobile Interactions

Mobile phones have benefited from many enhancements since several years, including an increasing number of their embedded sensors (accelerometer, light sensor...). Created for giving context-aware abilities, these sensors also allow new types of interaction and pave the way to a great expansion of multimodal mobile interactions. Unfortunately, the current context of mobile software development makes difficult the development of multimodal applications. We propose our model-based solution that aims to facilitate the development of multimodal mobile interfaces. We adopt the principles of the Model-Driven Engineering (MDE) which is particularly fitted for such context. Our proposition includes M4L modeling language, library of high-level sensor-based modalities and MIMIC framework. Our approach respects the main criteria of the MDE in order to define an efficient model-based approach. Results of the first evaluation suggest that using our approach facilitates the development of sensor-based multimodal mobile interfaces.

Experiments in Ubiquitous Computing for Communities of Practice Using Learning Resources

We present in this paper the interest of ubiquitous computing for the so called contextualized forums. These forums allow communities of practice exchanging and sharing experiences or situations already lived by using learning documents to classify these exchanges. These forums contain a lot of information which contribute to improve classical training. We study here ubiquitous computing possibilities to use this type of forums in order to improve professional training to business process by helping the user to put his/her acquisition into practice. The developed application is being tested and works mainly on smartphones.

Enhancing the communication value of UML models with graphical layers

UML is defined as a visual modeling language for specifying, constructing, and documenting software intensive systems. In that context, UML diagrams play a central role in the whole software engineering process, starting from early analysis, through implementation, to maintenance. Recent surveys of UML use in industry showed that software practitioners use it on a regular basis, and particularly for communication and as a mental-assist tool. However, they also pointed out the following weaknesses: the lack of context, graphical layout problems, and the languages inadequacy as a facility for communication between technical teams and their clients. In this paper, we present a general approach that addresses these problems by enhancing the effectiveness of UML models as a communication vehicle. Our approach is based on expressing stakeholder-specific viewpoints through the use of secondary notations. This involves the use of auxiliary visual variables (e.g., color, position, size) that are not formally specified in UML. To that end, we extend the traditional concept of layer found in many graphical editors to UML diagram editors. FlipLayers is an implementation of our approach. It is in the form of a plugin for the Papyrus modeling environment. One scenario with several case studies is presented in the paper to demonstrate the benefits of our approach and also to illustrate how to express viewpoints with FlipLayers.

Software Support Requirements for Awareness in Collaborative Modeling

To address issues of traditional modeling tools (installation, model versioning and lack of model repositories), Axellience has developed the first online UML modeling tool. In GenMyModel’s beta-phase, the most requested feature was collaboration. Supporting collaborative modeling involves addressing classical concerns of CSCW. These issues are usually classified through core dimensions like awareness and articulation work. We decided to focus our research on the most important dimension: awareness. Commercial modeling tools and research prototypes provide little support for awareness.To define the importance of awareness in modeling tools, we decided to study what awareness information is really required in collaborative modeling and to assess its importance according to articulation work types. To do this, we have implemented a basic collaboration system without constraint on articulation work. After a few months of use, we have identified three articulation work types present in more than 500 collaborative projects. This preliminary study allowed us to define awareness elements potentially needed for each articulation work type. As these elements are different for each articulation work type, we launched different surveys for each one of them. With these surveys, we have sorted awareness information by relevance according to articulation work types.