Annie Gentes

Multimodal Human Machine Interactions in Virtual and Augmented Reality

Virtual worlds are developing rapidly over the Internet. They are visited by avatars and staffed with Embodied Conversational Agents (ECAs). An avatar is a representation of a physical person. Each person controls one or several avatars and usually receives feedback from the virtual world on an audio-visual display. Ideally, all senses should be used to feel fully embedded in a virtual world. Sound, vision and sometimes touch are the available modalities. This paper reviews the technological developments which enable audio-visual interactions in virtual and augmented reality worlds. Emphasis is placed on speech and gesture interfaces, including talking face analysis and synthesis.

The game mechanics of pervasive applications: visiting the uncanny

“When I use a word”, Humpty Dumpty said, in a rather scornful tone, “it means just what I choose it to mean – neither more nor less.” “The question is”, said Alice, “whether you can make words mean so many different things”. “The question is”, said Humpty Dumpty, “which is to be master – thats all”. [Carroll 2007 (1865)] In pervasive games, a virtual layer with its own logic is added to the everyday, common meaning of objects, places, and people. The departure of the signifier (“things”) from the signified (what they mean on the virtual level) is the semiotic process that allows for a double reading of the environment. Such a divorce has been explored in other cultural products (books, movies). It has been qualified as the “uncanny mode”. This article analyzes how pervasive games use their own brand of uncanny. In particular, it shows how mobility becomes the way to uncover the hidden layers of the environment and serves as a tool to experience and eventually reduce or augment the uncanny. Finally, we can point to a typology of four main types of storytelling related to different types of mobility: games that use mimetic narrative strategies veering towards “realism”; games based on the “absurd”; “aloof” games that remain independent from places; games that present a specific play on this double reading of the world and develop the “Uncanny” as a genre.

Identifying the needs of children living with visual impairment: state of the art and French field-study

In this paper, we report the results of a study in the field, in a specialized institute, aiming to identify the specific needs of children living with visual impairment. The principal results show (1) that there are numerous uses of digital technologies by visually impaired children; (2) how reflective and ludic artefacts limit their frustrations and favorize their engagement; (3) the importance of the collaboration with teachers; (4) the ways stakeholders modify and adapt technologies to answer to the different needs. Those results may facilitate the decision process during the conception of interaction techniques for young people living with Visual Impairment, therefore providing techniques of higher quality and better suited to this population.

Beyond Rhetoric to Poetics in IT Invention

Two kinds of discourse typically define scientific productions: logical (epistemology of science) and rhetorical (sociology of science). We suggest that research projects can also be analyzed as poetical productions. While rhetorical strategies anticipate controversies and deploy techniques to defend projects and findings, poetical practices deepen the cultural and symbolic dimensions of technologies. Based on use cases that show different ways the poetics come to bear on research and development projects in information technology (IT), we discuss the play on words and images and how they contribute to the definition and creation of a new technology within research projects. Three cases of poetical practices are presented: naming technologies, christening projects, and designing logos. We give examples of naming and project identity formation to underscore how such a poetic stance impacts projects. Images and words help people imagine what the technology is about by giving imaginary traits and cultural substance. This paper’s analysis is a call for further work exposing the value of conscious use of poetical approaches to deepen the framing of IT projects.

RFID-based Distributed Shared Memory for Pervasive Games

The goal of our work is to give a user equipped with an RFID-enabled mobile handset (mobile phone, PDA, laptop...) the ability to access contents of distant elements of the system (tags or handsets), without physically moving to them and without using a Wireless Area Network. Our solution consists in an RFID-based Distributed Shared Memory (RDSM). After describing RDSM, we present its demonstration: a pervasive game which relies on RDSM to analyze data stored in distant elements (without any Wireless Area Network). We conclude by presenting the analysis of the players’ feed back.

Design as Meaning and Form Making: An Introduction

This chapter presents the scope and ambition of the research: to produce a model of design that accounts for the practices of designers, artists, and researchers in engineering. The goal is to reveal what connects these practices while respecting their respective contributions to the challenge of invention. The main question is what does it take to produce an original work of science, art, or design? According to the author, the answer lies in the humanities, in particular the use of semiotics and media studies that help to understand and produce the autonomous poetic space of design.

From Interactive Design to Reflective Design

This chapter offers to switch from the interactive to the reflective design paradigm. The book posits that interactive design relies on a model that focuses on activities and does not question the values and aesthetics of the artifact and therefore restricts the expansion into new norms, or new aesthetics. On the contrary, reflective design is about methods that generate new questions not only about functionalities, but also that explore new formats. Here media are considered as design tools and materials that contribute to “denaturalize” activities and forms and lay new foundations for a design project.

Design as Debate: The Thing Beyond the Object

This chapter introduces political philosophy – in particular Habermas’s theory of communication – and critical design. From the point of view of design, the question is how stakeholders organize a debate around their production and how it sustains the generativity of the design project. In this respect, designers not only produce objects, but produce “things” whose identities are in question, hence the need for expansive debates that contribute to the invention. Chapter Six examines three examples that shape the way artists, designers and researchers challenge their own perception and that of their users and audiences.

The Poetics of Invention

This chapter exposes the difference between the rhetoric of science analyzed by the sociology of sciences and technologies and the poetic of science better apprehended thanks to the humanities. Research is just as much about naming things and telling stories as elaborating theories and testing them. Engineering researchers are also poets who work on words, acronyms, expressions, to give an identity to their production and who use their narrative skills to project the invention in complex stories that weave the present and the future together. The chapter studies the generative properties of these poetic practices.

Creative Figures of Users

In this chapter, the reader will follow the trail of the “user”. Many disciplines claim to best represent her: ergonomics and engineering research focus on the system made of humans and machines, aesthetics concentrates on the sensitive experience, media studies focus on audiences and spectators. Rather than trying to catch the reality of the user, the chapter shows that the design process relies on several “figures” of the user. These figures are poetic productions and “indirect representations” of user models that support the expansion of the design project beyond a mere replication of standard uses.