Zsófia Ruttkay

Cross-cultural study on facial regions as cues to recognize emotions of virtual agents

This paper reports the preliminary results of a cross-cultural study on facial regions as cues to recognize the facial expressions of virtual agents. The experiment was conducted between Japan and Hungary using 18 facial expressions of cartoonish faces designed by Japanese. The results suggest the following: 1) cultural differences exist when using facial regions as cues to recognize cartoonish facial expressions between Hungary and Japan. Japanese weighed facial cues more heavily in the eye regions than Hungarians, who weighed facial cues more heavily in the mouth region than Japanese. 2) The mouth region is more effective for conveying the emotions of facial expressions than the eye region, regardless of country. Our findings can be used not only to derive design guidelines for virtual agent facial expressions when aiming at users of a single culture, but as adaptation strategies in applications with multicultural users.

Turk-2, a multi-modal chess player

In this paper we present Turk-2, a hybrid multi-modal chess player with a robot arm and a screen-based talking head. Turk-2 can not only play chess, but can see and hear the opponent, can talk to him and display emotions. We were interested to find out if a simple embodiment with human-like communication capabilities enhances the experience of playing chess against a computer. First, give an overview of the development road to multi-modal communication with computers. Then we motivate our research with a hybrid system, we introduce the architecture of Turk-2, we describe the human experiments and its evaluation. The results justify that multi-modal interaction makes game playing more engaging, enjoyable - and even more effective. These findings for a specific game situation provide yet another evidence of the power of human-like interaction in turning computer systems more attractive and easier to use.

Eloquence of eyes and mouth of virtual agents: cultural study of facial expression perception

This paper reports the results of a cross-cultural study on facial regions as cues to recognize facial expressions of virtual agents. The experiment was conducted between Japan and Hungary using 30 facial expressions of cartoonish faces designed by Hungarians. The results suggest the following: (1) cultural differences exist when using facial regions as cues to recognize cartoonish facial expressions between Hungary and Japan. Japanese weighed facial cues more heavily in the eye region than Hungarians, who weighed facial cues more heavily in the mouth region than Japanese. (2) The mouth region is more effective for conveying the emotions of facial expressions than the eye region, regardless of country. Our findings can be used not only to derive design guidelines for virtual agent facial expressions when aiming at users of a single culture, but as adaptation strategies in applications with users from various cultures.

Evaluation of Interactive Children Book Design

In spite of mushrooming of interactive books apps for kids, there is hardly any evidence on “what makes a good interactive book”. In this article we provide an in-depth analysis of design issues, and give account of the explorato-ry evaluation of experience with “Little Rooster”, an interactive book designed and implemented at our Lab, first of all for research purposes. We let 7-8 year old kids to “read” the interactive book, and/or a printed, traditional version with the same content. On the basis of analysis of the video recordings of the read-ing sessions and the interviews, we list our observations on what children found of the visual and sound design, how they used (or not) the interaction facilities for control and what strategy they followed in mixing reading and interacting. We also investigated the effect of the moving, interactive images on under-standing and remembering the narrative of the story, and on understanding con-cepts nowadays usually unfamiliar to children living in a town. We finish the article with discussing experimental methodological issues and summing up de-sign considerations.

Robots as Companions: What can we Learn from Servants and Companions in Literature, Theater, and Film?

Many researchers are working on developing robots into adequate partners, be it at the working place, be it at home or in leisure activities, or enabling elder persons to lead a self-determined, independent life. While quite some progress has been made in e.g. speech or emotion understanding, processing and expressing, the relations between humans and robots are usually only short-term. In order to build long-term, i.e. social relations, qualities like empathy, trust building, dependability, non-patronizing, and others will be required. But these are just terms and as such no adequate starting points to “program” these capacities even more how to avoid the problems and pitfalls in interactions between humans and robots. However, a rich source for doing this is available, unused until now for this purpose: artistic productions, namely literature, theater plays, not to forget operas, and films with their multitude of examples. Poets, writers, dramatists, screen-writers, etc. have studied for centuries the facets of interactions between persons, their dynamics, and the related snags. And since we wish for human-robot relations as master-servant relations - the human obviously being the master - the study of these relations will be prominent. A procedure is proposed, with four consecutive steps, namely Selection, Analysis, Categorization, and Integration. Only if we succeed in developing robots which are seen as servants we will be successful in supporting and helping humans through robots.

Processing for visual artists, by Andrew Glassner

appreciated are reproductions of Robert Bosch’s ornamental application of the Jordan curve or Dániel Erdély’s sculpture of a spidron polyhedron. Pickover also misses opportunities where the use of artworks can connect mathematics to culture or touch more vividly on the history embedding his milestones. His representation of Sangaku geometry, for example, is a graphic of one of the theorems proffered by these devotional tablets, rather than of the actual tablet mounted at a Shinto shrine. This is an unfortunate omission, since the tablets are less significant as mathematics than as illustrations of the role of mathematics in culture. In another example, he features a stock blow-up of modern plastic dice in his discussion of dice and chance. Since Pickover stresses the antiquity of dice and the study of chance, a far more effective image would be that of the ancient Iranian dice he cites, which in appearance are almost exactly like those of today (see ‘Chance and Necessity – Dice’ in [1, p. 308]). A significant problem in illustrating mathematics is that the topics are often so abstract as to defy physical presentation. (Very few mathematical artists create work about group theory.) An intriguing attempt to overcome this is Pickover’s choice to accompany Robert Langlands’ elegant conjecture linking analytics and algebra. He chose the image of the ornate interior of the cathedral of Notre-Dame de Fourviére in Lyons to wisely evoke the elegance, rather than portray the content he perceived in Langlands’ ideas. The choice of this cathedral with its intense decoration reveals something of The Math Book and its author in general. The book unfortunately does not identify the cathedral or its architect, Pierre Bossan. This devalues the image. As with so many of the book’s illustrations, its value is as a stock image — in this case chosen because a past writer compared the mathematical elegance of Langland’s work to that of a cathedral. Like so much of the imagery in the book, its inclusion is driven by a sophisticated impulse, but ill considered. The Neo-Byzantine cathedral seems important here because, despite its excesses, it does portray beauty that is joyful and celebratory and dedicated to a supreme entity. The ‘supreme’ beauty that Pickover finds in mathematics is more that of religious celebration and not ‘a beauty cold and austere’ like that seen by Bertrand Russell.

Pervasive computing for sustaining cultural heritage

As a computer scientist, I have been working for decades on challenging and highly useful domains of research and applications such as computer-aided manufacturing, health-care or remote communication via avatars. The tsunami-like development of communication technologies like smart phones, tiny sensors of all kinds and the 7/24 online existence has, in recent years, posed new challenges: How can we harvest the potentials of the constant development of digital technologies for the well-being of the society?; Who are the “we”, the ones to invent, design and present new applications in a way attractive for the “potential user”, especially, for the new generations? It is clear that these tasks are beyond the competences of programmers and engineers. At the Creative Technology Lab of MOME we work on experimental projects in interdisciplinary teams of visual artists, programmers, project managers and experts from humanities, pedagogy, and psychology, and often in strong cooperation with the envisioned user. Our aim is to invent novel applications which are appealing, inviting, playful and at the same time useful: making us aware of environmental or social problems, cultural heritage, help/motivate learning, strengthen communities and induce face-to-face dialogues.

ColourMirror—Connecting Visitors with Exhibits by an Interactive Installation

Museums are in transformation, along with today’s rapidly changing society. Digital technologies not only attract the young generations as potential visitors, but they also create entirely new ways of interpretation, engagement and outreach. As a case study, we introduce our ColourMirror, a multifunctional digital installation accompanying a museum exhibition in which objects are arranged by colour. In a ‘magical’ mirror, visitors get to see the exhibit that resembles their own colours the most. They may forward and distribute the image in the mirror. The collected data is the basis of animated visualizations. An empirical study of usage and visitor experience reveals that visitors enjoy the interaction, remember well and identify with the object they have been assigned, and feel motivated by digital installations to visit an exhibition. We sum up major lessons and potential further applications.

Renewal of the Museum in the Digital Epoch

At the beginning of the twenty-first century, digital technologies are radically changing the way young people communicate, learn and spend their free time. Museums, in order not to lose the next generations as visitors, must conform to the new expectations and needs. On a large scale, the museum must address young people, provide a forum for self-expression and participation and advertise itself by new means. On a smaller scale, the style and means of individual exhibitions must change, providing space for activity, emotions and multiple modalities besides text, personalized visits, interactive explorations and self-expression, evoking emotions but meanwhile also fulfilling educational objectives. Digital technologies—by the yet smaller, cheaper and more and more pervasive devices and services—provide ample means to reach these goals. In our article first we provide a conceptual framework, focussing on the Internet generation as new audience and traditional and new functions of museums. We show how digital technologies may be used to reach six major and general goals. For each issue, we discuss concrete recent examples, from international and own projects. Finally, we address the roles in the complex process of design, development and daily operation of digital applications, in the context of a digital strategy for the museum.

Which Avatars Comfort Children

In this paper we give an account of an empirical study related to avatar selection, whose purpose is to comfort children in the context of the TERENCE learning application. We investigated what Hungarian children of ages 6-11 like to play with, what stories they like to read, and what avatars – out of a selection of nine different designs – they like best. We found a statistically relevant correlation between age, gender, and the participants’ expertise and habits as regards ICT tools. Our studies thus provided relevant information on how to design avatars for this special age group, taking gender equality into consideration and increasingly subjective well-being in order to motivate learning.