Leonid Ivonin

Unconscious emotions: quantifying and logging something we are not aware of

Lifelogging tools aim to precisely capture daily experiences of people from the first-person perspective. Although there have been numerous lifelogging tools developed for users to record the external environment around them, the internal part of experience characterized by emotions seems to be neglected in the lifelogging field. However, the internal experiences of people are important and, therefore, lifelogging tools should be able to capture not only the environmental data, but also emotional experiences, thereby providing a more complete archive of past events. Moreover, there are implicit emotions that cannot be consciously experienced, but still influence human behaviors and memories. It has been proven that conscious emotions can be recognized from physiological signals of the human body. This fact may be used to enhance life-logs with information about unconscious emotions, which otherwise would remain hidden. On the other hand, it is not clear if unconscious emotions can be recognized from physiological signals and differentiated from conscious emotions. Therefore, an experiment was designed to elicit emotions (both conscious and unconscious) with visual and auditory stimuli and to record cardiovascular responses of 34 participants. The experimental results showed that heart rate responses to the presentation of the stimuli are unique for every category of the emotional stimuli and allow differentiation between various emotional experiences of the participants.

Automatic Recognition of the Unconscious Reactions from Physiological Signals

While the research in affective computing has been exclusively dealing with the recognition of explicit affective and cognitive states, carefully designed psychological and neuroimaging studies indicated that a considerable part of human experiences is tied to a deeper level of a psyche and not available for conscious awareness. Nevertheless, the unconscious processes of the mind greatly influence individuals’ feelings and shape their behaviors. This paper presents an approach for automatic recognition of the unconscious experiences from physiological data. In our study we focused on primary or archetypal unconscious experiences. The subjects were stimulated with the film clips corresponding to 8 archetypal experiences. Their physiological signals including cardiovascular, electrodermal, respiratory activities, and skin temperature were monitored. The statistical analysis indicated that the induced experiences could be differentiated based on the physiological activations. Finally, a prediction model, which recognized the induced states with an accuracy of 79.5%, was constructed.

Beyond cognition and affect: sensing the unconscious

In the past decade, research on human–computer interaction has embraced psychophysiological user interfaces that enhance awareness of computers about conscious cognitive and affective states of users and increase their adaptive capabilities. Still, human experience is not limited to the levels of cognition and affect but extends further into the realm of universal instincts and innate behaviours that form the collective unconscious. Patterns of instinctual traits shape archetypes that represent images of the unconscious. This study investigated whether seven various archetypal experiences of users lead to recognisable patterns of physiological responses. More specifically, the potential of predicting the archetypal experiences by a computer from physiological data collected with wearable sensors was evaluated. The subjects were stimulated to feel the archetypal experiences and conscious emotions by means of film clips. The physiological data included measurements of cardiovascular and electrodermal activities. Statistical analysis indicated a significant relationship between the archetypes portrayed in the videos and the physiological responses. Data mining methods enabled us to create between-subject prediction models that were capable of classifying four archetypes with an accuracy of up to 57.1%. Further analysis suggested that classification performance could be improved up to 70.3% in the case of seven archetypes by using within-subject models.

Traces of unconscious mental processes in introspective reports and physiological responses.

Unconscious mental processes have recently started gaining attention in a number of scientific disciplines. One of the theoretical frameworks for describing unconscious processes was introduced by Jung as a part of his model of the psyche. This framework uses the concept of archetypes that represent prototypical experiences associated with objects, people, and situations. Although the validity of Jungian model remains an open question, this framework is convenient from the practical point of view. Moreover, archetypes found numerous applications in the areas of psychology and marketing. Therefore, observation of both conscious and unconscious traces related to archetypal experiences seems to be an interesting research endeavor. In a study with 36 subjects, we examined the effects of experiencing conglomerations of unconscious emotions associated with various archetypes on the participants’ introspective reports and patterns of physiological activations. Our hypothesis for this experiment was that physiological data may predict archetypes more precisely than introspective reports due to the implicit nature of archetypal experiences. Introspective reports were collected using the Self-Assessment Manikin (SAM) technique. Physiological measures included cardiovascular, electrodermal, respiratory responses and skin temperature of the subjects. The subjects were stimulated to feel four archetypal experiences and four explicit emotions by means of film clips. The data related to the explicit emotions served as a reference in analysis of archetypal experiences. Our findings indicated that while prediction models trained on the collected physiological data could recognize the archetypal experiences with accuracy of 55 percent, similar models built based on the SAM data demonstrated performance of only 33 percent. Statistical tests enabled us to confirm that physiological observations are better suited for observation of implicit psychological constructs like archetypes than introspective reports.

Unspoken Emotions in Movies

Stories are the core of many types of entertainment.Although developers and designers strive to develop new forms of entertainment that deliver richer audiovisual experiences to the audience, one of the key elements – a good story – remains the same. Storytelling is intertwined in people’s lives and has deeply influenced human societies for generations. It ranges from the simplest forms, such as bedtime stories for children, to more complex forms such as plays, movies, and games. Just as children love often-told bedtime stories, sometimes adults enjoy watching classic movies over and over again and never tire of them. Storytelling is not only a matter of how well the stories are memorized but also the overall emotional experience over the course of the storyline. The link between the story and the corresponding emotions is an intriguing topic for developing new interactive storytelling systems.

Experience the world with archetypal symbols: a new form of esthetics.

This paper reports on the results of a user-based evaluation that was conducted on a 3D virtual environment that supports diverse interaction techniques. More specifically, the interaction techniques that were evaluated were touch, gestures (hands and legs) and the use of a smart object. The goal of the experiment was to assess the effectiveness of each interaction modes as a means for the user to complete common tasks within the application. A comparison is attempted in order to provide an insight to the suitability of each technique and direct future research in the area.According to the theories of symbolic interactionism, phenomenology of perception and archetypes, we argue that symbols play the key role in translating the information from the physical world to the human experience, and archetypes are the universal knowledge of cognition that generates the background of human experience (the life-world). Therefore, we propose a conceptual framework that depicts how people experience the world with symbols, and how archetypes relate the deepest level of human experience. This framework indicates a new direction of research on memory and emotion, and also suggests that archetypal symbolism can be a new resource of aesthetic experience design.

Lifelogging for hidden minds: interacting unconsciously

In this paper, we propose a conceptual design of an entertainment system, which would utilize a combination of lifelogging and bio-signal measuring techniques for capturing unconscious communication flow coming from a user. We expect this system could interpret the users unconsciousness and generate an emotionally rich feedback framed in a narrative, which would be tailored according to the universal symbolic patterns. This concept aims to explore a new area of entertainment computing. The discussed system looks promising to proceed with future work.

Enacting archetypes in movies: grounding the unconscious mind in emotion-driven media

Abstract The concept of emotion-driven media integrates affective computing into developing new types of digital media by including the human spectator as an essential part of the whole system. Enactive media, as a landmark in this direction, provides a theoretical basis that is rooted in the enactivist approach in cognitive science. Based on this thesis, it is required to establish a practical procedure for developing the media content through integrating psychological approaches. We demonstrate a systematic approach for developing the content for emotion-driven media. The theory of archetypes, one of the psychoanalytical approaches, was applied for editing movie clips as the media content, and coupled the content with emotional responses by using experimental psychological methodologies. The results showed that it is promising to utilise archetypal movie clips as the content of emotion-driven media and archetypal symbolism can be a useful resource for developing the content of media systems.

Feeling something without knowing why : measuring emotions toward archetypal content

To enhance communication among users through technology, we propose a framework that communicates ‘pure experience.’ This framework can be achieved by providing emotionally charged communication. To initiate this undertaking, we propose to explore materials for communicating human emotions. Research on emotion mainly focuses on emotions that are relevant to utilitarian concerns. Besides the commonly-known emotions like joy and fear, there are non-utilitarian emotions, such as aesthetic emotions, which are essential to our daily lives. Based on Jung’s theory of collective unconsciousness, we consider archetypal content as a new category of affective stimuli of non-utilitarian emotions. We collected pictures and sounds of the archetype of the self, and conducted an experiment with existing affective stimuli of utilitarian emotions. The results showed that archetypal content is potential to be a new category of affective content. It is promising to explore other affective content for further studies.

Measuring archetypal experiences with physiological sensors

The instincts and innate behaviors that form the collective unconscious can be recognized by a computer using physiological sensors.