Gert-Jan de Vries

A body-conforming tactile jacket to enrich movie viewing

Adding haptic stimulation to movies is a promising step in creating more emotionally immersive experiences. To explore the potential of this concept, we have created a wearable tactile jacket that is used to deliver movie-specific tactile stimuli to the viewers body that are specifically targeted to influence the viewers emotions. Immersion was evaluated in a user test using questionnaires and physiological measurements. The findings show promising effects of the haptic stimuli that need to be substantiated in further more refined user tests.

Emotion measurement platform for daily life situations

The growing interest in affective computing is expected to have its beneficial impact on consumer lifestyle products. Especially emphatic applications — applications that make you feel they really understand you — will serve the current consumer interest in enhanced and meaningful experiences. To do so, the applications will have to measure the users emotional experience. Well-established psychophysiological techniques bear promise, but so far have mainly been validated in laboratory situations. To also apply them in real-life situations, we built an emotion measurement platform. This platform shows that emotional experiences can be measured in a relatively unobtrusive way, while at the same time it enables us to gather knowledge on emotional experiences in everyday-life and it offers the opportunity to prototype emphatic application concepts and test them in relevant situations.

How to stay in the emotional rollercoaster: lessons learnt from designing EmRoll

Bodily expressions can be used to involve players in intense experiences with games. By physically moving, breathing, or increasing your pulse, you may start emotional processes that help create for a stronger experience of the narrative in the game. We have designed a system named EmRoll that poses riddles to pairs of players. The riddles can only be solved if the players are, or at least pretend to be, moving according to different emotional states: dancing happily, relaxed breathing and being scared. The system measures movement, breathing and sweat reactions from the two players. Lessons learnt were: playing in pairs is an important aspect as the two players influenced one-another, pulling each other into stronger experiences; getting excited through intense movement when involving your whole body worked well, as did relaxing through deep breathing; using the sweat response as an input mechanism worked less well; and finally, putting a Wizard (a human operator) into the loop can help bootstrap difficulty balancing and thereby increase emotional involvement.

Towards Emotion Classification Using Appraisal Modeling

The authors studied whether a two-step approach based on appraisal modeling could help in improving performance of emotion classification from sensor data that is typically executed in a one-stage approach in which sensor data is directly classified into a discrete emotion label. The proposed intermediate step is inspired by appraisal models in which emotions are characterized using appraisal dimensions, and subdivides the task in a person-dependent and person-independent stage. In this paper, the authors assessed feasibility of this second stage: the classification of emotion from appraisal data. They applied a variety of machine learning techniques and used visualization techniques to gain further insight into the classification task. Appraisal theory assumes the second step to be independent of the individual. Results obtained are promising, but do indicate that not all emotions can be equally well classified, perhaps indicating that the second stage is not as person-independent as proposed in the literature.

Experiences with adaptive statistical models for biosignals in daily life

We discuss the merits of adaptive statistical models for biosignals in a daily life context. Processing of this type of signals poses a number of challenges. First, it is clear that an adaptive model is needed to tailor for the differences in physiology between individuals, as well as adapt to someones current physiological state. Second, in a daily life setting we use unobtrusive measurement devices, which will lead to reduced signal quality compared to the laboratory setting. Third, low-power portable sensors allow for only limited data storage and data transmission. Two techniques to address these challenges are discussed in detail: the usage of the cumulative histogram and parametric models. We show applications to electroencephalogram (EEG), electrocardiogram (ECG) and skin conductance (SC) signals and we advise on how to obtain the most reliable results.

Prognostic value of psychosocial factors for first and recurrent hospitalizations and mortality in heart failure patients: insights from the OPERA-HF study.

Psychosocial factors are rarely collected in studies investigating the prognosis of patients with heart failure (HF), and only time to first event is commonly reported. We investigated the prognostic value of psychosocial factors for predicting first or recurrent events after discharge following hospitalization for HF.

The Tell-Tale Heart: Perceived Emotional Intensity of Heartbeats

Heartbeats are strongly related to emotions, and people are known to interpret their own heartbeat as emotional information. To explore how people interpret other’s cardiac activity, the authors conducted four experiments. In the first experiment, they aurally presented ten different levels of heart rate to participants and compare emotional intensity ratings. In the second experiment, the authors compare the effects of nine levels of heart rate variability around 0.10 Hz and 0.30 Hz on emotional intensity ratings. In the third experiment, they combined manipulations of heart rate and heart rate variability to compare their effects. Finally, in the fourth experiment, they compare effects of heart rate to effects of angry versus neutral facial expressions, again on emotional intensity ratings. Overall, results show that people relate increases in heart rate to increases in emotional intensity. These effects were similar to effects of the facial expressions. This shows possibilities for using human interpretations of heart rate in communication applications.

Employing Personal Health Records for Population Health Management

Linking various sources of medical data provides a wealth of data to researchers. Trends in society, however, have raised privacy concerns, leading to an increasing awareness of the value of data and data ownership. Personal Health Records address this concern by explicitly giving ownership of data to the patient and enabling the patient to choose whom to provide access to their data. We explored whether this paradigm still allows for population health management, including data analysis of large samples of patients, and built a working prototype to demonstrate this functionality. The creation and application of a readmission risk model for cardiac patients was used as carrier application to illustrate the functionality of our prototype platform.

Towards Process Mining of EMR Data - Case Study for Sepsis Management.

Imagine you have cold shivers and a racing heartbeat and high fever. Clear thinking is impossible! Ceiling lights flash by as you are rushed to the emergency department (ED). You feel your body is getting even sicker. Doctors are doing their utmost to treat this acute and threatening condition, while they work piece together all small parts of evidence to set the diagnosis and start targeted treatment. In this situation, the clinical staff depends on a clinical pathway protocol to streamline communication and deliver care according to the latest medical evidence. Today, such clinical pathways are mainly executed and tracked using paper. Hence, there is ample opportunity for technology in a supportive role. Automated process analysis can help improve these processes of delivering standardized care beyond their current level. In this paper, we provide insight into the steps required to perform process mining to EMR data in the challenging domain of sepsis treatment and provide learnings from our preliminary analysis of these data using process mining techniques.

Facial Expression Recognition Using Learning Vector Quantization

Although the detection of emotions from facial video or images has been topic of intense research for several years, the set of applied classification techniques seems limited to a few popular methods. Benchmark datasets facilitate direct comparison of methods. We used one such dataset, the Cohn-Kanade database, to build classifiers for facial expression recognition based upon Local Binary Patterns LBP features. We are interested in the application of Learning Vector Quantization LVQ classifiers to this classification task. These prototype-based classifiers allow to inspect of prototypical features of the emotion classes, are conceptually intuitive and quick to train. For comparison we also consider Support Vector Machine SVM and observe that LVQ performances exceed those reported in literature for methods based upon LBP features and are amongst the overall top performing methods. Most prominent features were found to originate, primarily, from the mouth region and eye regions. Finally, we explored the specific LBP features that were found most influential within these regions.