Kim Hartmann

Investigation of Speaker Group-Dependent Modelling for Recognition of Affective States from Speech

For successful human–machine-interaction (HCI) the pure textual information and the individual skills, preferences, and affective states of the user must be known. Therefore, as a starting point, the user’s actual affective state has to be recognized. In this work we investigated how additional knowledge, for example age and gender of the user, can be used to improve recognition of affective state. Two methods from automatic speech recognition are used to incorporate age and gender differences in recognition of affective state: speaker group-dependent (SGD) modelling and vocal tract length normalisation (VTLN). The investigations were performed on four corpora with acted and natural affected speech. Different features and two methods of classification (Gaussian mixture models (GMMs) and multi-layer perceptrons (MLPs)) were used. In addition, the effects of channel compensation and contextual characteristics were analysed. The results are compared with our own baseline results and with results reported in the literature. Two hypotheses were tested. First, incorporation of age information further improves speaker group-dependent modelling. Second, acoustic normalization does not achieve the same improvement as achieved by speaker group-dependent modelling, because the age and gender of a speaker affects the way emotions are expressed.

Investigating the Form-Function-Relation of the Discourse Particle “hm” in a Naturalistic Human-Computer Interaction

For a successful speech-controlled human-computer interaction (HCI) the pure textual information as well as individual skills, preferences, and affective states of the user have to be known. However, verbal human interaction consists of several information layers. Apart from pure textual information, further details regarding the speaker’s feelings, believes, and social relations are transmitted. The additional information is encoded through acoustics. Especially, the intonation reveals details about the speakers communicative relation and their attitude towards the ongoing dialogue.

Describing Human Emotions Through Mathematical Modelling

Abstract To design a companion technology we focus on the appraisal theory model to predict emotions and determine the appropriate system behaviour to support Human-Computer-Interaction. Until now, the implementation of emotion processing was hindered by the fact that the theories needed originate from diverging research areas, hence divergent research techniques and result representations are present. Since this difficulty arises repeatedly in interdisciplinary research, we investigated the use of mathematical modelling as an unifying language to translate the coherence of appraisal theory. We found that the mathematical category theory supports the modelling of human emotions according to the appraisal theory model and hence assists the implementation.

UAV exploitation: A new domain for cyber power

The risks of military unmanned aerial vehicles (UAVs) being subjected to electronic attack are well recognised, especially following high-profile incidents such as the interception of unencrypted video feeds from UAVs in Iraq and Israel, or the diversion and downing of a UAV in Iran. Protection of military UAV assets rightly focuses on defence against sophisticated cyber penetration or electronic attack, including data link intercepts and navigational spoofing. Offensive activity to counter adversary drone operations presumes a requirement for high-end electronic attack systems. However, combat operations in eastern Ukraine in 2014-16 have introduced an entirely new dimension to UAV and counter-UAV operations. In addition to drones with military-grade standards of electronic defence and encryption, a large number of civilian or amateur UAVs are in operation in the conflict. This presents both opportunities and challenges to future operations combating hybrid threats. Actual operations in eastern Ukraine, in combination with studies of potential criminal or terrorist use of UAV technologies, provide indicators for a range of aspects of UAV use in future conflict. However, apart from the direct link to military usage, UAVs are rapidly approaching ubiquity with a wide range of applications reaching from entertainment purposes to border patrol, surveillance, and research, which imposes an indirect security and safety threat. Issues associated with the unguarded use of drones by the general public range from potentially highly dangerous situations such as failing to avoid controlled airspace, to privacy violations. Specific questions include attribution of UAV activities to the individuals actually directing the drone; technical countermeasures against hacking, interception or electronic attack; and options for controlling and directing adversary UAVs. Lack of attribution and security measures protecting civilian UAVs against electronic attack, hacking or hijacking, with the consequent likelihood of unauthorised use or interception, greatly increases the complication of each of these concerns.

Emotion Detection in HCI: From Speech Features to Emotion Space

Abstract Control mechanisms in modern Human-Computer Interaction (HCI) underwent a paradigm shift from textual or display-based control to more intuitive control mechanisms, such as speech, gesture and mimic. Especially speech provides a high information density, delivering information about the speakers inner state as well as his intention and demand. While word-based analyses allow to understand the speakers request, further speech characteristics reveal the speakers emotion, intention and motivation. Therefore, emotion detection from speech became significant in modern HCI applications. However, the results from the disciplines involved in the emotion detection are not easily merged. Engineers developing voice controlled HCI systems work in “feature spaces”, relying on technically measurable acoustic and spectral features. Psychologists analysing and identifying emotions work in emotion categories, schemes or dimensional emotion spaces, describing emotions in terms of quantities and qualities of human notable expressions. While engineering methods notice the slightest variations in speech, emotion theories allow to compare and identify emotions, but must rely on human judgements. However, both perceptions are essential and must be combined to allow machines to allocate affective states during HCI. To provide a link between machine measurable variations in emotional speech and dimensional emotion theory, significant features describing emotions must be identified and analysed regarding their transferability to emotion space. In this article we present a justifiable feature selection for emotion detection from speech and show how to relate measurable features to emotions. We discuss our transformation model and validate both feature selection and model, based on a selection of the Emo-DB corpus.

Emotion and Disposition Detection in Medical Machines: Chances and Challenges

Machines designed for medical applications beyond usual data acquisition and processing need to cooperate with and adapt to humans in order to fulfill their supportive tasks. Technically, medical machines are therefore considered as affective systems, capable of detecting, assessing and adapting to emotional states and dispositional changes in users. One of the upcoming applications of affective systems is the use as supportive machines involved in the psychiatric disorder diagnose and therapy process. These machines have the additional requirement of being capable to control persuasive dialogues in order to obtain relevant patient data despite disadvantageous set-ups. These automated abilities of technical systems combined with enhanced processing, storage and observational capabilities raise both chances and challenges in medical applications. We focus on analyzing the objectivity, reliability and validity of current techniques used to determine the emotional states of speakers from speech and the arising implications. We discuss the underlying technical and psychological models and analyze recent machine assessment results of emotional states obtained through dialogues. Conclusively we discuss the involvement of affective systems as medical machines in the psychiatric diagnostics process and therapy sessions with respect to the technical and ethical circumstances.

On the security of international data exchange services for e-governance systems

The effects of globalisation and information welfare combined with the increasing mobility of individuals lead to a number of challenges to modern states. In order to guarantee a smooth, secure, uninterrupted organisational flow, governments and their subsidiaries need to cooperate and exchange data on individuals and organisations across national borders. However, insufficiently secured communication of such data imposes security threats which may endanger the individual’s privacy. Currently, several states within Europe develop and operate e-governance systems. These are primarily designed to allow the exchange of data within the institutions of one state. However, examples such as the Estonian e-governance backbone system X-Road strive towards an EU-wide expansion. Technical solutions for the transnational exchange of data between governmental institutions are an unavoidable part of the future of cyberspace. Despite the fact that EU specifications exist, the discrepancy between specification and implementation becomes immanent. This article explores some of the general aspects of the design of secure transnational data exchange frameworks. A comparative analysis of existing e-governance systems within Europe is given based on defined security aspects. It is explored how decisions made in the design may affect the security of the underlying network and its components. The challenges of transnational data exchange frameworks are discussed.

Shifting the Core

Ubiquitous and unconscious use of connected devices demand a fundamental shift in core assumptions about data security. Users cannot be expected to take responsibility for securing their devices and data when processes and data transfer are designed to take place without their active involvement or even awareness. Technological developments already in widespread use mean that privacy, encryption and national cyber security must all urgently be rethought.

Modeling Aspects in Human-Computer Interaction: Adaptivity, User Characteristics and Evaluation

During system interaction, the user’s emotions and intentions shall be adequately determined and predicted to recognize tendencies in his or her interests and dispositions. This allows for the design of an evolving search user interface (ESUI) which adapts to changes in the user’s emotional reaction and the users’ needs and claims.

ERM4CT 2016: 2nd international workshop on emotion representations and modelling for companion systems (workshop summary)

In this paper the organisers present a brief overview of the 2nd International Workshop on Emotion Representations and Modelling for Companion Systems (ERM4CT). The ERM4CT 2016 Workshop is held in conjunction with the 18th ACM International Conference on Multimodal Interaction (ICMI 2016) taking place Tokyo, Japan. The ERM4CT is the follow-up of three previous workshops on emotion modelling for affective human-computer interaction and companion systems. Apart from its usual focus on emotion representations and models, this years ERM4CT puts special emphasis on how to model adequate affective system behaviour. For the first time, this years ERM4CT gave out a dataset, which all attendees could investigate to jointly discuss their findings.