Stelios Hadjidimitriou

Sensorimotor cortical response during motion reflecting audiovisual stimulation: evidence from fractal EEG analysis

Sensorimotor activity in response to motion reflecting audiovisual titillation is studied in this article. EEG recordings, and especially the Mu-rhythm over the sensorimotor cortex (C3, CZ, and C4 electrodes), were acquired and explored. An experiment was designed to provide auditory (Modest Mussorgsky’s “Promenade” theme) and visual (synchronized human figure walking) stimuli to advanced music students (AMS) and non-musicians (NM) as a control subject group. EEG signals were analyzed using fractal dimension (FD) estimation (Higuchi’s, Katz’s and Petrosian’s algorithms) and statistical methods. Experimental results from the midline electrode (CZ) based on the Higuchi method showed significant differences between the AMS and the NM groups, with the former displaying substantial sensorimotor response during auditory stimulation and stronger correlation with the acoustic stimulus than the latter. This observation was linked to mirror neuron system activity, a neurological mechanism that allows trained musicians to detect action-related meanings underlying the structural patterns in musical excerpts. Contrarily, the response of AMS and NM converged during audiovisual stimulation due to the dominant presence of human-like motion in the visual stimulus. These findings shed light upon music perception aspects, exhibiting the potential of FD to respond to different states of cortical activity.

EmoActivity - An EEG-Based Gamified Emotion HCI for Augmented Artistic Expression: The i-Treasures Paradigm

There are important cultural differences in emotions that can be predicted and connected to each other in the light of cultural and artistic expressions. The main differences reflected at the affective space are expressed through initial response tendencies of appraisal and action readiness. Capturing and handling the emotions during artistic activities could be used as a dominant source of information to acquire and augment the cultural expression and maximize the emotional impact to the audience. This paper presents a novel EEG-based game-like application, to learn and handle affective states and transitions towards augmented artistic expression. According to the game scenario, the user has to reach and sustain one or more target affective states based on the level of the game, the difficulty setting and his/her current affective state. The game, although at its first version, has been demonstrated to a small group of potential users and has received positive feedback. Its use by a wider audience is anticipated within the realization of the i-Treasure FP7 EU Programme (2013-2017).

The i-Treasures Intangible Cultural Heritage dataset

In this paper, we introduce the i-Treasures Intangible Cultural Heritage (ICH) dataset, a freely available collection of multimodal data captured from different forms of rare ICH. More specifically, the dataset contains video, audio, depth, motion capture data and other modalities, such as EEG or ultrasound data. It also includes (manual) annotations of data, while in some cases additional features and metadata are provided, extracted using algorithms and modules developed within the i-Treasures project. We describe the creation process (sensors, capture setups and modules used), the dataset content and the associated annotations. An attractive feature of this ICH Database is that its the first of its kind, providing annotated multimodal data for a wide range of rare ICH types. Finally, some conclusions are drawn and the future development of the dataset is discussed.

On Supporting Parkinson's Disease Patients: The i-Prognosis Personalized Game Suite Design Approach

The use of serious games in health care interventions sector has grown rapidly in the last years, however, there is still a gap in the understanding on how these types of interventions are used for the management of the Parkinson Disease (PD), in particular. Targeting intelligent early detection and intervention in PD area, the Personalized Game Suite (PGS) design process approach is presented as part of the H2020 i-PROGNOSIS project that introduces the integration of different serious games in a unified platform (i.e., ExerGames, DietaryGames, EmoGames, and Handwriting/Voice Games). From the methodological point of view, to facilitate the visualization of 14 game-scenarios, the system interface and the PD contexts, the storyboarding technique was adopted here. Overall, the realization of the PGS sets the basis for establishing a holistic framework that could aim at improving motor and non-motor symptoms, in order to inform health care providers and policy makers for its inclusion in routine management for PD.

Towards a Practical Subject-Independent Affective State Recognition Based On Time-Domain EEG Feature Extraction

This work presents an evaluation of two time domain-based features, i.e., fractal dimension (FD) and higher-order crossings (HOC), for the subject-independent EEG-based recognition of four affective states as a preliminary step towards a practical real-time affective brain computer interface. EEG data were acquired from an experiment targeting the elicitation of four emotions using affective sounds. Features were computed for each electrode individually and tested in terms of classification using the k-nearest neighbors classifier. Results show that the valence of affective states can be recognized effectively, when the arousal level is specified. Moreover, an above chance level classification accuracy was achieved using a single electrode for the four affective states recognition. Both FD and HOC performed similarly, while the best classification rates were achieved from frontal electrode locations.

Monitoring of Musical ‘Motion’ in EEG Using Bispectral Analysis: A Mirror Neurons-based Approach

‘Motion’, as expressed through high-level features of music, combined with mirror neuron (MN) system activation is studied in this article. The mechanism of MN involved in the perception of musical structures is seen as a means for cueing the learner on ‘known’ factors that can be used for his/her knowledge scaffolding. To explore such relationships, EEG recordings, and especially the Mu-rhythm over the sensorimotor cortex that relates to the activation of MN, were acquired and explored. An experiment was designed to provide the auditory and visual stimuli to two groups of subjects, advanced music students and non-musicians as a control subject group. The musician group’s response to ‘motion’, implemented by Modest Mussorgsky’s ‘Promenade’ and a corresponding video clip, was monitored. The acquired signals, after appropriate averaging in the time domain, were analyzed in the bifrequency domain, using bispectral analysis. Experimental results showed that motion inherent in high-level features of music, could be associated with Mu-rhythm modulation. Such modulation provoked by the MNs could cause bispectral fluctuations, especially when visual stimulation is combined with an auditory one. These results pave the way for further exploitation of the role of MNs in music and, in general, knowledge perception.

Touchscreen typing-pattern analysis for detecting fine motor skills decline in early-stage Parkinson’s disease

Parkinson’s disease (PD) is a degenerative movement disorder causing progressive disability that severely affects patients’ quality of life. While early treatment can produce significant benefits for patients, the mildness of many early signs combined with the lack of accessible high-frequency monitoring tools may delay clinical diagnosis. To meet this need, user interaction data from consumer technologies have recently been exploited towards unsupervised screening for PD symptoms in daily life. Similarly, this work proposes a method for detecting fine motor skills decline in early PD patients via analysis of patterns emerging from finger interaction with touchscreen smartphones during natural typing. Our approach relies on low-/higher-order statistical features of keystrokes timing and pressure variables, computed from short typing sessions. Features are fed into a two-stage multi-model classification pipeline that reaches a decision on the subject’s status (PD patient/control) by gradually fusing prediction probabilities obtained for individual typing sessions and keystroke variables. This method achieved an AUC = 0.92 and 0.82/0.81 sensitivity/specificity (matched groups of 18 early PD patients/15 controls) with discriminant features plausibly correlating with clinical scores of relevant PD motor symptoms. These findings suggest an improvement over similar approaches, thereby constituting a further step towards unobtrusive early PD detection from routine activities.

A Natural User Interface for Gestural Expression and Emotional Elicitation to Access the Musical Intangible Cultural Heritage

This article describes a prototype natural user interface, named the Intangible Musical Instrument, which aims to facilitate access to knowledge of performers that constitutes musical Intangible Cultural Heritage using off-the-shelf motion capturing that is easily accessed by the public at large. This prototype is able to capture, model, and recognize musical gestures (upper body including fingers) as well as to sonify them. The emotional status of the performer affects the sound parameters at the synthesis level. Intangible Musical Instrument is able to support both learning and performing/composing by providing to the user not only intuitive gesture control but also a unique user experience. In addition, the first evaluation of the Intangible Musical Instrument is presented, in which all the functionalities of the system are assessed. Overall, the results with respect to this evaluation were very promising.

On Capturing Older Adults’ Smartphone Keyboard Interaction as a Means for Behavioral Change Under Emotional Stimuli Within i-PROGNOSIS Framework

The unobtrusive use of smartphone technology, as a facilitator and as a means of capturing the daily activities, can be seen as a great challenge in routine monitoring and in promoting behavioural change in older adults. In the present study, a protocol of a sequence of emotional stimuli database was combined with a sequence of emotion-free text typing using a dedicated keyboard of a smartphone and used for capturing the users’ patterns of typing, in terms of hold time (HT), alteration time (AT) and pressure (PR) of each key. Six older adults (three male/female) were employed in the study and sequences of images with facial expressions of Ekman’s six basic emotions (with the addition of the neutral one) were used as stimuli in a three-trial fashion. Statistical analysis of HT, AT and PR data revealed differences in the typing due to emotions alteration, setting a new domain for the analysis and behavioural modeling of older adults’ typing patterns under specific emotional stimuli. This combinatory approach amongst emotional and physical status could be adopted in the field of intelligent monitoring of the healthy ageing and could be extended to elderlies’ pathology cases, such as Parkinson’s disease, as approached by the i-PROGNOSIS initiative.

Intangible Cultural Heritage and New Technologies: Challenges and Opportunities for Cultural Preservation and Development

Intangible cultural heritage (ICH) is a relatively recent term coined to represent living cultural expressions and practices, which are recognised by communities as distinct aspects of identity. The safeguarding of ICH has become a topic of international concern primarily through the work of United Nations Educational, Scientific and Cultural Organization (UNESCO). However, little research has been done on the role of new technologies in the preservation and transmission of intangible heritage. This chapter examines resources, projects and technologies providing access to ICH and identifies gaps and constraints. It draws on research conducted within the scope of the collaborative research project, i-Treasures. In doing so, it covers the state of the art in technologies that could be employed for access, capture and analysis of ICH in order to highlight how specific new technologies can contribute to the transmission and safeguarding of ICH.