Charalampos Saitis

Exploring multimodal biosignal features for stress detection during indoor mobility

This paper presents a multimodal framework for assessing the emotional and cognitive experience of blind and visually impaired people when navigating in unfamiliar indoor environments based on mobile monitoring and fusion of electroencephalography (EEG) and electrodermal activity (EDA) signals. The overall goal is to understand which environmental factors increase stress and cognitive load in order to help design emotionally intelligent mobility technologies that are able to adapt to stressful environments from real-time biosensor data. We propose a model based on a random forest classifier which successfully infers in an automatic way (weighted AUROC 79.3%) the correct environment among five predefined categories expressing generic everyday situations of varying complexity and difficulty, where different levels of stress are likely to occur. Time-locating the most predictive multimodal features that relate to cognitive load and stress, we provide further insights into the relationship of specific biomarkers with the environmental/situational factors that evoked them.

Sound of Vision - Spatial Audio Output and Sonification Approaches

The paper summarizes a number of audio-related studies conducted by the Sound of Vision consortium, which focuses on the construction of a new prototype electronic travel aid for the blind. Different solutions for spatial audio were compared by testing sound localization accuracy in a number of setups, comparing plain stereo panning with generic and individual HRTFs, as well as testing different types of stereo headphones vs custom designed quadrophonic proximaural headphones. A number of proposed sonification approaches were tested by sighted and blind volunteers for accuracy and efficiency in representing simple virtual environments.

Identifying Urban Mobility Challenges for the Visually Impaired with Mobile Monitoring of Multimodal Biosignals

In this study, we aim to better the user experience of the visually impaired when navigating in unfamiliar outdoor environments assisted by mobility technologies. We propose a framework for assessing their cognitive-emotional experience based on ambulatory monitoring and multimodal fusion of electroencephalography, electrodermal activity, and blood volume pulse signals. The proposed model is based on a random forest classifier which successfully infers in an automatic way the correct urban environment among eight predefined categories (AUROC 93 %). Geolocating the most predictive multimodal features that relate to cognitive load and stress, we provide further insights into the relationship of specific biomarkers with the environmental/situational factors that evoked them.

Effect of task constraints on the perceptual evaluation of violins

Results from two previous studies that involved free-playing evaluative tasks showed that players are selfconsistent in their preference for violins and tend to agree of what particular qualities they look for in an instrument (in this case, “richness” and “dynamic range”). However, a significant lack of agreement between violinists was observed, likely because diff erent players evaluate the same perceptual attributes in diff erent ways. The present study thus investigated whether there will be more inter-player agreement if musicians evaluate violin richness and dynamic range by playing only certain notes in certain registers. Results showed that the more focused the task, the more self-consistent violinists are and the more they agree with each other. We further examined the evaluation of richness from playing versus listening tasks and observed that players were better able to discriminate between violins in the former than in the latter. Finally, the potential correlation of spectral centroid and tristimulus with violin richness were examined. Results showed that the perception of richness is likely associated with the relative amount of low- and mid-frequency partials in a given sound (i.e., low spectral centroid and high tristimulus 1 and 2), though more exploration would be necessary before drawing any conclusions.

Evaluating violin quality: How consistent are skilled players?

A perceptual experiment was designed to investigate how consistent violinists are at evaluating violin quality. The objective was to examine both intra‐ and inter‐subject consistency across a certain range of violins. Skilled classical violinists were asked to play a set of different violins, evaluate their quality, and order them by preference. Violins of different periods were used, varying from student to performance level. Low light conditions and dark sunglasses were used to hide the identity of the instruments as much as possible. Considering the bow as an extension of the player, violinists carried out the task using their own bow. Upon completing the task, participants had to comment on the ranking process and provide rationale for their choices. Preliminary results from a pilot study indicate that both intra‐ and inter‐subject consistency are high when violinists are presented a small group of instruments that are fairly distinct from one another. Results of a more in‐depth study (with more instruments, some very similar to one another) will be presented at the conference.

Cognitive Load Assessment from EEG and Peripheral Biosignals for the Design of Visually Impaired Mobility Aids

Reliable detection of cognitive load would benefit the design of intelligent assistive navigation aids for the visually impaired (VIP). Ten participants with various degrees of sight loss navigated in unfamiliar indoor and outdoor environments, while their electroencephalogram (EEG) and electrodermal activity (EDA) signals were being recorded. In this study, the cognitive load of the tasks was assessed in real time based on a modification of the well-established event-related (de)synchronization (ERD/ERS) index. We present an in-depth analysis of the environments that mostly challenge people from certain categories of sight loss and we present an automatic classification of the perceived difficulty in each time instance, inferred from their biosignals. Given the limited size of our sample, our findings suggest that there are significant differences across the environments for the various categories of sight loss. Moreover, we exploit cross-modal relations predicting the cognitive load in real time inferring on features extracted from the EDA. Such possibility paves the way for the design on less invasive, wearable assistive devices that take into consideration the well-being of the VIP.

The Role of Haptic Cues in Musical Instrument Quality Perception

We draw from recent research in violin quality evaluation and piano performance to examine whether the vibrotactile sensation felt when playing a musical instrument can have a perceptual effect on its judged quality from the perspective of the musician. Because of their respective sound production mechanisms, the violin and the piano offer unique example cases and diverse scenarios to study tactile aspects of musical interaction. Both violinists and pianists experience rich haptic feedback, but the former experience vibrations at more bodily parts than the latter. We observe that the vibrotactile component of the haptic feedback during playing, both for the violin and the piano, provides an important part of the integrated sensory information that the musician experiences when interacting with the instrument. In particular, the most recent studies illustrate that vibrations felt at the fingertips (left hand only for the violinist) can lead to an increase in perceived sound loudness and richness, suggesting the potential for more research in this direction.

Musical Haptics: Introduction

This chapter introduces to the concept of musical haptics, its scope, aims, challenges, as well as its relevance and impact for general haptics and human–computer interaction. A brief summary of subsequent chapters is given.

Concepts of timbre emerging from musician linguistic expressions

The metaphorical nature of the lexicon used by musicians to describe timbral qualities of instruments shows that they are not familiar with describing sound as a sensory experience in an acoustical terminology and share little knowledge about the perceptual dimensions of sound. Instead, they conceptualize and communicate sound qualities through different sensory domains, for instance, a sound felt, seen, or tasting as “velvety.” These metaphorical linguistic structures are central to the process of conceptualizing timbre by allowing the musician to communicate subtle acoustic variations in terms of other, more commonly shared sensory experiences. Their psycholinguistic analysis can be considered as one way to study the underlying cognitive representations empirically. An online listening test using short instrumental solo excerpts from recorded music was designed to obtain a rich corpus of free-format verbal descriptions of violin, clarinet, piano, and guitar timbre from instrumentalists describing their ...

Fractal Art: Closer to Heaven? Modern Mathematics, the Art of Nature, and the Nature of Art

Understanding nature has always been a reference point for both art and science. Aesthetics have put nature at the forefront of artistic achievement. Artworks are expected to represent nature, to work like it. Science has likewise been trying to explain the very laws that determine nature. Technology has provided both sides with the appropriate tools towards their common goal. Fractal art stands right at the heart of the art-science-technology triangle. This chapter examines the new perspectives brought to art by fractal geometry and chaos theory and how the study of the fractal character of nature offers promising possibilities towards art’s mission.