René van Egmond

3 – THE EXPERIENCE OF PRODUCT SOUNDS

Publisher Summary Many people use auditory cues unconsciously. People derive not only functional information from the sound but also information concerning an object’s physical properties. Studies have shown that people use sound to estimate the size and the shape of objects. This chapter explores product sounds, which can be considered a subdomain of environmental sounds. A number of studies have investigated memory, recognition, identification, and the descriptors of environmental sounds. Environmental sounds comprise the sounds that we hear during our daily life, or the sounds that surround us. This means that environmental sounds comprise a diversity of sounds. Among product sounds there is a difference between consequential and intentional sounds. Consequential sounds are a consequence of the moving parts of products and are affected by the interaction when a person uses a product. Intentional sounds are deliberately added to a product. These sounds are synthesized or recorded. They are radiated through a loudspeaker or by a vibrating surface. In addition, the term sound quality is often used in relation to reducing noise or in relation to psychoacoustic measures. However, recent discussions suggest that, dependent on the definition of quality, other aspects should be introduced. This chapter uses the term product sound experience as it is a more general concept related to the emotional, semantic, and sensorial experience, and because the term is more neutral. It also is a term that is related to the human aspect and not to how the sound can be described in terms of spectral composition or signal-to-noise ratios.

The effect of visual context on the identification of ambiguous environmental sounds

The influence of the specificity of the visual context on the identification of environmental sounds (i.e., product sounds) was investigated. Two different visual context types (i.e., scene and object contexts)-which varied in the specificity of the semantic information-and a control condition (meaningless images) were employed. A contextual priming paradigm was used. Identification accuracy and response times were determined in two context conditions and one control condition. The results suggest that visual context has a positive effect on sound identification. In addition, two types of product sounds (location-specific and event-specific sounds) were observed which exhibited different sensitivities to scene and object contexts. Furthermore, the results suggest that conceptual interactions exist between an object and a context that do not share the same perceptual domain. Therefore, context should be regarded as a network of conceptually associated items in memory.

Roughness in sound and vision

In three experiments the perceived roughness of visual and of auditory materials was investigated. In Experiment 1, the roughness of frequency-modulated tones was determined using a paired-comparison paradigm. It was found that using this paradigm similar results in comparison to literature were found. In Experiment 2, the perceived visual roughness of textures drawn from the CUReT database was determined. It was found that participants could systematically judge the roughness of the textures. In Experiment 3 the perceived pleasantness for the textures used in Experiment 2 was determined. It was found that two groups of participants could be distinguished. One group found rough textures unpleasant and smooth textures pleasant. The other group found rough textures pleasant and smooth textures unpleasant. Although for the latter groups the relation between relative roughness and perceived pleasantness was less strong.

Subjective and objective texture similarity for image compression

We focus on the evaluation of texture similarity metrics for structurally lossless or nearly structurally lossless image compression. By structurally lossless we mean that the original and compressed images, while they may have visible differences in a side-by-side comparison, they have similar quality so that one cannot tell which is the original. This is particularly important for textured regions, which can have significant point-by-point differences, even though to the human eye they appear to be the same. As in traditional metrics, texture similarity metrics are expected to provide a monotonic relationship between measured and perceived distortion. To evaluate metric performance according to this criterion, we introduce a systematic approach for generating synthetic texture distortions that model variations that occur in natural textures. Based on such distortions, we conducted subjective experiments with a variety of original texture images and different types and degrees of distortions. Our results indicate that recently proposed structural texture similarity metrics provide the best performance.

A new subjective procedure for evaluation and development of texture similarity metrics

In order to facilitate the development of objective texture similarity metrics and to evaluate their performance, one needs a large texture database accurately labeled with perceived similarities between images. We propose ViSiProG, a new Visual Similarity by Progressive Grouping procedure for conducting subjective experiments that organizes a texture database into clusters of visually similar images. The grouping is based on visual blending, and greatly simplifies pairwise labeling. ViSiProG collects subjective data in an efficient and effectivemanner, so that a relatively large database of textures can be accommodated. Experimental results and comparisons with structural texture similarity metrics demonstrate both the effectiveness of the proposed subjective testing procedure and the performance of the metrics.

Task Prioritization in Dual-Tasking: Instructions versus Preferences.

The role of task prioritization in performance tradeoffs during multi-tasking has received widespread attention. However, little is known on whether people have preferences regarding tasks, and if so, whether these preferences conflict with priority instructions. Three experiments were conducted with a high-speed driving game and an auditory memory task. In Experiment 1, participants did not receive priority instructions. Participants performed different sequences of single-task and dual-task conditions. Task performance was evaluated according to participants’ retrospective accounts on preferences. These preferences were reformulated as priority instructions in Experiments 2 and 3. The results showed that people differ in their preferences regarding task prioritization in an experimental setting, which can be overruled by priority instructions, but only after increased dual-task exposure. Additional measures of mental effort showed that performance tradeoffs had an impact on mental effort. The interpretation of these findings was used to explore an extension of Threaded Cognition Theory with Hockey’s Compensatory Control Model.

Hysteresis in Mental Workload and Task Performance The Influence of Demand Transitions and Task Prioritization

Objective: We examine how transitions in task demand are manifested in mental workload and performance in a dual-task setting. Background: Hysteresis has been defined as the ongoing influence of demand levels prior to a demand transition. Authors of previous studies predominantly examined hysteretic effects in terms of performance. However, little is known about the temporal development of hysteresis in mental workload. Method: A simulated driving task was combined with an auditory memory task. Participants were instructed to prioritize driving or to prioritize both tasks equally. Three experimental conditions with low, high, and low task demands were constructed by manipulating the frequency of lane changing. Multiple measures of subjective mental workload were taken during experimental conditions. Results: Contrary to our prediction, no hysteretic effects were found after the high- to low-demand transition. However, a hysteretic effect in mental workload was found within the high-demand condition, which degraded toward the end of the high condition. Priority instructions were not reflected in performance. Conclusion: Online assessment of both performance and mental workload demonstrates the transient nature of hysteretic effects. An explanation for the observed hysteretic effect in mental workload is offered in terms of effort regulation. Application: An informed arrival at the scene is important in safety operations, but peaks in mental workload should be avoided to prevent buildup of fatigue. Therefore, communication technologies should incorporate the historical profile of task demand.

Impact of sound on image-evoked emotions

In two experiments the effect of sound on visual information was investigated. In Experiment 1 the effect of the visual appearance of product types with an expensive deign and with an inexpensive design on the experience of the sound recordings of these products was investigated. Recordings and pictures were systematically interchanged. Thus, for example, the visual image of an expensive design was combined with a recording of the sound of an inexpensive and of an expensive design. It was found that product appearance did not affect the judgment on luxury, pleasantness, quality, and ease-of-use but that the experience of the sound dominated over the visual experience. In Experiment 2, pictures from the international affective pictures set were combined with frequency-modulated tones that varied in the amount of sensory pleasantness by manipulating the amount of roughness. The combination of sounds and pictures were rated on the valence and arousal dimensions of the circumplex model of core affect. It was found that the sounds only negatively affected the experience of the pictures on the valence dimension. The arousal level was not affected by the sounds. Both experiments show that sound can affect the perception and experience of pictures.In two experiments the effect of sound on visual information was investigated. In Experiment 1 the effect of the visual appearance of product types with an expensive deign and with an inexpensive design on the experience of the sound recordings of these products was investigated. Recordings and pictures were systematically interchanged. Thus, for example, the visual image of an expensive design was combined with a recording of the sound of an inexpensive and of an expensive design. It was found that product appearance did not affect the judgment on luxury, pleasantness, quality, and ease-of-use but that the experience of the sound dominated over the visual experience. In Experiment 2, pictures from the international affective pictures set were combined with frequency-modulated tones that varied in the amount of sensory pleasantness by manipulating the amount of roughness. The combination of sounds and pictures were rated on the valence and arousal dimensions of the circumplex model of core affect. It was found that the sounds only negatively affected the experience of the pictures on the valence dimension. The arousal level was not affected by the sounds. Both experiments show that sound can affect the perception and experience of pictures.

Designing and Deploying Meaningful Auditory Alarms for Control Systems

In the old days, spacecraft alarming notifications to operators were directed, upon arrival to ground, to one of those needle printers. Trained operators could tell, from the length and rhythm of the printer noise, what kind of alarm it was and therefore infer the criticality or the subject. Today, in monitoring and control systems (MCS) currently in use at the European Space Agency (ESA), there is no care to convey information in the sounds, and these alarm sounds have not been systematically designed to indicate the type of system failure and further elicit the desired and accurate operator response. Operators depend heavily on the graphical interfaces in order to pinpoint the source of alarm sounds (see Fig. 1) which further creates cognitive load. Similarly, switching cost from auditory perception to visual perception while finding the source of the information is undesirable when time can be a precious commodity for operators when monitoring valuable spacecraft. Therefore, ESA teamed up with Delft University of Technology and Plymouth University in order to investigate and design a new auditory display for the control rooms located in the European Space Operations Centre (ESOC), in Darmstadt, Germany

Sound meets image: freedom of expression in texture description

The use of sound was explored as means for expressing perceptual attributes of visual textures. Two sets of 17 visual textures were prepared: one set taken from the CUReT database, and one set synthesized to replicate the former set. Participants were instructed to match a sound texture with a visual texture displayed onscreen. A modified version of a Product Sound Sketching Tool was provided, in which an interactive physical interface was coupled to a frequency modulation synthesizer. Rather than selecting from a pre-defined set of sound samples, continuous exploration of the auditory space allowed for an increased freedom of expression. While doing so, participants were asked to describe what auditory and visual qualities they were paying attention to. It was found that participants were able to create sounds that matched visual textures. Based on differences in diversity of descriptions, synthetic textures were found to have less salient perceptual attributes than their original counterparts. Finally, three interesting sound synthesis clusters were found, corresponding with mutually exclusive description vocabularies.