Sebastian Merchel

Audiotactile Feedback Design for Touch Screens

The use of touch sensitive displays and touch surfaces is just emerging and they are more and more replacing physical buttons. If a physical button is pressed, audio and tactile feedback confirms the successful operation. The loss of audiotactile feedback in touch sensitive interfaces might create higher input error rates and user dissatisfaction. Therefore the design and evaluation of suitable signals is necessary. In literature different researchers discuss implementation and evaluation of audio and tactile feedback for mobile applications using small vibration actuators, e.g. [1,..., 12]. However in ticket machines or automated teller machines the size of the actuator is not a limiting factor. Thus arbitrary vibratory stimuli can be generated. In this study, the tactile feedback is generated using an electro-dynamic exciter which allows amplitudes comparable to physical buttons. Real buttons normally produce multimodal feedback. Therefore multimodal interaction is an important issue for the touch screens. In this study, psychophysical experiments were conducted to investigate the design and interaction issues of auditory and tactile stimuli for touch sensitive displays and the combined influence of auditory and tactile information (i.e. vibration) on the system quality.

Vibratory and Acoustical Factors in Multimodal Reproduction of Concert DVDs

Sound and vibration perception are always coupled in live music experience. Just think of a rock concert or hearing (and feeling) a church organ sitting on a wooden pew. Even in classical concerts kettledrum and double bass are sensed not only with our ears. The air-borne sound causes seat vibrations or excites the skin surface directly. For some instruments (e.g. an organ) structure-borne sound is transmitted directly from the instrument to the listener. If concert recordings are played back with multimedia hi-fi systems at home, these vibratory information is missing in the majority of cases. This is due to low reproduction levels or to the limited frequency range of conventional loudspeakers. The audio signal on todays DVDs contains an additional channel for low frequency effects (LFE), which is intended for reproduction using a subwoofer. The generation of tactile components is still very restricted. An enhancement of such a systems might be possible using an electrodynamical shaker which generates whole body vibration (WBV) for a seated person. This paper describes a system implementing this approach. The generation of a vibrotactile signal from the existing audio channels is analyzed. Different parameters during this process (amplitude of the vibration, frequency range) are examined in relation to their perceptual consequences using psychophysical experiments.

Cross-modal frequency matching: sound and whole-body vibration

Interest in human responses to whole-body vibration has grown, particularly due to the increasing usage of vehicles, e.g. cars, trucks, and helicopters etc. Another reason for growing interest in recent years is the importance of the vibrations generated by the performance of music for multimedia reproduction systems. There is a strong relationship between the frequency of the auditory stimulus and the frequency of the tactile stimulus, which simply results from the physical processes that generate the stimuli. The recordings in different vehicles or in different concert situations show that the whole-body vibration signal is like a low-pass filtered audio signal. The spectral contents, particularly low frequencies, are matched with each other. This correlation plays an important role in our integration mechanism of auditory and tactile information and in the perception of an immersive multimodal event. In this study, psychophysical experiments were conducted to investigate, if subjects are able to match the frequencies of two different sensory modalities with each other. In this experiment, sinusoidal sound and vibration signals were used. The auditory stimuli were presented to the subjects via headphones and the tactile stimuli were presented through a vibration seat. The task of the subject was to match the frequency of the whole-body vibration to the frequency of the auditory stimuli. The results show that the subjects are able to match the frequency of both modalities with some tolerances.

Perceptual evaluation of violin vibrations and audio-tactile interaction

When playing a violin, the musician communicates with his instrument not only through his ears but also his fingers, chin, shoulder, and eyes. He uses different sensory inputs, which are provided by different sensory channels, such as auditory, tactile, kinesthetic, and visual, to play his musical instrument. The perceived vibrations are useful for the player to feel and to control the instrument. The interaction between sound and vibration plays also a role on the overall instrument perception. In this study, violin vibrations and their interaction with violin sounds were evaluated. Therefore, the vibration amplitudes of the neck and the violin sounds were recorded simultaneously during normal playing. The vibration recordings were analyzed, and then additional stimuli were generated by filtering or modifying frequency components. In the first experimental session, the vibration stimuli, which were presented to the subjects via a mini electrodynamic shaker, were evaluated. In the second experimental sess...

Equal intensity contours for whole-body vibrations compared with vibrations cross-modally matched to isophones

In this study, two experiments were conducted to determine the curves of equal intensity perception for sinusoidal vertical whole-body vibrations (WBV) of seated subjects over the frequency range from 10Hz to 250Hz. Vibrations were presented to subjects using a flat hard seat. In total, 10 participants were asked to match the intensity of different vibrations, using a method of adjustment. The obtained contours were compared with the threshold of vibration and to vibrations cross-modally matched to tones from isophones. The shapes of the equal intensity contours in the present study show reasonable agreement with the contours from other studies despite the use of different methodologies and experimental questions. The contours show a characteristic similar to the perception threshold. No dependency of vibration magnitude on the shape of the contours was found in the applied dynamic range. However, large inter-individual variations were observed. The results imply that vibration curves that are cross-modally matched to isophones show similar characteristics.

Cross-modality matching of loudness and perceived intensity of whole-body vibrations

In this study, two experiments were conducted to determine the point of subjective intensity equality (PSE) of pure tones and sinusoidal whole-body vibrations (WBV) at various frequencies (50 Hz, 100 Hz and 200 Hz). In these experiments, sounds and vertical vibrations were simultaneously presented to subjects using circumaural headphones and a flat hard seat. In total, 10 participants were subjected to tones with a fixed loudness level (40 phon, 60 phon, 80 phon and 100 phon). The participants were asked to match the intensity of the vibration to the loudness of the tone, using the method of adjustment. In the first experiment, the participants were subjected to a vibration and tone with the same frequency. Alternatively, in the second experiment, the frequency of the vibration was maintained at 50 Hz, while that of the tone was varied. The results revealed that a 20 phon increase in loudness level resulted in a 5-6 dB increase in matched acceleration level at loudness levels greater than 40 phon. This result was reproducible with small intraindividual variations; however, large inter-individual differences were observed.

Identification and Evaluation of Perceptual Attributes for Periodic Whole-Body and Hand-Arm Vibration

When a systematic tactile design of a product is to be conducted, it is necessary to understand important perceptual properties of vibration from a user perspective. In everyday life humans are exposed to whole-body and hand-arm vibration, e.g. when driving a vehicle. Vibration of the steering wheel and seat is transmitted to the driver and conveys information about the condition of the road and vehicle. Depending on the spectral and temporal properties of the vibration signal certain attributes are elicited in the driver, e.g., “bumpy”. Such verbaliseable attributes describe the vibration from a perceptual perspective. In this study periodic whole-body and hand-arm vibration were presented to test subjects. The most common tactile attributes were collected using a free interview. Afterwards, these attributes were rated for their suitability to characterize the signal patterns. The results show a systematic relationship between physical parameters (level, carrier frequency and modulation frequency) and this suitability.

Auditory-Tactile Experience of Music

We listen to music not only with our ears. The whole body is present in a concert hall, during a rock event, or while enjoying music reproduction at home. This chapter discusses the influence of audio-induced vibrations at the skin on musical experience. To this end, sound and body vibrations were controlled separately in several psychophysical experiments. The multimodal perception of the resulting concert quality is evaluated, and the effect of frequency, intensity, and temporal variation of the vibration signal is discussed. It is shown that vibrations play a significant role in the perception of music. Amplifying certain vibrations in a concert venue or music reproduction system can improve the music experience. Knowledge about the psychophysical similarities and differences of the auditory and tactile modality help to develop perceptually optimized algorithms to generate music-related vibrations. These vibrations can be reproduced, e.g., using electrodynamic exciters mounted to the floor or seat. It is discussed that frequency shifting and intensity compression are important approaches for vibration generation.

Touchscreens and Musical Interaction

Touch-sensitive interfaces are more and more used for music production. Virtual musical instruments, such as virtual pianos or drum sets, can be played on mobile devices like phones. Audio tracks can be mixed using a touchscreen in a DJ set-up. Samplers, sequencers or drum machines can be implemented on tablets for use in live performances. The main drawback of traditional touch-sensitive surfaces is the missing haptic feedback. This chapter discusses if adding specifically designed vibrations helps improve the user interaction with touchscreens. An audio mixing application for touchscreens is used to investigate if tactile information is useful for interaction with virtual musical instruments and percussive loops. Additionally, the interaction of auditory and tactile perception is evaluated. The effect of loudness on haptic feedback is discussed using the example of touch-based musical interaction.

Tactile intensity perception compared to auditory loudness perception

The examination of auditory intensity perception has a long history, and comprehensive knowledge exists. However, tactile intensity perception has not been studied as thoroughly. A short literature review provides an overview of the current state of research, with a focus on perceived vibration magnitude. To broaden our knowledge, tactile intensity perception was investigated further in this study. The growth of perceived intensity of seat vibrations with increasing vibration level was compared to auditory loudness. Therefore, a magnitude estimation experiment was performed. Curves of equal vibration intensity have been determined.