Carlos César Loureiro Silva

Depth Cues and Perceived Audiovisual Synchrony of Biological Motion

Background Due to their different propagation times, visual and auditory signals from external events arrive at the human sensory receptors with a disparate delay. This delay consistently varies with distance, but, despite such variability, most events are perceived as synchronic. There is, however, contradictory data and claims regarding the existence of compensatory mechanisms for distance in simultaneity judgments. Principal Findings In this paper we have used familiar audiovisual events – a visual walker and footstep sounds – and manipulated the number of depth cues. In a simultaneity judgment task we presented a large range of stimulus onset asynchronies corresponding to distances of up to 35 meters. We found an effect of distance over the simultaneity estimates, with greater distances requiring larger stimulus onset asynchronies, and vision always leading. This effect was stronger when both visual and auditory cues were present but was interestingly not found when depth cues were impoverished. Significance These findings reveal that there should be an internal mechanism to compensate for audiovisual delays, which critically depends on the depth information available.

Judging Time-to-Passage of looming sounds: evidence for the use of distance-based information

Perceptual judgments are an essential mechanism for our everyday interaction with other moving agents or events. For instance, estimation of the time remaining before an object contacts or passes us is essential to act upon or to avoid that object. Previous studies have demonstrated that participants use different cues to estimate the time to contact or the time to passage of approaching visual stimuli. Despite the considerable number of studies on the judgment of approaching auditory stimuli, not much is known about the cues that guide listeners’ performance in an auditory Time-to-Passage (TTP) task. The present study evaluates how accurately participants judge approaching white-noise stimuli in a TTP task that included variable occlusion periods (portion of the presentation time where the stimulus is not audible). Results showed that participants were able to accurately estimate TTP and their performance, in general, was weakly affected by occlusion periods. Moreover, we looked into the psychoacoustic variables provided by the stimuli and analysed how binaural cues related with the performance obtained in the psychophysical task. The binaural temporal difference seems to be the psychoacoustic cue guiding participants’ performance for lower amounts of occlusion, while the binaural loudness difference seems to be the cue guiding performance for higher amounts of occlusion. These results allowed us to explain the perceptual strategies used by participants in a TTP task (maintaining accuracy by shifting the informative cue for TTP estimation), and to demonstrate that the psychoacoustic cue guiding listeners’ performance changes according to the occlusion period.

Immersiveness of Ubiquitous Computing Environments Prototypes: A Case Study

The development of ubiquitous computing ubicomp environments raises several challenges in terms of their evaluation. Ubicomp virtual reality prototyping tools enable users to experience the system to be developed and are of great help to face those challenges, as they support developers in assessing the consequences of a design decision in the early phases of development. Given the situated nature of ubicomp environments, a particular issue to consider is the level of realism provided by the prototypes. This work presents a case study where two ubicomp prototypes, featuring different levels of immersion desktop-based versus CAVE-based, were developed and compared. The goal was to determine the cost/benefits relation of both solutions, which provided better user experience results, and whether or not simpler solutions provide the same user experience results as more elaborate one.

Audiovisual perception in a virtual world: an application of human-computer interaction evaluation to the development of immersive environments

Understanding the mechanisms underlying audiovisual perception is crucial for the development of interactive audiovisual immersive environments. Some human perceptual mechanisms pose challenging problems that can now be better explored with the latest technology in computer-generated environments. Our main goal is to develop an interactive audiovisual immersive system that provides to its users a highly immersive and perceptually coherent interactive environment. In order to do this, we will perform user studies to get a better knowledge of the rules guiding audiovisual perception. This will allow improvements in the simulation of realistic virtual environments through the use of predictive human cognition models as guides for the development of an audiovisual interactive immersive system. This system will encompass the integration of two Virtual Reality systems: a Cave Automatic Virtual Environment-like (CAVE-like) system and a room acoustic modeling and auralization system. The interactivity between user and the audiovisual virtual world will be enabled by the using of a Motion Capture system as a user position tracker.

Audiovisual synchrony perception of walkers as a function of distance and depth cues

Audiovisual perception is still an intriguing phenomenon, especially when we think about the physical and neuronal differences underlying the perception of sound and light. Physically, there is a delay of ∼3 ms/m between the emission of a sound and its arrival to the observer but, on the other hand, we know that acoustic transduction is a very fast process (∼1 ms). Conversely, light speed makes negligible the physical delay while phototransduction is quite slow (∼50 ms). Audio and visual stimuli that are temporally mismatched can be perceived as a coherent audiovisual stimulus, but a sound delay is often required to achieve a better synchrony perception. In this study, we analyze the Point of Subjective Synchrony (PSS) as a function of stimulus distance to understand if individuals take into account sound velocity or if they compensate for differences in transduction time when judging synchrony. Using an audiovisual virtual-reality environment (CAVE-Like) with Point Light Walkers (PLW) as visual stimulus and sound of steps as audio stimulus, audiovisual sequences were presented from −285 to +300 ms of audio asynchrony, at different distances from the observer (10, 15, 20, 25, 30 and 35 m), and in three different conditions which differ only in the number of visual and auditory depth cues. The results show a relation between PSS and stimulus distance congruent with the differences in propagation velocity between sound and light. Depending on the number of depth cues presented, this relation appears to be increasingly closer to a model based on compensation for these physical differences.