Nadine E. Miner

Data collection and analysis techniques for evaluating the perceptual qualities of auditory stimuli

This paper describes a general methodological framework for evaluating the perceptual properties of auditory stimuli. The framework provides analysis techniques that can ensure the effective use of sound for a variety of applications, including virtual reality and data sonification systems. Specifically, we discuss data collection techniques for the perceptual qualities of single auditory stimuli including identification tasks, context-based ratings, and attribute ratings. In addition, we present methods for comparing auditory stimuli, such as discrimination tasks, similarity ratings, and sorting tasks. Finally, we discuss statistical techniques that focus on the perceptual relations among stimuli, such as Multidimensional Scaling (MDS) and Pathfinder Analysis. These methods are presented as a starting point for an organized and systematic approach for nonexperts in perceptual experimental methods, rather than as a complete manual for performing the statistical techniques and data collection methods. It is our hope that this paper will help foster further interdisciplinary collaboration among perceptual researchers, designers, engineers, and others in the development of effective auditory displays.

Computational Requirements and Synchronization Issues for Virtual Acoustic Displays

This paper addresses two main issues concerning virtual acoustic displays. First, we discuss the computational requirements including sound generation (or synthesis), environmental effects modeling, and three-dimensional (3-D) sound localization. The computational analysis reveals that acoustic processing delays of at least 66 ms are expected with todays technology. This analysis motivates the second issue: how much computational time is available for executing the acoustic process, assuming the requirement for perceptually perfect audiovisual synchronization? A psychoacoustic experiment designed to quantify the tolerable audiovisual delay indicates that an acoustic impact event must occur within an average of 175 ms of the visual event in order for the events to be perceived as synchronous. The most highly trained observers detect desynchrony with an audiovisual delay as low as 100 ms. The results of the computational requirement analysis and the psychoacoustic synchronization experiment provide important information for designers and researchers of virtual acoustic displays.

An application of shared virtual reality to situational training

This paper presents current research being undertaken at Sandia National Laboratories to develop a distributed, shared virtual reality simulation system. The architecture of the system is presented within the framework of an initial application: situational training of inspectors and escorts under programs to verify compliance with nuclear non-proliferation treaties.

An interactive virtual reality simulation system for robot control and operator training

Robotic systems are often very complex and difficult to operate, especially as multiple robots are integrated to accomplish difficult tasks. In addition, training the operators of these complex robotic systems is time-consuming and costly. In this paper a virtual reality based robotic control system is presented. The virtual reality system provides a means by which operators can operate, and be trained to operate, complex robotic systems in an intuitive, cost-effective way. Operator interaction with the robotic system is at a high, task-oriented, level. Continuous state monitoring prevents illegal robot actions and provides interactive feedback to the operator and real-time training for novice users.<<ETX>>

Using wavelets to synthesize stochastic-based sounds for immersive virtual environments

Stochastic, or nonpitched, sounds fill our real-world environment. Humans almost continuously hear stochastic sounds, such as wind, rain, motor sounds, and different types of impact sounds. Because of their prevalence in real-world environments, it is important to include these types of sounds for realistic virtual environment simulations. This paper describes a synthesis approach that uses wavelets for modeling stochastic-based sounds. Parameterizations of the wavelet models yield a variety of related sounds from a small set of models. The result is dynamic sound models that can change according to changes in the virtual environment. This paper contains a description of the sound synthesis process, several developed models, and the on-going perceptual experiments for validating the sound synthesis veracity. The developed models and results demonstrate proof of the concept and illustrate the potential of this approach.

Neurogenesis deep learning: Extending deep networks to accommodate new classes

Neural machine learning methods, such as deep neural networks (DNN), have achieved remarkable success in a number of complex data processing tasks. These methods have arguably had their strongest impact on tasks such as image and audio processing — data processing domains in which humans have long held clear advantages over conventional algorithms. In contrast to biological neural systems, which are capable of learning continuously, deep artificial networks have a limited ability for incorporating new information in an already trained network. As a result, methods for continuous learning are potentially highly impactful in enabling the application of deep networks to dynamic data sets. Here, inspired by the process of adult neurogenesis in the hippocampus, we explore the potential for adding new neurons to deep layers of artificial neural networks in order to facilitate their acquisition of novel information while preserving previously trained data representations. Our results on the MNIST handwritten digit dataset and the NIST SD 19 dataset, which includes lower and upper case letters and digits, demonstrate that neurogenesis is well suited for addressing the stability-plasticity dilemma that has long challenged adaptive machine learning algorithms.

A wavelet synthesis technique for creating realistic virtual environment sounds

This paper describes a new technique for synthesizing realistic sounds for virtual environments. The four-phase technique described uses wavelet analysis to create a sound model. Parameters are extracted from the model to provide dynamic sound synthesis control from a virtual environment simulation. Sounds can be synthesized in real time using the fast inverse wavelet transform. Perceptual experiment validation is an integral part of the model development process. This paper describes the four-phase process for creating the parameterized sound models. Several developed models and perceptual experiments for validating the sound synthesis veracity are described. The developed models and results demonstrate proof of the concept and illustrate the potential of this approach.

An Introduction to Wavelet Theory and Analysis

This report reviews the history, theory and mathematics of wavelet analysis. Examination of the Fourier Transform and Short-time Fourier Transform methods provides tiormation about the evolution of the wavelet analysis technique. This overview is intended to provide readers with a basic understanding of wavelet analysis, define common wavelet terminology and describe wavelet amdysis algorithms. The most common algorithms for performing efficient, discrete wavelet transforms for signal analysis and inverse discrete wavelet transforms for signal reconstruction are presented. This report is intended to be approachable by non- mathematicians, although a basic understanding of engineering mathematics is necessary.

Collision avoidance during teleoperation using whole-arm proximity sensors coupled to a virtual environment

This paper describes a collision avoidance system using Whole Arm Proximity (WHAP) sensors on an articulated robot arm. The capacitance-based sensors generate electric fields which completely encompass the robot arm and detect obstacles as they approach from any direction. The robot is moved through the workspace using a velocity command generated either by an operator through a force-sensing input device or a preprogrammed sequence of motions. The directional obstacle information gathered by the WHAP sensors is then used in a matrix column maximization algorithm that automatically selects the sensor closest to an obstacle during each robot controller cycle. The distance from this sensor to the obstacle is used to reduce the component of the command input velocity along the normal axis of the sensor, allowing graceful perturbation of the velocity command to prevent a collision.

Perceptual validation experiments for evaluating the quality of wavelet-synthesized sounds

This paper describes three psychoacoustic experiments that evaluated the perceptual quality of sounds generated from a new wavelet-based synthesis technique. The synthesis technique provides a method for modeling and synthesizing perceptually compelling sound. The experiments define a methodology for evaluating the effectiveness of any synthesized sound. An identification task and a context-based rating task evaluated the perceptual quality of individual sounds. These experiments confirmed that the wavelet technique synthesizes a wide variety of compelling sounds from a small model set. The third experiment obtained sound similarity ratings. Psychological scaling methods were applied to the similarity ratings to generate both spatial and network models of the perceptual relations among the synthesized sounds. These analysis techniques helped to refine and extend the sound models. Overall, the studies provided a framework to validate synthesized sounds for a variety of applications including virtual reality and data sonification systems.