Hengchin Yeh

Example-guided physically based modal sound synthesis

Linear modal synthesis methods have often been used to generate sounds for rigid bodies. One of the key challenges in widely adopting such techniques is the lack of automatic determination of satisfactory material parameters that recreate realistic audio quality of sounding materials. We introduce a novel method using prerecorded audio clips to estimate material parameters that capture the inherent quality of recorded sounding materials. Our method extracts perceptually salient features from audio examples. Based on psychoacoustic principles, we design a parameter estimation algorithm using an optimization framework and these salient features to guide the search of the best material parameters for modal synthesis. We also present a method that compensates for the differences between the real-world recording and sound synthesized using solely linear modal synthesis models to create the final synthesized audio. The resulting audio generated from this sound synthesis pipeline well preserves the same sense of material as a recorded audio example. Moreover, both the estimated material parameters and the residual compensation naturally transfer to virtual objects of different sizes and shapes, while the synthesized sounds vary accordingly. A perceptual study shows the results of this system compare well with real-world recordings in terms of material perception.

Sounding liquids: Automatic sound synthesis from fluid simulation

We present a novel approach for synthesizing liquid sounds directly from visual simulation of fluid dynamics. Our approach takes advantage of the fact that the sound generated by liquid is mainly due to the vibration of resonating bubbles in the medium and performs automatic sound synthesis by coupling physically-based equations for bubble resonance with multiple fluid simulators. We effectively demonstrate our system on several benchmarks using a real-time shallow-water fluid simulator as well as a hybrid grid-SPH simulator.

Wave-ray coupling for interactive sound propagation in large complex scenes

We present a novel hybrid approach that couples geometric and numerical acoustic techniques for interactive sound propagation in complex environments. Our formulation is based on a combination of spatial and frequency decomposition of the sound field. We use numerical wave-based techniques to precompute the pressure field in the near-object regions and geometric propagation techniques in the far-field regions to model sound propagation. We present a novel two-way pressure coupling technique at the interface of near-object and far-field regions. At runtime, the impulse response at the listener position is computed at interactive rates based on the stored pressure field and interpolation techniques. Our system is able to simulate high-fidelity acoustic effects such as diffraction, scattering, low-pass filtering behind obstruction, reverberation, and high-order reflections in large, complex indoor and outdoor environments and Half-Life 2 game engine. The pressure computation requires orders of magnitude lower memory than standard wave-based numerical techniques.

Synthesizing contact sounds between textured models

We present a new interaction handling model for physics-based sound synthesis in virtual environments. A new three-level surface representation for describing object shapes, visible surface bumpiness, and microscopic roughness (e.g. friction) is proposed to model surface contacts at varying resolutions for automatically simulating rich, complex contact sounds. This new model can capture various types of surface interaction, including sliding, rolling, and impact with a combination of three levels of spatial resolutions. We demonstrate our method by synthesizing complex, varying sounds in several interactive scenarios and a game-like virtual environment. The three-level interaction model for sound synthesis enhances the perceived coherence between audio and visual cues in virtual reality applications.

Real-Time Path Planning and Navigation for Multi-agent and Crowd Simulations

We survey some of our recent work on real-time path planning and navigation of multiple autonomous agents in dynamic environments. The driving application of our work are real-time crowd simulation for computer games, virtual environments, and avatar-based online 3D social networks. We also present extensions to these methods for accelerating the overall simulation and for modeling more complex behaviors. Finally, we present some preliminary results from our simulations.

Tracing Analytic Ray Curves for Light and Sound Propagation in Non-Linear Media

The physical world consists of spatially varying media, such as the atmosphere and the ocean, in which light and sound propagates along non-linear trajectories. This presents a challenge to existing ray-tracing based methods, which are widely adopted to simulate propagation due to their efficiency and flexibility, but assume linear rays. We present a novel algorithm that traces analytic ray curves computed from local media gradients, and utilizes the closed-form solutions of both the intersections of the ray curves with planar surfaces, and the travel distance. By constructing an adaptive unstructured mesh, our algorithm is able to model general media profiles that vary in three dimensions with complex boundaries consisting of terrains and other scene objects such as buildings. Our analytic ray curve tracer with the adaptive mesh improves the efficiency considerably over prior methods. We highlight the algorithms application on simulation of visual and sound propagation in outdoor scenes.

Outdoor sound propagation with analytic ray curve tracer and Gaussian beam

Outdoor sound propagation benefits from algorithms that can handle, in a computationally efficient manner, inhomogeneous media, complex boundary surfaces, and large spatial expanse. One recent work by Mo, Yeh, Lin, and Manocha [Appl. Acoust. 104, 142-151 (2016)] proposed a ray tracing method using analytic ray curves as tracing primitives, which improved the performance of propagation paths computation over rectilinear ray tracers. In this paper, an algorithm is developed that extends the performance improvement to field computation; it combines the analytic ray curve tracer with fast pressure computation based on the Gaussian beam model. The algorithm is validated against published results on benchmarks in atmospheric and ocean acoustics, and its application is demonstrated on a scene with terrains and buildings of realistic complexity and under a variety of atmospheric conditions. This algorithm is able to compute characteristic sound fields for fully general media profiles and complex three dimensional scenes at close-to-interactive speed.