Rishabh Ranjan

Natural listening over headphones in augmented reality using adaptive filtering techniques

Augmented reality (AR), which composes of virtual and real world environments, is becoming one of the major topics of research interest due to the advent of wearable devices. Today, AR is commonly used as assistive display to enhance the perception of reality in education, gaming, navigation, sports, entertainment, simulators, etc. However, most of the past works have mainly concentrated on the visual aspects of AR. Auditory events are one of the essential components in human perceptions in daily life but the augmented reality solutions have been lacking in this regard till now compared to visual aspects. Therefore, there is a need of natural listening in AR systems to give a holistic experience to the user. A new headphones configuration is presented in this work with two pairs of binaural microphones attached to headphones (one internal and one external microphone on each side). This paper focuses on enabling natural listening using open headphones employing adaptive filtering techniques to equalize the headset such that virtual sources are perceived as close as possible to sounds emanating from the physical sources. This would also require a superposition of virtual sources with the physical sound sources, as well as ambience. Modified versions of the filtered-x normalized least mean square algorithm (FxNLMS) are proposed in the paper to converge faster to the optimum solution as compared to the conventional FxNLMS. Measurements are carried out with open structure type headphones to evaluate their performance. Subjective test was conducted using individualized binaural room impulse responses (BRIRs) to evaluate the perceptual similarity between real and virtual sounds.

Fast continuous HRTF acquisition with unconstrained movements of human subjects

Head related transfer function (HRTF) is widely used in 3D audio reproduction, especially over headphones. Conventionally, HRTF database is acquired at discrete directions and the acquisition process is time-consuming. Recent works have been proposed to improve HRTF acquisition efficiency via continuous acquisition. However, these HRTF acquisition techniques still require subject to sit still (with limited head movement) in a rotating chair. In this paper, we further relax the head movement constraint during acquisition by using a head tracker. The proposed continuous HRTF acquisition technique relies on the activation based normalized least-mean-square (ANLMS) algorithm to extract HRTF on the fly. Experimental results validated the accuracy of the proposed technique, when compared with the standard static acquisition technique.

Fast and efficient real-time GPU based implementation of wave field synthesis

Wave Field Synthesis (WFS) aims to replicate true sound field in an extended listening area with the help of loudspeaker arrays. WFS practical setups are heavily computational, as they need to drive many loudspeakers to accurately render multiple virtual sources. Thus, performance bottleneck occurs due to the sequential implementation on PCs with few cores. In addition, real-time spatial audio reproduction systems like WFS are subjected to hard real-time constraints, limiting system throughput and require cascading of several PCs to improve performance. In this paper, a fast and efficient graphics processing unit (GPU) based implementation of WFS is proposed to enhance the system throughput by extracting maximum data parallelism in the algorithm. The proposed method, implemented on NVidia C2075 GPU, uses block based partitioning approach to achieve peak system throughput of 1,400 Msamples per second, while rendering up to 200 real-time sound sources.

Wave Field Synthesis: The Future of Spatial Audio

We all are used to perceiving sound in a three-dimensional (3-D) world. In order to reproduce real-world sound in an enclosed room or theater, extensive study on how spatial sound can be created has been an active research topic for decades. Spatial audio is an illusion of creating sound objects that can be spatially positioned in a 3-D space by passing original sound tracks through a sound-rendering system and reproduced through multiple transducers, which are distributed around the listening space. The reproduced sound field aims to achieve a perception of spaciousness and sense of directivity of the sound objects. Ideally, such a sound reproduction system should give listeners a sense of an immersive 3-D sound experience. Spatial audio can primarily be divided into three types of sound reproduction techniques, namely, loudspeaker stereophony, binaural technology, and reconstruction using synthesis of the natural wave field [which includes Ambisonics and wave field synthesis (WFS)], as shown in Fig. 1(a).

Fast HRFT measurement system with unconstrained head movements for 3D audio in virtual and augmented reality applications

Binaural audio plays an indispensable role in virtual reality (VR) and augmented reality (AR). Binaural audio recreates the sensation of the three dimensional auditory experience using Head- Related Transfer Functions (HRTFs). HRTFs are as unique as our fingerprint. To achieve an immersive audio experience, HRTFs measured from every particular user is required. Nowadays, the conventional methods for HRTF measurements requires a wellcontrolled environment, hardly any movement of the user, and projecting to the user a high level of unpleasant sound in a rather long duration. Such difficulties have greatly limited the use of individually measurement HRTFs and hinder the authenticity of immersive audio. To solve these problems, we proposed a fast and convenient HRTF measurement system that is an order of magnitude faster and more importantly, it does not place any constraints on the user’s movement. With the help of a head-tracker and advanced adaptive signal processing algorithms, this system is able to achieve satisfactory HRTF measurement accuracy. In this demonstration, we will present a fast real-time HRTF acquisition system and show how the individualized HRTFs improve the audio experience in VR/AR applications.

A hybrid speaker array-headphone system for immersive 3D audio reproduction

Spatial sound systems aim at rendering realistic sound experience to the listeners with uniform sound fields in the entire listening area. Today with the advancement of multichannel surround sound techniques, such systems are being practically realized, especially, at theatres, lecture halls, auditoriums, etc. Current practices, which are most widely used as home theatre systems, are based on multichannel stereophony, like 5.1, 10.2 and higher surround channel system. These systems require multiple loudspeakers to be placed in fixed configuration but often constrained by the room size. Sound reproduction systems like wave field synthesis (WFS) based on principle of natural propagation of sound waves, can create replica of true sound field uniformly over an extended listening area. However, WFS based systems too require hundreds of densely spaced loudspeakers enclosing the listener area and thus, difficult to realize in homes. In this paper, we introduce a new hybrid system by combining the WFS and binaural synthesis over headphones (based on active noise control techniques) to reduce the need of installing loudspeakers everywhere in a living room.