Muawiyath Shujau

Separation of speech sources using an Acoustic Vector Sensor

This paper investigates how the directional characteristics of an Acoustic Vector Sensor (AVS) can be used to separate speech sources. The technique described in this work takes advantage of the frequency domain direction of arrival estimates to identify the location, relative to the AVS array, of each individual speaker in a group of speakers and separate them accordingly into individual speech signals. Results presented in this work show that the technique can be used for real-time separation of speech sources using a single 20ms frame of speech, furthermore the results presented show that there is an average improvement in the Signal to Interference Ratio (SIR) for the proposed algorithm over the unprocessed recording of 15.1 dB and an average improvement of 5.4 dB in terms of Signal to Distortion Ratio (SDR) over the unprocessed recordings. In addition to the SIR and SDR results, Perceptual Evaluation of Speech Quality (PESQ) and listening tests both show an improvement in perceptual quality of 1 Mean Opinion Score (MOS) over unprocessed recordings.

Multisource DOA estimation based on time-frequency sparsity and joint inter-sensor data ratio with single acoustic vector sensor

By exploring the time-frequency (TF) sparsity property of the speech, the inter-sensor data ratios (ISDRs) of single acoustic vector sensor (AVS) have been derived and investigated. Under noiseless condition, ISDRs have favorable properties, such as being independent of frequency, DOA related with single valuedness, and no constraints on near or far field conditions. With these observations, we further investigated the behavior of ISDRs under noisy conditions and proposed a so-called ISDR-DOA estimation algorithm, where high local SNR data extraction and bivariate kernel density estimation techniques have been adopted to cluster the ISDRs representing the DOA information. Compared with the traditional DOA estimation methods with a small microphone array, the proposed algorithm has the merits of smaller size, no spatial aliasing and less computational cost. Simulation studies show that the proposed method with a single AVS can estimate up to seven sources simultaneously with high accuracy when the SNR is larger than 15dB. In addition, the DOA estimation results based on recorded data further validates the proposed algorithm.

Using in-air Acoustic Vector Sensors for tracking moving speakers

This paper investigates the use of an Acoustic Vector Sensor (AVS) for tracking a moving speaker in real time through estimation of the Direction of Arrival (DOA). This estimation is obtained using the MUltiple SIgnal Classification (MUSIC) [1] algorithm applied on a time-frame basis. The performance of the AVS is compared with a SoundField Microphone which has similar polar responses to the AVS using time-frames ranging from 20 ms to 1 s. Results show that for 20 ms frames, the AVS is capable of estimating the DOA for both mono-tone and speech signals, which are both stationary and moving, with an accuracy of approximately 1.6° and less than 5° in azimuth, for stationary and moving speech sources, respectively. The results also show that the DOA estimates using the SoundField microphone are significantly less accurate than those obtained from the AVS. Furthermore, the results suggest that for estimating the DOA for speech sources, a Voice Activity Detector (VAD) is critical to ensure accurate azimuth estimation.

Speech enhancement via separation of sources from co-located microphone recordings

This paper investigates multichannel speech enhancement for colocated microphone recordings based on Independent Component Analysis (ICA). Comparisons are made between co-located microphone arrays that contain microphones with mixed polar responses with traditional uniform linear arrays formed from omni-directional microphones. It is shown that polar responses of the microphones are a key factor in the performance of ICA applied to co-located microphones. Results from PESQ testing show a significant improvement in speech quality of ICA separated sources as a result of using an AVS rather than other types of microphone arrays.

Linear Predictive perceptual filtering for Acoustic Vector Sensors: Exploiting directional recordings for high quality speech enhancement

This paper investigates the performance of a new technique for speech enhancement which combines Linear Predictive (LP) spectrum-based perceptual filtering to the recordings obtained from an Acoustic Vector Sensor (AVS). The technique takes advantage of the directional polar responses of the AVS to obtain a significantly more accurate representation of the LP spectrum of a target speech signal in the presence of noise when compared to single channel, omni-directional recordings. Comparisons between the speech quality obtained from the proposed technique and existing beamforming-based speech enhancement techniques for the AVS are made through Perceptual Evaluation of Speech Quality (PESQ) tests and Mean Opinion Score (MOS) listening tests. Results show significant improvements in PESQ and MOS scores of 0.2 and 1.6, respectively, for the proposed enhancement technique. Being based on a miniature microphone array, the approach is particular suitable for hands free communication applications in mobile telephony.

Analysing the Quality of Experience of multisensory media from measurements of physiological responses

This paper investigates the Quality of Experience (QoE) of multisensory media by analysing biosignals collected by electroencephalography (EEG) and eye gaze sensors and comparing with subjective ratings. Also investigated is the impact on QoE of various levels of synchronicity between the sensory effect and target video scene. Results confirm findings from previous research that show sensory effects added to videos increases the QoE rating. While there was no statistical difference observed for the QoE ratings for different levels of sensory effect synchronicity, an analysis of raw EEG data showed 25% more activity in the temporal lobe during asynchronous effects and 20-25% more activity in the occipital lobe during synchronous effects. The eye gaze data showed more deviation for a video with synchronous effects and the EEG showed correlating occipital lobe activity for this instance. These differences in physiological responses indicate sensory effect synchronicity may affect QoE despite subjective ratings appearing similar.

Speech dereverberation based on Linear Prediction: An Acoustic Vector Sensor approach

This paper introduces a dereverberation algorithm based on Linear Prediction (LP) applied to the outputs of an Acoustic Vector Sensor (AVS). The approach applies adaptive beamforming to take advantage of the directional outputs of the AVS array to obtain a more accurate LP spectrum than can be obtained with a single channel or Uniform Linear Array (ULA) with a comparable number of channels. This is then used within a modified version of the Spatiotemporal Averaging Method for Enhancement of Reverberant Speech (SMERSH) algorithm derived for the AVS to enhance the LP residual signal. In a highly reverberant environment, the approach demonstrates a significant improvement compared to a ULA as measured by both the Signal to Reverberant Ratio (SRR) and Speech to Reverberation Modulation Energy Ratio (SRMR) for sources ranging from at 1m to 5m from the array.

Backward compatible spatialized teleconferencing based on squeezed recordings

Commercial teleconferencing systems currently available, although offering sophisticated video stimulus of the remote participants, commonly employ only mono or stereo audio playback for the user. However, in teleconferencing applications where there are multiple participants at multiple sites, spatializing the audio reproduced at each site (using headphones or loudspeakers) to assist listeners to distinguish between participating speakers can significantly improve the meeting experience (Baldis, 2001; Evans et al., 2000; Ward & Elko 1999; Kilgore et al., 2003; Wrigley et al., 2009; James & Hawksford, 2008). An example is Vocal Village (Kilgore et al., 2003), which uses online avatars to co-locate remote participants over the Internet in virtual space with audio spatialized over headphones (Kilgore, et al., 2003). This system adds speaker location cues to monaural speech to create a user manipulable soundfield that matches the avatar’s position in the virtual space. Giving participants the freedom to manipulate the acoustic location of other participants in the rendered sound scene that they experience has been shown to provide for improved multitasking performance (Wrigley et al., 2009). A system for multiparty teleconferencing requires firstly a stage for recording speech from multiple participants at each site. These signals then need to be compressed to allow for efficient transmission of the spatial speech. One approach is to utilise close-talking microphones to record each participant (e.g. lapel microphones), and then encode each speech signal separately prior to transmission (James & Hawksford, 2008). Alternatively, for increased flexibility, a microphone array located at a central point on, say, a meeting table can be used to generate a multichannel recording of the meeting speech A microphone array approach is adopted in this work and allows for processing of the recordings to identify relative spatial locations of the sources as well as multichannel speech enhancement techniques to improve the quality of recordings in noisy environments. For efficient transmission of the recorded signals, the approach also requires a multichannel compression technique suitable to spatially recorded speech signals.

Design and analysis of miniature and three tiered B-format microphones manufactured using 3D printing

This paper describes miniature and three-tiered B-format microphone array designs for accurate sound source localisation that are manufactured using 3D printing and MEMs devices. The implications of pressure gradient resolution reduction in spatial-temporal sampling on the accuracy of Direction of Arrival (DOA) estimation is analysed through simulated room impulse response measurements and characterized by the directional signal to noise ratio. It is shown how the capsule spacing can be optimally chosen based on microphone capsule sensitivity and the required DOA accuracy. Through this method a new three tiered B-format microphone array is proposed, where each tier optimally records 3D sound for a given frequency sub-band to achieve highly accurate DOA estimation for the full audible frequency range of 50 Hz to 20 kHz.