Michael M. Goodwin

Primary-Ambient Signal Decomposition and Vector-Based Localization for Spatial Audio Coding and Enhancement

Spatial audio coding and enhancement address the growing commercial need to store and distribute multichannel audio and to render content optimally on arbitrary reproduction systems. In this paper, we discuss a spatial analysis-synthesis scheme which applies principal component analysis to an STFT-domain representation of the original audio to separate it into primary and ambient components, which are then respectively analyzed for cues that describe the spatial percept of the audio scene on a per-tile basis; these cues are used by the synthesis to render the audio appropriately on the available playback system. The proposed framework can be tailored for robust spatial audio coding, or it can be applied directly to enhancement scenarios where there are no rate constraints on the intermediate spatial data and audio representation.

Audio segmentation by feature-space clustering using linear discriminant analysis and dynamic programming

We consider the problem of segmenting an audio signal into characteristic regions based on feature-set similarities. In the proposed method, a feature-space representation of the signal is generated; then, sequences of feature-space samples are aggregated into clusters corresponding to distinct signal regions. The clustering of feature sets is improved via linear discriminant analysis (LDA); dynamic programming (DP) is used to derive optimal cluster boundaries. The method avoids the heuristics employed in various feature-space segmentation schemes and is able to derive an optimal segmentation once the LDA and DP cost metrics have been chosen. We demonstrate that the method outperforms typical feature-space approaches described in the literature. We focus on an illustrative example of the basic segmentation task; however, by judicious design of the feature set, the training set, and the dynamic program, the method can be tailored for various applications such as speech/music discrimination, segmentation of audio streams for smart transport, or song structure analysis for thumbnailing.

Geometric signal decompositions for spatial audio enhancement

Decomposition of audio signals into primary and ambient components is useful for realizing spatial enhancements such as upmix and stereo widening. In this paper, we present several methods for primary-ambient decomposition of two-channel audio signals based on signal-space geometry. We discuss the performance of the various methods with respect to target orthogonality conditions on the estimated primary and ambient components, which cannot all be satisfied due to the need to constrain the model components to the signal subspace in light of limitations on implementation complexity.

Multiscale overlap-add sinusoidal modeling using matching pursuit and refinements

Sinusoidal models have proven useful for speech and audio analysis, modification, synthesis, and compression. We describe an adaptive multiscale overlap-add sinusoidal model based on the matching pursuit algorithm and various refinements such as conjugate pursuit and orthogonal subspace projection. The model is useful for scalable audio coding and signal modification.

A dynamic programming approach to audio segmentation and speech/music discrimination

We consider the problem of segmenting an audio signal into characteristic regions based on feature-set similarities. In the proposed approach, a feature-space representation of the signal is generated; sequences of these feature-space samples are then aggregated into clusters corresponding to distinct signal regions. The algorithm consists of using linear discriminant analysis (LDA) to condition the feature space and dynamic programming (DP) to identify data clusters. We consider the design of the dynamic program cost functions; we are able to derive effective cost functions without relying on significant prior information about the structure of the expected data clusters. We demonstrate the application of the LDA-DP segmentation algorithm to speech/music discrimination. Experimental results are given and discussed.

The STFT, Sinusoidal Models, and Speech Modification

Frequency-domain signal representations are used for a wide variety of applications in speech processing. In this Chapter, we first consider the short-time Fourier transform (STFT), presenting a number of interpretations of the analysis-synthesis process in a consistent mathematical framework. We then develop the sinusoidal model as a parametric extension of the STFT wherein the data in the STFT is compacted, sacrificing perfect reconstruction at the benefit of achieving a sparser and essentially more meaningful representation. We discuss several methods for sinusoidal parameter estimation and signal reconstruction, and present a detailed treatment of a matching pursuit algorithm for sinusoidal modeling. The final part of the Chapter addresses speech modifications such as filtering, enhancement, and time-scaling, for which both the STFT and the sinusoidal model are effective tools.

Multichannel Matching Pursuit and Applications to Spatial Audio Coding

Sparse signal modeling has been considered widely in the literature. In this paper, we discuss an extension of the matching pursuit sparse modeling algorithm to the case of simultaneously approximating multiple data signals; we outline the algorithm for general and for sinusoidal dictionaries. We then apply multichannel sinusoidal pursuit (M-SP) to spatial audio coding (SAC). In most SAC schemes, multichannel audio is coded by forming a downmix signal, compressing the down- mix with a legacy coder, and adding side information about spatial properties of the input audio. In the proposed M-SP system, a multichannel model of the input is used to derive the spatial information as well as a parametric model of an appropriate downmix signal. This joint spatial-parametric approach provides a different multichannel audio coding paradigm than that of previously described SAC methods.

Enhanced Microphone-Array Beamforming Based on Frequency-Domain Spatial Analysis-Synthesis

Distant-talking hands-free communication is hindered by reverberation and interference from unwanted sound sources. Microphone arrays have been used to improve speech reception in adverse environments, but small arrays based on linear processing such as delay-sum beamforming allow for only limited improvement due to low directionality and high-level sidelobes. In this paper, we propose a beamforming system that improves the spatial selectivity by forming multiple steered beams and carrying out a spatial analysis of the acoustic scene. The analysis derives a time-frequency mask that, when applied to a reference look-direction beam, enhances target sources and improves rejection of interferers that are outside of the specified target region. The performance of the system is demonstrated by simulations and audio examples.

Predicting and preventing unmasking incurred in coded audio post-processing

In modern audio compression algorithms, the masking properties of the auditory system are exploited to improve the coding gain, namely, quantization noise is introduced in the signal in time-frequency regions where it will be masked. However, since signal modifications change the characteristics of the masking regions, degradations may result if a decoded audio signal is modified. In this paper, we explain and demonstrate how modifying an audio signal can result in the unmasking of signal components that were imperceptible in the unmodified signal. We consider both pitch-shifting and linear filtering modifications; synthetic and natural audio examples are provided to verify the unmasking phenomenon. We discuss how modification of decoded audio may lead to unmasking of quantization noise, describe conditions for which such unmasking may occur, and propose a method for adjusting the masking threshold employed in the audio coder to make the decoded signal robust to quantization noise unmasking for a given set of signal modifications.

Efficient Designs for Broadband Linear Arrays

Linear electroacoustic arrays are useful for various applications in audio acquisition and reproduction. For robust broadband performance, a filter network is typically incorporated to achieve roughly frequency-invariant beamforming. In this paper, we are concerned with the design of cost-effective broadband arrays of limited size for consumer audio applications. Filter-based broadband beamformers are suboptimal for such applications due to high cost and the large number of elements generally needed to achieve invariance. Here, three alternative design methods are presented which lead to effective far-field broadband performance of small arrays at low cost: allpass weighting, optimal nonuniform spacing, and optimal delay dispersion. Performance improvements are demonstrated for each method and various design tradeoffs are considered