Andrew Nesbit

Audio source separation with a signal-adaptive local cosine transform

Audio source separation is a very challenging problem, and many different approaches have been proposed in attempts to solve it. We consider the problem of separating sources from two-channel instantaneous audio mixtures. One approach to this is to transform the mixtures into the time-frequency domain to obtain approximately disjoint representations of the sources, and then separate the sources using time-frequency masking. We focus on demixing the sources by binary masking, and assume that the mixing parameters are known. In this paper, we investigate the application of cosine packet (CP) trees as a foundation for the transform. We determine an appropriate transform by applying a computationally efficient best basis algorithm to a set of possible local cosine bases organised in a tree structure. We develop a heuristically motivated cost function which maximises the energy of the transform coefficients associated with a particular source. Finally, we evaluate objectively our proposed transform method by comparing it against fixed-basis transforms such as the short-time Fourier transform (STFT) and modified discrete cosine transform (MDCT). Evaluation results indicate that our proposed transform method outperforms MDCT and is competitive with the STFT, and informal listening tests suggest that the proposed method exhibits less objectionable noise than the STFT.

Benchmarking flexible adaptive time-frequency transforms for underdetermined audio source separation

We have implemented several fast and flexible adaptive lapped orthogonal transform (LOT) schemes for underdetermined audio source separation. This is generally addressed by time-frequency masking, requiring the sources to be disjoint in the time-frequency domain. We have already shown that disjointness can be increased via adaptive dyadic LOTs. By taking inspiration from the windowing schemes used in many audio coding frameworks, we improve on earlier results in two ways. Firstly, we consider non-dyadic LOTs which match the time-varying signal structures better. Secondly, we allow for a greater range of overlapping window profiles to decrease window boundary artifacts. This new scheme is benchmarked through oracle evaluations, and is shown to decrease computation time by over an order of magnitude compared to using very general schemes, whilst maintaining high separation performance and flexible signal adaptivity. As the results demonstrate, this work may find practical applications in high fidelity audio source separation.

Extension of Sparse, Adaptive Signal Decompositions to Semi-blind Audio Source Separation

We apply sparse, fast and flexible adaptive lapped orthogonal transforms to underdetermined audio source separation using the time-frequency masking framework. This normally requires the sources to overlap as little as possible in the time-frequency plane. In this work, we apply our adaptive transform schemes to the semi-blind case, in which the mixing system is already known, but the sources are unknown. By assuming that exactly two sources are active at each time-frequency index, we determine both the adaptive transforms and the estimated source coefficients using ***1 norm minimisation. We show average performance of 12---13 dB SDR on speech and music mixtures, and show that the adaptive transform scheme offers improvements in the order of several tenths of a dB over transforms with constant block length. Comparison with previously studied upper bounds suggests that the potential for future improvements is significant.

Oracle estimation of adaptive cosine packet transforms for underdetermined audio source separation

We address the problem of instantaneous, underdetermined audio source separation by time-frequency masking. Using oracle estimators, we determine experimental upper performance bounds, by assuming that we have reference sources available, and that we know, or have estimated, the mixing structure. Oracle estimation of four musical sources from two-channel mixtures demonstrates a potential for SDR improvements of up to 12.7 dB, compared to semi-blind methods. We also show that using adaptive cosine packet transforms, rather than fixed-basis STFTs, can improve performance by up to 2.2 dB. Finally, by allowing more than one non-zero source coefficient per time-frequency index, improvements of up to 7.7 dB could be possible.

Audio analysis using sparse representations.

The method of “sparse representations,” based on the idea that observations should be represented by only a few items chosen from a large number of possible items, has emerged recently as an interesting approach to the analysis of images and audio. New theoretical advances and practical algorithms mean that the sparse representations approach is becoming a potentially powerful signal processing and analysis method. Some of the key concepts in sparse representations will be introduced, including algorithms to find sparse representations of data. An overview of some applications of sparse representations in audio will be described, including for automatic music transcription and audio source separation, and pointers will be given for possible future directions in this area. [This work has been supported by grants and studentships from the UK Engineering and Physical Sciences Research Council.]