Wendong Huang

Content-based UEP: a new scheme for packet loss recovery in music streaming

Bandwidth efficiency and error robustness are two essential and conflicting requirements for streaming media content over error-prone channels, such as wireless channels. This paper describes a new scheme called content-based unequal error protection (C-UEP), which aims to improve the user-perceived QoS in the case of packet loss. We use music streaming as an example to show the effectiveness of the new concept. C-UEP requires only a small fraction of the redundancy used in existing forward error correction (FEC) methods. C-UEP classifies every audio segment (e.g. an encoding frame) into different classes to improve encoding efficiency. Salient transients such as drumbeats and note onsets are encoded with more redundancy in a secondary bitstream used to recover lost packets by the receiver. Formal perceptual evaluations show that our scheme improves audio quality significantly over simple muting and packet repetition baselines. This improvement is achieved with a negligible amount of redundancy, which is transmitted to the receiver ahead of playback.

A framework for robust and scalable audio streaming

We propose a framework to achieve bandwidth efficient, error robust and bitrate scalable audio streaming. Our approach is compatible with most audio compression format. The main contributions of this paper include: 1) the proposal of a Multi-Stage Interleaving (MSI) strategy which translates packet loss into loss of separate frequency components that are less perceptually significant; and 2) the design of a Layered Unequal-Sized Packetization (LUSP) scheme which enables bitrate scalability and prioritized packet transmission. The combination of the proposed MSI and LUSP allows the use of a set of simple yet effective methods of error concealment in the compressed domain. Our approach offers significant advantages over existing methods in terms of memory consumption (a savings of over 40 times in the sample MP3 implementation), and computational complexity, which are critical issues for battery-powered small devices.

An optimal speed control scheme supported by media servers for low-power multimedia applications

In this paper, we present a new concept of dynamic voltage scaling (DVS) for low-power multimedia decoding in battery-powered mobile devices. Most existing DVS techniques are suboptimal in achieving energy efficiency while providing the guaranteed playback quality of service, which is mainly due to the inherent limitations of client-only approaches. To address this problem, in this paper, we investigate the possibility of media server supported DVS techniques with smoothing mechanisms. Towards this new direction, we propose a generic offline bitstream analysis framework and an optimal speed control algorithm which achieves the maximal energy savings among all feasible speed profiles for the given buffers. The proposed scheme enables us to compute the buffer sizes of feasibility condition, which are the theoretical lower bound of buffer size requirement for a given media clip. More importantly, our scheme facilitates practical applications from four aspects. First, it does not require feedback information on clients’ configuration. This renders our scheme particularly suitable for broadcast or multicast applications. Second, the speed profile based on buffer sizes of feasibility condition can provide satisfactory energy efficiency. Third, the required buffer sizes are so small that they can be met by most mobile devices. Fourth, additional side information (i.e., speed profile) of the proposed scheme is negligible compared to the size of media content. These properties solve the diversity issue and feasibility issue of media server supported DVS schemes. Experimental results show that, in comparison with the representative existing techniques, our scheme improves the performance of DVS significantly.

A perception-aware low-power software audio decoder for portable devices

We propose a new software audio decoder for processors supporting multiple discrete voltage-frequency operating points. The proposed decoding scheme allows the user to switch between multiple output quality levels, where each level is associated with a different rate at which the processor consumes energy. This is an attractive feature in battery-powered portable audio players and mobile phones, where battery-life is often more crucial than the output quality, especially in noisy environments. Towards this, the frequency range of the decoder is partitioned into multiple groups, in accordance with their perceptual relevance. When a longer battery life is desired, only the most relevant frequency components are decoded, which allows the processor to be run at a lower voltage and frequency. We have implemented this scheme using the MP3 decoder and obtained up to 95% savings in the energy consumed by the processor for AM quality output (in contrast to CD quality output, which is associated with the maximum energy consumption). This scheme is easy to implement, has no runtime overhead and does not involve any runtime voltage or frequency scaling.

Power-aware bandwidth and stereo-image scalable audio decoding

We propose a new workload-scalable audio decoding scheme that would enable users to control the tradeoff between playback quality and power consumption in battery-powered portable audio players. Our objective is to give users a control at the decoder side, similar to the Long Play (LP) recording mode at the encoder side in many media recording devices. The main contribution of this paper is a proposal for a Bandwidth and Stereo-image Scalable (BSS) decoding scheme for single-layer audio formats such as MP3. The proposed scheme is based on an analysis of the perceptual relevance of different audio components in the compressed bitstream. The bandwidth and stereo-image scalability directly translates into scalability in terms of the computational workload generated by the decoder. This can be exploited by a voltage/frequency scalable processor to save energy and prolong the battery life.

A joint encoder–decoder framework for supporting energy efficient audio decoding

In comparison with the relatively slow progress of battery technology, semiconductor memory has improved much more rapidly, making storage a less critical limiting factor in designing low power embedded systems such as PDAs. To exploit such technology trends, we present a novel framework, a joint encoder–decoder framework (JEDF), which allows the decoder to tradeoff energy and memory consumption without sacrificing playback quality. We employ sum-of-powers-of-two (SOPOT) technique, an approximate signal processing (ASP) technique, in an MPEG AAC decoder to reduce the computational workload. The SOPOT introduces additional ASP noise (in the decoder) on top of the quantization noise introduced in the process of lossy compression (in the encoder). The sum of these two kinds of noise may become audible when it exceeds the masking threshold. We tackle this problem from a new perspective: the proposed JEDF allows the ASP and quantization noises to be shaped jointly to match the masking threshold. In the case that the perceptual room between the masking threshold and the quantization noise is insufficient for the ASP noise, the JEDF can reduce the quantization noise level which results in an increase in bitrate. To implement the proposed scheme, we have developed two new techniques: (1) SOPOT truncation noise shaping; (2) truncation noise allocation based on a perceptual model. Experimental results show the effectiveness of our approach.

A method for separating drum objects from polyphonic musical signals

An additional coding of auditory objects for packet loss concealment has been proven to be effective in music streaming applications. This paper describes a new extension to our previous method in separating drum objects from polyphonic music signals with improved performance. After a simple time domain separation method employed in our early system, we propose in this paper a novel frequency domain technique, a tonal-components tracking and attenuation (TTA), to suppress quasi-stationary auditory objects such as singing voice in the separated drum objects. Experimental results show that the new method is an effective pre-processing step to separate drum objects from polyphonic music signals. This method helps to improve accuracy of drum clustering and to mitigate the pitch and harmonic structure mismatch problem when applied in packet loss recovery in music streaming.

Efficient Partial Spectrum Reconstruction using an Asymmetric PQMF Algorithm for MPEG-Coded Stereo Audio

This paper presents a novel algorithm of a scalable and efficient pseudo-quadrature mirror filters (PQMF), which is employed for partial decoding a single-layer audio bitstream such as MP3, typically coded in joint/MS mode. The proposed algorithm is a new extension to our previous work on scalable audio decoding and is designed for asymmetric partial spectrum reconstruction (APSR), where perceptually irrelevant computations are removed. Furthermore, an efficient up-sampling operation is introduced for right channel output. The slight distortions introduced by our simple up-sampling method are inaudible according to a set of perceptual evaluations. Simulation results show that 64.6% energy savings can be achieved for a typical configuration in comparison to the standard PQMF algorithm employed by MPEG-1 audio