Xinglei Zhu

Real-Time Signal Estimation From Modified Short-Time Fourier Transform Magnitude Spectra

An algorithm for estimating signals from short-time magnitude spectra is introduced offering a significant improvement in quality and efficiency over current methods. The key issue is how to invert a sequence of overlapping magnitude spectra (a ldquospectrogramrdquo) containing no phase information to generate a real-valued signal free of audible artifacts. Also important is that the algorithm performs in real-time, both structurally and computationally. In the context of spectrogram inversion, structurally real-time means that the audio signal at any given point in time only depends on transform frames at local or prior points in time. Computationally, real-time means that the algorithm is efficient enough to run in less time than the reconstructed audio takes to play on the available hardware. The spectrogram inversion algorithm is parameterized to allow tradeoffs between computational demands and the quality of the signal reconstruction. The algorithm is applied to audio time-scale and pitch modification and compared to classical algorithms for these tasks on a variety of signal types including both monophonic and polyphonic audio signals such as speech and music.

Constructing Secure Content-Dependent Watermarking Scheme using Homomorphic Encryption

Content-dependent watermarking (CDWM) has been proposed as a solution to overcome the potential estimation attack aiming to recover and remove the watermark from the host signal. It has also been used for the application of content authentication. In this work, we first present an analysis on why some prior work on CDWM pose potential security problems due to their inherent cryptographic weakness. With the aim of achieving cryptographic level of security, we then propose a novel CDWM scheme based on homomorphic encryption and dirty paper precoding. The general idea is to introduce a decryption module before watermark detection to create some nonlinearity and thereby inhibit conventional watermark attacks based on linear operations. We conclude this paper by bringing up some thoughts on the integration of watermarking and cryptography.

Real-Time Iterative Spectrum Inversion with Look-Ahead

In this paper, we present an algorithm for real-time iterative spectrogram inversion (RTISI) with look-ahead (RTISI-LA). RTISI-LA reconstructs a time-domain signal from a given sequence of short-time Fourier transform magnitude (STFTM) spectra without phase information. Whereas RTISI reconstructs the current frame using only magnitude spectra information for previous frames and the current frame, RTISI-LA also uses magnitude spectra for a small number future frames. This allows RTISI-LA to achieve substantially higher signal-to-noise (SNR) performance than either RTISI or the Griffin & Lim method with an equivalent computational load, while retaining the real-time properties of RTISI

Application of a content-based percussive sound synthesizer to packet loss recovery in music streaming

This paper presents a novel method to recover lost packets in music streaming using a synthesizer to generate percussive sounds. As an improvement of the state-of-the-art system that uses a content-based audio codebook, the new method can greatly reduce the redundant information needed to recover perceptually critical lost packets.

Contemporary ubiquitous media services: Content recommendation and adaptation

Recent rapid advancement in both pervasive mobile computing and cloud service has been driving numerous new applications in the ubiquitous media consumption. To provide mobile user with richer experiences, it is desired for the service providers to offer highly relevant recommendations of media content to the users by exploring user-user, user-media and user-context relationships. Furthermore, contemporary media services also need to address potential presentation mismatch between high quality high resolution media data in the cloud data centers and low resolution terminal capability of the ubiquitous mobile users. This paper attempts to examine several relevant recent developments in ubiquitous media services, especially in the area of content recommendation and user centric content adaptation. Based on current technology trends, we envisage the bright future of ubiquitous media services as pervasive mobile computing and cloud service advance to next level of technology penetration.

A collusion resilient key management scheme for multi-dimensional scalable media access control

This paper proposes a novel key management scheme for multi-dimensional scalable multimedia access control. We build up a collusion-attack model and prove that the proposed scheme is indeed resilient to collusion attack. The lower bound of number of the segment keys is also given in the analysis. We consider the key management problem on partially order set and propose a general approach consists of a poset decomposition step and an element projection step. Under the proposed framework, a novel hierarchical 3D poset decomposition approach is further developed. The proposed scheme is the first provable collusion resilient scheme in 3-dimension (and higher) scenario. It can be easily adapted to other scalable dependence structure and higher dimension. Analysis on the proposed scheme proves its collusion-resilient property.

A joint layered scheme for reliable and secure mobile JPEG-2000 streaming

This article presents a novel joint layered approach to simultaneously achieve both reliable and secure mobile JPEG-2000 image streaming. With a priori knowledge of JPEG-2000 source coding and channel coding, the proposed joint system integrates authentication into the media error protection components to ensure that every source-decodable media unit is authenticated. By such a dedicated design, the proposed scheme protects both compressed JPEG-2000 codestream and the authentication data from wireless channel impairments. It is fundamentally different from many existing systems that consider the problem of media authentication separately from the other operations in the media transmission system. By utilizing the contextual relationship, such as coding dependency and content importance between media slices for authentication hash appending, the proposed scheme generates an extremely low authentication overhead. Under this joint layered coding framework, an optimal rate allocation algorithm for source coding, channel coding, and media authentication is developed to guarantee end-to-end media quality. Experiment results on JPEG-2000 images validate the proposed scheme and demonstrate that the performance of the proposed scheme is approaching its upper bound, in which case no authentication is applied to the media stream.

Flexible Layered Authentication Graph for Multimedia Streaming

In this paper, a new flexible layered authentication graph (FLAG) algorithm is proposed for multimedia streaming authentication. While maximizing the verification probability by avoiding authentication path overlapping, this algorithm allows flexible communication overhead in terms of the number of hash links, as well as flexible authentication group size. These flexibilities make FLAG an excellent candidate for multimedia streaming authentication, in that (i) in the sender buffering mode, it allows elastic sending delay required by multimedia streaming congestion control; (ii) in the receiver buffering mode, it facilitates adaptation to effective network bandwidth; (iii) it also has the potential to provide unequal authentication protection (UAP), which is a natural solution for multimedia code stream. Our analysis and experiment results further confirm the validity of our algorithm.

Exploiting Concept Association to Boost Multimedia Semantic Concept Detection

In the paper we study the efficiency of semantic concept association in multimedia semantic concept detection. We present an approach to automatically learn from the corpus the association strength between pair-wise semantic concepts. We discuss two usages of association strength: 1) applying positive concepts with high association strength for selecting expressive component in the model-based fusion and 2) applying negative concepts with low association strength as filters. We evaluate its efficiency on the task of semantic concept detection on the large-scale news video dataset from TRECVID 2005 development set. Our experimental results demonstrate that exploiting positive association reduces the size of feature dimension in the model-based fusion and significantly improves the rank performance of system. The mean average precision is increased to 0.215 on the validation set and 0.206 on the evaluation set. Compared to the traditional model-based fusion, the improvement is about 9.1% and 3.5%, respectively. The average feature dimension is reduced to 43 from 312.

A Joint Source-Channel Adaptive Scheme for Wireless H.264/AVC Video Authentication

Authentication has become an emerging issue for video streaming over lossy networks. Although the advanced video coding standards, such as H.264/AVC, efficiently reduce the amount of data to be transmitted, the coding dependency brings new challenges in designing efficient stream authentication scheme. In this paper, we propose a novel joint-designed-layered source-channel adaptive scheme that integrates authentication into source and channel coding components to sufficiently use the related information to efficiently address the coding dependency and to design the optimal rate allocation scheme for the sake of end-to-end video quality. The proposed layered framework is able to minimize end-to-end quality degradation incurred by both the wireless channel noise and the authentication failure. In particular, the competing requirements of high verification probability and low authentication overhead are concurrently satisfied by the elegant design of layered hash appending with efficient adaptation to the H.264 source coding and channel conditions. A joint source-channel-authentication rate allocation scheme is then developed to achieve optimal end-to-end video quality. The experimental results on H.264 video sequences confirm the efficacy of this joint adaptive scheme and demonstrate that it indeed outperforms the state-of-the-art graph-based authentication algorithms.