Trista Pei-chun Chen

Adaptive joint source-channel coding using rate shaping

We present in this paper an adaptive joint source-channel coding scheme using rate shaping on pre-coded video data. Rate shaping selectively drops portions of the video bitstream before transmitting them in order to satisfy the network bandwidth requirement. In wireless multimedia transport over heterogeneous networks, limited bandwidth is not the only issue. The high error rate of the channel should be considered as well, so channel coding is often applied. We propose a rate shaping method that drops not only the source-coding segments of the video bitstream, but also the channel-coding segments of the video bitstream, adaptively according to the network condition, in order to achieve the optimal rate-distortion performance. The proposed method is based on “discrete rate-distortion combination” to accomplish joint source-channel coding. We consider both the simulcast and multicast scenarios and show promising results.

Second-generation error concealment for video transport over error prone channels

Video transport over error prone channels may result in loss or erroneous decoding of the video. Error concealment is an effective mechanism to reconstruct the video content. In this paper, we review different error concealment methods and introduce a new framework, which we refer to as second-generation error concealment. All the error concealment methods reconstruct the lost video content by making use of some a priori knowledge about the video content. First-generation error concealment builds such a priori in a heuristic manner. The proposed second-generation error concealment builds the a priori by modeling the statistics of the video content. Context-based models are trained with the correctly decoded video content, and then used to replenish the lost video content. Trained models capture the statistics of the video content and thus reconstruct the lost video content better than reconstruction by heuristics.

Progressive image watermarking

Progressive transmission of images is very useful in many applications, especially in image transmission over the Internet. To view an image, people would want to see part of the image while the image transmission is in progress, rather than having to wait until the end of the image transmission. On the other hand, the ease of transmission and copying of images creates the need to use digital watermarking to embed the copyright information seamlessly into the media. We propose a progressive image watermarking scheme. In this scheme, the watermark is embedded in such a way that we can retrieve part of it even when the watermarked image is still being transmitted. As transmission progresses, the retrieved watermark has a decreasing bit error rate. Our proposed methods not only transmit the watermarked image progressively, but also intelligently select watermark embedding locations robust to various attacks.

A framework for optimal blind watermark detection

We propose a general framework for blind watermark detection. This framework contains a maximum-likelihood detector that utilizes the probability distribution of the original image. Other watermark detectors in literature are shown to be special cases of this framework. We demonstrate this framework in both the pixel domain and the transform domain, and show that our detector outperforms others because of 1) better modeling of the probability distribution of the original image, and 2) consideration to the human visual system in this framework.

Second-generation error concealment for video transport over error-prone channels

Video transport over error-prone channels may result in loss or erroneous decoding of the video data. Error concealment is an effective mechanism to reconstruct the video data. In this paper, we review error-concealment methods and introduce a new framework, which we refer to as second-generation error concealment. All the error-concealment methods reconstruct the lost video data by making use of certain a priori knowledge about the video content. First-generation error concealment builds such a priori in a heuristic manner. The proposed second-generation error concealment builds the a priori by modeling the statistics of the video content explicitly, typically in the region of interest (ROI). Context-based models are trained with the correctly received video data and then used to replenish the lost video data. Trained models capture the statistics of the video content and thus reconstruct the lost video data better than reconstruction by heuristics. A new dynamic model ‘updating principal components’ (UPC) is proposed as a model for second-generation error concealment. UPC can be applied to pixel values to conceal loss of pixel data. In addition, UPC can be applied to motion vectors, which results in ‘updating eigenflows’ (U-Eigenflow), to conceal loss of motion vectors. With UPC applied to both pixel values and motion vectors, hybrid temporal/spatial error concealment can be achieved. The proposed second-generation error-concealment method provides superior performances to first-generation error-concealment methods. Copyright

Error concealment aware rate shaping for wireless video transport

Streaming of video, which is both source- and channel-coded, over wireless networks faces the challenge of time-varying packet loss rate and fluctuating bandwidth. Rate shaping (RS) has been proposed to reduce the bit-rate of a precoded video bitstream to adapt to the real-time bandwidth variation. In our earlier work, rate shaping was extended to consider not only the bandwidth but also the packet loss rate variations. Rate-distortion optimized rate adaptation is performed on the precoded video that is a scalable coded bitstream protected by forward error correction codes. In this paper, we propose an RS scheme that further takes into account the error concealment (EC) method used at the receiver. We refer to this scheme as EC aware RS (ECARS). When performing ECARS, first ECARS needs to know the benefit/gain of sending each part of the precoded video, as opposed to not sending it but reconstructing it by EC. Then given a certain packet loss probability, the expected gain can be derived and be included in the rate-distortion optimization problem formulation. Finally, ECARS performs rate-distortion optimization to adapt the rate of the precoded video. A two-stage rate-distortion optimization approach is proposed to solve the ECARS rate-distortion optimization problem. In addition to ECARS, the precoding process can be EC aware to prioritize the precoded video based on the gain. We present an example EC aware precoding process by means of macroblock prioritization. Experiment results of ECARS together with EC aware precoding are shown to have excellent performance.

Shaping for video with frame dependency

Streaming of preceded video, which is both source- and channel-coded, over packet-loss networks faces challenges of the time-varying packet loss rate and fluctuating bandwidth. Rate shaping has been proposed to reduce the bit rate of a preceded video bitstream to adapt to the real-time bandwidth variation. In our earlier work, rate shaping was extended to consider not only the bandwidth but also the packet loss rate variations. In practice, the reconstructed result of the previous frame will affect the following frames if the video is predictive coded, and/or the error concealment method performed at the receiver utilizes temporal information. However, none prior work considers rate shaping for pre source- and channel-coded video with such frame dependency. In order to incorporate frame dependency into the rate shaping process, we propose to send the location (and mean) of the corrupted macroblock back to the sender, and use such feedback information to determine the distortion measure in the rate-distortion optimized rate shaping. Experiments have shown improved performance with the aids of the feedback information.

Life after Video Coding Standards: Rate Shaping and Error Concealment

Is there life after video coding standards? One might think that research has no room to advance with the video coding standards already defined. On the contrary, exciting research opportunities arise after the standards are specified. In this paper, we introduce two standard-related research areas: rate shaping and error concealment, as examples of interesting research that finds its context in standards. Experiment results are also shown.