Yen-Chi Lee

Encoder-assisted adaptive video frame interpolation

In low bandwidth video coding applications, frame rate is reduced to increase the spatial quality of the frames. However, video sequences that are encoded at low frame rates demonstrate motion jerkiness artifacts when displayed. Therefore, a mechanism is required at the decoder to increase the frame rate while keeping an acceptable level of spatial quality. In this paper, we present a new method to perform video frame interpolation by sending effective side information for frame rate up conversion applications. The proposed scheme encodes the skipped frames lightly by sending motion vectors and an important information map which indicates to the decoder the type of interpolation method to perform. We also propose a novel overhead reduction method to keep the side information cost low. Experimental results show that the proposed algorithm outperforms decoder-only frame rate up conversion methods and gives better performance in terms of PSNR and visual quality over encoding at full frame rate without frame skipping.

Coordinated application of multiple description scalar quantization and error concealment for error-resilient MPEG video streaming

Historically, multiple description coding (MDC) and postprocessing error concealment (ECN) algorithms have evolved separately. In this paper, we propose a coordinated application of multiple description scalar quantizers (MDSQ) and ECN, where the smoothness of the video signal helps to compensate for the loss of descriptions. In particular, we perform a reconstruction that is consistent with the data received at the decoder. When only a single description is available, the video is reconstructed in such a way that: 1) if we were to regenerate two descriptions (from the reconstructed video), one of them would be equivalent to the received description and 2) the reconstructed video is spatiotemporally smooth. Experimental results with several video sequences demonstrated a peak signal-to-noise ratio (PSNR) improvement of 0.9-2.8 dB for intracoded frames. The PSNR improvements for intercoded frames were negligible. However, for both cases, the visual improvements were much more striking than what the PSNR improvement suggested.

Efficient video data recovery for 3G-324M telephony over WCDMA networks

In this paper, we propose a novel cross-layer decoder design that efficiently recovers more video data in the presence of transmission errors for 3G-324M video telephony over WCDMA networks. The proposed method requires collaboration of the video decoder and multiplexing protocol. In particular, it considers loss characteristics of WCDMA channels and modifies demultiplexer to provide additional information to assist video decoder to locate the error position. Then, the video decoder performs error concealment only on the lost macroblocks (MBs), instead of all the MBs in a video slice. The simulation results show that the proposed method can reduce the number of concealed MBs by 22-27% and improve video quality by 0.3-0.7 dB, depending on the video contents and block error rates.

Low-complexity temporal error concealment by motion vector processing for mobile video applications

In this paper, we propose a novel motion vector processing (MVP) approach for temporal error concealment (TEC) in mobile video applications. Most existing TEC techniques estimate the lost motion vectors by minimizing a given distortion such as boundary variation or neighbor matching in pixel domain, which requires high power consumption compared to approaches that do not use pixel information. In wireless video applications, power consumption is a crucial factor and adoption of video postprocessing algorithms by mobile devices depends greatly on the complexity of the algorithms. The proposed algorithm uses only the received motion vectors without pixel data in the concealment process which achieves concealed video quality comparable to that of pixel domain approach but at much lower computational complexity. The performance gain is achieved by using new motion vector processing approaches that include (i) frame-to- frame motion detection, (ii) local motion classification and (iii) motion trajectory tracking. Furthermore, the proposed MVP-TEC approach provides better video quality compared to other TEC methods that do not perform motion analysis.