Wilson Kwok

Concealment of damaged block transform coded images using projections onto convex sets

An algorithm for lost signal restoration in block-based still image and video sequence coding is presented. Problems arising from imperfect transmission of block-coded images result in lost blocks. The resulting image is flawed by the absence of square pixel regions that are notably perceived by human vision, even in real-time video sequences. Error concealment is aimed at masking the effect of missing blocks by use of temporal or spatial interpolation to create a subjectively acceptable approximation to the true error-free image. This paper presents a spatial interpolation algorithm that addresses concealment of lost image blocks using only intra-frame information. It attempts to utilize spatially correlated edge information from a large local neighborhood of surrounding pixels to restore missing blocks. The algorithm is a Gerchberg (1974) type spatial domain/spectral domain constraint-satisfying iterative process, and may be viewed as an alternating projections onto convex sets method.

Multi-directional interpolation for spatial error concealment

An algorithm using multidirectional interpolation is presented for error concealment. In the algorithm, an edge classifier analyzes the values of pels in the blocks surrounding the missing block and determines which edge directions cut through the missing block. Interpolations are performed on a local pel neighborhood along the directions specified by the edge classifier. Then a mixing operation is used to restore the missing block by extracting the features obtained from the different directional interpolations and combining them together. The method of multidirectional interpolation and image mixing has demonstrated very good results when a sufficiently large neighborhood of correlated pels exists. This method of spatial interpolation can be combined with temporal interpolation to provide for a powerful error concealment technique for compressed video signal transmission. >

Error concealment algorithms for robust decoding of MPEG compressed video

Abstract This paper provides some research results on the topic of error resilience for robust decoding of MPEG (Moving Picture Experts Group) compressed video. It introduces and characterizes the performance of a general class of error concealment algorithms. Such receiver-based error concealment techniques are essential for many practical video transmission scenarios such as terrestrial HDTV broadcasting, packet network based teleconferencing/multimedia, and digital SDTV/HDTV delivery via ATM (asynchronous transfer mode). Error concealment is intended to ameliorate the impact of channel impairments (i.e., bit-errors in noisy channels, or cell-loss in ATM networks) by utilizing available picture redundancy to provide a subjectively acceptable rendition of affected picture regions. The concealment process must be supported by an appropriate transport format which helps to identify image pixel regions which correspond to lost or damaged data. Once the image regions (i.e., macroblocks, slices, etc.) to be concealed are identified, a combination of spatial and temporal replacement techniques may be applied to fill in lost picture elements. A specific class of spatio-temporal error concealment algorithms for MPEG video is described and alternative realizations are compared via detailed end-to-end simulations for both one- or two-tier transmission media. Several algorithm enhancements based on directional interpolation, ‘I-picture motion vectors’, and use of MPEG-2 ‘scalability’ features are also presented. In each case, achievable performance improvements are estimated via simulation. Overall, these results demonstrate that the proposed class of error concealment algorithms provides significant robustness for MPEG video delivery in the presence of channel impairments, permitting useful operation at ATM cell-loss rates in the region of 10 −4 to 10 −3 and 10 −2 to 10 −1 for one- and two-tier transmission scenarios, respectively.

MPEG coding performance improvement by jointly optimizing coding mode decisions and rate control

This paper presents a new algorithm for determining the optimal MPEG coding strategy in terms of the selection of macroblock coding modes and quantizer scales. In the algorithm proposed in the test model the rate control operates independently from the coding mode selection for each macroblock. The coding mode is decided based only upon the energy of predictive residues. Actually, the two processes of coding mode decision and rate control are intimately related to each other and should be determined jointly in order to achieve optimal coding performance. We formulate the constrained optimization problem and present solutions based upon rate-distortion characteristics, or R(D) curves, for all the macroblocks that compose the picture being coded. Distortion for the entire picture is assumed to be decomposable and expressible as a function of individual macroblock distortions, with this being the objective function to minimize. The determination of the optimal solution is complicated by the MPEG differential encoding of motion vectors and DC coefficients, which introduce dependencies that carry over from macroblock to macroblock for a duration equal to the slice length. As an approximation, a near optimum greedy algorithm is proposed. Once the upper bound in performance is calculated, it can be used to assess how well practical suboptimum methods perform. Finally, such a practical suboptimum algorithm is proposed and evaluated.

Rate control for VBR MPEG video on local area networks

A rate control algorithm for delivery of digital video traffic on local area networks that results on uniform video quality during normal traffic conditions and graceful degradation during periods of congestion is proposed. The algorithm controls a dual rate-control-mode MPEG-2 encoder. This encoder operates in either variable bit-rate (VBR) or joint rate controlled VBR mode.

Architectures for MPEG compressed bitstream scaling

The Moving Picture Expert Group (MPEG) video coding standard has been proposed for a variety of applications for video transmission and storage. Several applications such as video on demand, trick-play track on digital VTRs and extended-play recording VTRs motivate the idea of bitstream scaling which intends to reduce the compressed bitstream size according to the outgoing channel capacity. We propose several methods for implementation of bitstream scaling. The different methods have varying hardware implementation complexity, with each having its own degree of tradeoff between required hardware and resulting image quality.

Joint rate control for VBR MPEG video on PVC ATM links

Abstract. A joint rate control algorithm for variable bit rate (VBR) MPEG-compressed digital video on point-to-point permanent virtual circuit (PVC) ATM links is proposed. The algorithm controls the encoding mode of a number of video encoders that operate either on VBR or constant bit rate (CBR) mode. The algorithm selects the encoding mode based on the buffer occupancy of a multiplexer co-located with the encoders that interfaces them to the PVC link. VBR encoding is the predominant encoding mode used during congestion-free periods. CBR mode is used only during congested periods. When CBR is selected, new bit rates are jointly assigned to the encoders based on their relative encoding complexities. The bit rate assigned to the encoders are enforced by a CBR rate control local to each encoder. The performance of the joint rate control algorithm is evaluated through simulation of a packet multiplexer, where a number of connections are multiplexed onto a fixed-capacity channel. The performance is compared to that of multiplexing conventional CBR and open-loop VBR from the video quality and bandwidth efficiency points of view. Simulation results show that the proposed algorithm improves performance over multiplexed conventional CBR or open-loop VBR MPEG video, without significantly increasing implementation complexity. The application of the algorithm to video-on-demand over ATM is discussed.

Temporal and spatial projection onto the convex set (POCS) based error concealment algorithm for the MPEG-encoded video sequence

The paper presents an algorithm attempting to combine both intra-frame and interframe information to reconstruct lost macroblocks due to imperfect communication channels when decoding a MPEG bitstream. The algorithm is a POCS-based (Projection Onto the Convex Set) iterative restoration algorithm incorporating both the temporal and spatial constraints derived from a set of analysis performed on the picture sequence. Often the use of the temporal information in the restoration process is complicated by the scene changes or large random motion activities. To reliably utilize the temporal information, we formulate a series of test to determine the usefulness of the temporal information. In addition, the tests yield a temporal constraint if the temporal information is deemed good. Along with the spatial constraints as described in [1], the temporal constraint is used in the proposed iterative restoration algorithm.