Michael Tok

A block-adaptive skip mode for inter prediction based on parametric motion models

Motion compensated prediction (MCP) in hybrid video coding estimates a translational motion vector for a given block which is then used for residual computation. However, when complex motion like zoom, rotation, and perspective transformation occur, the translational model assumption does not always hold. This may result in higher residual energy and splitting of blocks, respectively. This paper proposes a skip mode based on higher-order parametric motion models. Often, these models provide a better prediction quality resulting in lower residual energy and larger block sizes. The proposed technique estimates a higher-order motion model between two given pictures. The encoder decides in terms of rate-distortion optimization whether to use the new skip mode for a block and therefore not to transfer any additional information like coefficient data. Experimental evaluation shows that the proposed technique can improve the coding performance of next generation video coding standards significantly.

Adaptive Global Motion Temporal Filtering for High Efficiency Video Coding

Coding artifacts in video codecs can be reduced using several spatial in-loop filters that are part of the emerging video coding standard High Efficiency Video Coding (HEVC). In this paper, we introduce the concept of global motion temporal filtering. A theoretical framework for a concept combining the temporal overlapping of several noisy versions of the same signal is introduced. This includes a model of the motion estimation error. As an important result, it is shown that an optimum number of frames N for filtering exists. An implementation of the concept based on several versions of the HEVC test model using global motion-compensated temporal filtering shows that significant gains can be achieved.

Feature-based global motion estimation using the Helmholtz principle

Global motion estimation is an important task for various video processing techniques. The estimation itself has to be robust in presence of arbitrarily moving foreground objects. For that task, two different kinds of estimation methods exist. On the one hand, pixel-based approaches deliver more precise results and work more robust on video sequences with foreground objects. On the other hand, when working on encoded video streams, block-based methods can be used for a much faster but often less precise estimation. We propose a two step estimation method based on the determination and tracking of feature points of video frames and robust motion model estimation using the Helmholtz principle. Therefore, good trackable features are detected and tracked in video sequences. Subsequently, a perspective motion model is derived from the resulting correspondencies by removing feature pairs not belonging to global motion.

Compressed domain global motion estimation using the Helmholtz Tradeoff Estimator

Several algorithms for global motion estimation in video sequences using pixel- or block-based approaches have been published. Most known pixel-based methods lack in performance while when using block-based algorithms working on motion vectors, robustness to outliers and accuracy is missing. In this paper we present the fundamentals of a significantly improved, robust block-based method for global motion estimation in compressed domain following the generic Helmholtz principle. To this aim, we use motion vector fields as provided by MPEG data streams. Background PSNR values for four motion compensated test sequences show that our new method delivers results comparable to more complex algorithms.

Monte-Carlo-Based Parametric Motion Estimation Using a Hybrid Model Approach

Parametric motion estimation is an important task for various video processing applications, such as analysis, segmentation, and coding. The process for such an estimation has to satisfy three requirements. It has to be fast, accurate, and robust in the presence of arbitrarily moving foreground objects. We introduce a two-step simplification scheme, suitable for Monte-Carlo-based perspective motion model estimation. For complexity reduction, the Helmholtz tradeoff estimator as well as random sample consensus are enhanced with this scheme and applied on Kanade–Lucas–Tomasi features as well as on video stream macroblock motion vector fields. For the feature-based estimation, good trackable features are detected and tracked on raw video sequences. For the block-based approach, motion vector fields from encoded H.264/AVC video streams are used. Results indicate that the complexity of the whole estimation process can be reduced by a factor of up to 10000 compared to state-of-the-art methods without losing estimation precision.

Motion-based object segmentation using hysteresis and bidirectional inter-frame change detection in sequences with moving camera

We present an unsupervised motion-based object segmentation algorithm for video sequences with moving camera, employing bidirectional inter-frame change detection. For every frame, two error frames are generated using motion compensation. They are combined and a segmentation algorithm based on thresholding is applied. We employ a simple and effective error fusion scheme and consider spatial error localization in the thresholding step. We find the optimal weights for the weighted mean thresholding algorithm that enables unsupervised robust moving object segmentation. Further, a post processing step for improving the temporal consistency of the segmentation masks is incorporated and thus we achieve improved performance compared to the previously proposed methods. The experimental evaluation and comparison with other methods demonstrate the validity of the proposed method.

Parametric motion vector prediction for hybrid video coding

Motion compensated prediction still is the main technique for redundancy reduction in modern hybrid video codecs. However, the resulting motion vector fields are highly redundant as well. Thus, motion vector prediction and difference coding are used for compressing. One drawback of all common motion vector prediction techniques is, that they are not able to predict complex motion as rotation and zoom efficiently. We present a novel parametric motion vector predictor (PMVP), based on higher-order motion models to overcome this issue. To transmit the needed motion models, an efficient compression scheme is utilized. This scheme is based on transformation, quantization and difference coding. By incorporating this predictor into the HEVC test model HM 3.2 gains of up to 2.42% are achieved.

Short-term motion-based object segmentation

Motion-based segmentation approaches employ either longterm motion information, which is computationally demanding, or suffer from lack of accuracy when employing short-term information. We present an automatic motion-based object segmentation algorithm for video sequences with moving camera, employing short-term motion information solely. For every frame, two error frames are generated using motion compensation. They are combined and a thresholding segmentation algorithm is applied. Recent advances in the field of global motion estimation enable outlier elimination in the background area, and thus a more precise definition of the foreground is achieved. We propose a simple and effective error frame generation and consider spatial error localization. Thus, we achieve improved performance compared with a previously proposed short-term motion-based method and provide subjective as well as objective evaluation.

Motion modeling for motion vector coding in HEVC

During the standardization of HEVC, new motion information coding and prediction schemes such as temporal motion vector prediction have been investigated to reduce the spatial redundancy of motion vector fields used for motion compensated inter prediction. In this paper a general motion model based vector coding scheme is introduced. This scheme includes estimation, coding and dynamic recombination of parametric motion models to generate vector predictors and merge candidates for all common HEVC inter coding settings. Bit rate reductions of up to 4.9% indicate that higher order motion models can increase the efficiency of motion information coding in modern hybrid video coding standards.

A Parametric Merge Candidate for High Efficiency Video Coding

Block based motion compensated prediction still is the main technique used for temporal redundancy reduction in modern hybrid video codecs. However, the resulting motion vector fields are highly redundant as well. So, motion vector prediction and difference coding are used to compress such vector fields. A drawback of common motion vector prediction techniques is their inability to predict complex motion such as rotation and zoom in an efficient way. We present a novel Merge candidate for improving already existing vector prediction techniques based on higher order motion models to overcome this issue. To transmit the needed models, an efficient compression scheme is utilized. The improvement results in bit rate savings of 1.7% in average and up to 4% respectively.