Stephen James Mein

Non-rigid structure from motion with incremental shape prior

Most existing approaches to the non-rigid structure from motion problem use batch type algorithms with all the data collected before 3D shape reconstruction takes place. Such a methodology is not suitable for real-time applications. Concurrent on-line estimation of the camera position and 3D structure, based only on the measurements up to that moment is much more a challenging problem. In this paper, a novel approach is proposed for recursive recovery of non-rigid structures from image sequences captured by an orthographic camera. The main novelty in the proposed method is an adaptive algorithm for construction of shape constraints imposing stability on the on-line reconstructed shapes. The proposed, adaptively learned constraints have two aspects, consisting of constraints imposed on the basic shapes, the basic “building blocks” from which shapes are reconstructed, as well as constraints imposed on the mixing coefficients in a form of their probability distribution. The constraints are updated when the current model inadequately represents new shapes. This is achieved by means of Incremental Principal Component Analysis (IPCA). Results of the proposed method are shown on synthetic and real data of articulated face.

Fast prediction algorithm for multiview video coding

Abstract. The H.264/multiview video coding (MVC) standard has been developed to enable efficient coding for three-dimensional and multiple viewpoint video sequences. The inter-view statistical dependencies are utilized and an inter-view prediction is employed to provide more efficient coding; however, this increases the overall encoding complexity. Motion homogeneity is exploited here to selectively enable inter-view prediction, and to reduce complexity in the motion estimation (ME) and the mode selection processes. This has been accomplished by defining situations that relate macro-blocks’ motion characteristics to the mode selection and the inter-view prediction processes. When comparing the proposed algorithm to the H.264/MVC reference software and other recent work, the experimental results demonstrate a significant reduction in ME time while maintaining similar rate-distortion performance.

Fast Adaptive Hierarchical Prediction Algorithm for H.264/AVC Scalable Video Coding

The objective of scalable video coding (SVC) is to enable the generation of a unique bitstream that can adapt to various bit-rates, transmission channels and display capabilities. The scalability is categorised in terms of temporal, spatial, and quality. In order to improve encoding efficiency, the SVC scheme incorporates inter-layer prediction mechanisms to complement the H.264/AVC very refined Motion Estimation (ME) and mode decision processes. However, this further increases the overall encoding complexity of the scalable coding standard. In this paper several conditional probabilities are established relating motion estimation characteristics and the mode distribution at different layers of the H.264/SVC. An evaluation of these probabilities is used to structure a low-complexity prediction algorithm for Group of Pictures (GOP) in H.264/SVC, reducing computational complexity whilst maintaining similar RD performance. When compared to the JSVM software, the proposed algorithm achieves a significant reduction of encoding time, with a negligible average PSNR loss and bit-rate increase in temporal, spatial and SNR scalability. Experiments are conducted to provide a comparison between our method and recently developed fast mode selection algorithms. These demonstrate the proposed method achieves appreciable time savings for scalable spatial and scalable quality video coding, while maintaining similar PSNR and bit rate.

Fast mode decision for the H.264/AVC video coding standard based on frequency domain motion estimation

The H.264 video coding standard achieves high performance compression and image quality at the expense of increased encoding complexity. Consequently, several fast mode decision and motion estimation techniques have been developed to reduce the computational cost. These approaches successfully reduce the computational time by reducing the image quality and/or increasing the bitrate. In this paper we propose a novel fast mode decision and motion estimation technique. The algorithm utilizes preprocessing frequency domain motion estimation in order to accurately predict the best mode and the search range. Experimental results show that the proposed algorithm significantly reduces the motion estimation time by up to 97%, while maintaining similar rate distortion performance when compared to the Joint Model software.

Low Complexity Hierarchical Prediction Algorithm for H.264/SVC

In the scalable video coding extension of the H.264/AVC standard, an exhaustive search technique is used to select the best coding mode for each macroblock. This technique achieves the highest possible coding efficiency, but it demands a higher video encoding computational complexity which constrains its use in many practical applications. This paper proposes combined fast sub-pixel motion estimation and a fast mode decision algorithm for inter-frame coding for temporal, spatial, and coarse grain signal-to-noise ratio scalability. It makes use of correlation between the macroblock and its enclosed partitions at different layers. Experimental results show that the scheme reduces the computational complexity significantly with negligible coding loss and bit-rate increases when compared to JSVM 9.15 and recently reported fast mode decision algorithms.

Inter-view Reference Frame Selection in Multi-view Video Coding

Summary form only given. Multiple video cameras are used to capture the same scene simultaneously to acquire the multiview view coding data, obviously, over-large data will affect the coding efficiency. Due to the video data is acquired from the same scene, the inter-view similarities between adjacent camera views are exploited for efficient compression. Generally, the same objects with different viewpoints are shown on adjacent views. On the other hand, containing objects at different depth planes, and therefore perfect correlation over the entire image area will never occur. Additionally, the scene complexity and the differences in brightness and color between the video of the individual cameras will also affect the current block to find its best match in the inter-view reference picture. Consequently, the temporal-view reference picture is referred more frequently. In order to gain the compression efficiency, it is a core part to disable the unnecessary inter-view reference. The idea of this paper is to exploit the phase correlation to estimate the dependencies between the inter-view reference and the current picture. If the two frames with low correlation, the inter-view reference frame will be disabled. In addition, this approach works only on non-anchor pictures. Experimental results show that the proposed algorithm can save 16% computational complexity on average, with negligible loss of quality and bit rate. The phase correlation process only takes up 0.1% of the whole process.

Fast multilayered prediction algorithm for group of pictures in H.264/SVC

The objective of scalable video coding is to enable the generation of a unique bitstream that can adapt to various bitrates, transmission channels and display capabilities. The scalability is categorised in terms of temporal, spatial, and quality. To improve encoding efficiency, the SVC scheme incorporates inter-layer prediction mechanisms which increases complexity of overall encoding. In this paper several conditional probabilities are established relating motion estimation characteristics and the mode distribution at different layers of the H.264/SVC. An evaluation of these probabilities is used to structure a low-complexity prediction algorithm for Group of Pictures (GOP) in H.264/SVC, reducing computational complexity whilst maintaining similar performance. When compared to the JSVM software, this algorithm achieves a significant reduction of encoding time, with a negligible average PSNR loss and bit-rate increase in temporal, spatial and SNR scalability. Experiments are conducted to provide a comparison between our method and a recently developed fast mode selection algorithm. These demonstrate our method achieves appreciable time savings for scalable spatial and scalable quality video coding, while maintaining similar PSNR and bit rate.

A simplified rate control algorithm for H.264/SVC

The objective of scalable video coding is to enable the generation of a unique bitstream that can adapt to various bitrates, transmission channels and display capabilities. The scalability is categorised in terms of temporal, spatial, and quality. Effective Rate Control (RC) has important ramifications for coding efficiency, and also channel bandwidth and buffer constraints in real-time communication. The main target of RC is to reduce the disparity between the actual and target bit-rates. In order to meet the target bitrate, a predicted Mean of Absolute Difference (MAD) between frames is used in a rate-quantisation model to obtain the Quantisation Parameter (QP) for encoding the current frame. The encoding process exploits the interdependencies between video frames; therefore the MAD does not change abruptly unless the scene changes significantly. After the scene change, the MAD will maintain a stable slow increase or decrease. Based on this observation, we developed a simplified RC algorithm. The scheme is divided in two steps; firstly, we predict scene changes, secondly, in order to suppress the visual quality, we limit the change in QP value between two frames to an adaptive range. This limits the need to use the rate-quantisation model to those situations where the scene changes significantly. To assess the proposed algorithm, comprehensive experiments were conducted. The experimental results show that the proposed algorithm significantly reduces encoding time whilst maintaining similar rate distortion performance, compared to both the H.264/SVC reference software and recently reported work.

Phase correlation based adaptive mode decision for the H.264/AVC

The H.264 video coding standard achieves high performance compression and image quality at the expense of increased encoding complexity, due to the very refined Motion Estimation (ME) and mode decision processes. This paper focuses on decreasing the complexity of the mode selection process by effectively applying a novel fast mode decision algorithm. Firstly the phase correlation is analysed between a macroblock and its prediction obtained from the previously encoded adjacent block. Relationships are established between the correlation value and object size and also best fit motion vector. From this a novel fast mode decision and motion estimation technique has been developed utilising preprocessing frequency domain ME in order to accurately predict the best mode and the search range. We measure the correlation between a macroblock and the corresponding prediction. Based on the result we select the best mode, or limit the mode selection process to a subset of modes. Moreover the correlation result is also used to select an appropriate search range for the ME stage. Experimental results show that the proposed algorithm significantly reduces the motion estimation time whilst maintaining similar Rate Distortion performance, when compared to both the H.264/AVC Joint Model (JM) reference software and recently reported work.

Fast motion prediction algorithm for multiview video coding

Multiview Video Coding (MVC) is an extension to the H.264/MPEG-4 AVC video compression standard developed with joint efforts by MPEG/VCEG to enable efficient encoding of sequences captured simultaneously from multiple cameras using a single video stream. Therefore the design is aimed at exploiting inter-view dependencies in addition to reducing temporal redundancies. However, this further increases the overall encoding complexity In this paper, the high correlation between a macroblock and its enclosed partitions is utilised to estimate motion homogeneity, and based on the result inter-view prediction is selectively enabled or disabled. Moreover, if the MVC is divided into three layers in terms of motion prediction; the first being the full and sub-pixel motion search, the second being the mode selection process and the third being repetition of the first and second for inter-view prediction, the proposed algorithm significantly reduces the complexity in the three layers. To assess the proposed algorithm, a comprehensive set of experiments were conducted. The results show that the proposed algorithm significantly reduces the motion estimation time whilst maintaining similar Rate Distortion performance, when compared to both the H.264/MVC reference software and recently reported work.