Mainak Biswas

Real-Time Affine Global Motion Estimation Using Phase Correlation and its Application for Digital Image Stabilization

We propose a fast and robust 2D-affine global motion estimation algorithm based on phase-correlation in the Fourier-Mellin domain and robust least square model fitting of sparse motion vector field and its application for digital image stabilization. Rotation-scale-translation (RST) approximation of affine parameters is obtained at the coarsest level of the image pyramid, thus ensuring convergence for a much larger range of motions. Despite working at the coarsest resolution level, using subpixel-accurate phase correlation provides sufficiently accurate coarse estimates for the subsequent refinement stage of the algorithm. The refinement stage consists of RANSAC based robust least-square model fitting for sparse motion vector field, estimated using block-based subpixel-accurate phase correlation at randomly selected high activity regions in finest level of image pyramid. Resulting algorithm is very robust to outliers such as foreground objects and flat regions. We investigate the robustness of the proposed method for digital image stabilization application. Experimental results show that the proposed algorithm is capable of estimating larger range of motions as compared to another phase correlation method and optical flow algorithm.

A novel motion estimation algorithm using phase plane correlation for frame rate conversion

Frame rate conversion is an important application in video processing. It allows the application to perform processes such as slow motion, standard conversion and display conversion. The ability to perform high quality interpolation would improve the quality of all these processes. The requirement for accurate motion vectors is extremely important for the interpolator. The phase plane correlation approach achieves this with far less operation count than different block matching algorithms. Two approaches have been discussed to find accurate motion vectors as well as the application in interpolation of intermediate frames. In the first approach, hierarchical block matching (HBM) with smoothness constraint is used to find robust motion vectors for interpolation of spatio-temporal filters. In the second approach, the motion vector from phase plane correlation (PPC) method is used that assigns multiple motion vectors to a block thus achieving the result of object based approaches with much fewer operation counts.

Spatio-temporal texture synthesis and image inpainting for video applications

In this paper we investigate the application of texture synthesis and image inpainting techniques for video applications. Working in the non-parametric framework, we use 3D patches for matching and copying. This ensures temporal continuity to some extent which is not possible to obtain by working with individual frames. Since, in present application, patches might contain arbitrary shaped and multiple disconnected holes, fast Fourier transform (FFT) and summed area table based sum of squared difference (SSD) calculation (S.L. Kilthau, et al, 2002) cannot be used. We propose a modification of above scheme which allows its use in present application. This results in significant gain of efficiency since search space is typically huge for video applications.

Efficient phase correlation motion estimation using approximate normalization

Motion estimation methods based on phase correlation have been in use for almost three decades. Its ability to measure large motions with subpixel accuracy has made it very appealing in many applications. Significant computational steps involved are FFT, normalized cross-power spectrum and IFFT. While efficient hardware and software solutions exist for FFT and IFFT, normalized cross-power spectrum computation has so far evaded efficient solutions. In this paper, we propose a novel method for computing cross-power spectrum based on approximate normalization of complex scalar, which is very efficient and hardware friendly as compared to existing methods. Proposed algorithm along with its generalization to normalization of complex vectors, results in significant reduction in precision requirements on FFT and IFFT as well. It is related to block floating point representation of numbers but overhead due to exponents have been avoided. Simulation results show that proposed algorithm holds potential for making phase correlation an accessible tool for low power devices as well as applications dealing with very large images.

A novel de-interlacing technique based on phase plane correlation motion estimation

In this paper, a de-interlacing technique using motion compensated interpolation is proposed. In the proposed scheme motion estimation is first performed between the same parity field i.e., reference and current field from either odd or even fields. Compared to other motion adaptive and motion compensated interpolation technique this method results in a better visual quality due to an accurate motion estimation technique. For further improvement a motion vector validation method based on motion parameters like pan, zoom or tilt is used to reduce the interpolation error caused due to the incorrect motion vector. Simulation results confirm that the proposed method has better visual performance compared to the conventional method.

Linear system analysis of motion compensated de-interlacing

The majority of the work on deinterlacing techniques is ad hoc in nature and lacks theoretical justification. The designs are heuristic and the results are in most cases experimental. It is very difficult to quantify the overall performance of a deinterlacing system based on its performance over a few video sequences. In this paper, we present a linear system analysis for a motion-compensated video upconversion system, analyze the effect of motion vector accuracy in system performance and investigate switching the algorithm between fixed and motion-compensated deinterlacing.