Theodore Vlachos

High-Accuracy Sub-Pixel Motion Estimation From Noisy Images in Fourier Domain

In this paper, we propose a new method for estimating sub-pixel motion via exploiting the principle of phase correlation in the Fourier domain. The method is based on linear weighting of the height of the main peak on the one hand and the difference between its two neighboring side-peaks on the other. Using both synthetic and real data we show that the proposed method outperforms many established approaches and achieves improved accuracy even in the presence of noisy samples.

A study of sub-pixel motion estimation using phase correlation

We propose a method for obtaining high-accuracy sub-pixel motion estimates using phase correlation. Our method is motivated by recently published analysis according to which the Fourier inverse of the normalized cross-power spectrum of pairs of images which have been mutually shifted by a fractional amount can be approximated by a two-dimensional sinc function. We use a modified version of such a function to obtain a sub-pixel estimate of motion by means of variable-separable fitting in the vicinity of the maximum peak of the phase correlation surface. We demonstrate that our method outperforms, in terms of sub-pixel accuracy, not only other surface fitting techniques but also the state-of-the-art in motion estimation using phase correlation including the technique that motivated our work in the first place. Furthermore our method performs particularly well in the presence of artificially induced additive white Gaussian noise and also offers better motion vector coherence in terms of zero-order entropy.

Cut detection in video sequences using phase correlation

A novel algorithm for the detection of cuts in video sequences is proposed. The algorithm uses phase correlation to obtain a measure of content similarity for temporally adjacent frames and responds very well to scene cuts. The algorithm is insensitive to the presence of global illumination changes and noise and outperforms established methods for cut detection. As the proposed scheme is implemented in the frequency domain, the availability of fast hardware makes the scheme attractive for interactive and on-line applications.

Flicker correction for archived film sequences using a nonlinear model

A new model for the correction of flicker in archived film sequences is formulated from first principles. The model takes into account the Hurter-Driffield density versus log exposure characteristic to estimate image intensity errors caused by exposure inconsistencies. The model is shown to be highly nonlinear and can accommodate typical grayscale manipulations performed during film-to-video transfers. Motivated by the above model, an algorithm for the estimation of a correction profile across the available grayscale is developed. This profile is obtained from histogrammed gray-level differences between a reference and a degraded frame and is robust with respect to intensity changes due to scene and camera motion. Our results show that the proposed algorithm is very effective toward removing flicker for a variety of archived sequences and compares favorably with other state-of-the-art techniques developed for the same purpose. Additionally, the parametric form of the profile makes it particularly suitable for metadata use and allows fast inversion of the restoration process and recovery of the original material.

Motion estimation for the correction of twin-lens telecine flicker

Twin-lens telecine flicker is considered as a special type of artefact in film-to-video conversions. It is introduced by telecine equipment using separate optic light paths to register each of the two fields of a line-interlaced video frame. Motion estimation and compensation algorithms suitable for the elimination of this type of artefact are developed. Their application achieves considerable improvement in picture quality both subjectively and in terms of measured mean-square error.

Efficient detection of temporally impulsive dirt impairments in archived films

We propose a novel approach for the detection of temporally impulsive dirt impairments in archived film sequences. Our method does not require motion compensation and uses raw differences between the current frame and each of the previous and next frames to extract a confidence signal which is used to localize and label dirt regions. A key feature of our method is the removal of false alarms by local region-growing. Unlike other work utilizing manually added dirt impairments, we test our method on real film sequences with objective ground truth obtained by infrared scanning. With confidence information extracted from color channels, dirt areas of low contrast to the corresponding gray image can be successfully detected by our method when motion-based methods fail. Comparisons with established algorithms demonstrate that our method offers more efficient, robust and accurate dirt detection with fewer false alarms for a wide range of test material.

Using gradient correlation for sub-pixel motion estimation of video sequences

A highly accurate and computationally efficient method is presented suitable for the estimation of motion in video sequences. The method is based on the maximization of the spatial gradient cross-correlation function, which is computed in the frequency domain and therefore can be implemented by fast transformation algorithms. We present enhancements to the baseline gradient-correlation algorithm, which further improve performance, especially in the presence of manually induced additive Gaussian noise. We also present a comparative performance analysis, which demonstrates that the proposed method outperforms the state-of-the-art in frequency-domain motion estimation, in the shape of phase correlation.

On the estimation of subpixel motion using phase correlation

We propose a method for obtaining high-accuracy subpixel motion estimates using phase correlation. Our method is motivated by recently published analysis according to which the Fourier inverse of the normalized cross-power spectrum of pairs of images that have been mutually shifted by a fractional amount can be approximated by a two-dimensional sinc function. We propose a modified version of such a function to obtain a subpixel estimate of motion by means of variable-separable fitting in the vicinity of the maximum peak of the phase correlation surface. We demonstrate that our method outperforms, in terms of subpixel accuracy, not only other surface-fitting techniques but also the state-of-the-art in motion estimation using phase correlation including the technique that motivated our work in the first place. Furthermore, our method performs particularly well in the presence of artificially induced additive white Gaussian noise and also offers better motion vector coherence in terms of zero-order entropy.

Motion-compensation using variable-size block-matching with binary partition trees

A new approach to variable size block matching is proposed, based on the binary partitioning of blocks. If a particular block does not allow for accurate motion compensation, then it is split into two using the horizontal or vertical line that achieves the maximum reduction in motion compensation error. This method causes partitioning to occur along motion boundaries, thus substantially reducing blocking artifacts. In addition, small blocks are placed in regions of complex motion, while large blocks cover regions of uniform motion. The proposed technique provides significant gains in picture quality of 1.5 to 3.0 dB, when compared to fixed size block matching at the same total rate.

Gradient-based subspace phase correlation for fast and effective image alignment

To prove the efficacy of subspace phase correlation in estimating 2D image offsets.To prove more robust results yielded from subspace approach under zero-mean noise.To prove our method robust to non-zero-mean noise caused by non-overlapped regions.To prove higher peaks yielded by our method for robustness with reduced complexity.To validate the effectiveness with various synthetic data and noisy MRI images. Phase correlation is a well-established frequency domain method to estimate rigid 2-D translational motion between pairs of images. However, it suffers from interference terms such as noise and non-overlapped regions. In this paper, a novel variant of the phase correlation approach is proposed, in which 2-D translation is estimated by projection-based subspace phase correlation (SPC). Conventional wisdom has suggested that such an approach can only amount to a compromise solution between accuracy and efficiency. In this work, however, we prove that the original SPC and the further introduced gradient-based SPC can provide robust solution to zero-mean and non-zero-mean noise, and the latter is also used to model the interference term of non-overlapped regions. Comprehensive results from synthetic data and MRI images have fully validated our methodology. Due to its substantially lower computational complexity, the proposed method offers additional advantages in terms of efficiency and can lend itself to very fast implementations for a wide range of applications where speed is at a premium.