Sarp Ertürk

Digital image stabilization with sub-image phase correlation based global motion estimation

This paper presents digital image stabilization with sub-image phase correlation based global motion estimation and Kalman filtering based motion correction. Global motion is estimated from the local motions of four sub-images each of which is detected using phase correlation based motion estimation. The global motion vector is decided according to the peak values of sub-image phase correlation surfaces, instead of impartial median filtering. The peak values of sub-image phase correlation surfaces reveal reliable local motion vectors, as poorly matched sub images result in considerably lower peaks in the phase correlation surface due to spread. The utilization of sub-images enables fast implementation of phase correlation based motion estimation. The global motion vectors of image frames are accumulated to obtain global displacement vectors, that are Kalman filtered for stabilization.

Two-bit transform for binary block motion estimation

One-bit transforms (1BTs) have been proposed for low-complexity block-based motion estimation by reducing the representation order to a single bit, and employing binary matching criteria. However, as a single bit is used in the representation of image frames, bad motion vectors are likely to be resolved in 1BT-based motion estimation algorithms particularly for small block sizes. It is proposed in this paper to utilize a two-bit transform (2BT) for block-based motion estimation. Image frames are converted into two-bit representations by a simple block-by-block two bit transform based on multithresholding with mean and linearly approximated standard deviation values. In order to avoid blocking effects at block boundaries during the block-by-block transformation while enabling the two-bit representation to be constructed according to local detail, threshold values are computed within a larger window surrounding the transforming block. The 2BT makes use of lower bit-depth and binary matching criteria properties of 1BTs to achieve low-complexity block motion estimation. The 2BT improves motion estimation accuracy and seriously reduces the amount of bad motion vectors compared to 1BTs, particularly for small block sizes. It is shown that the proposed 2BT-based motion estimation technique improves motion estimation accuracy in terms of peak signal-to-noise ratio of reconstructed frames and also results in visually more accurate frames subsequent to motion compensation compared to the 1BT-based motion estimation approach.One-bit transforms (1BTs) have been proposed for low-complexity block-based motion estimation by reducing the representation order to a single bit, and employing binary matching criteria. However, as a single bit is used in the representation of image frames, bad motion vectors are likely to be resolved in 1BT-based motion estimation algorithms particularly for small block sizes. It is proposed in this paper to utilize a two-bit transform (2BT) for block-based motion estimation. Image frames are converted into two-bit representations by a simple block-by-block two bit transform based on multithresholding with mean and linearly approximated standard deviation values. In order to avoid blocking effects at block boundaries during the block-by-block transformation while enabling the two-bit representation to be constructed according to local detail, threshold values are computed within a larger window surrounding the transforming block. The 2BT makes use of lower bit-depth and binary matching criteria properties of 1BTs to achieve low-complexity block motion estimation. The 2BT improves motion estimation accuracy and seriously reduces the amount of bad motion vectors compared to 1BTs, particularly for small block sizes. It is shown that the proposed 2BT-based motion estimation technique improves motion estimation accuracy in terms of peak signal-to-noise ratio of reconstructed frames and also results in visually more accurate frames subsequent to motion compensation compared to the 1BT-based motion estimation approach.

Hyperspectral Image Classification Using Relevance Vector Machines

This letter presents a hyperspectral image classification method based on relevance vector machines (RVMs). Support vector machine (SVM)-based approaches have been recently proposed for hyperspectral image classification and have raised important interest. In this letter, it is genuinely proposed to use an RVM-based approach for the classification of hyperspectral images. It is shown that approximately the same classification accuracy is obtained using RVM-based classification, with a significantly smaller relevance vector rate and, therefore, much faster testing time, compared with SVM-based classification. This feature makes the RVM-based hyperspectral classification approach more suitable for applications that require low complexity and, possibly, real-time classification.

Multiplication-Free One-Bit Transform for Low-Complexity Block-Based Motion Estimation

A multiplication-free one-bit transform (1BT) for low-complexity block-based motion estimation is presented in this letter. A novel filter kernel is utilized to construct the 1BT of image frames using addition and shift operations only. It is shown that the proposed approach provides the same motion estimation accuracy at macro-block level and even better accuracy for smaller block sizes compared to previously proposed 1BT methods. Because the proposed 1BT approach does not require multiplication operations, it can be implemented in integer arithmetic using addition and shifts only, reducing the computational complexity, processing time, as well as power consumption

Constrained One-Bit Transform for Low Complexity Block Motion Estimation

One-bit transform (1BT)- and two-bit transform (2BT)-based block motion estimation (ME) schemes have been proposed in the literature to reduce the computational complexity of the ME process by enabling simple Boolean ex-or matching of lower bit depth representations of image frames. Recently a multiplication-free 1BT (MF-1BT) has been proposed to facilitate 1BT to be carried out with integer arithmetic using addition and shifts only. Thresholding schemes are typically used in order to construct the lower bit depth representations utilized in 1BT and 2BT. In our experience we have observed that one problem with such schemes is that pixel values that lie on directly opposite sides of the threshold are categorized into separate classes and are, therefore, counted as a nonmatch in the search process even if they are close in value. A constrained 1BT (C-1BT) that restricts pixels with values adjacent to the transform threshold during 1BT matching, counting them as a match regardless of their 1BT value, is proposed in this paper. It is shown that the proposed C-1BT approach improves the ME accuracy of 1BT-based ME and even outperforms 2BT-based ME at macroblock level

Real-Time Digital Image Stabilization Using Kalman Filters

T his paper presents a novel, real-time stabilization system that uses Kalman filters to remove short-term image fluctuations with retained smooth gross movements. The global camera motion is defined in terms of constant acceleration motion and constant velocity motion models, and Kalman filtering is employed to facilitate smooth operation. It is shown that the process noise variance has a direct effect on stabilization performance, and that it is possible to implement an efficient and robust stabilization system by adaptively changing the process noise variance value according to long-term camera motion dynamics.

Image sequence stabilisation: motion vector integration (MVI) versus frame position smoothing (FPS)

Image sequence stabilisation is the task of removing unwanted camera movements (jitter) from a video sequence. While these jitter can be translational, rotational or zoom based, translational cases have a larger influence on visual perception and are most likely to result in visual appearance degradation that will upset the viewer. While most of the research in the area of image stabilisation has been concentrated in the motion estimation part, with the goal of accurate global motion estimation, the correction part has been given less attention. Two techniques have been reported for the correction of translational jitter: motion vector integration (MVI) and frame position smoothing (FPS). While most reported stabilisation systems have employed MVI, this paper shows that FPS outperforms this technique in that jitter stabilisation is accomplished while successfully preserving intentional camera motions, without the need to compromise as is the case in MVI.

Sast digital image stabilization using one bit transform based sub-image motion estimation

This paper presents a fast image sequence stabilization system that uses one bit transform (IBT) matching based global motion estimation and Kalman filtering based global motion correction. In order to facilitate a fast motion estimation process, image frames are initially converted into a single bit-plane representation using the IBT, and then motion vectors are evaluated matching the IBTs of two consecutive frames using Boolean operations. Because a binary representation is established, the computational complexity of motion estimation is significantly reduced providing a proportionate reduction in power consumption of custom hardware. The global motion vector is computed using local motion vectors of sub-image blocks to further increase the motion estimation speed. For the motion correction part, global motion vectors are accumulated to yield absolute frame displacements and Kalman filtering is employed to achieve stabilization.

A Low-Complexity Approach for the Color Display of Hyperspectral Remote-Sensing Images Using One-Bit-Transform-Based Band Selection

This paper presents a new approach for the color display of hyperspectral images. It is proposed to use the one-bit transform (1BT) of hyperspectral image bands to select three suitable bands for red, green, and blue (RGB) display. The proposed approach has low complexity and is very suitable for hardware implementation. A dedicated hardware architecture that computes the transitions in the 1BT of hyperspectral image bands to determine bands that contain more information and the corresponding field-programmable gate array implementation of the proposed architecture are presented. In the proposed approach, less-structured bands are initially eliminated using the total number of transitions in the 1BT of hyperspectral image bands. Then, three suitable bands are selected from within this remaining set of well-structured bands for RGB color display. The proposed approach provides a new method for facilitating the color display of hyperspectral images, which has very low complexity.

Efficient Hardware Implementations of Low Bit Depth Motion Estimation Algorithms

In this paper, we present efficient hardware implementation of multiplication free one-bit transform (MF1BT) based and constraint one-bit transform (C-1BT) based motion estimation (ME) algorithms, in order to provide low bit-depth representation based full search block ME hardware for real-time video encoding. We used a source pixel based linear array (SPBLA) hardware architecture for low bit depth ME for the first time in the literature. The proposed SPBLA based implementation results in a genuine data flow scheme which significantly reduces the number of data reads from the current block memory, which in turn reduces the power consumption by at least 50% compared to conventional 1BT based ME hardware architecture presented in the literature. Because of the binary nature of low bit-depth ME algorithms, their hardware architectures are more efficient than existing 8 bits/pixel representation based ME architectures.