Jong Beom Ra

A fast multi-resolution block matching algorithm and its LSI architecture for low bit-rate video coding

We propose a fast multi-resolution block-matching algorithm (BMA) using multiple motion vector (MV) candidates and spatial correlation in MV fields, called a multi-resolution motion search algorithm (MRMCS). The proposed MRMCS satisfies high estimation performance and efficient LSI implementation. This paper describes the MRMCS with three resolution levels. At the coarsest level, two MV candidates are obtained on the basis of minimum matching error for the next search level. At the middle level, the two candidates selected at the coarsest level and the other one based on spatial MV correlation at the finest level are used as center points for local searches, and a MV candidate is chosen for the next search level. Then, at the finest level, the final MV is obtained from local search around the single candidate obtained at the middle level. This paper also describes an efficient LSI architecture based on the proposed algorithm for low bit-rate video coding. Since this architecture requires a small number of processing elements (PEs) and a small size on-chip memory, MRMCS can be implemented with a much smaller number of gates than other conventional architectures for full-search BMA while keeping a negligible degradation of coding performance. Moreover, the proposed motion estimator can support an advanced prediction mode (8/spl times/8 prediction mode) for H.263 and MPEG-4 video encoding. We implement this architecture with about 25 K gates and 288 bytes of RAM for a search range of [-16.0, +15.5] by using a synthesizable VHDL.

A fast search algorithm for vector quantization using L/sub 2/-norm pyramid of codewords

Vector quantization for image compression requires expensive encoding time to find the closest codeword to the input vector. This paper presents a fast algorithm to speed up the closest codeword search process in vector quantization encoding. By using an appropriate topological structure of the codebook, we first derive a condition to eliminate unnecessary matching operations from the search procedure. Then, based on this elimination condition, a fast search algorithm is suggested. Simulation results show that with little preprocessing and memory cost, the proposed search algorithm significantly reduces the encoding complexity while maintaining the same encoding quality as that of the full search algorithm. It is also found that the proposed algorithm outperforms the existing search algorithms.

Automatic registration between 3D intra-operative ultrasound and pre-operative CT images of the liver based on robust edge matching

The registration of a three-dimensional (3D) ultrasound (US) image with a computed tomography (CT) or magnetic resonance image is beneficial in various clinical applications such as diagnosis and image-guided intervention of the liver. However, conventional methods usually require a time-consuming and inconvenient manual process for pre-alignment, and the success of this process strongly depends on the proper selection of initial transformation parameters. In this paper, we present an automatic feature-based affine registration procedure of 3D intra-operative US and pre-operative CT images of the liver. In the registration procedure, we first segment vessel lumens and the liver surface from a 3D B-mode US image. We then automatically estimate an initial registration transformation by using the proposed edge matching algorithm. The algorithm finds the most likely correspondences between the vessel centerlines of both images in a non-iterative manner based on a modified Viterbi algorithm. Finally, the registration is iteratively refined on the basis of the global affine transformation by jointly using the vessel and liver surface information. The proposed registration algorithm is validated on synthesized datasets and 20 clinical datasets, through both qualitative and quantitative evaluations. Experimental results show that automatic registration can be successfully achieved between 3D B-mode US and CT images even with a large initial misalignment.

Three-Dimensional Blood Vessel Quantification via Centerline Deformation

It is clinically important to quantify the geometric parameters of an abnormal vessel, as this information can aid radiologists in choosing appropriate treatments or apparatuses. Centerline and cross-sectional diameters are commonly used to characterize the morphology of vessel in various clinical applications. Due to the existence of stenosis or aneurysm, the associated vessel centerline is unable to truly portray the original, healthy vessel shape and may result in inaccurate quantitative measurement. To remedy such a problem, a novel method using an active tube model is proposed. In the method, a smoothened centerline is determined as the axis of a deformable tube model that is registered onto the vessel lumen. Three types of regions, normal, stenotic, and aneurysmal regions, are defined to classify the vessel segment under-analyzed by use of the algorithm of a cross-sectional-based distance field. The registration process used on the tube model is governed by different region-adaptive energy functionals associated with the classified vessel regions. The proposed algorithm is validated on the 3-D computer-generated phantoms and 3-D rotational digital subtraction angiography (DSA) datasets. Experimental results show that the deformed centerline provides better vessel quantification results compared with the original centerline. It is also shown that the registered model is useful for measuring the volume of aneurysmal regions.

Robust CCD and IR Image Registration Using Gradient-Based Statistical Information

This letter presents a robust similarity measure for registering charged-couple device (CCD) and infrared (IR) images. The measure is based on the entropy obtained from a 3-D joint histogram incorporating edginess and modified generalized gradient vector flow (GGVF). To make a reliable mapping relationship between the edge regions of two images, the concept of edginess was adopted so registration performance would be affected mainly by gradient existences rather than their magnitudes. In addition, by adopting the GGVF, we relaxed a narrow capture range problem in conventional gradient-based measures. Experimental results showed that the proposed measure performs more robustly than existing measures.

Non-rigid registration between 3D ultrasound and CT images of the liver based on intensity and gradient information

In order to utilize both ultrasound (US) and computed tomography (CT) images of the liver concurrently for medical applications such as diagnosis and image-guided intervention, non-rigid registration between these two types of images is an essential step, as local deformation between US and CT images exists due to the different respiratory phases involved and due to the probe pressure that occurs in US imaging. This paper introduces a voxel-based non-rigid registration algorithm between the 3D B-mode US and CT images of the liver. In the proposed algorithm, to improve the registration accuracy, we utilize the surface information of the liver and gallbladder in addition to the information of the vessels inside the liver. For an effective correlation between US and CT images, we treat those anatomical regions separately according to their characteristics in US and CT images. Based on a novel objective function using a 3D joint histogram of the intensity and gradient information, vessel-based non-rigid registration is followed by surface-based non-rigid registration in sequence, which improves the registration accuracy. The proposed algorithm is tested for ten clinical datasets and quantitative evaluations are conducted. Experimental results show that the registration error between anatomical features of US and CT images is less than 2 mm on average, even with local deformation due to different respiratory phases and probe pressure. In addition, the lesion registration error is less than 3 mm on average with a maximum of 4.5 mm that is considered acceptable for clinical applications.

Image distortion correction for lenticula misalignment in three-dimensional lenticular displays

In manufacturing a lenticular display system, precise align- ment of the lenticular sheet on the LCD panel may not be practically achievable. Hence, observed view images inevitably produce unwanted distortion. We propose a novel method to alleviate the display distortion of each observed view image in a lenticular 3-D display. We first derive the relationship between subpixel values on the LCD pixel array and the image to be observed at each viewing zone in terms of system design parameters and the viewers eye position. Based on this relationship, we analyze the distortion between the observed and original view images. We then derive a compensation algorithm to minimize the distortion and generate high-quality 3-D images. To verify the proposed scheme, we examine displayed results from several 3-D images of synthetic and real scenes. The results demonstrate that the proposed scheme significantly reduces distortions and improves the image quality in the lenticular dis- play system.

Contrast-Enhanced Fusion of Multisensor Images Using Subband-Decomposed Multiscale Retinex

In this paper, we propose a novel pixel-level multisensor image fusion algorithm with simultaneous contrast enhancement. In order to accomplish both image fusion and contrast enhancement simultaneously, we suggest a modified framework of the subband-decomposed multiscale retinex (SDMSR), our previous contrast enhancement algorithm. This framework is based on a fusion strategy that reflects the multiscale characteristics of the SDMSR well. We first apply two complementary intensity transfer functions to source images in order to effectively utilize hidden information in both shadows and highlights in the fusion process. We then decompose retinex outputs into nearly nonoverlapping spectral subbands. The decomposed retinex outputs are then fused subband-by-subband, by using global weighting as well as local weighting to overcome the limitations of the pixel-based fusion approach. After the fusion process, we apply a space-varying subband gain to each fused SD retinex output according to the subband characteristic so that the contrast of the fused image can be effectively enhanced. In addition, in order to effectively manage artifacts and noise, we make the degree of enhancement of fused details adjustable by improving a detail adjustment function. From experiments with various multisensor image pairs, the results clearly demonstrate that even if source images have poor contrast, the proposed algorithm makes it possible to generate a fused image with highly enhanced contrast while preserving visually salient information contained in the source images.

Example-Based Super-Resolution via Structure Analysis of Patches

In example-based super-resolution, it is difficult to determine appropriate high-frequency (HF) patches from a training database by using only the information of one input image. In this letter, we utilize the sharpness of high-resolution (HR) patch candidates for the reliable determination of HF patches. For each input patch, we first preselect a sufficient number of HF patch candidates and produce HR patches by adding the candidates to the input patch. After removing the outlier patches, we then reselect several HF patches according to the patch characteristic for producing the final HR image. This reselection procedure is optimized for edge patches and non-edge patches, respectively. Experimental results show that the proposed algorithm provides sharper details compared to the existing algorithms.

Frame Rate Up Conversion Based on Variational Image Fusion

This paper presents a new framework for motion compensated frame rate up conversion (FRUC) based on variational image fusion. The proposed algorithm consists of two steps: 1) generation of multiple intermediate interpolated frames and 2) fusion of those intermediate frames. In the first step, we determine four different sets of the motion vector field using four neighboring frames. We then generate intermediate interpolated frames corresponding to the determined four sets of the motion vector field, respectively. Multiple sets of the motion vector field are used to solve the occlusion problem in motion estimation. In the second step, the four intermediate interpolated frames are fused into a single frame via a variational image fusion process. For effective fusion, we determine fusion weights for each intermediate interpolated frame by minimizing the energy, which consists of a weighted- L1-norm based data energy and gradient-driven smoothness energy. Experimental results demonstrate that the proposed algorithm improves the performance of FRUC compared with the existing algorithms.