Sangjin Kim

Optical flow-based real-time object tracking using non-prior training active feature model

This paper presents a feature-based object tracking algorithm using optical flow under the non-prior training (NPT) active feature model (AFM) framework. The proposed tracking procedure can be divided into three steps: (i) localization of an object-of-interest, (ii) prediction and correction of the objects position by utilizing spatio-temporal information, and (iii) restoration of occlusion using NPT-AFM. The proposed algorithm can track both rigid and deformable objects, and is robust against the objects sudden motion because both a feature point and the corresponding motion direction are tracked at the same time. Tracking performance is not degraded even with complicated background because feature points inside an object are completely separated from background. Finally, the AFM enables stable tracking of occluded objects with maximum 60% occlusion. NPT-AFM, which is one of the major contributions of this paper, removes the off-line, preprocessing step for generating a priori training set. The training set used for model fitting can be updated at each frame to make more robust objects features under occluded situation. The proposed AFM can track deformable, partially occluded objects by using the greatly reduced number of feature points rather than taking entire shapes in the existing shape-based methods. The on-line updating of the training set and reducing the number of feature points can realize a real-time, robust tracking system. Experiments have been performed using several in-house video clips of a static camera including objects such as a robot moving on a floor and people walking both indoor and outdoor. In order to show the performance of the proposed tracking algorithm, some experiments have been performed under noisy and low-contrast environment. For more objective comparison, PETS 2001 and PETS 2002 datasets were also used.

Contrast Enhancement Using Dominant Brightness Level Analysis and Adaptive Intensity Transformation for Remote Sensing Images

This letter presents a novel contrast enhancement approach based on dominant brightness level analysis and adaptive intensity transformation for remote sensing images. The proposed algorithm computes brightness-adaptive intensity transfer functions using the low-frequency luminance component in the wavelet domain and transforms intensity values according to the transfer function. More specifically, we first perform discrete wavelet transform (DWT) on the input images and then decompose the LL subband into low-, middle-, and high-intensity layers using the log-average luminance. Intensity transfer functions are adaptively estimated by using the knee transfer function and the gamma adjustment function based on the dominant brightness level of each layer. After the intensity transformation, the resulting enhanced image is obtained by using the inverse DWT. Although various histogram equalization approaches have been proposed in the literature, they tend to degrade the overall image quality by exhibiting saturation artifacts in both low- and high-intensity regions. The proposed algorithm overcomes this problem using the adaptive intensity transfer function. The experimental results show that the proposed algorithm enhances the overall contrast and visibility of local details better than existing techniques. The proposed method can effectively enhance any low-contrast images acquired by a satellite camera and is also suitable for other various imaging devices such as consumer digital cameras, photorealistic 3-D reconstruction systems, and computational cameras.

Multifocusing and Depth Estimation Using a Color Shift Model-Based Computational Camera

This paper presents a novel approach to depth estimation using a multiple color-filter aperture (MCA) camera and its application to multifocusing. An image acquired by the MCA camera contains spatially varying misalignment among RGB color channels, where the direction and length of the misalignment is a function of the distance of an object from the plane of focus. Therefore, if the misalignment is estimated from the MCA output image, multifocusing and depth estimation become possible using a set of image processing algorithms. We first segment the image into multiple clusters having approximately uniform misalignment using a color-based region classification method, and then find a rectangular region that encloses each cluster. For each of the rectangular regions in the RGB color channels, color shifting vectors are estimated using a phase correlation method. After the set of three clusters are aligned in the opposite direction of the estimated color shifting vectors, the aligned clusters are fused to produce an approximately in-focus image. Because of the finite size of the color-filter apertures, the fused image still contains a certain amount of spatially varying out-of-focus blur, which is removed by using a truncated constrained least-squares filter followed by a spatially adaptive artifacts removing filter. Experimental results show that the MCA-based multifocusing method significantly enhances the visual quality of an image containing multiple objects of different distances, and can be fully or partially incorporated into multifocusing or extended depth of field systems. The MCA camera also realizes single camera-based depth estimation, where the displacement between multiple apertures plays a role of the baseline of a stereo vision system. Experimental results show that the estimated depth is accurate enough to perform a variety of vision-based tasks, such as image understanding, description, and robot vision.

Wavelet-domain color image enhancement using filtered directional bases and frequency-adaptive shrinkage

This paper presents a novel wavelet-domain color image enhancement using filtered directional bases and frequency-adaptive shrinkage. Most traditional noise reduction methods tend to over-suppress high-frequency details. For overcoming this problem we first decompose the input image into flat and edge regions, and remove noise using the alpha map computed from wavelet transform coefficients of LH, HL, and HH bands. After removing noise in the flat region, we further remove noise in edge regions by adaptively shrinking wavelet coefficients based on the entropy. Moreover, we present a new directional transform using wavelet basis and Gaussian low pass filters. The wavelet coefficients of edge regions are inverse transformed by using the filtered wavelet bases. Experimental results show the proposed algorithm can reduce noise without losing sharp details and is suitable for commercial low-cost imaging systems, such as digital cameras, CCTV, and surveillance system.

Color shift model-based image enhancement for digital multifocusing based on a multiple color-filter aperture camera

In this paper, we present a novel image enhancement approach using a color shift model-based multiple color-filter aperture (MCA) camera for digital multifocusing. The proposed image enhancement algorithm consists of three steps; (i) cluster-based region-of-interest (ROI) estimation, (ii) image registration using phase correlation matching (PCM) and fusion, and (iii) image enhancement using spatially adaptive noise smoothing based on the alpha map. The image acquired by the MCA configuration contains color misalignment, which provides additional depth information of objects at different distances. This color misalignment can also provide additional information for blur estimation. The proposed cluster-based image segmentation method can effectively classify ROIs according to the distance from the camera. The segmented regions are aligned by using PCM, and they are fused to generate an in-focused image. For further enhancement of the color-registered image, we use spatially adaptive noise smoothing based on the alpha map. Experimental results show the proposed image enhancement method can significantly enhance the visual quality of the MCA output image, and can be fully or partially incorporated into multifocusing or extended depth of field (EDoF) systems in the form of the finite impulse response (FIR) filter structure.

Real-time bayer-domain image restoration for an extended depth of field (EDoF) camera

In this paper, we present a novel real-time image restoration approach using a truncated constrained least-squares (TCLS) filter and spatially adaptive noise smoothing (SANS) algorithm based on alpha map for the extended depth of field (EDoF) system in an image signal processing (ISP) chain. The proposed TCLS filter and the alpha map-based SANS algorithm can be implemented in the Bayer-domain by using a general finite impulse response (FIR) structure. The TCLS filter coefficients are priori determined according to the given point-spread-function (PSF) by optical lens simulation, and deconvolution is performed in the Bayer-domain instead of the RGB-domain. The SANS algorithm can successfully remove noise in flat regions without affecting the sharply restored details. Based on the extended set of experimental results the proposed algorithm is proved to be able to restore an accurately focused image in real-time, and is suitable for commercial low-cost, high-quality imaging devices such as a digital camera and a camcorder.

Real-time image restoration for digital multifocusing in a multiple color-filter aperture camera

A multiple color-filter aperture (MCA) can provide a single camera with depth information and multifocusing. However, the original version of the MCA system exhibits inherent limitations such as manual, empirical tuning parameters for the color channel registration and fusion (CRF) process. Furthermore, a CRF output image still contains undesired out-of-focus blur because of the finite-sized apertures and the lateral displacement of each color-filter aperture, which results in low exposure, color mixing, deviation of color convergence, and divergence of light rays. For overcoming these problems, we present a real-time image processing solution for digital multifocusing in a MCA system.

Feature fusion-based multiple people tracking

This paper presents a feature fusion-based tracking algorithm using optical flow under the non-prior training active feature model (NPT-AFM) framework. The proposed object tracking procedure can be divided into three steps: (i) localization of human objects, (ii) prediction and correction of the object’s location by utilizing spatio-temporal information, and (iii) restoration of occlusion using the NPT-AFM[15]. Feature points inside an ellipsoidal shape including objects are estimated instead of its shape boundary, and are updated as an element of the training set for the AFM. Although the proposed algorithm uses the greatly reduced number of feature points, the proposed feature fusion-based multiple people tracking algorithm enables the tracking of occluded people in complicated background.

Ringing Artifact Removal in Digital Restored Images Using Multi- Resolution Edge Map

This paper presents a novel approach to reducing ringing artifact in digitally restored images by using multi-resolution edge map. The performance of reducing ringing artifacts depends on the accurate classification of the local features in the image. The discrete wavelet transform (DWT) provides effective insight into both spatial and frequency characteristics of an image. Through the DWT analysis, we show that ringing artifacts can be suppressed to a great extent by using multiple-level edge maps, which provide enhanced matching to local edges. Base on the experimental results, the proposed method can reduce ringing artifacts with minimized edge degradation by using DWT analysis.

Optical flow-based tracking of deformable objects using a non-prior training active feature model

This paper presents a feature point tracking algorithm using optical flow under the non-prior training active feature model (NPT- AFM) framework. The proposed algorithm mainly focuses on analysis of deformable objects, and provides real-time, robust tracking. The pro- posed object tracking procedure can be divided into two steps: (i) opti- cal flow-based tracking of feature points and (ii) NPT-AFM for robust tracking. In order to handle occlusion problems in object tracking, feature points inside an object are estimated instead of its shape boundary of the conventional active contour model (ACM) or active shape model (ASM), and are updated as an element of the training set for the AFM. The pro- posed NPT-AFM framework enables the tracking of occluded objects in complicated background. Experimental results show that the proposed NPT-AFM-based algorithm can track deformable objects in real-time.