Man Hee Lee

Design and Performance Evaluation of Image Processing Algorithms on GPUs

In this paper, we construe key factors in design and evaluation of image processing algorithms on the massive parallel graphics processing units (GPUs) using the compute unified device architecture (CUDA) programming model. A set of metrics, customized for image processing, is proposed to quantitatively evaluate algorithm characteristics. In addition, we show that a range of image processing algorithms map readily to CUDA using multiview stereo matching, linear feature extraction, JPEG2000 image encoding, and nonphotorealistic rendering (NPR) as our example applications. The algorithms are carefully selected from major domains of image processing, so they inherently contain a variety of subalgorithms with diverse characteristics when implemented on the GPU. Performance is evaluated in terms of execution time and is compared to the fastest host-only version implemented using OpenMP. It is shown that the observed speedup varies extensively depending on the characteristics of each algorithm. Intensive analysis is conducted to show the appropriateness of the proposed metrics in predicting the effectiveness of an application for parallel implementation.

Active 3D shape acquisition using smartphones

In this paper, we propose an active 3D shape acquisition method based on photometric stereo using smartphones camera and flash. A pair of smartphones collaborates as the master and slave, in which the slave projects illumination from different locations while the master captures the images and processes photometric stereo algorithm to reconstruct 3D shape. In order to reduce the error, the smart-phones camera is calibrated to overcome the effect of lens distortion and nonlinear camera sensor response. We apply SURF feature matching and five-point algorithm to estimate the relative pose between the master and slave smart-phones. Then the lighting direction is estimated to run photometric stereo algorithm. All procedures are implemented on an off-the-shelf smartphone. Experimental result shows that the proposed system enables us to use smartphone as a 3D shape capturing device with low cost and reasonable quality.

Accelerating depth image-based rendering using GPU

In this paper, we propose a practical method for hardware-accelerated rendering of the depth image-based representation (DIBR) object, which is defined in MPEG-4 Animation Framework eXtension (AFX). The proposed method overcomes the drawbacks of the conventional rendering, i.e. it is slow since it is hardly assisted by graphics hardware and surface lighting is static. Utilizing the new features of modern graphic processing unit (GPU) and programmable shader support, we develop an efficient hardware-accelerated rendering algorithm of depth image-based 3D object. Surface rendering in response of varying illumination is performed inside the vertex shader while adaptive point splatting is performed inside the fragment shader. Experimental results show that the rendering speed increases considerably compared with the software-based rendering and the conventional OpenGL-based rendering method.

Mobile photo collage

In this paper, we propose an efficient technique for creating a visually appealing collage on a mobile platform from a set of input images. The proposed algorithm consists of four main modules, namely image ranking, region of interest (ROI) selection, packing, and blending. Each of the four modules is designed using a greedy and localized approach. The modules are further optimized during implementation for efficient porting on a mobile phone processor. Experimental results show the effectiveness of the proposed algorithm with visually appealing results on an off-the-shelf mobile phone.

Efficient design and implementation of visual computing algorithms on the GPU

In this paper, we explore the key factors in the design and implementation of visual computing (image processing and computer vision) algorithms on the massive parallel GPU (graphics processing units). The goal of the exploration is to provide common perspective and guidelines of using GPU for visual computing applications. We have selected three nontrivial applications (multiview stereo matching, linear feature extraction, and JPEG2000 image encoding) for the benchmarks, which show different characteristics in GPU parallel computing. Intensive analysis is performed to evaluate the characteristic of each algorithm and its effect on the performance. Based on this, we draw general guidelines of using GPU for the visual computing algorithms.

Feature description using local neighborhoods

A novel local neighborhoods based feature description method is proposed.The proposed method handles significant viewpoint change and shape deformation.An efficient similarity function is proposed to achieve fast matching.The proposed method can be easily adapted in conventional graph matching methods. Feature description and matching is an essential part of many computer vision applications. Numerous feature description algorithms have been developed to achieve reliable performance in image matching, e.g. SIFT, SURF, ORB, and BRISK. However, their descriptors usually fail when the images have undergone large viewpoint changes or shape deformation. To remedy the problem, we propose a novel feature description and similarity measure based on local neighborhoods. The proposed descriptor and similarity is useful for a wide range of matching methods including nearest neighbor matching methods and popular graph matching algorithms. Experimental results show that the proposed method detects reliable matches for image matching, and performs robustly to viewpoint changes and shape deformation.

Performance Evaluation of Local Descriptors for Affine Invariant Region Detector

Local feature descriptors are widely used in many computer vision applications. Over the past couple of decades, several local feature descriptors have been proposed which are robust to challenging conditions. Since they show different characteristics in different environment, it is necessary to evaluate their performance in an intensive and consistent manner. However, there has been no relevant work that addresses this problem, especially for the affine invariant region detectors which are popularly used in object recognition and classification. In this paper, we present a useful and rigorous performance evaluation of local descriptors for affine invariant region detector, in which MSER (maximally stable extremal regions) detector is employed. We intensively evaluate local patch based descriptors as well as binary descriptors, including SIFT (scale invariant feature transform), SURF (speeded up robust features), BRIEF (binary robust independent elementary features), FREAK (fast retina keypoint), Shape descriptor, and LIOP (local intensity order pattern). Intensive evaluation on standard dataset shows that LIOP outperforms the other descriptors in terms of precision and recall metric.

Robust feature description and matching using local graph

Feature detection and matching are essential parts in most computer vision applications. Many researchers have developed various algorithms to achieve good performance, such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features). However, they usually fail when the scene has considerable out-of-plane rotation because they only focus on in-plane rotation and scale invariance. In this paper, we propose a novel feature description algorithm based on local graph representation and graph matching based, which is more robust to out-of-plane rotation. The proposed local graph encodes the geometric correlation between the neighboring features. In addition, we propose an efficient score function to compute the matching score between the local graphs. Experimental result shows that the proposed algorithm is more robust to out-of-plane rotation than conventional algorithms.

Photorealistic 3D face modeling on a smartphone

In this paper, we propose an efficient method for creating a photorealistic 3D face model on a smartphone. Major features of human face such as eyes, nose, lip, cheek, chin, and profile boundary are extracted automatically from the front and profile images, in which ACM (active contour model) and deformable ICP (iterative closest point) methods are employed. A 3D face model is generated by deforming a generic model so that the 3D face model is correctly corresponded to the extracted facial features. Skin texture map is created from the input image, which is mapped on the deformed 3D face model. All procedures are implemented and optimized efficiently on a smartphone with limited processing power and memory capability. Experimental results show that photorealistic 3D face models are created successfully on a variety of test samples. It takes about 6 seconds on an off-the-shelf smartphone.

Performance evaluation of local descriptors for maximally stable extremal regions

Abstract Visual feature descriptors are widely used in most computer vision applications. Over the past several decades, local feature descriptors that are robust to challenging environments have been proposed. Because their characteristics differ according to the imaging condition, it is necessary to compare their performance consistently. However, no pertinent research has attempted to establish a benchmark for performance evaluation, especially for affine region detectors, which are mainly used in object classification and recognition. This paper presents an intensive and informative performance evaluation of local descriptors for the state-of-the-art affine-invariant region detectors, i.e., maximally stable extremal region detectors. We evaluate patch-based and binary descriptors, including SIFT, SURF, BRIEF, FREAK, the shape descriptor, LIOP, DAISY, GSURF, RFDg, and CNN descriptors. The experimental results reveal the relative performance and characteristics of each descriptor.