Jung-Hwan Ko

Real-time stereo object tracking system by using block matching algorithm and optical binary phase extraction joint transform correlator

In this paper, we proposed a new adaptive stereo object tracking system that can control the convergence angle and pan/tilt of cameras by using optical binary phase extraction joint transform correlator and can extract the tracking object from a complex background and foreground noises by using the block-based mean square error algorithm. From the experimental results, the proposed stereo tracking system is found to track the object adaptively under the complex circumstances and changing background noises and the possibility of real-time implementation of the proposed system by using the optical system is also suggested.

Intelligent mobile robot system for path planning using stereo camera-based geometry information

In this paper, a new real-time and intelligent mobile robot system for path planning and navigation using stereo camera embedded on the pan/tilt system is proposed. In the proposed system, face area of a moving person is detected from a sequence of the stereo image pairs by using the YCbCr color model and using the disparity map obtained from the left and right images captured by the pan/tilt-controlled stereo camera system and depth information can be detected. And then, the distance between the mobile robot system and the face of the moving person can be calculated from the detected depth information. Accordingly, based-on the analysis of these data, three-dimensional objects can be detected. Finally, by using these detected data, 2-D spatial map for a visually guided robot that can plan paths, navigate surrounding objects and explore an indoor environment is constructed. From some experiments on target tracking with 480 frames of the sequential stereo images, it is analyzed that error ratio between the calculated and measured values of the relative position is found to be very low value of 1.4 % on average. Also, the proposed target tracking system has achieved a high speed of 0.04 sec/frame for target detection and 0.06 sec/frame for target tracking.

Implementation of stereo camera-based automatic unmanned ground vehicle system for adaptive target detection

In this paper, an integrated automatic stereo target-detection system is proposed, in which a stereo camera embedded on the mobile robot system is employed and using this system a potential 3D target object is detected and discriminated from the other objects and backgrounds, and then its moving trajectory and direction data is transferred to the host vehicle so as to warn a potential collision with a target to the UGV (unmanned ground vehicle). The face area of a target person can be tracked by using the YCbCr color model, centroid method and the depth map. In addition, a distance and area coordinates of the target face can be also calculated, in which the calculated coordinates are used for extraction of 3D information that can be used for plan paths, construction of the coordinates maps and exploration of the indoor environment. From some experiments on person tracking by using the sequential stereo image pairs of 1,280 frames, it is analyzed that error ratio between the calculated and measured values of the distance is found to be very low value of 0.7% on average.

Stereo image reconstruction using regularized adaptive disparity estimation

In this paper, stereo image reconstruction using regularized adaptive disparity estimation is proposed. That is, by adaptively predicting the mutual correlation between stereo images pair using the proposed algorithm, the bandwidth of stereo input images pair can be compressed to the level of a conventional 2D image and a predicted image also can be effectively reconstructed using a reference image and disparity vectors. Especially, in the proposed algorithm, the first feature values are extracted from the input stereo images pair. Then, a matching window for stereo matching is adaptively selected depending on the magnitude of these feature values. That is, for the region having larger feature values, a smaller matching window is selected, while, for the opposite case, a larger matching window is selected by comparing predetermined threshold values. This approach is not only able to reduce a mismatching of disparity vectors, which occurs in the conventional dense disparity estimation with a small matching window, but is also able to reduce blocking effects that occur in the coarse disparity estimation with a large matching window. In addition, in this paper, a new regularized adaptive disparity estimation technique is proposed. That is, by regularizing the estimated disparity vector with the neighboring disparity vectors, problems of the conventional adaptive disparity estimation scheme might be solved, and the predicted stereo image can be more effectively reconstructed. From experiments using stereo sequences of “Man”, “Fichier”, “Manege”, and “Tunnel”, it is shown that the proposed algorithm improves the PSNRs of a reconstructed image to about 6.90 dB on average at ±30 search ranges as compared to those of conventional algorithms. Also, it is found that there is almost no difference between an original image and a reconstructed image through the proposed algorithm by comparison to that of conventional algorithms.

Intelligent person identification system using stereo camera-based height and stride estimation

In this paper, a stereo camera-based intelligent person identification system is suggested. In the proposed method, face area of the moving target person is extracted from the left image of the input steros image pair by using a threshold value of YCbCr color model and by carrying out correlation between the face area segmented from this threshold value of YCbCr color model and the right input image, the location coordinates of the target face can be acquired, and then these values are used to control the pan/tilt system through the modified PID-based recursive controller. Also, by using the geometric parameters between the target face and the stereo camera system, the vertical distance between the target and stereo camera system can be calculated through a triangulation method. Using this calculated vertical distance and the angles of the pan and tilt, the targets real position data in the world space can be acquired and from them its height and stride values can be finally extracted. Some experiments with video images for 16 moving persons show that a person could be identified with these extracted height and stride parameters.

A real-time face detection and tracking for surveillance system using pan/tilt controlled stereo camera

In this paper, a new real-time and intelligent face tracking system basing on the pan/tilt controlled stereo camera is proposed. In the proposed system moving human face is firstly detected from a sequence of the stereo image captured by the stereo camera system using a threshold value of YCbCr color model and its distance and area coordinates are measured and then, finally the stereo camera is controlled by using the pan/tilt system for tracking the moving face in real-time. Especially, color information of the face and geometric coordinate information of the stereo camera in which the pan/tilt system is embedded are used for extraction of 3D information of the humans face. From some experiments using a sequence of 1,280 frames of stereo input image, it is analyzed that the horizontal and vertical displacement on the center position of the face after tracking are kept to be very low values of 0.6% for 1,280 frames on average. Also, the error ratio between the measured and computed values is kept to be very low value of 0.5% on average in cartesian coordinates. This good experimental result finally suggests a possibility of implementing a new face tracking system having a high degree of accuracy and a very fast response time on the target by using the proposed algorithm.

Reconstruction of stereoscopic image using adaptive disparity estimation scheme

In this paper, a new stereoscopic image reconstruction technique using an adaptive disparity estimation algorithm is proposed and its performance is analyzed in terms of PSNR through comparison to that of the conventional disparity estimation algorithms. In the proposed method, the feature-based disparity estimation method in which Canny mask operator is used for detecting the edge information from the input stereo pair are used for extracting the feature value. And, the matching window size for reconstruction of stereoscopic image is adaptively selected depending on the magnitude of the feature value of the input stereo pair by comparing with the predetermined threshold value. That is, coarse matching is carried out in the region having a small feature value while dense matching is carried out in the region having a large feature value. This new approach can not only reduce mismatching possibility of the disparity vector mostly happened in the conventional dense disparity estimation with a small matching window size, but also reduce the blocking effect occurred in the disparity estimation with a large matching window size. From some experimental results, it is found that the proposed algorithm improves PSNR of the reconstructed image about 5.36~7.76 dB on the average than that of the conventional algorithms.

Convergence control of a moving object using hierarchical algorithms in a stereo vision system

In this paper, a new stereo tracking system using the hierarchical opto-digital algorithms is proposed and implemented. In the proposed method, a moving target object is initially extracted from the input image by removing the background noises by applying the region-based SAD algorithm to the sequential left image. And then, the location coordinates of the moving target for each of the sequential input frames are extracted through performing the optical BPEJTC between the reference image of the extracted target object and the stereo input image. These extracted coordinate values are finally used for controlling the convergence angle and the pan/tilt embedded to the conventional stereo target tracking system. From some experimental results with the 20 frames of the stereo input image pairs, the proposed system is found to be able to effectively extract the area where the target object is located from the stereo input image regardless of the background noises. With the location values of the tracking object obtained from the execution of the optical BPEJTC, the convergence angle and the pan/tilt of the stereo cameras are found to be successfully controlled. Therefore, in this paper, a feasibility test for implementing the stereo tele-working system or the stereo robot vision system using the proposed algorithm is suggested.

Real-time stereo object tracking system using the block-based window mask and optical BPEJTC

In this paper, we proposed a stereo object tracking system that can control the convergence angle and pan/tilt of cameras by using optical binary phase extraction joint transform correlator (BPEJTC) and can extract the tracking object from a complex background and foreground noises by using the block matching-based window mask. It is used to perceive and extract the tracking object from the foreground and complex background by using window mask of the block matching-based SAD(sum of absolute difference), and by using the optical BPEJTC of the phase type, which has improved the correlating properties of the conventional optical JTC(joint transform correlator) with the adaptive object tracking ability, the position values of moving target on the left and right images can be calculated. And, we can be controlling the convergence angle and pan/tilt of cameras by using this values. Therefore, real time stereo object tracking system, which could adapt to the changes in surrounding, can be implemented. From the experimental results, the proposed stereo tracking system is found to track the object adaptively under the complex circumstances and changing background noises and the possibility of real-time implementation of the proposed system by using the optical system is also suggested.

Effective Route Decision of an Automatic Moving Robot(AMR) using a 2D Spatial Map of the Stereo Camera System

This paper proposes a method for an effective intelligent route decision for automatic moving robots(AMR) using a 2D spatial map of a stereo camera system. In this method, information about depth and disparity map are detected in the inputting images of a parallel stereo camera. The distance between the automatic moving robot and the obstacle is detected, and a 2D spatial map is obtained from the location coordinates. Then the relative distances between the obstacle and other objects are deduced. The robot move automatically by effective and intelligent route decision using the obtained 2D spatial map. From experiments on robot driving with 240 frames of stereo images, it was found that the error ratio of the calculated distance to the measured distance between objects was very low, 1.52[%] on average.