Jilin Liu

Real-time stereo vision system using adaptive weight cost aggregation approach

Many vision applications require high-accuracy dense disparity maps in real time. Due to the complexity of the matching process, most real-time stereo applications rely on local algorithms in the disparity computation. These local algorithms generally suffer from matching ambiguities as it is difficult to find appropriate support for each pixel. Recent research shows that algorithms using adaptive cost aggregation approach greatly improve the quality of disparity map. Unfortunately, although these improvements are excellent, they are obtained at the expense of high computational. This article presents a hardware implementation for speeding up these methods. With hardware friendly approximation, we demonstrate the feasibility of implementing this expensive computational task on hardware to achieve real-time performance. The entire stereo vision system, includes rectification, stereo matching, and disparity refinement, is realized using a single field programmable gate array. The highly parallelized pipeline structure makes system be capable to achieve 51 frames per second for 640 × 480 stereo images. Finally, the success of accuracy improvement is demonstrated on the Middlebury dataset, as well as tests on real scene.

Guided depth enhancement via a fast marching method

Range imaging sensors such as Kinect and time-of-flight cameras can produce aligned depth and color images in real time. However, the depth maps captured by such sensors contain numerous invalid regions and suffer from heavy noise. These defects more or less influence the use of depth information in practical applications. In order to enhance the depth maps, this paper proposes a new inpainting approach based on the fast marching method (FMM). We extend the inpainting model and the propagation strategy of FMM to incorporate color information for depth inpainting. An edge-preserving guided filter is further applied for noise reduction. To validate our algorithm, we perform experiments on both Kinect data and Middlebury dataset which, respectively, provide qualitative and quantitative results. Meanwhile, we also compare it to the original FMM and other two state-of-the-art depth enhancement methods. Experimental results show that our method performs better than the local methods in terms of both visual and metric qualities, and it achieves visually comparable results to the time-consuming global method.

Polarization-Based Water Hazards Detection for Autonomous Off-road Navigation

A polarization-based method for water hazards detection is presented. The concept of polarization is introduced to computer vision to detect water hazards for autonomous off-road navigation. This method is based on the physical principle that the light reflected from water surface is partial linearly polarized and the polarization phases of them are more similar than those from the scenes around. Water hazards can be detected by comparison of polarization degree and similarity of the polarization phases. Experiments show that the method has good performance in water detection in complex natural backgrounds, especially when there is vegetation reflected in the water. This method has significant complementary advantages with respect to existing techniques, is computationally efficient, and can be easily implemented with existing imaging technology.

Improved real-time correlation-based FPGA stereo vision system

In this paper, we describe the design and implementation of Field Programmable Gate Array (FPGA) based stereo vision system. Implementation of correlation-based algorithm is well designed to exploit the parallelism in FPGA. The system is capable of running at more than 70 frames per second with 512×512 images. To improve the inherent weakness of simple correlation-based algorithm, two methods have been added. The multi-window approach improves disparity quality at object borders and cross checking reduces possible errors in general. The experimental comparison shows the improvement on final results.

Multi-Feature Fusion Based Outdoor Water Hazards Detection

Water hazards such as ponds usually threaten the unmanned ground vehicle autonomous off-road navigation. The water hazards detection is a great challenge when using the machine vision. The method of multi-feature fusion is applied here for this kind of detection. By using the color cameras, we extract the features of brightness and texture from the non-reflection regions. Further more, a new stereo vision based method is also proposed to get the height and the distance information of the reflection regions, and then the corresponding water regions can be acquired. Finally all the features are fused together and the accurate water regions from the images can be detected. The experimental results demonstrate the efficiency of this method.

An adaptive fast search algorithm for block motion estimation in H.264

Motion estimation is an important issue in H.264 video coding systems because it occupies a large amount of encoding time. In this paper, a novel search algorithm which utilizes an adaptive hexagon and small diamond search (AHSDS) is proposed to enhance search speed. The search pattern is chosen according to the motion strength of the current block. When the block is in active motion, the hexagon search provides an efficient search means; when the block is inactive, the small diamond search is adopted. Simulation results showed that our approach can speed up the search process with little effect on distortion performance compared with other adaptive approaches.

Stereo vision based SLAM in outdoor environments

Simultaneous localization and mapping is a well studied problem as it is considered by many to be an essential capability for autonomous robots. In this paper, we present an algorithm using a stereo camera based on Rao-Blackwelltion particle filter, It can realize three dimensional stereo vision SLAM for mobile robot in unknown outdoor environments. Firstly, we determine the initial motion estimation between two adjacent frames through the multiple view geometry utilizing the matched SIFT point pairs. Then, the 3D positions of landmarks are constructed directly through triangulation methodology. With accurate data association, the cameras state and the landmark positions are updated recursively. Finally, an efficient particle resamping algorithm MPR (the modified particle resampling algorithm) is proposed to deal with the degeneracy of particles.

Polarization-based water hazards detection for autonomous off-road navigation

A polarization-based method for water hazards detection is presented. The concept of polarization is introduced to computer vision to detect water hazards for autonomous off-road navigation. This method is based on the physical principle that the light reflected from water surface is partial linearly polarized and the polarization phases of them are more similar than those from the scenes around. Water hazards can be detected by comparison of polarization degree and similarity of the polarization phases. Experiments show that the method has good performance in water detection in complex natural backgrounds, especially when there is vegetation reflected in the water. This method has significant complementary advantages with respect to existing techniques, is computationally efficient, and can be easily implemented with existing imaging technology.

Visual traversability analysis for micro planetary rover

Traversability analysis is a significant and challenging problem for planetary rovers navigation and safeguard on rocky and bumpy terrain. Moreover, the growing requirement for the micro rover in future planetary missions demands robust traversability analysis upon limited sensors and field of perception. In this paper, a framework for stereo based terrain mapping and traversability analysis in autonomous navigation is developed for the micro planetary rover. In this work, a Joint Reliability based Disparity Expansion (JRDE) algorithm is proposed to obtain accurate 3D information of the local terrain. Focusing on the low complexity and robust performance, we report a hierachical Digital Elevation Map (DEM) based method to achieve the traversability map with relative index value, which consists of elevation thresholding and estimation of slope and roughness. Path planning is subsequently fulfilled upon the traversability map. This method is validated with a set of planet-surface like scenes, while complete onboard system has also been proved fast and effective in artificial rocky environment.

Lane detection in critical shadow conditions based on double A/D convertors camera

Lane detection in unstructured environments is the basis for navigation of mobile robots. A method for detecting lane in critical shadow conditions is proposed. Based on the color information of the unstructured lane, an improved region-growing algorithm is employed to segment the image. To enhance the image quality and the accuracy of the algorithm, a double A/D convertors camera is used to recover the color space information of the environments in critical shadow conditions. The results demonstrate that proposed method segments the lane effectively, and is robust against shadows, noises and varied illuminations.