ZuWhan Kim

Robust Lane Detection and Tracking in Challenging Scenarios

A lane-detection system is an important component of many intelligent transportation systems. We present a robust lane-detection-and-tracking algorithm to deal with challenging scenarios such as a lane curvature, worn lane markings, lane changes, and emerging, ending, merging, and splitting lanes. We first present a comparative study to find a good real-time lane-marking classifier. Once detection is done, the lane markings are grouped into lane-boundary hypotheses. We group left and right lane boundaries separately to effectively handle merging and splitting lanes. A fast and robust algorithm, based on random-sample consensus and particle filtering, is proposed to generate a large number of hypotheses in real time. The generated hypotheses are evaluated and grouped based on a probabilistic framework. The suggested framework effectively combines a likelihood-based object-recognition algorithm with a Markov-style process (tracking) and can also be applied to general-part-based object-tracking problems. An experimental result on local streets and highways shows that the suggested algorithm is very reliable.

Vision-based road-following using a small autonomous aircraft

This paper describes the vision-based control of a small autonomous aircraft following a road. The computer vision system detects natural features of the scene and tracks the roadway in order to determine relative yaw and lateral displacement between the aircraft and the road. Using only the vision measurements and onboard inertial sensors, a control strategy stabilizes the aircraft and follows the road. The road detection and aircraft control strategies have been verified by hardware in the loop (HIL) simulations over long stretches (several kilometers) of straight roads and in conditions of up to 5 m/s of prevailing wind. Hardware experiments have also been conducted using a modified radio-controlled aircraft. Successful road following was demonstrated over an airfield runway under variable lighting and wind conditions. The development of vision-based control strategies for unmanned aerial vehicles (UAVs), such as the ones presented here, enables complex autonomous missions in environments where typical navigation sensor like GPS are unavailable.

Real time object tracking based on dynamic feature grouping with background subtraction

Object detection and tracking has various application areas including intelligent transportation systems. We introduce an object detection and tracking approach that combines the background subtraction algorithm and the feature tracking and grouping algorithm. We first present an augmented background subtraction algorithm which uses a low-level feature tracking as a cue. The resulting background subtraction cues are used to improve the feature detection and grouping result. We then present a dynamic multi-level feature grouping approach that can be used in real time applications and also provides high-quality trajectories. Experimental results from video clips of a challenging transportation application are presented.

Autonomous Searching and Tracking of a River using an UAV

Surveillance operations include inspecting and monitoring river boundaries, bridges and coastlines. An autonomous unmanned aerial vehicle (UAV) can decrease the operational costs, expedite the monitoring process and be used in situations where a manned inspection is not possible. This paper addresses the problem of searching and mapping such littoral boundaries using an autonomous UAV based on visual feedback. Specifically, this paper describes an exploration system that equips a fixed wing UAV to autonomously search a given area for a specified structure (could be a river, a coastal line etc.), identify the structure if present and map the coordinates of the structure based on the images from the onboard sensor(could be vision or near infra-red). Experimental results with a fixed wing UAV searching and mapping the coordinates of a 2 mile stretch of a river with a cross track error of around 9 meters are presented.

Flight Demonstrations of Self-directed Collaborative Navigation of Small Unmanned Aircraft

The development of small, autonomous UAVs that can operate in complex environments as part of large coordinated groups will enable many new applications at fractions of the cost of current systems. A fleet of fixed-wing aircraft has been developed to create an intelligent aerial platform that has demonstrated various autonomous capabilities. By means of a downward-looking camera, a single aircraft autonomously follows a roadway using the natural features of the scene in conjunction with onboard sensors, without the use of GPS or prior knowledge of the road’s coordinates. A forward looking camera is used to perceive obstacles in the aircraft’s flight path by segmenting images into sky/no-sky regions and classifying no-sky regions above the horizon as obstacles. The tracking of friendly ground vehicles – for which GPS information is known but path information is not – is performed using circular and sinusoidal orbits to maintain desired proximity regardless of ground vehicle motion. Teams of two or three aircraft demonstrate convoy protection by providing persistent surveillance around a moving ground vehicle. The team either flies in rigid formation, providing a large area of coverage around the ground vehicle, or the aircraft coordinate several separate actions that provide both lateral (side to side) and longitudinal (front to back) surveillance of the ground vehicle’s path. Details of the hardware platform and each capability are described and results of flight demonstrations are presented.

Spatio-Temporal Traffic Scene Modeling for Object Motion Detection

Moving object detection is an important component of a traffic surveillance system. Usual background subtraction approaches often poorly perform on a long outdoor traffic video due to vehicles waiting at an intersection and gradual changes of illumination and background shadow position. We present a fast and robust background subtraction algorithm based on unified spatio-temporal background and foreground modeling. The correlation between neighboring pixels provides high levels of detection accuracy in the dynamic background scene. Our Bayesian fusion method, which establishes the traffic scene model, combines both background and foreground models and considers prior probabilities to adapt changes of background in each frame. We explicitly model both temporal and spatial information based on the kernel density estimation (KDE) formulation for background modeling. Then, we use a Gaussian formulation to describe the spatial correlation of moving objects for foreground modeling. In the updating step, a fusion background frame is generated, and reasonable updating rates are also proposed for the traffic scene. The experimental results show that the proposed method outperforms the previous work with less computation and is better suited for the traffic scenes.

Pseudoreal-time activity detection for railroad grade-crossing safety

It is important to understand the factors underlying grade-crossing crashes and to examine potential solutions. We have installed a camera in front of a locomotive to examine grade-crossing accidents (or near accidents). We present a computer vision system that automatically extracts possible near-accident scenes by detecting the activity of vehicles crossing in front of the train after signals are ignited. We present a fast algorithm to detect moving objects recorded by a moving camera with minimal computation. The moving object is detected by: 1) estimating the ego motion of the camera and 2) detecting and tracking feature points whose motion is inconsistent with the camera motion. We introduce a pseudoreal-time ego-motion (camera-motion) estimation method with a robust optimization algorithm. We present experiments on ego-motion estimation and moving-object detection. Our algorithm works in pseudoreal-time and we expect that our algorithm can be applied to real-time applications such as collision warning in the near future, with the development of hardware technology.

Evaluation of feature-based vehicle trajectory extraction algorithms

Vehicle trajectories are and can be used in various intelligent transportation systems applications including driver behavior modelling and safety. Video-based approaches have been used to extract a large number of non-cooperative trajectories. However, it is difficult to evaluate the accuracies of the resulting trajectories. An algorithm-specific simulation tool is developed to evaluate the feature-grouping algorithm. We introduce a Kalman smoothing model to estimate vehicle trajectories and compare it with our previous rescaling-based trajectory estimation algorithm using the simulation tool. A comparison with GPS (WAAS) on real video clip is also presented. Our evaluation shows that the feature-based algorithms provide more accurate trajectories than those by previous approaches including one for the NGSIM system.

Pseudo-realtime activity detection for railroad grade crossing safety

It is important to understand the factors underlying grade crossing crashes, and to examine potential solutions. We have installed a camera in front of a locomotive to examine grade crossing accidents (or near accidents). We present a computer vision system that automatically extracts possible near accidents scenes by detecting the activity of vehicles crossing in front of the train after the signals are ignited. We presented a fast algorithm to detect moving objects that is recorded by a moving camera with minimal computation. The moving object is detected by 1) estimating ego-motion of the camera and 2) detecting and tracking feature points whose motion is inconsistent with the camera motion. We introduce a pseudo-realtime ego-motion (camera motion) estimation method with a robust optimization algorithm. We present experiments on ego-motion estimation and moving object detection. Our algorithm works in pseudo-realtime and we expect that our algorithm can be applied to realtime applications, such as collision warning, in the near future with the development of hardware technology.

Bicyclist Intersection Crossing Times

In support of efforts to improve traffic signal timing to accommodate bicyclists’ needs, observations were made of the timing of bicyclists’ intersection crossing maneuvers. Video recordings were made of bicyclists’ crossings and the video images were processed to extract the bicyclists’ trajectories. These trajectories were synchronized with video images of the traffic signals so that the timing of the bicyclists’ maneuvers could be determined relative to the signal phases. The processed data yielded cumulative distributions of the crossing speeds of bicyclists who did not have to stop at the intersection and the start-up times and final crossing speeds of the bicyclists who had to cross from a standing start. A unique feature of these data is the timing information relative to the traffic signal, which is used to define recommended signal times to permit most bicyclists to cross wide arterial intersections safely.