Silviu Bota

Stereo-Based Pedestrian Detection for Collision-Avoidance Applications

Pedestrians are the most vulnerable participants in urban traffic. The first step toward protecting pedestrians is to reliably detect them. We present a new approach for standing- and walking-pedestrian detection, in urban traffic conditions, using grayscale stereo cameras mounted on board a vehicle. Our system uses pattern matching and motion for pedestrian detection. Both 2-D image intensity information and 3-D dense stereo information are used for classification. The 3-D data are used for effective pedestrian hypothesis generation, scale and depth estimation, and 2-D model selection. The scaled models are matched against the selected hypothesis using high-performance matching, based on the Chamfer distance. Kalman filtering is used to track detected pedestrians. A subsequent validation, based on the motion fields variance and periodicity of tracked walking pedestrians, is used to eliminate false positives.

Real-time obstacle detection using dense stereo vision and dense optical flow

Accurate detection of moving obstacles from a moving vehicle is at the core of safe autonomous driving research. Stereo vision based sensors have been extensively used for this task as they are passive and provide a large amount 3D and 2D data. However, since no motion information is revealed, in intersections or crowded urban areas, static and dynamic objects immediately next to each other, or closely positioned obstacles moving in different directions are often merged into a single obstacle leading to dangerous misinterpretations. In this paper we address these problems through a powerful fusion between dense stereo vision and dense optical flow in a depth-adaptive occupancy grid framework. The proposed fusion model is presented and then applied for obstacle detection in an intersection assistance system.

Stereovision-Based Sensor for Intersection Assistance

The intersection scenario imposes radical changes in the physical setup and in the processing algorithms of a stereo sensor. Due to the need for a wider field of view, which comes with distortions and reduced depth accuracy, increased accuracy in calibration and dense stereo reconstruction is required. The stereo matching process has to be performed on rectified images, by a dedicated stereo board, to free processor time for the high-level algorithms. In order to cope with the complex nature of the intersection, the proposed solution perceives the environment in two modes: a structured approach, for the scenarios where the road geometry is estimated from lane delimiters, and an unstructured approach, where the road geometry is estimated from elevation maps. The structured mode provides the parameters of the lane, and the position, size, speed and class of the static and dynamic objects, while the unstructured mode provides an occupancy grid having the cells labeled as free space, obstacle areas, curbs and isles.

Intersection safety using lidar and stereo vision sensors

In this paper, we describe our approach for intersection safety developed in the scope of the European project INTERSAFE-2. A complete solution for the safety problem including the tasks of perception and risk assessment using on-board lidar and stereo-vision sensors will be presented and interesting results are shown.

Tracking multiple objects in urban traffic environments using dense stereo and optical flow

Stereo vision based sensors provide large amounts of data, a fact which is advantageous when trying to extract semantic information about the imaged scene. However, this data is corrupted by errors, caused especially by the uncertainties in the stereo reconstruction process. Temporal information can be used in order to minimize these errors. This paper presents an advanced object model, a novel association mechanism and the design of a Kalman filter based tracking algorithm, for tracking multiple objects, in complex, urban traffic scenarios.

Real time stereo vision based pedestrian detection using full body contours

There are many approaches to pedestrian detection in collision avoidance systems depending on the sensors (visible light, thermal infrared, RADAR, LASER scanner) used for acquiring the data and the features (depth, shape, motion) used for detection. In this paper we present a method for shape based pedestrian detection in traffic scenes using a stereo vision system for acquiring the image frames and a contour matching technique for classifying the scene objects as belonging or not to the pedestrian class. The 3D information is used for determining the foreground points of each 2D object and a contour extraction and merging algorithm is performed on these points. A 2D image filtering is performed and edges are extracted and used for objects contours refinement. A hierarchy of pedestrian full body contours and a matching technique are used for classifying the extracted objects contours from the scene.

Camera motion estimation using monocular and stereo-vision

A preliminary step for many computer vision algorithms is the estimation of camera motion. In this paper we describe and compare the results of three algorithms for camera motion estimation, which require various degrees of knowledge about the camera(s) used and the scenario. The first algorithm uses a single uncalibrated camera and it is useful for applications such as content based image retrieval. This method provides only qualitative results. The second algorithm uses a single calibrated camera in order to obtain quantitative results. The third approach uses a calibrated stereo camera pair, together with a hardware stereo reconstruction system.

On-board stereo sensor for intersection driving assistance architecture and specification

This paper presents the specifications and architecture for a stereovision sensor to be used in intersection assistance. The intersection problem imposes a wide field of view, reasonable accuracy for the typical intersection length, and fast response time. The image and 3D data provided by the low level routines are used to generate two kinds of environment descriptions - an unstructured description, composed of elevation maps, occupancy grids, and polylines delimiting obstacle areas and curbs, and a structured description, composed of lanes, cuboidal objects, and classified pedestrians. The descriptions can be further combined, and additional data sources can be used, in order to provide a complete and accurate description of the intersection.

Spatio-temporal reasoning for traffic scene understanding

In this paper we introduce a system for semantic understanding of traffic scenes. The system detects objects in video images captured in real vehicular traffic situations, classifies them, maps them to the OpenCyc1 ontology and finally generates descriptions of the traffic scene in CycL or cvasi-natural language. We employ meta-classification methods based on AdaBoost and Random forest algorithms for identifying interest objects like: cars, pedestrians, poles in traffic and we derive a set of annotations for each traffic scene. These annotations are mapped to OpenCyc concepts and predicates, spatiotemporal rules for object classification and scene understanding are then asserted in the knowledge base. Finally, we show that the system performs well in understanding traffic scene situations and summarizing them. The novelty of the approach resides in the combination of stereo-based object detection and recognition methods with logic based spatio-temporal reasoning.

Vision based obstacle tracking in urban traffic environments

Stereo vision based sensors provide large amounts of data, a fact which is advantageous when trying to extract semantic information about the imaged scene. However, this data is corrupted by errors, caused especially by the uncertainties in the stereo reconstruction process. Temporal information can be used in order to minimize these errors. This paper presents an advanced object model, a novel association mechanism and the design of a Kalman filter based tracking algorithm, for tracking multiple objects, in complex, urban traffic scenarios.