Siniša Šegvić

A mapping and localization framework for scalable appearance-based navigation

This paper presents a vision framework which enables feature-oriented appearance-based navigation in large outdoor environments containing other moving objects. The framework is based on a hybrid topological-geometrical environment representation, constructed from a learning sequence acquired during a robot motion under human control. At the higher topological layer, the representation contains a graph of key-images such that incident nodes share many natural landmarks. The lower geometrical layer enables to predict the projections of the mapped landmarks onto the current image, in order to be able to start (or resume) their tracking on the fly. The desired navigation functionality is achieved without requiring global geometrical consistency of the underlying environment representation. The framework has been experimentally validated in demanding and cluttered outdoor environments, under different imaging conditions. The experiments have been performed on many long sequences acquired from moving cars, as well as in large-scale real-time navigation experiments relying exclusively on a single perspective vision sensor. The obtained results confirm the viability of the proposed hybrid approach and indicate interesting directions for future work.

Determining the absolute orientation in a corridor using projective geometry and active vision

The capability of a mobile robot to determine its position in the environment (self-localization) is a prerequisite for achieving autonomous navigation. An approach is proposed for determining the absolute orientation of an autonomous robot in a system of corridors, based on the projective geometry and active computer vision. In the proposed approach, the common direction of longitudinal corridor edges is inferred by detecting the vanishing point of the corresponding straight line segments in the image. It is assumed that the knowledge about the vertical direction in the scene is available, so that the image coordinates of these vanishing points are considerably constrained. However, longitudinal corridor edges are not visible in images acquired for many viewing directions, so that the processing in a localization procedure has to be performed on a sequence of images acquired from the given position, for regularly arranged orientations of the camera. Extensive experimentation was performed on real scenes and the obtained results are provided.

Large scale vision-based navigation without an accurate global reconstruction

Autonomous cars will likely play an important role in the future. A vision system designed to support outdoor navigation for such vehicles has to deal with large dynamic environments, changing imaging conditions, and temporary occlusions by other moving objects. This paper presents a novel appearance-based navigation framework relying on a single perspective vision sensor, which is aimed towards resolving of the above issues. The solution is based on a hierarchical environment representation created during a teaching stage, when the robot is controlled by a human operator. At the top level, the representation contains a graph of key-images with extracted 2D features enabling a robust navigation by visual servoing. The information stored at the bottom level enables to efficiently predict the locations of the features which are currently not visible, and eventually (re-)start their tracking. The outstanding property of the proposed framework is that it enables robust and scalable navigation without requiring a globally consistent map, even in interconnected environments. This result has been confirmed by realistic off-line experiments and successful real-time navigation trials in public urban areas.

Enhancing the point feature tracker by adaptive modelling of the feature support

We consider the problem of tracking a given set of point features over large sequences of image frames. A classic procedure for monitoring the tracking quality consists in requiring that the current features nicely warp towards their reference appearances. The procedure recommends focusing on features projected from planar 3D patches (planar features), by enforcing a conservative threshold on the residual of the difference between the warped current feature and the reference. However, in some important contexts, there are many features for which the planarity assumption is only partially satisfied, while the true planar features are not so abundant. This is especially true when the motion of the camera is mainly translational and parallel to the optical axis (such as when driving a car along straight sections of the road), which induces a permanent increase of the apparent feature size. Tracking features containing occluding boundaries then becomes an interesting goal, for which we propose a multi-scale monitoring solution striving to maximize the lifetime of the feature, while also detecting the tracking failures. The devised technique infers the parts of the reference which are not projected from the same 3D surface as the patch which has been consistently tracked until the present moment. The experiments on real sequences taken from cars driving through urban environments show that the technique is effective in increasing the average feature lifetimes, especially in sequences with occlusions and large photometric variations.

Exploiting temporal and spatial constraints in traffic sign detection from a moving vehicle

This paper addresses detection, tracking and recognition of traffic signs in video. Previous research has shown that very good detection recalls can be obtained by state-of-the-art detection algorithms. Unfortunately, satisfactory precision and localization accuracy are more difficultly achieved. We follow the intuitive notion that it should be easier to accurately detect an object from an image sequence than from a single image. We propose a novel two-stage technique which achieves improved detection results by applying temporal and spatial constraints to the occurrences of traffic signs in video. The first stage produces well-aligned temporally consistent detection tracks by managing many competing track hypotheses at once. The second stage improves the precision by filtering the detection tracks by a learned discriminative model. The two stages have been evaluated in extensive experiments performed on videos acquired from a moving vehicle. The obtained experimental results clearly confirm the advantages of the proposed technique.

Real-time active visual tracking system

The paper describes implementation of a real-time visual tracking system equipped with an active camera. The system is intended for indoor human motion tracking. Real-time tracking is achieved using simple and fast motion detection procedures based on frame differencing and camera motion compensation. Results of on-line person tracking are presented.

Generative modeling of spatio-temporal traffic sign trajectories

We consider the task of automatic detection and recognition of traffic signs in video. We show that successful off-the-shelf detection (Viola-Jones) and classification (SVM) systems yield unsatisfactory results. Our main concern are high false positive detection rates which occur due to sparseness of the traffic signs in videos. We address the problem by enforcing spatio-temporal consistency of the detections corresponding to a distinct sign in video. We also propose a generative model of the traffic sign motion in the image plane, which is obtained by clustering the trajectories filtered by an appropriate procedure. The contextual information recovered by the proposed model will be employed in our future research on recognizing traffic signs in video.

A computer vision assisted geoinformation inventory for traffic infrastructure

Geoinformation inventories are often employed as a tool for providing a comprehensive view onto the required state of traffic control infrastructure. They are especially important in road safety inspection where, in combination with georeferenced video, they enable repeatable off-line and off-site assessments as an attractive aternative to classic onsite inspection. Nevertheless, manual assessments are tedious and time-consuming even when performed off-line, and this seriously impairs the potential of the geoinformation inventory concept. This paper therefore researches a hypothesis that suitable georeferenced video processing techniques would allow reliable automation of the following operations: i) creation of the traffic inventory from the given video, and ii) assessing the video against the state in the inventory. Prominent computer vision approaches have been rigorously and systematically evaluated and the obtained results are presented. The results seem to support the hypothesis, although further work is required for a more definite answer.

Experimental Evaluation of Autonomous Driving Based on Visual Memory and Image-Based Visual Servoing

In this paper, the performance of a topological-metric visual-path-following framework is investigated in different environments. The framework relies on a monocular camera as the only sensing modality. The path is represented as a series of reference images such that each neighboring pair contains a number of common landmarks. Local 3-D geometries are reconstructed between the neighboring reference images to achieve fast feature prediction. This condition allows recovery from tracking failures. During navigation, the robot is controlled using image-based visual servoing. The focus of this paper is on the results from a number of experiments that were conducted in different environments, lighting conditions, and seasons. The experiments with a robot car show that the framework is robust to moving objects and moderate illumination changes. It is also shown that the system is capable of online path learning.

Online/Realtime Structure and Motion for General Camera Models

This paper presents a novel algorithm for online structure and motion estimation. The algorithm works for general camera models and minimizes object space error, it does not rely on gradient-based optimization, and it is provably globally convergent. In comparison to previous work, which reports cubic complexity in the number of frames, our major contribution is a significant reduction of complexity. The new algorithm requires constant time per frame and can thus be used in online applications. Experimental results show high reconstruction accuracy with respect to simulated ground truth data. We also present two applications in artificial marker reconstruction and handheld augmented reality.