Tobias Schwarze

Robust scale estimation for monocular visual odometry using structure from motion and vanishing points

While monocular visual odometry has been widely investigated, one of its key issues restrains its broad appliance: the scale drift. To tackle it, we leverage scene inherent information about the ground plane to estimate the scale for usage on Advanced Driver Assistance Systems. The algorithm is conceived so that it is independent of the unscaled ego-motion estimation, augmenting its adaptability to other frameworks. A ground plane estimation using Structure From Motion techniques is complemented by a vanishing point estimation to render our algorithm robust in urban scenarios. The method is evaluated on the KITTI dataset, outperforming state of the art algorithms in areas where urban scenery is dominant.

A Camera-Based Mobility Aid for Visually Impaired People

We present a wearable assistance system for visually impaired persons that perceives the environment with a stereo camera system and communicates obstacles and other objects to the user in form of intuitive acoustic feedback. The system is designed to complement traditional assistance aids. We describe the core techniques of scene understanding, head tracking, and sonification and show in an experimental study that it enables users to walk in unknown urban terrain and to avoid obstacles safely.

Detection of ascending stairs using stereo vision

Environment perception is an important task in computer vision for many applications in robotics. Especially for robots navigating through different levels of a building, stair detection constitutes an important perception task. In this paper, we propose a stair detection algorithm using range data. Firstly, we introduce a parameter, which describes local surface orientations w.r.t. a global reference. Secondly, a matched filter is used to detect relevant edges in the orientation data. Afterwards, line segments are determined using these edge data which are further used to estimate stairs. The proposed method is invariant against rotations of the sensor. We show that the system can handle typical outdoor stair types and outperforms the accuracy of state-of-the-art stair detection methods. Moreover, the method is used in real time to assist visually impaired people who wear the camera system on a helmet.

Robust ground plane tracking in cluttered environments from egocentric stereo vision

Estimating the ground plane is often one of the first steps in geometric reasoning processes as it offers easily accessible context knowledge. Especially unconstrained platforms that capture video from egocentric viewpoints can benefit from such knowledge in various ways. A key requirement here is keeping orientation, which can be greatly achieved by keeping track of the ground. We present an approach to keep track of the ground plane in cluttered inner-urban environments using stereo vision in real-time. We fuse a planar model fit in low-resolution disparity data with the direction of the vertical vanishing point. Our experiments show how this effectively decreases the error of plane attitude estimation compared to classic least-squares fitting and allows to track the plane with camera configurations in which the ground is not visible. We evaluate the approach using ground-truth from an inertial measurement unit and demonstrate long-term stability on a dataset of challenging inner city scenes.

Stair detection and tracking from egocentric stereo vision

In this work we present a real-time approach to capture the properties of staircases from a free moving, head mounted stereo-camera. A variety of systems can profit from such ability, examples range from robots in multi-floor exploration scenarios to wearable assistance systems for the visually impaired. We introduce a light-weight method to measure the individual steps and use this information to update a minimal stair model while approaching and traversing the stair. Results are evaluated on an in- and outdoor scenario and show competitive accuracy to state of the art approaches working on precise lidar-sensors.

An Intuitive Mobility Aid for Visually Impaired People Based on Stereo Vision

We present a wearable assistance system for visually impaired persons that perceives the environment with a stereo camera and communicates obstacles and other objects to the user. We develop our idea of combining perception on an increased level of scene understanding with acoustic feedback to obtain an intuitive mobility aid. We describe our core techniques of scene modelling, object tracking, and acoustic feedback and show in an experimental study how our system can help improving the mobility and safety of visually impaired users.

Geometry Estimation of Urban Street Canyons Using Stereo Vision from Egocentric View

We investigate the problem of estimating the local geometric scene structure of urban street canyons captured from an egocentric viewpoint with a small-baseline stereo camera setup. We model the facades of buildings as planar surfaces and estimate their parameters based on a dense disparity map as only input. After demonstrating the importance of considering the stereo reconstruction uncertainties, we present two approaches to solve this model-fitting problem. The first approach is based on robust planar segmentation using random sampling, the second approach transforms the disparity into an elevation map from which the main building orientations can be obtained. We evaluate both approaches on a set of challenging inner city scenes and show how visual odometry can be incorporated to keep track of the estimated geometry in real-time.

Role-based identity recognition for telecasts

Semantic queries involving image understanding aspects require the exploitation of multiple clues, namely the (inter-)relations between objects and events across multiple images, the situational context, and the application context. A prominent example for such queries is the identification of individuals in video sequences. Straightforward face recognition approaches require a model of the persons in question and tend to fail in ill conditioned environments. Therefore, an alternative approach is to involve contextual conditions of observations in order to determine the role a person plays in the current context. Due to the strong relation between roles, persons and their identities, knowing either often allows inferring about the other. This paper presents a system that implements this approach: First, robust face detection localizes the actors in the video. By clustering similar face instances the relative frequency of their appearance within a sequence is determined. In combination with a coarse textual annotation manually created by the broadcast stations archivist the roles and consequently the identities can be assigned and labeled in the video. Starting with unambiguous assignments and cascading appropriately most of the persons can be identified and labeled successfully. The feasibility and performance of the role-based person identification is demonstrated on basis of several programs of a popular German TV show, which consists of various elements like interview scenes, games and musical show acts.

Role-based identity recognition for TV broadcasts

Semantic queries involving image understanding aspects require the exploitation of multiple clues, namely the (inter-) relations between objects and events across multiple images, the situational context, and the application context. A prominent example for such queries is the identification of individuals in video sequences. Straightforward face recognition approaches require a model of the persons in question and tend to fail in ill-conditioned environments. Therefore, an alternative approach is to involve contextual conditions of observations in order to determine the role a person plays in the current context. Due to the strong relation between roles, persons and their identities, knowing either often allows inferring about the other. This paper presents a system that implements this approach: First, robust face detection localizes the actors in the video. By clustering similar face instances the relative frequency of their appearance within a sequence is determined. In combination with a coarse textual annotation manually created by the broadcast station’s archivist the roles and consequently the identities can be assigned and labeled in the video. Starting with unambiguous assignments and cascading, most of the persons can be identified and labeled successfully. The feasibility and performance of the role-based person identification is demonstrated on the basis of several programs of a popular German TV show, which consists of various elements like interview scenes, games and musical show acts.

Ein intuitives kamerabasiertes System zur Assistenz sehbehinderter Menschen

Zusammenfassung Viele blinde und sehbehinderte Menschen sind stark beeinträchtigt in ihrer individuellen Mobilität. Die technischen Fortschritte im Bereich der Umfelderfassung intelligenter Systeme könnten hier einen substantiellen Beitrag leisten. In dieser Arbeit stellen wir ein tragbares Assistenzsystem für blinde Menschen vor, das die Umgebung mit einer binokularen Kamera erfasst und durch intuitives akustisches Feedback an den Benutzer übermittelt. Wir beschreiben die Algorithmen zum Szenenverstehen und zur Bestimmung der Eigenbewegung und gehen auf die Sonifikation der interpretierten Umgebung ein. In einer experimentellen Studie zeigen wir, wie die intelligente Umgebungswahrnehmung zur Sicherheit und Mobilität von Blinden beitragen kann.