Werner Krybus

Towards real-time camera egomotion estimation and three-dimensional scene acquisition from monocular image streams

The three-dimensional reconstruction of rigid scenes from monocular image streams is based on the former calculation of the relative camera pose between at least two successive image frames. This egomotion estimation has not been solved satisfactorily by relying only on corresponding image features, such as points or lines, due to noise, critical motion patterns or special point configurations. This paper describes a framework for incorporating inertial measurements from gyroscopes, accelerometers and magnetometers to achieve an improved performance of the estimation of camera motion and scene structure in terms of accuracy, robustness and computational costs. The framework is designed as a dual-track system containing a visual and a inertial route.

Probabilistic Scene Analysis for Robust Stereo Correspondence

Most area-based approaches for stereo correspondence are leading to a large set of non-correct matches in the generated disparity-map. These are mainly caused by low textured areas, half occlusions, discontinuities in depth and the occurrence of repetitive patterns in the observed scene. This paper proposes a novel framework where non salient regions inside the stereo pair are identified previously to the matching, whereat the decision about the involvement of particular areas in the correspondence analysis is realized based on the fusion of separate confidence maps. They describe the possibility for a correct matching based on different criteria.

Experiences with LEGO MINDSTORMS as an Embedded and Robotics Plattform within the Undergraduate Curriculum

This paper introduces a concept for the integration of the LEGO MINDSTORMS system into the undergraduate curriculum for electrical engineering students. The concept proposes an optional project-oriented module for students in the first semester and contains typical topics from programming, embedded design and mobile robotics. Besides the positive effects to the professional skills of the students, also secondary virtues, such as team-building and motivation are addressed by the course. This article summarises the experiences from the installation of the additional module and shows possible opportunities for additional MINDSTORMS-based projects and courses.

A visual-inertial approach for camera egomotion estimation and simultaneous recovery of scene structure

The estimation of a cameras egomotion is a highly desireable goal in many different application fields such as augmented reality (AR), visual navigation, robotics or entertainment. Especially for real-time modeling the former estimation of the camera trajectory is an elementary step towards the generation of three dimensional scene models. This paper presents a framework for simultaneous recovery of scene structure and camera motion by combining visual and inertial cues. For this purpose two different system designs are proposed: a loosely-coupled system and a monolithic design, which adapts ideas from non-linear state estimation as extended Kalman filtering (EKF) for structure and motion recovery.

Spatio-Temporal Scene Analysis Based on Graph Algorithms to Determine Rigid and Articulated Objects

We propose a novel framework in the context of structure and motion for representing and analyzing three-dimensional motions particularly for human heads and faces. They are captured via a stereo camera system and a scene graph is constructed that contains low and high-level vision information. It represents and describes the observed scene of each frame. By creating graphs of successive frames it is possible to match, track and segment main important features and objects as a structure of each scene and reconstruct these features into the three dimensional space. The cue-processor extracts feature information like 2D- and 3D-position, velocity, age, neighborhood, condition, or relationship among features that are stored in the vertices and weights of the graph to improve the estimation and detection of the features and/or objects in the continuous frames. The structure and change of the graph leads to a robust determination and analysis of changes in the scene and to segment and determine these changes even for temporal and partial occluded objects over a long image sequence.

Solving the PnP Problem for Visual Odometry – An Evaluation of Methodologies for Mobile Robots

The general procedure of visual odomentry (VO) for a mobile robot based on a monocular image stream can be subdivided into different subtasks. The minimal configuration of a VO framework illustrated in the following figure contains three major subtasks: feature handling, structure recovery and motion recovery. The motion recovery is solved by incorporation of general ideas from the fields of photogrammety and stereo vision, where homologous image information is used to derive the geometrical (epipolar) relations between two different images captured from different viewpoints.

Visual-Inertial 2D Feature Tracking based on an Affine Photometric Model

The robust tracking of point features throughout an image sequence is one fundamental stage in many different computer vision algorithms (e.g. visual modelling, object tracking, etc.). In most cases, this tracking is realised by means of a feature detection step and then a subsequent re-identification of the same feature point, based on some variant of a template matching algorithm. Without any auxiliary knowledge about the movement of the camera, actual tracking techniques are only robust for relatively moderate frame-to-frame feature displacements. This paper presents a framework for a visual-inertial feature tracking scheme, where images and measurements of an inertial measurement unit (IMU) are fused in order to allow a wider range of camera movements. The inertial measurements are used to estimate the visual appearance of a feature’s local neighbourhood based on a affine photometric warping model.

Dynamic World Modelling by Dichotomic Information Sets and Graphical Inference

This report establishes a novel concept for tracking complex and articulated objects in the presence of high observation uncertainties utilising Markov random fields Markov chains (MRFMCs) and a novel paradigm of modelling visual perception. The approach is rooted in ideas from information fusion and cognitive sciences. The problem is to track non-rigid and articulated objects in the 3D space. The aim is to precisely estimate landmarks with high certainty for fitting accurate object models and secondary states like the orientation under partial occlusions. The targeted system is characterised by a high degree of generality. Previous solutions are relatively limited in robustness and accuracy. The new concept is motivated by the fact that all previous tracking approaches rely on semantic information, that is classified signal signatures, while neglecting all further non-classifiable and thus semantically unrelated information present in the scene herein abstracted as structure. By observing salient cues in structure and by learning and incorporating topological relations between salient cues and semantic features it is intended to tackle the major problem of visual tracking, namely accurate and robust inference in the presence of high observation uncertainties. The notion of the dichotomy of semantic and structure is not covered in previous literature. The new concept constitutes a novel direction in the design and implementation of visual perception and tracking networks. While the ideas of dynamic world modelling and intelligent forgetting stem from principles of information fusion, the principle of fusing semantical with structural information from intelligent exploring is an entirely original contribution and is inspired by ideas from cognitive sciences and linguistics. It is deduced from the inherent yet unrevealed principle of appearance modelling, which is based on incorporating object-related appearance information without classification. In this report the presented system is applied to high-level facial pose tracking and compared to a state-of-the-art reference method.

A New Approach on Spatio-temporal Scene Analysis for Driver Observation

Advanced Driver Assistance Systems are, due to their potentials regarding security and markets, in the focus of future developments within the automotive industry. The visual observation of the car interior is gaining attention due to the increasing efficiency of methods and technologies in digital image processing. Special weight is put on the visual driver observation, which measures diversion and fatigue of the driver and notifies about endangering behavior. This is accomplished by utilizing complex image-processing systems. The spatial positions and orientations of head and eyes are measured and evaluated. This report presents in detail and coherently the motivation and the current status of available approaches and systems. Following, a new concept for spatio-temporal modeling and tracking of partially rigid objects is developed and described. This concept is based on methods for spatio-temporal scene analysis, graph theory, adaptive information fusion and multi-hypothesis tracking. Our original contributions are the detailed representation of the available procedures and systems in this certain field and the development of a new concept and related prototypes.