Franz Kummert

ERNEST: a semantic network system for pattern understanding

The authors give a detailed account of a system environment for treating general problems of image and speech understanding. A framework for representing declarative and procedural knowledge based on a suitable definition of a semantic network is provided. The syntax and semantics of the network are clearly defined. The pragmatics of the network in its use for pattern understanding are defined by several rules which are problem-independent. This allows one to formulate problem-independent control algorithms. Complete software environments are available to handle the described structures. The general applicability of the network system is demonstrated by short descriptions of three applications from different task domains. >

Methods for automatic microarray image segmentation

This paper describes image processing methods for automatic spotted microarray image analysis. Automatic gridding is important to achieve constant data quality and is, therefore, especially interesting for large-scale experiments as well as for integration of microarray expression data from different sources. We propose a Markov random field (MRF) based approach to high-level grid segmentation, which is robust to common problems encountered with array images and does not require calibration. We also propose an active contour method for single-spot segmentation. Active contour models describe objects in images by properties of their boundaries. Both MRFs and active contour models have been used in various other computer vision applications. The traditional active contour model must be generalized for successful application to microarray spot segmentation. Our active contour model is employed for spot detection in the MRF score functions as well as for spot signal segmentation in quantitative array image analysis. An evaluation using several image series from different sources shows the robustness of our methods.

Long-term vehicle motion prediction

Future driver assistance systems will have to cope with complex traffic situations, especially in the road crossing scenario. To detect potentially hazardous situations as early as possible, it is therefore desirable to know the position and motion of the ego-vehicle and vehicles around it for several seconds in advance. For this purpose, we propose in this study a long-term prediction approach based on a combined trajectory classification and particle filter framework. As a measure for the similarity between trajectories, we introduce the quaternion-based rotationally invariant longest common subsequence (QRLCS) metric. The trajectories are classified by a radial basis function (RBF) classifier with an architecture that is able to process trajectories of arbitrary non-uniform length. The particle filter framework simultaneously tracks and assesses a large number of motion hypotheses (∼102), where the class-specific probabilities estimated by the RBF classifier are used as a-priori probabilities for the hypotheses of the particle filter. The hypotheses are clustered with a mean-shift technique and are assigned a likelihood value. Motion prediction is performed based on the cluster centre with the highest likelihood. While traditional motion prediction based on curve radius and acceleration is inaccurate especially during turning manoeuvres, we show that our approach achieves a reasonable motion prediction even for long prediction intervals of 3 s for these complex motion patterns.

Control and explanation in a signal understanding environment

To interpret sensor signals like images, image sequences, or continuous speech the representation and use of task-specific knowledge is necessary. The paper presents a framework for the representation of declarative and procedural knowledge using a suitable definition of a semantic network. Based on that formalism a problem-independent control algorithm for the interpretation of sensor signals is presented. It provides both data-driven and model-driven control structures which can easily be combined to perform any mixed strategy. An explanation facility is available which makes the development of complex knowledge bases easier and increases the acceptance of such a knowledge-based analysis system.

Automatic Segmentation of Unstained Living Cells in Bright-Field Microscope Images

The automatic subcellular localisation of proteins in living cells is a critical step in determining their function. The evaluation of fluorescence images constitutes a common method of localising these proteins. For this, additional knowledge about the position of the considered cells within an image is required. In an automated system, it is advantageous to recognise these cells in bright-field microscope images taken in parallel with the regarded fluorescence micrographs. Unfortunately, currently available cell recognition methods are only of limited use within the context of protein localisation, since they frequently require microscopy techniques that enable images of higher contrast (e.g. phase contrast microscopy or additional dyes) or can only be employed with too low magnifications. Therefore, this article introduces a novel approach to the robust automatic recognition of unstained living cells in bright-field microscope images. Here, the focus is on the automatic segmentation of cells.

Spatial ray features for real-time ego-lane extraction

In order to support driver assistance systems in unconstrained environments, we propose to extend local appearance-based road classification with a spatial feature generation and classification. Therefore, a hierarchical approach consisting of multiple low level base classifiers, the novel spatial feature generation, as well as a final road terrain classification, is used. The system perceives a variety of local properties of the environment by means of base classifiers operating on patches extracted from monocular camera images, each represented in a metric confidence map. The core of the proposed approach is the computation of spatial ray features (SPRAY) from these confidence maps. With this, the road-terrain classifier can decide based on local visual properties and their spatial layout in the scene. In order to show the feasibility of the approach, the extraction and evaluation of the metric ego-lane driving corridor on an inner city stream is demonstrated. This is a challenging task because on a local appearance level, ego-lane is not distinguishable from other asphalt parts on the road. However, by incorporating the proposed SPRAY features the distinction is possible without requiring an explicit lane model. Due to the parallel structure of this bottom-up approach, the implemented system operates in real-time with approximately 25 Hz on a GPU.

Recognition of situation classes at road intersections

The recognition and prediction of situations is an indispensable skill of future driver assistance systems. This study focuses on the recognition of situations involving two vehicles at intersections. For each vehicle, a set of possible future motion trajectories is estimated and rated based on a motion database for a time interval of 2–4 seconds ahead. Realistic situations are generated by a pairwise combination of these individual motion trajectories and classified according to nine categories with a polynomial classifier. In the proposed framework, situations are penalised for which the time to collision significantly exceeds the typical human reaction time. The correspondingly favoured situations are combined by a probabilistic framework, resulting in a more reliable situation recognition and collision detection than obtained based on independent motion hypotheses. The proposed method is evaluated on a real-world differential GPS data set acquired during a test drive of 10 km, including three road intersections. Our method is typically able to recognise the situation correctly about 1–2 seconds before the distance to the intersection centre becomes minimal.

Monocular Road Terrain Detection by Combining Visual and Spatial Information

For future driver assistance systems and autonomous vehicles, the road course, i.e., the width and shape of the driving path, is an important source of information. In this paper, we introduce a new hierarchical two-stage approach for learning the spatial layout of road scenes. In the first stage, base classifiers analyze the local visual properties of patches extracted from monocular camera images and provide metric confidence maps. We use classifiers for road appearance, boundary appearance, and lane-marking appearance. The core of the proposed approach is the computation of SPatial RAY (SPRAY) features from each metric confidence map in the second stage. A boosting classifier selecting discriminative SPRAY features can be trained for different types of road terrain and allows capturing the local visual properties together with their spatial layout in the scene. In this paper, the extraction of road area and ego-lane on inner-city video streams is demonstrated. In particular, the detection of the ego-lane is a challenging semantic segmentation task showing the power of SPRAY features, because on a local appearance level, the ego-lane is not distinguishable from other lanes. We have evaluated our approach operating at 20 Hz on a graphics processing unit on a publicly available data set, demonstrating the performance on a variety of road types and weather conditions.

A speech understanding and dialog system with a homogeneous linguistic knowledge base

This article presents the speech understanding and dialog system EVAR. All levels of linguistic knowledge are used both to control the analysis process and for the interpretation of an utterance. All kinds of knowledge are integrated in a homogeneous knowledge base. The control algorithm used for the analysis is defined within the representation scheme and does not depend on the application. One of the aims of EVAR is to develop a system structure where linguistic and nonlinguistic expectations could be used not only for the interpretation but also as predictions for the recognition process. >

Vehicle tracking and motion prediction in complex urban scenarios

The recognition of potentially hazardous situations on road intersections is an indispensable skill of future driver assistance systems. In this context, this study focuses on the task of vehicle tracking in combination with a long-term motion prediction (1-2 s into the future) in a dynamic scenario. A motion-attributed stereo point cloud obtained using computationally efficient feature-based methods represents the scene, relying on images of a stereo camera system mounted on a vehicle. A two-stage mean-shift algorithm is used for detection and tracking of the traffic participants. A hierarchical setup depending on the history of the tracked object is applied for prediction. This includes prediction by optical flow, a standard kinematic prediction, and a particle filter based motion pattern method relying on learned object trajectories. The evaluation shows that the proposed system is able to track the road users in a stable manner and predict their positions at least one order of magnitude more accurately than a standard kinematic prediction method.