Hans-Joachim Novak

Generating a coherent text describing a traffic scene

If a system that embodies a reference semantic for motion verbs and prepositions is to generate a coherent text describing the recognized motions it needs a decision procedure to select the events. In NAOS event selection is done by use of a specialization hierarchy of motion verbs. The strategy of anticipated visualization is used for the selection of optional deep cases. The system exhibits low-level strategies which are based on verbinherent properties that allow the generation of a coherent descriptive text.

On the Selection of Verbs for Natural Language Description of Traffic Scenes

A vocabulary for the description of real-world image sequences of a traffic scene is introduced. Its descriptive power and completeness is discussed. Different knowledge sources like knowledge about trajectories, special places in the domain, standard properties and situations and knowledge associating pragmatic situations with observations of motion, are shown to determine the applicability of verbs. The selection of a verb out of several applicable verbs for a given image sequence is discussed.

On Verbalizing Real-World Events: An Interface of Natural Language and Vision

The process of verbalization is first viewed from a psychological point of view. Then the architecture of the NAOS system is described and is shown to reflect the earlier findings. Especially it is shown that verbalization is a cognitive decision process which is guided by pragmatic context.

Natural Language Inquiries abouth Motion in an Automatically Analyzed Traffic Scene

This contribution is concerned with natural language dialogue about scenes with moving objects. Two systems are connected, a natural language dialogue system originally conceived for static scenes and an emerging scene analysis system for real-world TV-frame sequences. The latter produces time dependent object descriptions which serve as a referential database for inquiries. The time intervals relevant for answering the questions are determined from domain specific parameters, the context of the dialogue, the tense of the verbs and time adverbials. For checking the correspondence between a verbally specified motion and a trajectory, predicates are evaluated which can be deduced from the verb’s case-frame.

A Relational Matching Strategy for Temporal Event Recognition

In this contribution a relational matching strategy is described. It allows to handle temporal information like the beginning and end of events, durativity and non-durativity correctly. The implementation and use of this strategy for temporal event recognition in the NAOS system is discussed.