Masato Kurokawa

Method for retrieving sequences of images on the basis of motion analysis

This paper describes a method for retrieving sequences of images in a motion image database using motion information as a key. This information is obtained by application of the motion analysis technique. The proposed method has three steps: automatic extraction of motion vectors through frame sequences, motion description using spatio-temporal space, and retrieval of sequences of images. The motion vectors of objects are extracted on the basis of block motion estimation. Block motion estimation tracks each block of a scene throughout a period. This tracking procedure takes account of the traceability of each block and the reliability of its calculated motion vector. Extracted motion vectors are mapped using spatio- temporal space (x-y-t). The motion of each block is represented as one vector in the feature space, and motion vectors in a scene are aggregated into several representative vectors by statistical analysis. The retrieval of sequences of images is divided into two parts: specification of the query conditions, and matching between the query conditions and the motion database. The query conditions are input from a stroke device. The user can interactively manipulate the query conditions, such as the number, period, and starting time of objects. The data are gathered as an x-y series at a given interval, and then normalized. Similarity is defined in spatio-temporal space as the distance between the query conditions and the stored motion index. Candidate sequences of images are ranked in order of distance. An experimental system has been developed for retrieving sequences of images on a workstation.

Representation and retrieval of video scene by using object actions and their spatio-temporal relationships

In this paper we present a method for representing and retrieving video sequences based on the domain-specific behavior of objects present in the scenes. The representation includes three parts: (a) Action description, representing the action performed by a single object. (b) Interaction description, which describes interactions between multiple objects and is mapped directly to the event semantics in the content domain. (c) Event Structure, which provides a set of spatial and temporal relationship functions, along with a syntax to define the necessary conditions that a particular interaction should meet. Retrieval is performed by processing Event Structures, interpreting object relationships and selecting the relevant combinations of Action descriptions which match the conditions defined in the Event Structures. We describe an implementation of this system for retrieving scenes of soccer plays from among several soccer video sequences.

Estimation of motion vectors and their application to scene retrieval

This paper describes a method for retrieving scenes from a motion picture database by using motion information as a key. The method has three steps: the automatic estimation of motion vectors in frame sequences, the description of motions in spatio-temporal space, and the retrieval of sequences of images. The motion vectors are estimated by block matching. Estimated motion vectors are mapped in spatio-temporal space (x, y, t). Motion vectors in a scene are aggregated into several representative vectors by statistical analysis. The retrieval of scenes is divided into two parts: the specification of query conditions, and matching between the query conditions and the motion database. Similarity is defined in spatio-temporal space as the distance between the query conditions and the stored motion index. Candidate scenes are ranked in order of distance. Experimental results indicate that the proposed method is effective.