Atsuo Yoshitaka

A survey on content-based retrieval for multimedia databases

Conventional database systems are designed for managing textual and numerical data, and retrieving such data is often based on simple comparisons of text/numerical values. However, this simple method of retrieval is no longer adequate for multimedia data, since the digitized representation of images, video, or data itself does not convey the reality of these media items. In addition, composite data consisting of heterogeneous types of data also associates with the semantic content acquired by a users recognition. Therefore, content-based retrieval for multimedia data is realized taking such intrinsic features of multimedia data into account. Implementation of the content-based retrieval facility is not based on a single fundamental, but is closely related to an underlying data model, a priori knowledge of the area of interest, and the scheme for representing queries. This paper surveys recent studies on content-based retrieval for multimedia databases from the point of view of three fundamental issues. Throughout the discussion, we assume databases that manage only nontextual/numerical data, such as image or video, are also in the category of multimedia databases.Conventional database systems are designed for managing textual and numerical data, and retrieving such data is often based on simple comparisons of text/numerical values. However, this simple meth...

Knowledge-assisted content based retrieval for multimedia databases

Unlike conventional databases, which manage only text and numerical data, multimedia databases must evaluate audio and visual properties of data. We propose a system of querying and content-based retrieval that considers audio or visual properties of multimedia data.<<ETX>>

VIOLONE: Video Retrieval by Motion Example

Abstract Recent progress in hardware and operating system technologies has made it possible to manage multimedia data consisting of text, static images, sound and/or video. Video data is considered to be the most informative of these types of data. It presents a scene consisting of objects and the motion of objects conveying particular meaning of the scene. Thus, the inherent feature of video data lies in the motion of objects. In this paper, we present a system that retrieves video data by means of the motion of objects observed in the video data in the database. The system accepts a query for a video database, which is specified by drawing an example trajectory of an object, and retrieves video data by extracting a moving object observed in the video data. The proposed way of specifying a query condition is superior to other ways of representing a condition, e.g. by text, in the sense that it is suitable for representing the difference of motion.

IconicBrowser: An iconic retrieval system for object-oriented databases

From the point of view of the ability to manage complex data, object-oriented database systems are superior to conventional database systems. Query languages for object-oriented databases, however, tend to be overly complicated for the casual user unfamiliar with sophisticated database manipulation. This paper describes an IconicBrowser which allows the user to retrieve objects in a database by means of icons. Icons represent classes and objects in the database. Queries are specified by overlapping one icon over another. The system then interprets them into the database operations depending on their combination, and at the same time generates predicate-based (text-base) queries which can be used in other applications of the database.

V-QBE: video database retrieval by means of example motion of objects

Multimedia databases consist of text, static images, sound, and/or video. Video data is considered to be the most informative among those types of data. It conveys the meaning of a scene, the existence and/or motion of certain objects, and others. We present a system that retrieves video data by means of the motions of objects observed in video data in the database. That is, a query for a video database is specified by drawing an example trajectory of an object.

Scene detection by audio-visual features

Compared with cut detection of video data, scene boundary detection is more complicated since the transition of an atmosphere and/or visual scenery need to be recognized. Scene boundary detection by video processing may contain ambiguity because scenes are not always classified with clear distinction. We propose the noble way of scene boundary detection by audio and video features, which takes editing pattern of movies and audio features into account. Back-ground noise extraction and its classification is cooperatively evaluated with visual feature extraction in order to detect the sequence of shots as a single scene. In this paper, we show that the audio analysis to detect the transition of scene improves the accuracy of scene boundary detection as well.

Shot Type and Replay Detection for Soccer Video Parsing

Parsing the structure of soccer video plays an important role in semantic analysis of soccer video. In this paper, we present a shot classification method based on the detection of grass field pixels and size of players. In addition, a replay detection algorithm is proposed. First, the candidate logo images are identified by using contrast feature and histogram difference. The contrast logo template is calculated to detect logo frames. Finally, replay segments are identified by pairing and finding the beginning and the end of logo transition. Experiments on three soccer matches showed that our method is effective and applicable for higher level semantic analysis.

Content-based retrieval of video data based on spatiotemporal correlation of objects

Unlike image data, video (animation) data contain the motion of objects. The spatiotemporal correlation of objects may represent certain semantic contents, which are not perceived by focusing only on the motion of an individual objects independently. We propose a content-based retrieval framework based on the spatiotemporal correlation of objects with a set of description factors of spatiotemporal relations of objects.

Soccer video summarization based on cinematography and motion analysis

Summarization of soccer videos has been widely studied due to its worldwide viewers and potential commercial applications. Most existing methods focus on searching for highlight events in soccer videos such as goals, penalty kicks and generating a summary as a list of such events. However, besides highlight events, scenes of intensive competition between players of two teams and emotional moments are also interesting. In this paper, we propose a soccer summarization system which is able to capture highlight events, scenes of intensive competition, and emotional moments. Based on the flow of soccer games, we organize a video summary as follows: first, scenes of intensive competition, second, what events happened, third, who were involved in the events, and finally how players or audience reacted to the events. With this structure, the generated summary is more complete and interesting because it provides both game play and emotional moments. Our system takes broadcast video as input, and divides it into multiple clips based on cinematographic features such as sport video production techniques, the transition of shots, and camera motions. Then, the system evaluates the interest level of each clip to generate a summary. Experimental results and subjective evaluation are carried out to evaluate the quality of the generated summary and the effectiveness of our proposed interest level measure.

Rendition-based video editing for public contents authoring

Previous video editing is performed by specifying cutting points in a sequence of video frames and combining the trimmed shots together. In order to emphasize a nonverbal, emotional information of a scene effectively, we need to edit a video sequence so that it follows ‘film grammar’ or cinematography, which is the technique for effective expression of the scene in a film. In this paper, we propose a new framework of rendition-based video editing. Instead of specifying cutting points of a video sequence, an editor specifies the type of emotional expression that he/she likes to emphasize. Consequently, desirable transition of shot duration and shot combination are determined, and video contents are semi-automatically edited so as to follow the film grammar.