Norio Katayama

The SR-tree: an index structure for high-dimensional nearest neighbor queries

Recently, similarity queries on feature vectors have been widely used to perform content-based retrieval of images. To apply this technique to large databases, it is required to develop multidimensional index structures supporting nearest neighbor queries efficiently. The SS-tree had been proposed for this purpose and is known to outperform other index structures such as the R*-tree and the K-D-B-tree. One of its most important features is that it employs bounding spheres rather than bounding rectangles for the shape of regions. However, we demonstrate in this paper that bounding spheres occupy much larger volume than bounding rectangles with high-dimensional data and that this reduces search efficiency. To overcome this drawback, we propose a new index structure called the SR-tree (Sphere/Rectangle-tree) which integrates bounding spheres and bounding rectangles. A region of the SR-tree is specified by the intersection of a bounding sphere and a bounding rectangle. Incorporating bounding rectangles permits neighborhoods to be partitioned into smaller regions than the SS-tree and improves the disjointness among regions. This enhances the performance on nearest neighbor queries especially for high-dimensional and non-uniform data which can be practical in actual image/video similarity indexing. We include the performance test results the verify this advantage of the SR-tree and show that the SR-tree outperforms both the SS-tree and the R*-tree.

Topic threading for structuring a large-scale news video archive

We are building a broadcast news video archive where topics of interest can be retrieved and tracked easily. This paper introduces a structuring method applied to the accumulated news videos. First they are segmented into topic units and then threaded according to their mutual relations. A user interface for topic thread-based news video retrieval is also introduced. Since the topic thread structure is formed so that it has fewer number of emerging links from each topic than a simple link structure of related topics, it should lessen the tedious selection during a tracking process by a user. Although evaluation of the effect of threading and user study on the interface is yet to be done, we have found the interface informative to understand the details of a topic of interest.

Distinctiveness-sensitive nearest-neighbor search for efficient similarity retrieval of multimedia information

Nearest neighbor (NN) search in high dimensional feature space is widely used for similarity retrieval of multimedia information. However recent research results in the database literature reveal that a curious problem happens in high dimensional space. Since high dimensional space has a high degree of freedom, points could be scattered so that every distance between them might yield no significant difference. In this case, we can say that the NN is indistinctive because many points exist at the similar distance. To make matters worse, indistinctive NNs require more search cost because search completes only after choosing the NN from plenty of strong candidates. In order to circumvent the handful effect of indistinctive NNs, the paper presents a new NN search algorithm which determines the distinctiveness of the NN during search operation. This enables us not only to cut down search cost but also to distinguish distinctive NNs from indistinctive ones. These advantages are especially beneficial to interactive retrieval systems.

Threading news video topics

We introduce a method to extract topic threads throughout a large-scale news video corpus as well as an interface that provides the users with the facility to browse through the corpus guided by the thread structure. The thread-based interface enables thorough understanding of a topic of interest, and moreover, it is designed to reduce the number of candidates provided during the tracking process which lightens the burden of selection imposed on the users. In this paper, we introduce the details of topic segmentation, tracking/threading, followed by the evaluation of the segmentation and the introduction of a prototype interface. The evaluation showed practical ability, and the trial use of the interface turned out to be effective and informative.

Mining large-scale broadcast video archives towards inter-video structuring

The current computer technology enables us to build huge broadcast video archives which had been a future dream. Today, even the hard disk recorders on the market are capable of recording several hundred hours of broadcast video. It is naturally perceived that a huge amount of broadcast video would be a useful corpus for multimedia indexing and mining research. Based on this viewpoint, we designed and constructed a broadcast video archive system having sufficient capacity and functionality to serve as the testbed for indexing and mining research. The system can capture multiple channels (currently seven channels) all-day broadcast video streams simultaneously, up to 6000 hours, and program-specific broadcasts, currently a news program for more than three years so far. This paper discusses design and implementation issues of the video archive system and then introduces our research efforts utilizing the archives as huge multimedia corpora.

trackThem : exploring a large-scale news video archive by tracking human relations

We propose a novel retrieval method for a very large-scale news video archive based on human relations extracted from the archive itself. This paper presents the idea and the implementation of the method, and also introduces the trackThem interface that enables the retrieval and at the same time track down the relations. Although detailed evaluations are yet to be done, we have found interesting relations through the exploration of the archive by making use of the proposed interface.

Topic-based inter-video structuring of a large-scale news video corpus

We propose a topic-based inter-video news video corpus structuring method and a visual interface to efficiently browse through the structured corpus. Such inter-video structuring was not deeply sought in previous works. The topic-based structure is analyzed by closed-caption text analysis; topic segmentation and tracking. The visual interface provides the ability to 1) search and select a topic by query terms and 2) track a topic thread interactively referring to the text analysis results. Although topic retrieval is somewhat similar to conventional video retrieval methods, the combination with topic tracking makes it remarkably easy to narrow down the results that match a users interest and moreover reveal underlying content-based structures, where the structure itself contains rich information.

SR-tree: An index structure for nearest-neighbor searching of high-dimensional point data

Similarity search methods using feature vectors are employed widely for implementation of content-based retrieval of visual data, and appropriate index structures were explored to accelerate the search. Methods proposed hitherto have used the R*-tree and the SS-tree. This study offers a faster index structure namely, the SR-tree (sphere/rectangle-tree). The main feature of the proposed method is that both bounding spheres and bounding rectangles are used in combination. Bounding spheres and rectangles have been already employed in the SS-tree and the R*-tree, respectively. Experiments carried out as a part of the present study show, however, that as dimensionality grows high, both methods become problematic. Thus, when using bounding rectangles, the difference between the length of the edge of the rectangle and the diagonal becomes too large; with bounding spheres, the volume increases considerably as compared to rectangles. On the other hand, the SR-tree method uses both spheres and triangles, which ensures more efficient partitioning as compared to the SS-tree or R*-tree. Evaluation experiments proved that the proposed method outperforms both SS-tree and R*-tree in terms of CPU time and number of disk accesses.

Key image extraction from a news video archive for visualizing its semantic structure

Recently, it has become possible to handle a large amount of video data with a video archive system. It is very important that a video data is structured based on semantics for useful access to a large video archive. Video data is consisted of images, sounds and texts. Therefore video data should be structured by using their information in multi-modality. In this paper, we introduce a method to extract the key images for visualizing the semantic structure of news video archive by analyzing semantic structure using both images and texts.

An efficient implementation and evaluation of robust face sequence matching

This paper presents a robust and efficient matching method for face sequences obtained from videos. Face information is quite important especially for news programs, dramas, and movies. Face sequence matching for such videos enables many multimedia applications including content-based face retrieval, automated face annotation, automated video authoring, etc. However face sequences in videos are subject to variations in lighting conditions, pose, face expression, etc., which cause difficulty in face matching. These problems are tackled to achieve robust face sequence matching applicable to real video domains, and its efficient implementation is presented. The paper proves the proposed method achieves good performance in actual video domains. In addition, by combination with the high-dimensional index structure, the algorithm achieves practical computational time, as well as scalability against increase of the number of faces.