Naoko Nitta

Video Summarization for Large Sports Video Archives

Video summarization is defined as creating a shorter video clip or a video poster which includes only the important scenes in the original video streams. In this paper, we propose two methods of generating a summary of arbitrary length for large sports video archives. One is to create a concise video clip by temporally compressing the amount of the video data. The other is to provide a video poster by spatially presenting the image keyframes which together represent the whole video content. Our methods deal with the metadata which has semantic descriptions of video content. Summaries are created according to the significance of each video segment which is normalized in order to handle large sports video archives. We experimentally verified the effectiveness of our methods by comparing the results with man-made video summaries

PriSurv: privacy protected video surveillance system using adaptive visual abstraction

Recently, video surveillance has received a lot of attention as a technology to realize a secure and safe community. Video surveillance is useful for crime deterrence and investigations, but may cause the invasion of privacy. In this paper, we propose a video surveillance system named PriSurv, which is characterized by visual abstraction. This system protects the privacy of objects in a video by referring to their privacy policies which are determined according to closeness between objects in the video and viewers monitoring it. A prototype of PriSurv is able to protect the privacy adaptively through visual abstraction.

Intermodal collaboration: a strategy for semantic content analysis for broadcasted sports video

This paper presents intermodal collaboration: a strategy for semantic content analysis for broadcasted sports video. The broadcasted video can be viewed as a set of multimodal streams such as visual, auditory, text (closed caption) and graphics streams. Collaborative analysis for the multimodal streams is achieved based on temporal dependency between their streams, in order to improve the reliability and efficiency for semantic content analysis such as extracting highlight scenes from sports video and automatically generating annotations of specific scenes. A couple of case studies are shown to experimentally confirm the effectiveness of intermodal collaboration.

Extracting actors, actions and events from sports video -a fundamental approach to story tracking

To effectively deal with the vast amount of videos, we need to construct a content-based representation for each video. As a step towards this goal, this paper proposes a method to automatically generate the semantical annotations for a sports video by integrating the text(c1osed-caption) and image stream. we first segment the text data and extract segments which are meaningful to grasp the story of the video, then extract the actors, the actions and the events of each scene which are useful for information retrieval by using the linguistic cues and the domain knowledge. We also segment the image stream so that each segment can associate with each text segment extracted above by using the image cues. Finally we can annotate the video by associating the text segments with the image segments. Some experimental results are presented and discussed in this paper.

Automatic video summarization of sports videos using metadata

Video summarization is defined as creating a shorter video clip or a video poster which includes all but only the important scenes in an original video stream. In this paper, we propose two methods of generating a summary of arbitrary length for sports videos. One is to create a concise video clip by temporally compressing the amount of the video data. The other is to provide a video poster by spatially presenting the image keyframes which represent the whole video content. Both methods deal with the videos with metadata to summarize the video semantically. We experimentally verified the effectiveness of our method by comparing the results with man-made video summaries as well as by conducting the questionnaires to the users.

Automatic personalized video abstraction for sports videos using metadata

Video abstraction is defined as creating a video abstract which includes only important information in the original video streams. There are two general types of video abstracts, namely the dynamic and static ones. The dynamic video abstract is a 3-dimensional representation created by temporally arranging important scenes while the static video abstract is a 2-dimensional representation created by spatially arranging only keyframes of important scenes. In this paper, we propose a unified method of automatically creating these two types of video abstracts considering the semantic content targeting especially on broadcasted sports videos. For both types of video abstracts, the proposed method firstly determines the significance of scenes. A play scene, which corresponds to a play, is considered as a scene unit of sports videos, and the significance of every play scene is determined based on the play ranks, the time the play occurred, and the number of replays. This information is extracted from the metadata, which describes the semantic content of videos and enables us to consider not only the types of plays but also their influence on the game. In addition, user’s preferences are considered to personalize the video abstracts. For dynamic video abstracts, we propose three approaches for selecting the play scenes of the highest significance: the basic criterion, the greedy criterion, and the play-cut criterion. For static video abstracts, we also propose an effective display style where a user can easily access target scenes from a list of keyframes by tracing the tree structures of sports games. We experimentally verified the effectiveness of our method by comparing our results with man-made video abstracts as well as by conducting questionnaires.

Recoverable privacy protection for video content distribution

This paper presents a method which attains recoverable privacy protection for video content distribution. The method is based on discrete wavelet transform (DWT), which generates scaling coefficients and wavelet coefficients. In our method, scaling coefficients, which can be regarded as a low-resolution image of an original image, are used for producing privacy-protected image. On the other hand, wavelet coefficients, which can be regarded as privacy information, are embedded into the privacy-protected image via information hiding technique. Therefore, privacy protected image can be recovered by authorized viewers if necessary. The proposed method is fully analyzed through experiments from the viewpoints of the amount of the embedded privacy information, the deterioration due to the embedding, and the computational time.

Generating Semantic Descriptions of Broadcasted Sports Videos Based on Structures of Sports Games and TV Programs

This paper presents a model to represent a broadcasted sports video in a semantic way and proposes a method of automatically generating semantic descriptions of significant scenes. Representation of a video should clarify the semantic content of the video as accurately as possible. Our model structurizes the video and specifies suitable semantic descriptions for video segments paying attention to the structure of both a sports game and a sports TV program. As the elements of these semantic descriptions, the proposed method tries to obtain the information about the plays and their related players from the closed-caption stream by searching key phrases. Finding the corresponding segments of the video by means of template matching for the image stream attaches these textual descriptions to the proper portion of the video. In this paper, we discuss some experimental results of our method and the potentiality for integrating these results into the standardized MPEG-7 description tools.

Real-World Event Detection Using Flickr Images

This paper proposes a real-world event detection method by using the time and location information and text tags attached to the images in Flickr. Events can generally be detected by extracting images captured at the events which are annotated with text tags frequently used only in specific times and locations. However, such approach can not detect events where only a small number of images were captured. We focus on the fact that semantically related events often occur around the same time at different locations. Considering a group of these events as an event class, the proposed method firstly detects event classes from all images in Flickr based on their similarity of the captured time and text tags. Then, from the images consisting each event class, events are detected based on their similarity of the captured locations. Such two-step approach enables us to detect events where a small number of images were captured.

A discrete wavelet transform based recoverable image processing for privacy protection

This paper presents a novel scheme of a recoverable image processing for privacy protection in real-time video surveillance system. The privacy information is embedded into the video using information hiding. Thus, the original privacy information can be recoverable with secrete key if necessary. In the proposed system, the privacy information is defined as information of objects that consist of detailed data to recover the original image of objects. The scheme is based on discrete wavelet transform (DWT) which is used for generating privacy-protected low resolution image, as well as the high resolution data including privacy information. An amplitude modulo modulation based information hiding scheme is used to hide the privacy information. Experimental results have shown that the proposed system can reduce the amount of the privacy information significantly, and allows the privacy information to be revealed after being embedded in real time.