Hyeokman Kim

An efficient graphical shot verifier incorporating visual rhythm

Recent publications regarding video shot change detection algorithms show that abrupt shot changes are detected fairly well. However, gradual shot changes, including wipes and dissolves, are often missed or falsely detected. A robust shot change detection system, therefore, must include a shot verification step to further enhance the overall system performance. In this paper, we introduce the concept of a visual rhythm, an abstraction of the video. A visual rhythm is a single image, a sub-sampled version of a full video in which the sampling is performed in a pre-determined and in a systematic fashion. It is basically a representation of the video which includes the overall content of the video, but most importantly, the visual rhythm contains patterns or visual features that allow the viewer/operator to distinguish and classify many different types of video effects (edits and otherwise): cuts, wipes, dissolves, fades, camera motions, object motions, flashlights, zooms, etc. The different video effects manifest themselves as different patterns on the visual rhythm. Using the visual rhythm, it becomes possible, without sequentially playing the entire video, to find false positive shots as well as undetected shots. Thus, inclusion of the visual rhythm in the shot verification process will aid the operator to verify detected shots as well as to find undetected shots fast and efficiently. For this purpose, we have developed a new tool, a shot verifier that includes the visual rhythm. The usefulness of the visual rhythm during the shot verification process is presented.

Visual Rhythm and Shot Verification

Typical result of an automatic shot change detection algorithm expectedly includes a certain number of undetected shots as well as falsely detected shots. Even though automatic shot change detection algorithms are continuing to improve, the ultimate goal of automatically detecting all shot changes without false alarms may never be achieved. Thus, allowing a human operator to intervene—to review and verify the result of a shot change detection algorithm, to delete falsely detected shots as well as to find undetected shots—may be the most viable approach currently available for increasing the accuracy of the overall shot detection process. For this exact purpose, we propose a shot verifier based on the visual rhythm.The visual rhythm, an abstraction of the video, is a single image, a sub-sampled version of a full video in which the sampling is performed in a pre-determined and in a systematic fashion. It is a representation of the video, which includes the overall content of the video. But most importantly, the visual rhythm contains patterns or visual features that allow the viewer/operator to distinguish and classify many different types of video effects (edits and otherwise): cuts, wipes, dissolves, fades, camera motions, object motions, flashlights, zooms, etc. The different video effects manifest themselves as different patterns on the visual rhythm. Using the visual rhythm, it becomes possible, without sequentially playing the entire video, to find false positive shots as well as undetected shots. Thus, inclusion of the visual rhythm in the shot verification process will aid the operator to verify detected shots as well as to find undetected shots fast and efficiently.Our newly developed shot verifier based on the visual rhythm has been designed for operator efficiency. The design of our shot verifier presented and the usefulness of the visual rhythm during the shot verification process will be demonstrated.

Coverage-time optimized dynamic clustering of networked sensors for pervasive home networking

This paper proposes a novel energy-efficient coverage-time optimized dynamic clustering scheme for two- tiered wireless sensor networks (WSNs) used in an outdoor monitoring application of home networking systems. The coverage-time is defined as the time until one of cluster heads (CHs) runs out of energy in clustering-based WSNs, thereby resulting in an incomplete coverage. DC-CTO scheme regulates cluster radii for balanced energy consumption among CHs to maximize coverage-time. There are several advantages of using the DC-CTO scheme. The first advantage is balanced energy consumption among CHs under NLP- based computations. The second advantage is minimized energy consumption in each CH to extend coverage-time. The last one is the guarantee of perfect coverage. If there is no area which is uncontrolled by a CH in a networked sensing field, we call it a perfect coverage. If there is an incomplete coverage, the event in the area cannot be identified. Therefore perfect coverage is important to robust monitoring systems. The novelty of DC-CTO scheme is demonstrated by various simulation-based performance evaluations.

Real-Time Video Indexing System for Live Digital Broadcast TV Programs

In this paper, we introduce a real-time metadata service system that is implemented for live digital broadcast TV programs. The system is composed of three parts: an indexing host which indexes broadcast programs in real-time, a broadcaster where the segmentation metadata delivered from the indexing host is multiplexed into the broadcast stream and transferred to clients, and a client PVR that receives the metadata and locates a segment of interest from the recorded stream according to the time description of the delivered metadata. We propose to utilize broadcasting time for a time description of the segmentation metadata, so as to be free from the media localization problems in broadcast environment. In addition, we utilize a spatiotemporal visual pattern of a video for a verification tool of real-time indexing, such that we can reduce the false alarms of video segmentation caused by lack of an efficient tool for verifying video segment. As a result, we show the real experiments that are performed without requiring a return channel and demonstrate the feasibility of the proposed system.

Fast Text Caption Localization on Video Using Visual Rhythm

In this paper, a fast DCT-based algorithm is proposed to efficiently locate text captions embedded on specific areas in a video sequence through visual rhythm, which can be fast constructed by sampling certain portions of a DC image sequence and temporally accumulating the samples along time. Our proposed approach is based on the observations that the text captions carrying important information suitable for indexing often appear on specific areas on video frames, from where sampling strategies are derived for a visual rhythm. Our method then uses a combination of contrast and temporal coherence information on the visual rhythm to detect text frames such that each detected text frame represents consecutive frames containing identical text strings, thus significantly reducing the amount of text frames needed to be examined for text localization from a video sequence. It then utilizes several important properties of text caption to locate the text caption from the detected frames.

Implementation of Encoder and Decoder for MPEG-7 BiM

In the paper, we implemented a software system that encodes XML instance documents conforming to a schema document according to the MPEG-7 BiM compression method, and decodes the encoded documents vice versa. We designed software structures of BiM encoder and decoder as class hierarchies, and then implemented the structures. The implemented BiM encoder shows a compression ratio of 9.44% on the average. The BiM encoder is a general-purpose XML compressor that can encode any instance documents conforming to a schema document described in XML Schema language including the MPEG-7 schema. The BiM encoder thus can be used in many application fields including digital broadcasting environment, where encoding XML instance documents is needed.

Design of color-code system for time-stamping broadcast video

For the purpose of video indexing and monitoring of broadcast TV streams, it is needed to associate any frame of a broadcast stream with its current broadcasting time. We propose a new time-stamping method to associate any frame of a broadcast stream accurately with its corresponding broadcasting time by stamping a structured color-code visually representing the current broadcasting time on each frame of a broadcast stream. The experimental results show that a structured color-code with brightness calibration can be perfectly recognized in real-time. The robustness of the structured color-code is also demonstrated by implementing a real-time video indexing system that can generate metadata containing segmentation information described by broadcasting time. TV viewers can interactively view broadcast streams recorded in their set-top boxes with the corresponding metadata delivered.

Time-Series Analysis for Price Prediction of Opportunistic Cloud Computing Resources

Cloud computing resources are offered in various forms, and surplus of computing resources are provided at cheaper price. A leading cloud computing vendor, Amazon Web Services, provides such opportunistic resources as EC2 spot instance whose price changes dynamically based on the resource demand from users. We analyze the spot instance price logs and apply various predictive analysis algorithms to better predict future spot instance price. By applying various train dataset modeling heuristics, we uncover that the SARIMA algorithm achieves the best prediction accuracy in spot price prediction; it shows 17% more accuracy than other algorithms that are widely used for spot instance applications. By applying contributions in this paper, we expect that spot instance users can decrease monetary cost while improving system stability.

Sleep Control Game for Wireless Sensor Networks

In wireless sensor networks (WSNs), each node controls its sleep to reduce energy consumption without sacrificing message latency. In this paper we apply the game theory, which is a powerful tool that explains how each individual acts for his or her own economic benefit, to analyze the optimal sleep schedule for sensor nodes. We redefine this sleep control game as a modified version of the Prisoner’s Dilemma. In the sleep control game, each node decides whether or not it wakes up for the cycle. Payoff functions of the sleep control game consider the expected traffic volume, network conditions, and the expected packet delay. According to the payoff function, each node selects the best wake-up strategy that may minimize the energy consumption and maintain the latency performance. To investigate the performance of our algorithm, we apply the sleep control game to X-MAC, which is one of the recent WSN MAC protocols. Our detailed packet level simulations confirm that the proposed algorithm can effectively reduce the energy consumption by removing unnecessary wake-up operations without loss of the latency performance.

A graphical browser for XML schema documents

Recently, tools for browsing XML Schema documents have become popular, but they cannot support advanced browsing functionalities. We have implemented a new graphical schema browser which provides traversals not only along a composition hierarchy but also along a type hierarchy defined in the documents. In this demonstration, we show how users can quickly and easily understand the semantic structures by freely navigating to any datatypes or elements located at any position in the hierarchies.