Yasuyuki Nakajima

Universal scene change detection on MPEG-coded data domain

In this paper, we propose scene decomposition algorithm from MPEG compressed video data. As a preprocessing for scene decomposition, partial reconstruction methods of DC image for P- and B-pictures as well as I-pictures directly from MPEG bitstream are used. As for detection algorithms, we have exploited several methods for detection of abrupt scene change, dissolve and wipe transitions using comparison of DC images between frames and coding information such as motion vectors. It is also proposed the method for exclusion of undesired detection such as flashlight in order to enhance scene change detection accuracy. It is shown that more than 95 percent of decomposition accuracy has been obtained in the experiment using more than one hour TV program. It is also found that in the proposed algorithm scene change detection can be performed more than 5 times faster than normal playback speed using 130MIPS workstation.

A study on rate distortion optimization scheme for JVT coder

This paper focuses on the encoder of ITU-T H.264 | ISO/IEC MPEG-4 Part-10 AVC (hereafter “JVT”), especially the Lagrange parameter for the rate distortion optimization (RDO) scheme. Since the Lagrange optimization method is very effective from the viewpoint of coding efficiency optimization, it is specified as an essential experimental condition for performance evaluation in JVT standardization activity. After discussing the background of this mechanism, we verify this method and the currently used parameter theoretically. Computer simulation results show that the Lagrange multiplier might not necessarily be original value; smaller values provide better performance in QCIF, for example.

Complete Video Quality-Preserving Data Hiding

Although many data hiding methods are proposed in the literature, all of them distort the quality of the host content during data embedding. In this paper, we propose a novel data hiding method in the compressed video domain that completely preserves the image quality of the host video while embedding information into it. Information is embedded into a compressed video by simultaneously manipulating Mquant and quantized discrete cosine transform coefficients, which are the significant parts of MPEG and H.26x-based compression standards. To the best of our knowledge, this data hiding method is the first attempt of its kind. When fed into an ordinary video decoder, the modified video completely reconstructs the original video even compared at the bit-to-bit level. Our method is also reversible, where the embedded information could be removed to obtain the original video. A new data representation scheme called reverse zerorun length (RZL) is proposed to exploit the statistics of macroblock for achieving high embedding efficiency while trading off with payload. It is theoretically and experimentally verified that RZL outperforms matrix encoding in terms of payload and embedding efficiency for this particular data hiding method. The problem of video bitstream size increment caused by data embedding is also addressed, and two independent solutions are proposed to suppress this increment. Basic performance of this data hiding method is verified through experiments on various existing MPEG-1 encoded videos. In the best case scenario, an average increase of four bits in the video bitstream size is observed for every message bit embedded.

MPEG audio bit rate scaling on coded data domain

Formerly, once the audio data is compressed, transcoding is used to scale the bit rate, where decoding and re-encoding are taking place. Therefore, data manipulation of coded data has been very complex and time consuming work. We describe three algorithms for bit rate scaling in the coded MPEG data domain. One is a bandwidth limitation method cutting higher frequency components until the target data rate is satisfied. The other two use a re-quantization process where a quantization step in each subband is modified. One of them reflects the psychoacoustic model from bit allocation information obtained in the bitstream in order to improve the bit rate scaling efficiency. The simulation results show that the re-quantization process provides a very high conversion efficiency and a nearly equal sound quality can be obtained as directly coding from PCM by reflecting the psychoacoustic model. It is also shown that a very fast scaling (factor of six) have been achieved when compared with the transcoding method.

A fast scene change detection on MPEG coding parameter domain

This paper proposes a very fast and accurate scene change detection algorithm on MPEG coding parameter domain. The fast operation can be obtained by spatio-temporally subsampling coding information and by exploiting only coding parameters extracted in variable length decoding (VLD) stage, while the accurate detection is accomplished by examining the statistical characteristics of various scene changes on the coding parameter domain. The computer simulation shows that the proposed algorithm can accomplish detection more than 5 times faster than that of real-time playback for MPEG-2 video sequences using a standard workstation. It is also shown that most of abrupt scene changes, dissolve transitions, and wipe transitions have been successfully detected.

MPEG encoding algorithm with scene adaptive dynamic GOP structure

Although a GOP (group of picture) length is often fixed in MPEG, this may not guarantee the best picture quality from the point of view of coding efficiency. Furthermore, it is also well known that P-picture interval M=3 does not provide the best quality for all sequences. We propose an MPEG encoding algorithm with scene adaptive dynamic GOP structure to enhance the picture quality in the cases mentioned above. Using macroblock characteristics such as luminance activity and simple motion compensation on the activity domain, N and M values are dynamically determined.

Moving-object detection from MPEG coded data

We describe a method of moving object detection directly from MPEG coded data. Since motion information in MPEG coded data is determined in terms of coding efficiency point of view, it does not always provide real motion information of objects. We use a wide variety of coding information including motion vectors and DCT coefficients to estimate real object motion. Since such information can be directly obtained from coded bitstream, very fast operation can be expected. Moving objects are detected basically analyzing motion vectors and spatio-temporal correlation of motion in P-, and B-pictures. Moving objects are also detected in intra macroblocks by analyzing coding characteristics of intra macroblocks in P- and B-pictures and by investigating temporal motion continuity in I-pictures. The simulation results show that successful moving object detection has been performed on macroblock level using several test sequences. Since proposed method is very simple and requires much less computational power than the conventional object detection methods, it has a significant advantage as motion analysis tool.

Rewritable Data Embedding on MPEG Coded Data Domain

In this paper, we propose a rewritable data embedding scheme on MPEG coded data domain for content managements including DRM, content controlling and indexing. Data embedding is performed in a block by block basis, where the length of zero run and the value of dummy AC component of quantized DCT coefficients are used as a data carrier. In the detection process, we can reconstruct the MPEG coded data that is very close to the original one, which enables us not only to rewrite embedded data but also retain the original MPEG video quality. In the experiment, we show that up to a few kbits/frame data embedding without any large PSNR penalty and data recovery can be realized using typical MPEG-1 coded stream.

Moving object detection and identification from MPEG coded data

Moving object detection is a useful tool for such purposes as intelligent video browsing/analysis and video surveillance systems. Previously several algorithms have been reported on moving object detection from compressed video data. However, in these algorithms, flat regions are often falsely detected. In addition if multiple moving objects are involved, each moving object may not be identified and therefore object tracking becomes very difficult. We describe a fast moving detection and identification method from MPEG coded data. The experimental results show that each moving object is successfully identified for several surveillance sequences.

Fast dissolve operations for MPEG video contents

Although NLE (nonlinear editing) becomes widely used for multimedia contents creation, its deployment for MPEG has not been exploited much due to the inter-frame coding structure of MPEG. We propose two fast algorithms in order to realize dissolve operation of MPEG coded data. One is based on the baseband method (BBM) where dissolving is performed in the pixel domain. The other uses the transformed domain method (TDM). Since motion estimation is altered due to the transition and its re-calculation process greatly increases the computational complexity, we use texture information to determine dominant motion during dissolving and avoid the motion estimation calculation. In the experiment, it is observed that fast dissolve processing using BBM can be achieved at less than half of the conventional BBM with at most 1 dB penalty. As for TDM, although it achieves the same processing speed of that of the proposed BBM, larger degradation is found due to the restricted motion accuracy.