Yu-Hsun Lin

Quality Assessment of Stereoscopic 3D Image Compression by Binocular Integration Behaviors

The objective approaches of 3D image quality assessment play a key role for the development of compression standards and various 3D multimedia applications. The quality assessment of 3D images faces more new challenges, such as asymmetric stereo compression, depth perception, and virtual view synthesis, than its 2D counterparts. In addition, the widely used 2D image quality metrics (e.g., PSNR and SSIM) cannot be directly applied to deal with these newly introduced challenges. This statement can be verified by the low correlation between the computed objective measures and the subjectively measured mean opinion scores (MOSs), when 3D images are the tested targets. In order to meet these newly introduced challenges, in this paper, besides traditional 2D image metrics, the binocular integration behaviors-the binocular combination and the binocular frequency integration, are utilized as the bases for measuring the quality of stereoscopic 3D images. The effectiveness of the proposed metrics is verified by conducting subjective evaluations on publicly available stereoscopic image databases. Experimental results show that significant consistency could be reached between the measured MOS and the proposed metrics, in which the correlation coefficient between them can go up to 0.88. Furthermore, we found that the proposed metrics can also address the quality assessment of the synthesized color-plus-depth 3D images well. Therefore, it is our belief that the binocular integration behaviors are important factors in the development of objective quality assessment for 3D images.

A Digital Blind Watermarking for Depth-Image-Based Rendering 3D Images

The content protection for image-based 3D data is getting more importance with the advance of low cost 3D display devices. The depth-image-based rendering (DIBR) 3D image is one of the image-based 3D data which consists of the center image and the depth image generated by the content provider. The left-eye image and the right-eye image are rendered from the center image and the depth image at the content consumer side. The blind watermarking for DIBR 3D image is rarely studied in the literature. In this paper, a novel blind multiple watermarking scheme is proposed to deal with the content protection problem of DIBR 3D images. Besides the usual requirement of mutual orthogonality among reference patterns for multiple watermark embedding, we found that proper embedding order plays an even more important role in watermarking the DIBR 3D images. Experimental results show that the proposed scheme is robust against the JPEG compression and noise adding attacks. More interestingly, it is found that the proposed watermarking can also tolerate large range variations of the depth image during rendering.

A Novel Privacy Preserving Location-Based Service Protocol With Secret Circular Shift for K-NN Search

Location-based service (LBS) is booming up in recent years with the rapid growth of mobile devices and the emerging of cloud computing paradigm. Among the challenges to establish LBS, the user privacy issue becomes the most important concern. A successful privacy-preserving LBS must be secure and provide accurate query [e.g., -nearest neighbor (NN)] results. In this work, we propose a private circular query protocol (PCQP) to deal with the privacy and the accuracy issues of privacy-preserving LBS. The protocol consists of a space filling curve and a public-key homomorphic cryptosystem. First, we connect the points of interest (POIs) on a map to form a circular structure with the aid of a Moore curve. And then the homomorphism of Paillier cryptosystem is used to perform secret circular shifts of POI-related information (POI-info), stored on the server side. Since the POI-info after shifting and the amount of shifts are encrypted, LBS providers (e.g., servers) have no knowledge about the users location during the query process. The protocol can resist correlation attack and support a multiuser scenario as long as the predescribed secret circular shift is performed before each query; in other words, the robustness of the proposed protocol is the same as that of a one-time pad encryption scheme. As a result, the security level of the proposed protocol is close to perfect secrecy without the aid of a trusted third party and simulation results show that the k-NN query accuracy rate of the proposed protocol is higher than 90% even when is large.

A Depth Information Based Fast Mode Decision Algorithm for Color Plus Depth-Map 3D Videos

In this paper, we propose a fast mode decision algorithm for color plus depth-map 3D videos. By utilizing the relationship of coding modes between color frame and associated depth frame, an early SKIP mode decision scheme and a mode selection mechanism have been developed to speed up the encoding process of color videos. Experimental results show that the proposed algorithm is faster than other existing mode decision approaches with similar coding efficiency. The entire encoding time is reduced by 72% on average compared with conventional 3D video coding algorithm, while introducing only negligible loss in peak signal-to-noise ratio value and small increase in bit rates.

Appearance-Based QR Code Beautifier

Quick Response (QR) code is a widely used matrix bar code with the increasing population of smart phones. However, QR code usually consists of random textures which are not suitable for incorporating with other visual designs (e.g. name card and business advertisement poster). In order to overcome the shortcomings of noise-like looks of QR codes, we propose a systematic QR code beautification framework where the visual appearance of QR code is composed of visually meaningful patterns selected by users, and more importantly, the correctness of message decoding is kept intact. Our work makes QR code from machine decodable only (i.e. standardized random texture) to a personalized form with human visual pleasing appearance. We expect the proposed QR code beautifier can inspire more visual-pleasant mobile multimedia applications.

Unseen visible watermarking for color plus depth map 3D images

Unseen visible watermarking (UVW) is a novel data hiding scheme that imitates real-world watermarks and maintains advantages of both visible and invisible watermarking. One important feature of UVW is that specific extraction module is not required during watermarking decoding. The UVW for 2D image is studied based on imaging functions, e.g. gamma-correction in LCD monitors. On the other hand, due to the great success of 3D movies and low-priced 3D display devices, 3D contents are booming up in recent years. In this work, we propose a UVW for color plus depth map 3D images in which the watermark extraction is realized by changing of the rendering conditions. Under normal rendering conditions, the watermarked cover can be perceived exactly the same as the original one. Limitations and future extensions of the proposed 3D UVW are also addressed in this work.

A novel blind watermarking scheme for depth-image-based rendering 3D images

The content protection for image-based 3D data is getting more importance with the advance of low cost 3D display devices. The depth-image-based rendering (DIBR) 3D image is one of the image-based 3D data which consists of the center image and the depth image generated by the content provider. The left-eye image and the right-eye image are rendered from the center image and the depth image at the content consumer side. The blind watermarking for DIBR 3D image is rarely studied in the literature. In this paper, a novel blind multiple watermarking scheme is proposed to deal with the content protection problem of DIBR 3D images. Besides the usual requirement of mutual orthogonality among reference patterns for multiple watermark embedding, we found that proper embedding order plays an even more important role in watermarking the DIBR 3D images. Experimental results show that the proposed scheme is robust against the JPEG compression and noise adding attacks. More interestingly, it is found that the proposed watermarking can also tolerate large range variations of the depth image during rendering.

Efficient Real-Time Distributed Video Coding by Parallel Progressive Side Information Regeneration

Distributed video coding (DVC) provides a different compression paradigm that consists of lightweight encoder and complex decoder. This compression paradigm shed a light for nowadays mobile devices which equipped with limited battery and computing resources while the corresponding multimedia service is backed up by cloud computing infrastructures. After years of developing, the coding performance and the decoding speed of DVC are still facing serious challenges, for example, DVC’s rate-distortion (RD) performance is far behind the traditional prediction based video codec (e.g., H.264) and the required decoding complexity handicaps its value in practical usage. In order to meet these challenges, we propose an efficient real-time DVC framework with progressive side information regeneration by taking GPGPU-based parallel computing into account. The proposed system can resolve one of the achilles’ heels, i.e., the large group of picture scenario, of DVC codec with good RD performance and much improved decoding speed. The Bjontegaard Delta peak signal-to-noise ratio improvement of our system is up to 2.87 dB as compared with DISCOVER codec, which is also comparable with that of the state-of-the-art optical flow-based DVC codecs. The average decoding speed goes up to 22.44 frames per second (FPS) and even up to 76.35 FPS for Hall sequence. To the best of our knowledge, this is the first DVC codec which tackles the RD performance and the decoding speed issues simultaneously.

Rendering Lossless Compression of Depth Image

The compression issue of 3D images is getting more and more important with the booming of 3D multimedia content. The color plus depth image (color+depth, for short) is one of the 3D image formats. The left-eye image and the right-eye image are rendered by the color image and the depth image. The compression induced distortion of the depth image is quite different from that of the color image, because the depth image will be used for rendering operations in 3D imaging. In this paper, a novel rendering loss less compression scheme for the depth image is proposed, in which the characteristic of depth image based rendering is fully utilized to achieve promising compression gain. This work provides a different thinking of depth image compression which is in a perspective on loss less rendering. By introducing the concept of loss less rendering, we can compress the depth image efficiently and keep the outcome of rendering operation intact.

Seam Carving for Color-Plus-Depth 3D Image

Color-plus-Depth 3D images are booming up with the advance of depth-sensing camera (e.g., Kinect). This new 3D visual content imposes new challenges on image resizing since we have to resize both the color and the depth images simultaneously. In order to resolve these newly introduced challenges of color-plus-depth 3D images, we propose a new energy function with salient depth cues consideration for seam carving operation. We further incorporate super-pixel over-segmentation and depth remapping for achieving an object-based and 3D viewing comfort zone aware resizing framework. Experimental results show the proposed framework can effectively generate the resized images by maintaining the salient regions and providing a comfortable 3D visual perception, at the same time.