Akira Kojima

Mobile and multi-device interactive panorama video distribution system

Distribution of reality of the event site through broadband network is expected for a future video service. We propose video distribution system for providing such a realistic user experience in the mobile and home environments, upon recent trends of wireless broadband network LTE and Wi-Fi, and multi-touch wide screen mobile terminal. In the system the user can see high quality video of the expected area within the huge panorama video which represents the whole scene while moving and zooming in interactively the viewing area under limited bandwidth of wireless network. In addition, multi-device video streaming to the TV terminal and mobile terminal is supported. The user can experience more sensational interactive viewing with more high quality video and sound while keeping stress-free multi-touch interface to control the viewing area.

Improving item recommendation based on social tag ranking

Content-based filtering is a popular framework for item recommendation. Typical methods determine items to be recommended by measuring the similarity between items based on the tags provided by users. However, because the usefulness of tags depends on the annotators skills, vocabulary and feelings, many tags are irrelevant. This fact degrades the accuracy of simple content-based recommendation methods. To tackle this issue, this paper enhances content-based filtering by introducing the idea of tag ranking, a state-of-the-art framework that ranks tags according to their relevance levels. We conduct experiments on videos from a video-sharing site. The results show that tag ranking significantly improves item recommendation performance, despite its simplicity.

Self-similarity matching with predictive linear upsampling for depth map

We propose a real-time upsampling scheme for depth maps. The proposed scheme contains two upsampling stages; one is self-similarity matching (SSM), and the other is predictive linear upsampling (PLU). SSM accelerates cost volume filtering by using a variant of joint bilateral upsampling, which utilizes high-dimensional vectors, which is neighborhoods of an RGB image and a depth map. The high-dimensional upsampling suppresses edge blurring and scattering problems. PLU generates smooth surfaces with keeping edges guided by the results of SSM. Experimental results show that the proposed scheme has higher accuracy than the state-of-the-art upsampling. Additionally, the proposed method has real-time performance on a multi-core CPU.

Highly realistic 3D display system for space composition telecommunication

In this paper, we describe a highly realistic 3D display system that generates composites of current and remote places for telecommunication purposes. It uses a 3D projector and head tracking to display a person in a remote place as a life-size stereoscopic image against background scenery. Since it generates displayed images that correspond to the observers viewing position, it reproduces well the fidelity of existence and the feel of materials. We also describe a simple, fast, and high quality background scenery generation method, the development of which was inspired by the visual effects of depth-fused 3D (DFD) displays, and applied to a single direction conceptual demonstration system. Our system is a promising means for achieving real-time communication between two different places in cases where a sense of reality is required.

Multi-layered image retargeting

Pixel domain methods for content-aware image retargeting have a performance limitation in that they distort important objects when an image is shrunken too much. To address this problem, we have designed and developed a novel 3D space domain image retargeting which rearranges objects with occlusions and deforms objects. With this method image formation is modeled as a quasi-3D space. The method also reduces image width by shrinking a part of the image space. Instead of directly optimizing the position and shape of objects, it closes objects by shrinking the gap space between them and deforms objects by shrinking spaces that include parts of them. It is capable of being extended to higher dimension problems such as multi-view image retargeting because it needs to optimize only one quasi-3D space whether the input is ordinary images or more complex ones such as video and stereo images. We have also designed a simple algorithm that makes use of dynamic programming. An experiment we conducted demonstrated the algorithms effectiveness.

MoPaCo: High telepresence video communication system using motion parallax with monocular camera

In video communication, social telepresence is essential for enabling users to communicate naturally in the same manner as in face-to-face meetings. In an attempt to enhance social telepresence, we propose an advanced video communication system with a simple equipment setup that imparts motion parallax which adjusts to the viewpoint position of the user in addition to providing life-size, eye contact, and high quality images. Evaluation experiment results on the system showed that motion parallax enhances social telepresence.

Design and Evaluation of Mirror Interface MIOSS to Overlay Remote 3D Spaces

The MIOSS mirror interface can overlay two remote spaces, enabling users to feel as if they are in the same room and thereby to share 3D objects in the spaces. MIOSS imparts motion parallax through a mirror that adjusts to the viewpoint of the user, in addition to providing geometrical consistency in the occlusion, size, and positional relationships in the two remote spaces. Experimental evaluations of an implemented MIOSS system show that users can recognize the exact positions of shared objects in the partner’s space via the mirror video.

Scene-independent super-resolution for plenoptic cameras

classroom use is granted without fee provided that copies are not made or distributed for commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. SIGGRAPH 2014, August 10 – 14, 2014, Vancouver, British Columbia, Canada. 2014 Copyright held by the Owner/Author. ACM 978-1-4503-2958-3/14/08 Scene-independent Super-resolution for Plenoptic Cameras

Depth-based weighted bi-prediction for video plus depth map coding

Video with depth is attracting attention as a promising approach to future 3D video. Unlike conventional video coding technologies, video with depth should take the correlations between video and depth into account. This paper proposes a novel bi-prediction scheme for video coding that exploits the structure similarity between video and depth. In the proposed scheme, a prediction image is generated by the pixel-wise weighted average of motion compensated reference images, and weights are decided by considering pixel similarities in the depth domain. The proposed pixel-wise weighted bi-prediction offers implicit depth-based block partitioning with consideration of the pixel correlations between predicted and reference blocks. Compared to using explicit, arbitrary shaped block partitioning, the proposed method offers better prediction for non-rigid objects, and requires no increase in motion information. Experiments show that the proposed method can achieve up to 17% bitrate reduction for bi-predictive frames and about 8% on average for 6 sequences relative to the existing biprediction scheme.

Acquiring perceptually diffuse shading from general objects in actual scenes

Recently, augmented reality has started to be used in interactively simulating clothes, furnitures, and so on in actual scenes. In such applications, the shading of synthesized CG objects should be matched to those scenes to achieve a sense of reality. Although many techniques have been proposed to match the shading of CG objects to actual scenes, typically they require the use of light probes such as mirrored spheres [Debevec 1998]. However, such light probes are not always available in interactive simulation situations.