Keisuke Nonaka

Robust moving camera calibration for synthesizing free viewpoint soccer video

In this paper, a robust moving camera calibration method is proposed in order to synthesize a free viewpoint soccer video with a high degree of accuracy. The main problem in video registration-based moving camera calibration is that the calibration accuracy is very low if the detected feature points are from moving objects. In order to solve this problem, the proposed method tracks the feature points along video frames to construct a trajectory matrix of feature points, and the trajectory matrix is decomposed into a low-rank matrix representing global camera motion and a sparse matrix representing local individual motion. Therefore, according to such decomposition, the individual motions of dynamic feature points that are from moving objects could be suppressed and removed. Experimental results show that the proposed method achieves more accurate calibration result and the visual quality of a synthesized free viewpoint soccer video is also improved by the proposed method.

Automatic camera self-calibration for immersive navigation of free viewpoint sports video

In recent years, the demand of immersive experience has triggered a great revolution in the applications and formats of multimedia. Particularly, immersive navigation of free viewpoint sports video has become increasingly popular, and people would like to be able to actively select different viewpoints when watching sports videos to enhance the ultra realistic experience. In the practical realization of immersive navigation of free viewpoint video, the camera calibration is of vital importance. Especially, automatic camera calibration is very significant in real-time implementation and the accuracy of camera parameter directly determines the final experience of free viewpoint navigation. In this paper, we propose an automatic camera self-calibration method based on a field model for free viewpoint navigation in sports events. The proposed method is composed of three parts, namely, extraction of field lines in a camera image, calculation of crossing points, determination of the optimal camera parameter. Experimental results show that the camera parameter can be automatically estimated by the proposed method for a fixed camera, dynamic camera and multi-view cameras with high accuracy. Furthermore, immersive free viewpoint navigation in sports events can also be completely realized based on the camera parameter estimated by the proposed method.

Semi-Automatic Generation of Free Viewpoint Video Contents for Sport Events: Toward Real-time Delivery of Immersive Experience

Free viewpoint technology makes it possible to view video of sports content from any angle or position, but creating such content is currently a time-consuming process that can prevent real-time delivery. To address this problem, the authors present an application framework that implements semi-automatic camera calibration, object extraction, object tracking, and object separation to seamlessly generate high-quality free viewpoint sports videos for handheld devices.

Generalized image retargeting via convex optimization

Image retargeting is a technique for displaying an image on various devices adaptively. However, the conventional retargeting methods restrict target displays shape as rectangular. For expanding their application field, generalization of the shape of target display is useful. Yet, generalization of the existing methods is non-trivial due to the fact that the basic idea of their algorithm deeply rooted in this shape restriction. On the other hand, one of the retargeting approaches, so called warping based method that have high affinity with optimization problem, is proposed. In this paper, we propose a warping based generalized image retargeting. For generalizing the warping method, the main problem is to solve a repositioning problem of regions of interest (ROIs) in input image. By restricting their movable regions, we convexificate this problem and it allows us to use a conventional convex optimization algorithm. The obtained result shows that our method outperforms the conventional method.