Sei Ikeda

High-resolution video mosaicing for documents and photos by estimating camera motion

Recently, document and photograph digitization from a paper is very important for digital archiving and personal data transmission through the internet. Though many people wish to digitize documents on a paper easily, now heavy and large image scanners are required to obtain high quality digitization. To realize easy and high quality digitization of documents and photographs, we propose a novel digitization method that uses a movie captured by a hand-held camera. In our method, first, 6-DOF(Degree Of Freedom) position and posture parameters of the mobile camera are estimated in each frame by tracking image features automatically. Next, re-appearing feature points in the image sequence are detected and stitched for minimizing accumulated estimation errors. Finally, all the images are merged as a high-resolution mosaic image using the optimized parameters. Experiments have successfully demonstrated the feasibility of the proposed method. Our prototype system can acquire initial estimates of extrinsic camera parameters in real-time with capturing images.

Extrinsic Camera Parameter Recovery from Multiple Image Sequences Captured by an Omni-Directional Multi-camera System

Recently, many types of omni-directional cameras have been developed and attracted much attention in a number of different fields. Especially, the multi-camera type of omni-directional camera has advantages of high-resolution and almost uniform resolution for any direction of view. In this paper, an extrinsic camera parameter recovery method for a moving omni-directional multi-camera system (OMS) is proposed. First, we discuss a perspective n-point (PnP) problem for an OMS, and then describe a practical method for estimating extrinsic camera parameters from multiple image sequences obtained by an OMS. The proposed method is based on using the shape-from-motion and the PnP techniques.

Extrinsic camera parameter estimation based-on feature tracking and GPS data

This paper describes a novel method for estimating extrinsic camera parameters using both feature points on an image sequence and sparse position data acquired by GPS. Our method is based on a structure-from-motion technique but is enhanced by using GPS data so as to minimize accumulative estimation errors. Moreover, the position data are also used to remove mis-tracked features. The proposed method allows us to estimate extrinsic parameters without accumulative errors even from an extremely long image sequence. The validity of the method is demonstrated through experiments of estimating extrinsic parameters for both synthetic and real outdoor scenes.

High-resolution panoramic movie generation from video streams acquired by an omnidirectional multi-camera system

Telepresence systems using an omnidirectional image sensor enable us to experience remote sites with rich sensation. An omnidirectional multi-camera system is more useful for acquiring high-resolution omnidirectional images of outdoor scenes than a monocular camera system. However, exact calibration of the camera system is necessary to generate a panoramic movie. In this paper, we describe a panoramic movie generation method based on geometric and photometric calibration of the omnidirectional multi-camera system. A prototype of a telepresence system is also shown as applications of generated panoramic movies. This high-resolution telepresence system has been proven to enable us to experience remote sites with rich presence.

Remote assistance using visual prompts for demented elderly in cooking

This paper presents a smart kitchen system for supporting demented elderly cooking in a kitchen. In this system, a remote supporter provides a user with verbal and visual prompts manually. As a prompting method for this system projection should be chosen as a priority because it enables direct pointing of targets in a real scene. Even if projection is used as a priority, this system should be designed to solve three practical problems: 1) there are many areas where sharp images cannot be projected in a kitchen, 2) kitchen is a too complex environment to display visual prompts using only a single method, 3) remote supporters do not know the kitchen environment well enough. In this paper we propose a novel system overcoming the above problems as follows, respectively.1) The system first tries indirect projection of circular markers as visual prompts around target objects.2) Proper method is automatically selected from using projector, embedded monitor and illumination devices based on environmental information.3) The system automatically acquires flat surfaces to project on and their reflectance properties as the environmental information. Based on an observation of two elderly suffering from dementia, we discuss the feasibility and concrete problems in displaying visual prompt and how to solve them technically.

An immersive telepresence system with a locomotion interface using high-resolution omnidirectional movies

This paper describes a novel telepresence system which enables users to walk through a photorealistic virtualized environment by actual walking. To realize such a system, a wide-angle high-resolution movie is projected on an immersive multi-screen display to present users the virtualized environments and a treadmill is controlled according to detected users locomotion. In this study, we use an omnidirectional multi-camera system to acquire images of a real outdoor scene. The proposed system provides users with rich sense of walking in a remote site.

Video mosaicing based on structure from motion for distortion-free document digitization

This paper presents a novel video mosaicing method capable of generating a geometric distortion-free mosaic image using a hand-held camera. For a document composed of curved pages, mosaic images of virtually flattened pages are generated. The process of our method is composed of two stages : real-time stage and off-line stage. In the realtime stage, image features are automatically tracked on the input images, and the viewpoint of each image as well as the 3-D position of each image feature are estimated by a structure-from-motion technique. In the offline stage, the estimated viewpoint and 3-D position of each feature are refined and utilized to generate a geometric distortion-free mosaic image. We demonstrate our prototype system on curved documents to show the feasibility of our approach.

Panoramic movie generation using an omnidirectional multi-camera system for telepresence

Telepresencesystems using an omnidirectional image sensor enable us to experienceremote sites with rich sensation. An omnidirectional multi-camerasystem is more useful to acquire high-resolution omnidirectional images of outdoor scenes than a monocular camera system. How ever, exactalibration of the camera system is necessary to generate a panoramic movie. In this paper, we describ e panormnic movie generation method based on geometric and photometric calibration of the omnidirectional multi-camerasystem. Some prototype of telepresence system are also shown as applications of generated panoramic movies. These high-resolution telepresence systems have been proven to enable us to experienceremote sites with high presence.

Calibration method for an omnidirectional multicamera system

Telepresence systems using an omnidirectional image sensor enable us to experience remote site. A omnidirectional multi-camera system is more useful to acquire outdoor scenes than a monocular camera system, because the multi-camera system can easily capture high-resolution omnidirectional images. However, exact calibration of the camera system is necessary to virtualize the real world accurately. In this paper, we describe a geometric and photometric camera calibration and a panorama movie generation method for the omnidirectional multi-camera system. In the geometric calibration, intrinsic and extrinsic parameters of each camera are estimated using a calibration board and a laser measurement system called total station. In the photometric calibration, the limb darkening and color balances among the cameras are corrected. The result of the calibration is used in the panorama movie generation. In experiments, we have actually calibrated the multi-camera system and have generated spherical panorama movies by using the estimated camera parameters. A telepresence system was prototyped in order to confirm that the panorama movie can be used for telepresence well. In addition, we have evaluated the discontinuity in generated panoramic images.

Video Mosaicing for Curved Documents Based on Structure from Motion

Various methods for video mosaicing have been already investigated by many researchers. Most of these methods, however, assume that the target object is flat or very far from the camera to avoid the disparity problem. This paper describes a novel video mosaicing method for curved documents based on 3-D reconstruction. With the proposed method, the mosaic image of the geometrically restored target document is generated, even if the document has a curved surface. Experiments on curved documents have shown the feasibility of the proposed method.