Toshio Moriya

A human tracking mobile-robot with face detection

In this paper, we propose the face detection method for tracking a human by a mobile-robot. We obtain images from a web camera, and detect faces by focusing on skin colors and eyes as facial features. If we detect faces from images, we trace the detected human, take a picture of him/her, and print it automatically by using the mobile-robot. In order to show the effectiveness of the proposed method, we show the experimental results. Firstly, in the face detection, we show the face detection accuracy. Then, in the human tracking with mobile-robot by using face detection, we show the tracking performance.

A video-based virtual reality system

We introduce a new environment to make and play interactive contents with more than video game quality. The system consists of a projector array, a viewer and an editor for the special contents. The projector array projects multiple digital images seamlessly both in time and space, thus a very high quality video projection system. The viewer features a function to composite a passive video and interactive CG in real time. The editor is a high-end non-linear editing system combined with some new plug-in software to pre-compute the information necessary for real-time compositing. A new method of digital image recognition assisted by human operators is used. Unlike general purpose computer vision algorithms, it minimizes the error of 3D estimation at the compositing position. We call this approach V2R or the video-based virtual reality. It allows the operator to experience interactive communications with objects in a very high quality video.

Solving the rotation-estimation problem by using the perspective three-point algorithm

We present a closed-form solution for the rotation-estimation problem, by introducing the same algorithm used for solving the perspective three-point problem. The rotation-estimation problem is defined as estimating the relative rotation of two camera images when the translation is known. The perspective three-point problem, in which the pose of a single camera is estimated from the location of three points, is a classical problem in the field of computer vision, and a closed-form solution for it has already been established. The two problems are not obviously similar, but we show by making some parameter exchanges that they are essentially equivalent. Actual applications using the rotation-estimation problem are also explained.

Integral videography of high-density light field with spherical layout camera array

We propose a spherical layout for a camera array system when shooting images for use in Integral Videography (IV). IV is an autostereoscopic video image technique based on Integral Photography (IP) and is one of the preferred autostereoscopic techniques for displaying images. There are many studies on autostereoscopic displays based on this technique indicating its potential advantages. Other camera arrays have been studied, but their purpose addressed other issues, such as acquiring high-resolution images, capturing a light field, creating contents for non-IV-based autostereoscopic displays and so on. Moreover, IV displays images with high stereoscopic resolution when objects are displayed close to the display. As a consequence, we have to capture high-resolution images in close vicinity to the display. We constructed the spherical layout for the camera array system using 30 cameras arranged in a 6 by 5 array. Each camera had an angular difference of 6 degrees, and we set the cameras to the direction of the sphere center. These cameras can synchronously capture movies. The resolution of the cameras is a 640 by 480. With this system, we determined the effectiveness of the proposed layout of cameras and actually captured IP images, and displayed real autostereoscopic images.

Mobile dual-arm robot for automated order picking system in warehouse containing various kinds of products

We have prototyped a mobile dual-arm robot and an automated order picking system including the robot for warehouses that contain various kinds of products. By using self-localization, model-based object recognition and arm trajectory planning, the robot can autonomously move to the front of the shelf that has a target product, fetch the product from the shelf and put the product in a carton transported by an automated guided vehicle. Especially for adjusting to various kinds of products and their storage situations, the robot makes two arms collaborate and mounts tables to lift and rotate two arms and four different types of end effectors. In experimental tests, the robot successfully picked out bottles in a case and picked up two different sizes of boxes directly placed on shelves with different heights. In the result, our proposed system can automate whole order picking operations in warehouses where the operations are currently performed by workers.

Furniture layout AR application using floor plans based on planar object tracking

In this paper, we propose a new approach of Augmented Reality (AR) system for the furniture layout based on a planar object tracking. The planar object tracking methods using natural features are effective methods to estimate the objects pose and position in the AR applications because we are able to use the natural images. However, most of the feature descriptors have a lot of matching procedure. Therefore, by using an efficient feature point descriptor which is very fast both to build and to match, we track the planar objects. Especially, we use floor plans as the planar objects, and then furniture CG models are overlaid on the floor plans. This is because the floor plans are presented in the selection of rooms for rent or buy. Therefore, this system helps borrowers and buyers to select some mansion or apartment rooms. In this system, we propose to use human whistle sounds and color rectangles recognition to operate the furniture layout. In order to show the effectiveness of our proposed system, we perform some planar object tracking experiments when we applied the proposed system to some floor plans.

Real-Time Updating of 2D Map for Autonomous Robot Locomotion Based on Distinction Between Static and Semi-Static Objects

Abstract Autonomous mobile robots are increasingly being used in complex 2D environments such as factories, warehouses, and offices. For such environments, we propose a real-time technique for updating an environmental map for a robot’s self-localization using a bearing-range sensor in situations where a basis map can be preliminarily prepared. These environments include many semi-static objects such as cardboard boxes, and the locations of these objects change frequently. Therefore, the self-localization needs to reflect the changes in both the existence and position of semi-static objects in the map in real-time. However, if the robot uses a traditional technique that updates all objects and if it keeps updating the map for a long period, static objects such as walls will move slightly on the map due to errors of both measurement and self-localization, and the map will be distorted. Therefore, our technique distinguishes between static and semi-static objects on the map, and it defines the changeability of the occupancy probability of every spatial grid in order to update the map without changing the occupancy probabilities of grids around static objects. By doing so, we prevented the map from being distorted. In addition, by estimating the grids’ statuses during two observations of the same grids and by changing the probabilities of the objects’ fixedness based on the statuses, our technique can robustly distinguish the objects on the map even if the timing of observing grids is irregular.

The proposal of model-based initial frame alignment using real-coded GA for mixed reality

In this paper, we propose a model-based alignment system using real-coded genetic algorithm (RGA) for mixed reality (MR) or augmented reality (AR). MR/AR is a technique for superimposing useful virtual information on the real world to provide users more effective views. To realize the MR/AR, the alignment between real space and virtual space is a serious problem. In particular, an initial frame alignment for real-time tracking is needed in the previous knowledge based MR/AR. Therefore, we propose an initial frame alignment method using RGA. In this paper, we use the multiple RGA generation alternation models, crossovers, and mutations to design an optimum GA for the alignment system. This paper shows the expanding version of the pre-proposed method by our system. In order to show the effectiveness of proposed method, we perform some matching simulations when we applied to the real space.

A proposal of model-based alignment using swarm intelligence and condensation

In this paper, we propose a model-based alignment system using swarm intelligence and condensation for mixed reality (MR) and augmented reality (AR). MR and AR are techniques to overlay useful virtual information on the real world and display them to provide users more effective views. To realize the MR and AR, alignment between real space and virtual space is a serious problem. In particular, an initial frame alignment for real-time tracking is needed in the previous knowledge based alignment. Therefore, we propose an alignment method using swarm intelligence for the initial frame alignment. In this paper, we use the multiple swarm intelligence methods for designing an effective method of alignment system. This paper shows the expanding version of the pre-proposed method by our system. Moreover, we conduct tracking of the alignment target using condensation based on the initial frame alignment results. By using this system, a model-based alignment can be conducted in a room-type environment such as being hard to acquire many feature points. In order to show the effectiveness of a proposed method, we perform some alignment experiments when we applied the alignment system to a real space.

Dynamic 3D stabilization for video CG composite

We propose a method to estimate camera parameters from video sequences for the purpose of natural CG composite remarking the problem of stabilization. When the video is taken under wrong condition, it is inescapable that the estimated parameters contain nonnegligible errors which cause an unexpected shaking of CG object. To avoid this problem, we introduce a smoothing technique taking account of a property of synthesizing object. The method is based on an idea that the estimated camera parameters need not to be correct so far as composite is natural, and can be varied according to the condition of CG object even though the given images are same. Introducing the factor of locational and temporal smoothness to estimate the camera parameters, we can accomplish an appropriate stabilization for particular CG composite.