Tomoya Ishikawa

In-Situ 3D Indoor Modeler with a Camera and Self-contained Sensors

We propose a 3D modeler for supporting in-situ indoor modeling effectively. The modeler allows a user easily to create models from a single photo by interaction techniques taking advantage of features in indoor space and visualization techniques. In order to integrate the models, the modeler provides automatic integration functions using Visual SLAM and pedestrian dead-reckoning (PDR), and interactive tools to modify the result. Moreover, for preventing shortage of texture images to be used for the models, our modeler automatically searches from 3D models created by the user for un-textured regions and intuitively visualizes shooting positions to take a photo for the regions. These functions make it possible that the user easily create photorealistic indoor 3D models that have enough textures on the fly.

A method of pedestrian dead reckoning using action recognition

We present a method of estimating the location and orientation of a pedestrian which simultaneously recognizing his/her actions with a single low-cost inertial measurement unit (IMU) mounted at the waist of the user. Some of the actions other than walking locomotion, such as standing up from/sitting down on a chair, and bending over to slip through obstacles, taken by the pedestrians can be mostly seen at the particular locations where the objects and building facilities to induce the actions are placed. Conversely, by knowing the current location and its attribute about possibly taken actions, the action recognition process can be improved with the contextual information since prior knowledge about occurrence of actions is given as the attribute in the map. Additionally, when the posture (such as sitting, standing and getting to one knee) of the pedestrians is known, falsely recognized actions can be rejected. Experimental results show that accuracy of the action recognition on six types of the action (forward walking, backward walking, side stepping, sitting down on/standing up from a chair, going downstairs/upstairs and bending over) is more than 95% by cross validation test on the training data set, and the results also show that error rate of the PDR localization is reduced from 4% of the walking distance to 2% in the total scenario within the office environment by using the results of action recognition to adjust the estimated location.

Economic and Synergistic Pedestrian Tracking System for Indoor Environments

We describe an indoor pedestrian tracking system that can economically improve the tracking performance and the quality and value of services by incorporating other services synergistically. Our tracking system utilizes existing infrastructures to be used for security services such as surveillance cameras and active RFID tags which are sparsely put in place and generally used in plants, offices, and commercial facilities for estimating user’s walking parameters and correction of tracking errors without significantly increasing costs, and realizes the improvement of the tracking performance by them. Furthermore, by sharing not only surveillance videos and RFID signals from security services but also the tracking information and models from 3D environment modeling services among services, each service can enhance the quality and value of the service and relatively reduce the costs for creating the data and realizing functions.

Indoor-Outdoor Navigation System for Visually-Impaired Pedestrians: Preliminary Evaluation of Position Measurement and Obstacle Display

This paper reports on a preliminary evaluation of position measurement and obstacle display in a navigation system for visually-impaired pedestrians on a real-city course including indoor and outdoor areas.

Economic and Synergistic Pedestrian Tracking System with Service Cooperation for Indoor Environments

This paper describes an indoor pedestrian tracking system that can economically improve the tracking performance and the quality and value of services by incorporating other services synergistically. The tracking system obtains position, orientation, and action of pedestrians continuously and accurately in large indoor environments by utilizing surveillance cameras and active RFID tags for security services and 3-D environment models for navigation services. Considering service cooperation and co-creative intelligence cycles, this system can improve both the tracking performance and the quality of services without significant increase of costs by sharing the existing infrastructures and the 3-D models among services. The authors conducted an evaluation of the tracking system in a large indoor environment and confirmed that the accuracy of the system can be improved by utilizing the infrastructures and the 3-D models. Synergistic services utilizing the tracking system and service cooperation can also enhance the quality and value of services.

Service cooperation and co-creative intelligence cycles based on mixed-reality technology

Mixed Reality (MR) technology has the potential to support the improvement of the service productivity by augmenting the experience and intuition of service users and providers. However, in order to make the MR technology including Augmented-Reality (AR) widespread in the service industry, it is necessary to raise the ease of the social implementation by reducing the costs on sensing and content management. In general, service cooperation and co-creative intelligence cycles are well-suited to establish a framework of load/cost sharing and functional enhancement among services and stakeholders. By applying such a framework to MR and AR, digital content and service-operation log can improve the performance of real-world sensing, and meanwhile the real-world sensing can contribute the efficiency of service operation, content gathering and authoring. In this paper, we introduce our own works to enjoy such synergy by facilitating cooperation among services such as pedestrian navigation, patrol inspection, interactive modeling, behavior analysis and service redesign based on MR, and also by co-creatively circulating knowledge among consumers, service providers, and other stakeholders.

Interactive 3-D indoor modeler for virtualizing service fields

This paper describes an interactive 3-D indoor modeler that effectively creates photo-realistic 3-D indoor models from multiple photographs. This modeler supports the creation of 3-D models from photographs by implementing interaction techniques that use geometric constraints estimated from photographs and visualization techniques that help to easily understand shapes of 3-D models. We evaluated the availability and usability by applying the modeler to model service fields where actual workers provide services and an experience-based exhibit. Our results confirmed that the modeler enables the creation of large-scale indoor environments such as hot-spring inns and event sites at a relatively modest cost. We also confirmed that school children could learn modeling operations and create 3-D models from a photograph for approximately 20xa0min because of the easy operations. In addition, we describe additional functions that increase the effectiveness of 3-D modeling based on knowledge from service-field modeling. We present applications for behavior analysis of service workers and for 3-D indoor navigation using augmented virtuality (AV)-based visualization realized by photo-realistic 3-D models.

Photo-shoot localization of a mobile camera based on registered frame data of virtualized reality models

This paper presents a study of a method for estimating the position and orientation of a photo-shoot in indoor environments for augmented reality applications. Our proposed localization method is based on registered frame data of virtualized reality models, which are photos with known photo-shoot positions and orientations, and depth data. Because registered frame data are secondary product of modeling process, additional works are not necessary to create registered frame data especially for the localization. In the method, a photo taken by a mobile camera is compared to registered frame data for the localization. Since registered frame data are linked with photo-shoot position, orientation, and depth data, 3D coordinates of each pixel on the photo of registered frame data is available. We conducted experiments with employing five techniques of the estimation for comparative evaluations.

Hybrid inpainting algorithm with superpixels and hash table for inpainting the 3D model

This paper discusses the new inpainting algorithm which uses the superpixels and the hash tables. The proposed algorithm is called as a hybrid method because it combines the effectiveness of two popular inpainting algorithms namely, the Exemplar-Based method and the Edge-Based image restoration method and rectifies the drawbacks in these two methods. The tedious patch search in the Exemplar-Based method is simplified by the hash table which holds the classified textures. The Edge-Based method can handle the structures only in the form of lines and circles whereas the proposed method can handle any type of curves by using the splines. This new inpainting method is developed to inpaint the images in the 3D model. The untextured regions in the 3D model are textured by this proposed inpainting method which guarantees the structure propagation by using the superpixel constraints.

PDRplus: Human Behaviour Sensing Method for Service Field Analysis

This chapter presents a novel method of estimating position, orientation, and multiple actions of a worker in a service field. In general, pedestrian dead reckoning (PDR) is appropriate for effectively estimating the position and orientation of a pedestrian in an indoor environment. However, in actual service fields, PDR is not as accurate for workers’ behaviour sensing when a number of actions for their work other than walking are taking place. Moreover, common sensors for PDR have less information for multiple action recognition other than walking. For realizing human behaviour sensing for service process analysis, we propose a method which integrates human localization and action recognition with the complementary use, named “PDRplus”. In service fields, since position, orientation, and action of a human usually show strong correlation with her or his situation, both the PDR and action recognition can be improved with complementary use of the PDR and action recognition. In this chapter, in order to ensure the effect of the complementary use of the PDR and action recognition, we conducted two types of experiments in real service industry fields. Firstly, we compared accuracies of the action recognition both with and without using the PDR in the restaurant kitchen, and average recognition rate of five types of actions was improved about 19 % points. Secondly, we compared accuracies of the PDR both with and without using the action recognition in house-assembly plants, and average position error was reduced by 19.5 %.