Hiromasa Yoshimoto

Real-time human motion sensing based on vision-based inverse kinematics for interactive applications

Vision-based human motion sensing has a strong merit that it does not impose any physical restrictions on humans, which provides a natural way of measuring human motion. However, its real-time processing is not easy to realize, because a human body has a high degrees of freedom, whose vision-based analysis is not simple and is usually time consuming. Here, we have developed a method in which human postures are analyzed from a limited number of visual cues. It is a combination of numerical analysis of inverse kinematics and visual search. Our method is based on a general framework of inverse kinematics, and, therefore, we can use relatively complex human figure model, which can generate natural human motion. In our experimental studies, we show that our implemented system works in real-time on a PC-cluster.

Vision-based real-time motion capture system using multiple cameras

In this paper, we discuss a vision-based real-time motion capture system, which is constructed on a PC-cluster. Vision-based motion capture does not impose physical restrictions on humans, which provides a natural way of measuring human motion. However, there are several issues to be solved, which are robust estimation of human motion from a limited number of visual cues and computation cost of the estimation algorithm. To deal with these issues, we have developed multi-view-based algorithms using multiple cameras and have implemented the algorithms on a PC-cluster to solve the computation problem. In this paper, we present our experimental study on vision-based real-time motion capture with emphasis on 3D human posture estimation.

Real-time image processing on IEEE1394-based PC cluster

In this paper we introduce a new real-time parallel image processing system based on an IEEE1394-based PC-cluster. PC-clusters are becoming standard tools in the field of computer vision, especially for real-time multi-view image processing. To reduce the cost of system construction, we have employed IEEE1394 bus as a base network of the PC-cluster. IEEE1394 bus has the throughput of up to 400Mbps, which is large enough to transfer uncompressed video data, and supports not only asynchronous data transfer but isochronous one, which is inevitable for real-time data transfer. In addition, since we have several products of IEEE1394-based digital cameras, we can easily integrate real-time image processing system including camera systems. This paper presents an overview of our real-time parallel image processing system on an IEEE1394based PC-cluster, referring to basic features of IEEE1394. We also show some experimental results to evaluate the performance of the PC-cluster, comparing it with the performance of our previous Myrinet-based PC-cluster. Then we show a prototypical application of the system, which is real-time volume reconstruction from multi-view images by visual cone intersection method.

Confidence-driven architecture for real-time vision processing and its application to efficient vision-based human motion sensing

In this paper, we discuss a real-time vision architecture which provides a mechanism of controlling trade-off between the accuracy and the latency of vision systems. In vision systems, to acquire accurate information from input-images, the huge amount of computation power is usually required. On the other hand, to realize real-time processing, we must reduce the latency. Therefore, under given hardware resources, we must make difficult trade-off between the accuracy and the latency so that the quality of the systems output keeps appropriateness. To solve the problem, we propose confidence-driven scheme, which enables us to control the trade-off dynamically and easily without rebuilding vision systems. In the confidence-driven architecture, the trade-off can be controlled by specifying a generalized parameter called confidence, which relatively indicates how accurate the analysis should be. Here, we present the concept of confidence-driven architecture, and then, we show a shared memory which uses confidence-driven scheme. Using confidence-driven memory, we can use imprecise computation model to reduce the latency without a large decrease of accuracy.

Real-time communication for distributed vision processing based on imprecise computation model

In this paper we propose an efficient real-time communication mechanism for distributed vision processing. One of the biggest problems of distributed vision processing, as is the same as in other distributed systems, is how to reduce the overhead of communication among computation nodes. In vision processing, we have to deal with a lot of time varying variables, some of which are large in size, and, therefore, the efficiency of sending and receiving of those variables is essential. To solve the problem, we propose Accuracy-driven Memory architecture, whose key idea is based on imprecise computation model and predictive coding. Here, we will present the basic framework of Accuracy-driven Memory architecture and show its efficiency based on some simulation results.

Cubistic Representation for Real-Time 3D Shape and Pose Estimation of Unknown Rigid Object

This paper introduces Cubistic Representation as a novel 3D surface shape model. Cubistic representation is a set of 3D surface fragments, each fragment contains subjects 3D surface shape and its color and redundantly covers the subject surface. By laminating these fragments using a given pose parameter, the subjects appearance can be synthesized. Using cubistic representation, we propose a real-time 3D rigid object tracking approach by acquiring the 3D surface shape and its pose simultaneously. We use the particle filter scheme for both shape and pose estimation, each fragment is used as a partial shape hypothesis and is sampled and refined by a particle filter. We also use the RANSAC algorithm to remove wrong fragments as outliers to refine the shape. We also implemented an online demonstration system with GPU and a Kinect sensor and evaluated the performance of our approach in a real environment.

Performance evaluation of vision-based real-time motion capture

In this paper, we discuss a vision-based real-time motion capture system, which is constructed on a PC-cluster. Vision-based motion capture is a merit that it does not impose any physical restrictions on humans, which provides a natural way of measuring human motion. However, there are several issues to be solved, which are robust estimation of human motion from limited number of visual cues, computation cost of the estimation algorithm. To deal with these issues, we have developed multi-view-based algorithms using multiple cameras and we have implemented the algorithms on a PC-cluster to solve the computation problem. In this paper, we present our experimental study on vision-based real-time motion capture with emphasis on 3D human posture estimation.

Cell-based 3D Video Capture of a Freely-moving Object Using Multi-viewpoint Active Cameras

We propose a method to capture 3D video of an object that moves in a large area using active cameras. Our main ideas are to partition a desired target area into hexagonal cells, and to control active cameras based on these cells. Accurate camera calibration and continuous capture of the object with at least one third of the cameras are guaranteed regardless of the objects motion. We show advantages of our method over an existing capture method using fixed cameras. We also show that our method can be applied to a real studio.

Cell-Based 3D Video Capture Method with Active Cameras

This paper proposes a 3D video capture method with active cameras, which enables us to produce 3D video of a moving object in a widespread area. Most existing capture methods use fixed cameras and have strong restrictions on allowable object motion; an object cannot move in a wide area. To solve this problem, our method partitions a studio space into a set of subspaces named “cells”, and conducts the camera calibration and control for object tracking based on the cells. We first formulate our method as an optimization problem and then propose an algorithm to solve it.

Multi-modal sensing and analysis of poster conversations with smart posterboard

Conversations in poster sessions in academic events, referred to as poster conversations, pose interesting, and challenging topics on multi-modal signal and information processing. We have developed a smart posterboard for multi-modal recording and analysis of poster conversations. The smart posterboard has multiple sensing devices to record poster conversations, so we can review who came to the poster and what kind of questions or comments he/she made. The conversation analysis incorporates face and eye-gaze tracking for effective speaker diarization. It is demonstrated that eye-gaze information is useful for predicting turn-taking and also improving speaker diarization. Moreover, high-level indexing of interest and comprehension level of the audience is explored based on the multi-modal behaviors during the conversation. This is realized by predicting the audiences speech acts such as questions and reactive tokens.