Itaru Kitahara

ViSE: Visual Search Engine Using Multiple Networked Cameras

We propose a visual search engine (ViSE) as a semi-automatic component in a surveillance system using networked cameras. The ViSE aims to assist the monitoring operation of huge amounts of captured video streams, which tracks and finds people in the video based on their primitive features with the interaction of a human operator. We address the issues of object detection and tracking, shadow suppression and color-based recognition for the proposed system. The experimental results on a set of video data with ten subjects showed that ViSE retrieves correct candidates with 83% recall at 83% precision

Live mixed-reality 3D video in soccer stadium

This paper proposes a method to realize a 3D video display system that can capture video from multiple cameras, reconstruct 3D models and transmit 3D video data in real time. We represent a target object with a simplified 3D model consisting of a single plane and a 2D texture extracted from multiple cameras. This 3D model is simple enough to be transmitted via a network. We have developed a prototype system that can capture multiple videos, reconstruct 3D models, transmit the models via a network, and display 3D video in real time. A 3D video of a typical soccer scene that includes a dozen players was processed at 26 frames per second.

A nested marker for augmented reality

A Nested Marker, a novel visual marker for camera calibration in augmented reality (AR), enables accurate calibration even when the observer is moving very close to or far away from the marker. Our proposed Nested Marker has a recursive layered structure. One marker at an upper layer contains four smaller markers at the lower layer. Smaller markers can also have lower-layer markers nesting inside them. Each marker can be identified by its inside pattern, so the system can select a proper calibration parameter set for the marker. When the observer views the marker close-up, the lowest layer marker will work. When the observer views the marker from a distance, the top-layer marker will work. It is also possible to simultaneously utilize all visible markers in different layers for more stable calibration. Note that Nested Marker can be used in a standard ARToolkit framework. We have also developed an AR system to demonstrate the ability of Nested Marker

3D image display with motion parallax by camera matrix stereo

We propose a 3D image display system which can present real scenes with realistic motion parallax. In the sensing system, a scene is observed by using a camera matrix. An excellent stereo algorithm SEA which utilizes 3/spl times/3 image matrix recovers the depth information of the scene with the density and the sharpness required for high quality image generation. In the display system, following the viewing position of the observer, 2D images with proper notion parallax are generated and presented. A novel algorithm to determine the view parameters suitable for reproducing the motion parallax on a fixed screen and an image generation algorithm which can cope with arbitrary viewing positions are described. A prototype system has been developed to demonstrate the feasibility of the proposed algorithms.

Live 3D Video in Soccer Stadium

This paper proposes a method to realize a 3D video system that can capture video data from multiple cameras, reconstruct 3D models, transmit 3D video streams via the network, and display them on remote PCs. All processes are done in real time. We represent a player with a simplified 3D model consisting of a single plane and a live video texture extracted from multiple cameras. This 3D model is simple enough to be transmitted via a network. A prototype system has been developed and tested at actual soccer stadiums. A 3D video of a typical soccer scene, which includes more than a dozen players, was processed at video rate and transmitted to remote PCs through the internet at 15–24 frames per second.

A Nested Marker for Augmented Reality

A Nested Marker, a novel visual marker for camera calibration in augmented reality (AR), enables accurate calibration even when the observer is moving very close to or far away from the marker. Our proposed Nested Marker has a recursive layered structure. One marker at an upper layer contains four smaller markers at the lower layer. Smaller markers can also have lower-layer markers nesting inside them. Each marker can be identified by its inside pattern, so the system can select a proper calibration parameter set for the marker. When the observer views the marker close-up, the lowest layer marker will work. When the observer views the marker from a distance, the top-layer marker will work. It is also possible to simultaneously utilize all visible markers in different layers for more stable calibration. Note that Nested Marker can be used in a standard ARToolkit framework. We have also developed an AR system to demonstrate the ability of Nested Marker

Robust foreground extraction technique using Gaussian family model and multiple thresholds

We propose a robust method to extract silhouettes of foreground objects from color video sequences. To cope with various changes in the background, the background is modeled as generalized Gaussian Family of distributions and updated by the selective running average and static pixel observation. All pixels in the input video image are classified into four initial regions using background subtraction with multiple thresholds, after which shadow regions are eliminated using color components. The final foreground silhouette is extracted by refining the initial region using morphological processes. We have verified that the proposed algorithm works very well in various background and foreground situations through experiments.

Scalable 3D representation for 3D video display in a large-scale space

The authors introduce their research for realizing a 3D video display system in a very large-scale space such as a soccer stadium, concert hall, etc. They propose a method for describing the shape of a 3D object with a set of planes in order to synthesize a novel view of the object effectively. The most effective layout of the planes can be determined based on the relative locations of an observers viewing position, multiple cameras, and 3D objects. A method is described for controlling the LOD of the 3D representation by adjusting the orientation, interval, and resolution of planes. The data size of the 3D model and the processing time can be reduced drastically. The effectiveness of the proposed method is demonstrated by experimental results.

Face-to-Face Tabletop Remote Collaboration in Mixed Reality

This paper proposes a novel remote face-to-face mixed reality (MR) system that enables two people in distant places to share MR space. Challenging issues to realize such an MR system include capturing, sending, and rendering each users appearance in real time. We developed a method to represent users upper body and hands on the table as a single deformed-billboard. An MR Othello game is implemented as a test bed of the remote face-to-face MR system. Users can play the tabletop game as if their opponent were sitting across from the table, despite being physically separated. By detecting and sending the status of each real game board to the other site, both users feel that they are sharing tabletop objects.

Robust foreground extraction technique using background subtraction with multiple thresholds

We propose a robust method to extract silhouettes of foreground objects from color-video sequences. To cope with various changes in the background, we model the background as a Laplace distribution and update it with a selective running average and static pixel observation. All pixels in the input video image are classified into four initial regions using background subtraction with multiple thresholds. Shadow regions are eliminated using color components, and the final foreground silhouette is extracted by smoothing the boundaries of the foreground and eliminating errors inside and outside of the regions. Experimental results show that the proposed algorithm works very well in various background and foreground situations.