Shinsaku Hiura

Bokode: imperceptible visual tags for camera based interaction from a distance

We show a new camera based interaction solution where an ordinary camera can detect small optical tags from a relatively large distance. Current optical tags, such as barcodes, must be read within a short range and the codes occupy valuable physical space on products. We present a new low-cost optical design so that the tags can be shrunk to 3mm visible diameter, and unmodified ordinary cameras several meters away can be set up to decode the identity plus the relative distance and angle. The design exploits the bokeh effect of ordinary cameras lenses, which maps rays exiting from an out of focus scene point into a disk like blur on the camera sensor. This bokeh-code or Bokode is a barcode design with a simple lenslet over the pattern. We show that a code with 15μm features can be read using an off-the-shelf camera from distances of up to 2 meters. We use intelligent binary coding to estimate the relative distance and angle to the camera, and show potential for applications in augmented reality and motion capture. We analyze the constraints and performance of the optical system, and discuss several plausible application scenarios.

Depth measurement by the multi-focus camera

In this paper, we first introduce the multi-focus camera, a new image sensor used for depth from defocus (DFD) range measurement. It can capture three images with different focus values simultaneously. We then propose two different depth measurement methods using the camera. The first method, an augmented version of the one proposed by N. Asada et al. (1998), employs a noniterative optimization process to compute depth values on edge points. The second one incorporates a coded aperture with the camera; and applies model-based pattern matching to estimate depth values of textured surfaces. Here we propose two types of coded apertures and corresponding analysis algorithms: 1D Fourier analysis to acquire a depth map and a blur-free image from three defocused images taken with a pair of pinholes, and 2D convolution based model matching for the fast and precise depth measurement using a coded aperture with four pinholes. Experimental results showed that the multi-focus camera works well as a practical DFD range sensor and that the coded apertures much improve its range estimation capability for real world scenes.

Dynamic memory: architecture for real time integration of visual perception, camera action, and network communication

In a Cooperative Distributed Vision system a group of communicating Active Vision Agents (AVA, in short, i.e. real time image processor with an active video camera and high speed network interface) cooperate to fulfil a meaningful task such as moving object tracking and dynamic scene visualization. A key issue to design and implement an AVA rests in the dynamic integration of Visual Perception, Camera Action, and Network Communication. This paper proposes a novel dynamic system architecture named Dynamic Memory Architecture, where perception, action, and communication modules share what we call the Dynamic Memory. It maintains not only temporal histories of state variables such as pan-tilt angles of the camera and the target object location but also their predicted values in the future. Perception, action, and communication modules are implemented as parallel processes which dynamically read from and write into the memory according to their own individual dynamics. The dynamic memory supports such asynchronous dynamic interactions (i.e. data exchanges between the modules) without wasting time for synchronization. This no-wait asynchronous module interaction capability greatly facilitates the implementation of real time reactive systems such as moving object tracking. Moreover, the dynamic memory supports the virtual synchronization between multiple AVAs, which facilitates the cooperative object tracking by communicating AVAs. A prototype system for real time moving object tracking demonstrated the effectiveness of the proposed idea.

Fusing Depth from Defocus and Stereo with Coded Apertures

In this paper we propose a novel depth measurement method by fusing depth from defocus (DFD) and stereo. One of the problems of passive stereo method is the difficulty of finding correct correspondence between images when an object has a repetitive pattern or edges parallel to the epipolar line. On the other hand, the accuracy of DFD method is inherently limited by the effective diameter of the lens. Therefore, we propose the fusion of stereo method and DFD by giving different focus distances for left and right cameras of a stereo camera with coded apertures. Two types of depth cues, defocus and disparity, are naturally integrated by the magnification and phase shift of a single point spread function (PSF) per camera. In this paper we give the proof of the proportional relationship between the diameter of defocus and disparity which makes the calibration easy. We also show the outstanding performance of our method which has both advantages of two depth cues through simulation and actual experiments.

Krill-eye: Superposition Compound Eye for Wide-Angle Imaging via GRIN Lenses

We propose a novel wide angle imaging system inspired by compound eyes of animals. Instead of using a single lens, well compensated for aberration, we used a number of simple lenses to form a compound eye which produces practically distortion-free, uniform images with angular variation. The images formed by the multiple lenses are superposed on a single surface for increased light efficiency. We use GRIN (gradient refractive index) lenses to create sharply focused images without the artifacts seen when using reflection based methods for X-ray astronomy. We show the theoretical constraints for forming a blur-free image on the image sensor, and derive a continuum between 1 ∶ 1 flat optics for document scanners and curved sensors focused at infinity. Finally, we show a practical application of the proposed optics in a beacon to measure the relative rotation angle between the light source and the camera with ID information.

3-D tele-direction interface using video projector

We developed a direction system for assisting the work in a real world from a distant site. At first, the 3-D shape of the object is measured and sent to the distant PC. A supervisor at the distant site can observe the CG of the object and draw annotation figures on it. The figures of the direction message are projected onto the object using projectors. The worker is free from any wearing equipment, ex. HMD, and multi projectors avoid the problem of occlusion by the worker body.

User Interface by Virtual Shadow Projection

This paper introduced a user interface based on virtual shadow derived by projector for spatial augmented reality (SAR) environment. Shadow is a daily phenomenon in our daily life and may contribute to build an effective and intuitive connection between physical and digital world. Taking advantage of spatial and optical characteristics of shadow, user can realize remote interaction for ubiquitous interface. The prototype system is implemented and adaptive image processing algorithms for shadow are proposed. The authors has demonstrated in this paper that shadow has possibility to realize simple but effective interface between human and computer systems, which may yield to many useful applications without massive device resources

Calibration of a 3D endoscopic system based on active stereo method for shape measurement of biological tissues and specimen.

For endoscopic medical treatment, measuring the size and shape of the lesion, such as a tumor, is important for the improvement of diagnostic accuracy. We are developing a system to measure the shapes and sizes of living tissue by active stereo method using a normal endoscope on which a micro pattern projector is attached. In order to perform 3D reconstruction, estimating the intrinsic and extrinsic parameters of the endoscopic camera and the pattern projector is required. Particularly, calibration of the pattern projector is difficult. In this paper, we propose a simultaneous estimation method of both intrinsic and extrinsic parameters of the pattern projector. This simplifies the calibration procedure required in practical scenes. Furthermore, we have developed an efficient user interface to intuitively operate the calibration and reconstruction procedures. Using the developed system, we measured the shape of an internal tissue of the soft palate of a human and a biological specimen.

Image destabilization: Programmable defocus using lens and sensor motion

We propose a novel camera setup in which both the lens and the sensor are perturbed during the exposure. We analyze the defocus effects produced by such a setup, and use it to demonstrate new methods for simulating a lens with a larger effective aperture size (i.e., shallower depth of field) and methods for achieving approximately depth-independent defocus blur size. We achieve exaggerated, programmable, and pleasing bokeh with relatively small aperture sizes such as those found on cell phone cameras. Destabilizing the standard alignment of the sensor and lens allows us to introduce programmable defocus effects and achieve greater flexibility in the image capture process.

Eigen space approach for a pose detection with range images

An application of the pose detection using range images usually uses characteristic matching of the geometrical model but this method has two problems: selecting characteristics from range images is difficult; and it is difficult to make a geometrical model for a complicated shape. Previously a parametric eigen-space method was proposed for pose detection. This method makes object recognition and pose detection possible, but this parametric eigen-space method has a problem that intensity-images depend on a variety of light conditions therefore learning images must include variation of light conditions.