Moon-Hyun Lee

Multispectral Imaging Using Multiplexed Illumination

Many vision tasks such as scene segmentation, or the recognition of materials within a scene, become considerably easier when it is possible to measure the spectral reflectance of scene surfaces. In this paper, we present an efficient and robust approach for recovering spectral reflectance in a scene that combines the advantages of using multiple spectral sources and a multispectral camera. We have implemented a system based on this approach using a cluster of light sources with different spectra to illuminate the scene and a conventional RGB camera to acquire images. Rather than sequentially activating the sources, we have developed a novel technique to determine the optimal multiplexing sequence of spectral sources so as to minimize the number of acquired images. We use our recovered spectral measurements to recover the continuous spectral reflectance for each scene point by using a linear model for spectral reflectance. Our imaging system can produce multispectral videos of scenes at 30fps. We demonstrate the effectiveness of our system through extensive evaluation. As a demonstration, we present the results of applying data recovered by our system to material segmentation and spectral relighting.

Surface-Independent direct-projected augmented reality

Some issues on direct-projected augmented reality (DirectAR) are addressed: the projection may be geometrically distorted due to the non-planar surface (geometric distortion); the projection cannot be seen to user as intended because the position of the projector is not the same as that of the user’s viewpoint (viewpoint-ignorant projection); the projection may be modulated by surface color (radiometric distortion); the projected area may not have uniform brightness when the projection is obliquely headed for the surface (uneven projection). We propose an integrated framework for handling all the problems. Experimental results demonstrate that the problems unavoidable in surface-independent DirectAR can be successfully resolved.

Optimal illumination for discriminating objects with different spectra.

Color differences are determined by illumination, the spectral reflectance of objects, and the spectral sensitivity of the imaging sensor. We explore the optimal illumination conditions that best separate one object from another. Given two objects with distinct spectra, we derive the optimal illumination spectrum to maximize their color distance with a plain RGB camera. In practice, it is crucial to compose the most appropriate illuminations using available lighting sources, since creating an arbitrary illumination spectrum is unrealistic. Therefore, we derive the optimal linear combination of the provided illumination sources. Finally, we verify the effectiveness of the methods through experiments.

Undistorted projection onto dynamic surface

For projector-based augmented reality systems, geometric correction is a crucial function. There have been many researches on the geometric correction in the literature. However, most of them focused only on static projection surfaces and could not give us a solution for dynamic surfaces (with varying geometry in time). In this paper, we aim at providing a simple and robust framework for projecting augmented reality images onto dynamic surfaces without image distortion. For this purpose, a new technique for embedding pattern images into the augmented reality images, which allows simultaneous display and correction, is proposed and its validity is shown in experimental results.

Simultaneous Geometric and Radiometric Adaptation to Dynamic Surfaces With a Mobile Projector-Camera System

Existing geometric and radiometric compensation methods for direct-projected augmented reality focus on static projection surfaces rather than dynamic surfaces (with varying geometry in time). We aim at providing an effective framework for projecting a sequence of augmented reality images onto dynamic surfaces without geometric and radiometric distortion. We present our design of a special pattern image for simultaneous geometric and radiometric compensation and evaluate two different techniques for embedding the pattern image into augmented reality images. The validity of the proposed method is examined through a variety of experiments with a mobile projector-camera system.

Efficient mobile museum guidance system using augmented reality

We propose a mobile augmented reality (AR) system for helpful and efficient museum guidance. The system can provide not only useful information of exhibitions such as their inside view, origination, and how-to-use, but also a variety of interactions with the exhibitions. The system prototype is implemented on ultra mobile personal computer (UMPC) which is equipped with a camera, an ultrasound receiver, and a gyro sensor. The system function consists of two parts: tracking part, and event handling and visualization part. The tracking part consists of two parts again: global pose tracking part using ultrasound sensors and a gyro sensor, and local pose tracking part using a vision sensor which computes effectively the camera pose under less-controlled environments by combining edge information with feature point information of the camera images. The usefulness of the proposed system and user satisfaction are evaluated through the experimental results in various scenarios.

Direct-Projected AR Based Interactive User Interface for Medical Surgery

In the field of computer aided surgery, augmented reality (AR) technology has been successfully used for enhancing accuracy of surgery and making surgeons convenient by visually assisting them in performing a number of complicated and time-consuming medical operations. However, there are still medical operations that do not receive the benefit of AR technology. As a representative one, surgeons still use an ink pen when they mark surgical targets for scheduling an operation. The ink pen is inconvenient because the mark drawn by the foreign matter is not easily modified or deleted. And the ink pen is also unlikely to be sanitary. In this paper, we propose an interactive user interface based on direct-projected augmented reality (DirectAR) technology for handling all these problems with the ink pen and its validity is shown in experimental results.

Contrast Enhancement in Direct-Projected Augmented Reality

In direct-projected augmented reality, use of projector makes it possible to utilize 3-D real and large objects as displays and frees from discomforts incidental to wearing a device such as HMD. However, the resulting augmentation usually has poor depth resolution due to projectors with low contrast. In direct-projected augmented reality, the radiometric compensation, which is originally employed to recover the color properties of the projector input image in direct-projected augmented reality, seems to be another cause of decreasing the final contrast of the augmentation. In this paper, a contrast enhancement method is proposed that combines a hue- and saturation-preserving histogram equalization with the radiometric compensation in direct-projected augmented reality. The method guarantees that the radiometrically compensated color is maintained while the brightness (or intensity) contrast is enhanced. Experimental results demonstrate the validity of our method

Optimal illumination spectrum for endoscope

Color differences are determined by illumination, the spectral reflectance of objects, and the spectral sensitivity of the imaging sensor. In this paper, we explore the optimal illumination conditions that best separate one object from another. Given two objects with distinct spectra, we derive the optimal illumination spectrum to maximize their color distance with a plain RGB camera. In practice, it is crucial to compose the most appropriate illuminations using available lighting sources since creating arbitrary illumination spectrum is unrealistic. Therefore, we derive the optimal linear combination of the provided illumination sources. Finally, we verify the effectiveness of the methods through experiments.

Projection-Based Diminished Reality System

Diminished reality is a technique that provides visual convenience by virtually hiding an object. Despite that the promising advances of projection-based vision techniques provided good solutions for vision applications, there has no report of the development of diminished reality systems using projection-based vision techniques. As a first attempt, this paper proposes a projection-based diminished reality system using two essential techniques: image completion technique for removing a target object; radiometric compensation technique for seamless projection onto the target object. Potential of the proposed system is demonstrated through experiments.