Shing-Tung Yau

High-resolution, real-time 3D absolute coordinate measurement based on a phase-shifting method

We describe a high-resolution, real-time 3D absolute coordinate measurement system based on a phase-shifting method. It acquires 3D shape at 30 frames per second (fps), with 266K points per frame. A tiny marker is encoded in the projected fringe pattern, and detected by software from the texture image and the gamma map. Absolute 3D coordinates are obtained from the detected marker position and the calibrated system parameters. To demonstrate the performance of the system, we measure a hand moving over a depth distance of approximately 700 mm, and human faces with expressions. Applications of such a system include manufacturing, inspection, entertainment, security, medical imaging.

Generic nonsinusoidal phase error correction for three-dimensional shape measurement using a digital video projector

A structured light system using a digital video projector is widely used for 3D shape measurement. However, the nonlinear gamma of the projector causes the projected fringe patterns to be nonsinusoidal, which results in phase error and therefore measurement error. It has been shown that, by using a small look-up table (LUT), this type of phase error can be reduced significantly for a three-step phase-shifting algorithm. We prove that this algorithm is generic for any phase-shifting algorithm. Moreover, we propose a new LUT generation method by analyzing the captured fringe image of a flat board directly. Experiments show that this error compensation algorithm can reduce the phase error to at least 13 times smaller.

Multilevel quality-guided phase unwrapping algorithm for real-time three-dimensional shape reconstruction

A multilevel quality-guided phase unwrapping algorithm for real-time 3D shape measurement is presented. The quality map is generated from the gradient of the phase map. Multilevel thresholds are used to unwrap the phase level by level. Within the data points in each level, a fast scan-line algorithm is employed. The processing time of this algorithm is approximately 18.3 ms for an image size of 640x480 pixels in an ordinary computer. We demonstrate that this algorithm can be implemented into our real-time 3D shape measurement system for real-time 3D reconstruction. Experiments show that this algorithm improves the previous scan-line phase unwrapping algorithm significantly although it reduces its processing speed slightly.

GPU-assisted high-resolution, real-time 3-D shape measurement

This paper describes a Graphics Processing Unit (GPU)-assisted real-time three-dimensional shape measurement system. Our experiments demonstrated that the absolute coordinates calculation and rendering speed of a GPU is more than four times faster than that of a dual CPU workstation with the same graphics card. By implementing the GPU into our system, we realized simultaneous absolute coordinate acquisition, reconstruction and display at 30 frames per second with a resolution of approximately 266K points per frame. Moreover, a 2+1 phase-shifting algorithm was employed to alleviate the measurement error caused by motion. Applications of the system include medical imaging, manufacturing, entertainment, and security.

High-speed three-dimensional shape measurement system using a modified two-plus-one phase-shifting algorithm

This paper describes a high-resolution, real-time, three- dimensional shape measurement system using the modified two-plus- one phase-shifting algorithm. The data acquisition speed is as high as 60 frames/s with an image resolution of 640480 pixels per frame. Ex- periments demonstrated that the system was able to acquire the dy- namic changing objects such as facial geometric shape changes when the subject is speaking, and the modified two-plus-one phase-shifting algorithm can further alleviate the error due to motion. Applications of this system include manufacturing, online inspection, medical imaging, compute vision, and computer graphics.

High Dynamic Range Scanning Technique

Measuring objects with a high variation range of surface reflectivity is challenging for any optical method: This paper addresses a high dynamic range scanning technique that can measure this type of object. It takes advantage of one merit of a phase-shifting algorithm: pixel-by-pixel phase retrieval. For each measurement, a sequence of fringe images with different exposures are taken: the brightest ones have good fringe quality in the darkest areas while the darkest ones have good fringe quality in the brightest areas. They are arranged from brighter to darker (i.e., from higher exposure to lower exposure). The final fringe images, used for phase retrieval, are produced pixel-by-pixel by choosing the brightest but unsaturated corresponding pixel from one exposure. A phase-shifting algorithm is employed to compute the phase, which can be further converted to coordinates. Our experiments demonstrate that the proposed technique can successfully measure objects with high dynamic range of surface reflectivity variation.

Three-dimensional shape measurement using a structured light system with dual cameras

A structured light system for three-dimensional shape measurement with single camera has the shortcoming of camera occlusion. To alleviate this problem, this paper introduces a structured light system with dual cameras for three-dimensional shape measurement. We discuss (1) system description, (2) system calibration, (3) three-dimensional data registration using the iterative closest-point (ICP) algorithm, and (4) three-dimensional data merging using holoimage. The principle of the system is introduced, and experiments are presented to verify its performance.

Absolute phase-assisted three-dimensional data registration for a dual-camera structured light system

For a three-dimensional shape measurement system with a single projector and multiple cameras, registering patches from different cameras is crucial. Registration usually involves a complicated and time-consuming procedure. We propose a new method that can robustly match different patches via absolute phase without significantly increasing its cost. For y and z coordinates, the transformations from one camera to the other are approximated as third-order polynomial functions of the absolute phase. The x coordinates involve only translations and scalings. These functions are calibrated and only need to be determined once. Experiments demonstrated that the alignment error is within RMS 0.7 mm.

Three-dimensional data merging using Holoimage

Three-dimensional data merging is vital for full-field three-dimensional (3D) shape measurement. All 3D range data patches, acquired from either different sensors or the same sensor in different viewing angles, have to be merged into a single piece to facilitate future data analysis. A novel method for 3D data merging using Holoimage is proposed. Similar to the 3D shape measurement system using a phase-shifting method, Holoimage is a phase-shifting–based computer synthesized fringe image. The 3D information is retrieved from Holoimage using a phase-shifting method. If two patches of 3D data with overlapping areas are rendered by OpenGL, the overlapping areas are resolved by the graphics pipeline, that is, only the front geometry can be visualized. Therefore, the merging is performed if the front geometry information can be obtained. Holoimage is to obtain the front geometry by projecting the fringe patterns onto the rendered scene. We also demonstrated that each point of the geometry in the overlapping area can be obtained by averaging the corresponding point of the geometries reconstructed from Holoimage for each patch. Moreover, using Holoimage, the texture can also be obtained. Both simulation and experiments demonstrated the success of the proposed method.

High-resolution, real-time-geometry video acquisition system

The geometric video acquisition system can be applied in many fields. One immediate application is 3-D modeling, for example, facial expressions modeling. Applications can be found in gaming. Since the acquisition, reconstruction, and display can be done in real-time, digitizing the player and putting his/her face into the virtual game world will be feasible in the near future. Another application is animation and movies, the real captured video data can be used in the animation world immediately. The system can also be applied to plastic surgery. Moreover, the same technology can be applied to homeland security. The geometry of the object overcomes many shortcomings of security checking techniques based on 2-D images, such as facial recognition, finger print, etc. Other applications include motion capture, fast prototyping, virtual reality, feature films, human computer interaction, industrial measurement, and inspection.