Shijie Feng

High-speed three-dimensional profilometry for multiple objects with complex shapes.

This paper describes an easy-to-implement three-dimensional (3-D) real-time shape measurement technique using our newly developed high-speed 3-D vision system. It employs only four projection fringes to realize full-field phase unwrapping in the presence of discontinuous or isolated objects. With our self-designed pattern generation hardware and a modified low-cost DLP projector, the four designed patterns can be generated and projected at a switching speed of 360 Hz. Using a properly synchronized high-speed camera, the high-speed fringe patterns distorted by measured objects can be acquired and processed in real-time. The resulting system can capture and display high-quality textured 3-D data at a speed of 120 frames per second, with the resolution of 640 × 480 points. The speed can be trebled if a camera with a higher frame rate is employed. We detail our shape measurement technique, including the four-pattern decoding algorithm as well as the hardware design. Some evaluation experiments have been carried out to demonstrate the validity and practicability of the proposed technique.

Optimized pulse width modulation pattern strategy for three-dimensional profilometry with projector defocusing

Three-dimensional profilometry by sinusoidal fringe projection using phase-shifting algorithms is usually distorted by the nonlinear intensity response of commercial video projectors. To overcome this problem, several methods including sinusoidal pulse width modulation (SPWM) were proposed to generate sinusoidal fringe patterns with binary ones by defocusing the project to some certain extent. However, the residual errors are usually nonnegligible for highly accurate measurement fields, especially when the defocusing level is insufficient. In this work, we propose two novel methods to further improve the defocusing technique. We find that by properly optimizing SPWM patterns according to some criteria, and combining SPWM technique with four-step phase-shifting algorithm, the dominant undesired harmonics will have no impact on the phase obtained. We also propose a new sinusoidal fringe generation technique called tripolar SPWM, which can generate ideal sinusoidal fringe patterns with a very small degree of defocusing. Simulations and experiments are presented to verify the performance of these two proposed techniques.

Real-time 3-D shape measurement with composite phase-shifting fringes and multi-view system

In recent years, fringe projection has become an established and essential method for dynamic three-dimensional (3-D) shape measurement in different fields such as online inspection and real-time quality control. Numerous high-speed 3-D shape measurement methods have been developed by either employing high-speed hardware, minimizing the number of pattern projection, or both. However, dynamic 3-D shape measurement of arbitrarily-shaped objects with full sensor resolution without the necessity of additional pattern projections is still a big challenge. In this work, we introduce a high-speed 3-D shape measurement technique based on composite phase-shifting fringes and a multi-view system. The geometry constraint is adopted to search the corresponding points independently without additional images. Meanwhile, by analysing the 3-D position and the main wrapped phase of the corresponding point, pairs with an incorrect 3-D position or a considerable phase difference are effectively rejected. All of the qualified corresponding points are then corrected, and the unique one as well as the related period order is selected through the embedded triangular wave. Finally, considering that some points can only be captured by one of the cameras due to the occlusions, these points may have different fringe orders in the two views, so a left-right consistency check is employed to eliminate those erroneous period orders in this case. Several experiments on both static and dynamic scenes are performed, verifying that our method can achieve a speed of 120 frames per second (fps) with 25-period fringe patterns for fast, dense, and accurate 3-D measurement.

Graphics processing unit–assisted real-time three-dimensional measurement using speckle-embedded fringe

This paper presents a novel two-frame fringe projection technique for real-time, accurate, and unambiguous three-dimensional (3D) measurement. One of the frames is a digital speckle pattern, and the other one is a composite image which is generated by fusing that speckle image with sinusoidal fringes. The contained sinusoidal component is used to obtain a wrapped phase map by Fourier transform profilometry, and the speckle image helps determine the fringe order for phase unwrapping. Compared with traditional methods, the proposed pattern scheme enables measurements of discontinuous surfaces with only two frames, greatly reducing the number of required patterns and thus reducing the sensitivity to movements. This merit makes the method very suitable for inspecting dynamic scenes. Moreover, it shows close performance in measurement accuracy compared with the phase-shifting method from our experiments. To process data in real time, a Compute Unified Device Architecture-enabled graphics processing unit is adopted to accelerate some time-consuming computations. With our system, measurements can be performed at 21 frames per second with a resolution of 307,000 points per frame.

Micro Fourier Transform Profilometry (μFTP): 3D shape measurement at 10,000 frames per second

Abstract Fringe projection profilometry is a well-established technique for optical 3D shape measurement. However, in many applications, it is desirable to make 3D measurements at very high speed, especially with fast moving or shape changing objects. In this work, we demonstrate a new 3D dynamic imaging technique, Micro Fourier Transform Profilometry (μFTP), which can realize an acquisition rate up to 10,000 3D frame per second (fps). The high measurement speed is achieved by the number of patterns reduction as well as high-speed fringe projection hardware. In order to capture 3D information in such a short period of time, we focus on the improvement of the phase recovery, phase unwrapping, and error compensation algorithms, allowing to reconstruct an accurate, unambiguous, and distortion-free 3D point cloud with every two projected patterns. We also develop a high-frame-rate fringe projection hardware by pairing a high-speed camera and a DLP projector, enabling binary pattern switching and precisely synchronized image capture at a frame rate up to 20,000 fps. Based on this system, we demonstrate high-quality textured 3D imaging of 4 transient scenes: vibrating cantilevers, rotating fan blades, flying bullet, and bursting balloon, which were previously difficult or even unable to be captured with conventional approaches.

Motion-oriented high speed 3-D measurements by binocular fringe projection using binary aperiodic patterns

Fringe projection is an extensively used technique for high speed three-dimensional (3-D) measurements of dynamic objects. To precisely retrieve a moving object at pixel level, researchers prefer to project a sequence of fringe images onto its surface. However, the motion often leads to artifacts in reconstructions due to the sequential recording of the set of patterns. In order to reduce the adverse impact of the movement, we present a novel high speed 3-D scanning technique combining the fringe projection and stereo. Firstly, promising measuring speed is achieved by modifying the traditional aperiodic sinusoidal patterns so that the fringe images can be cast at kilohertz with the widely used defocusing strategy. Next, a temporal intensity tracing algorithm is developed to further alleviate the influence of motion by accurately tracing the ideal intensity for stereo matching. Then, a combined cost measure is suggested to robustly estimate the cost for each pixel and lastly a three-step framework of refinement follows not only to eliminate outliers caused by the motion but also to obtain sub-pixel disparity results for 3-D reconstructions. In comparison with the traditional method where the effect of motion is not considered, experimental results show that the reconstruction accuracy for dynamic objects can be improved by an order of magnitude with the proposed method.

Coded multi-angular illumination for Fourier ptychography based on Hadamard codes

Fourier ptychographic microscopy (FPM) is a newly developed super-resolution technique, which employs angularly varying illumination and a phase retrieval algorithm to surpass the diffraction limit of the objective lens. Specifically, FP captures a set of low-resolution (LR) images, under angularly varying illuminations, and stitches them together in the Fourier domain. However, because the requisite large number of incident illumination angles, the long capturing process becomes an obvious limiting factor. Furthermore, in order to acquire high-dynamic range images, the time can be increased several times over. In this work, utilizing the Hadamard code principle, we propose a highly efficient method, which applies coded multi-angular illumination for FPM, to shorten the exposure time of each raw image. High acquisition efficiency is achieved by employing an optimal multi-angular illumination scheme by using two set of Hadamard coded multiplexing patterns. Both simulation and experimental results indicate that the proposed multi-angular illumination process could shorten the acquisition time of conventional FPM.

Robust and efficient multi-frequency temporal phase unwrapping: optimal fringe frequency and pattern sequence selection

Temporal phase unwrapping (TPU) is an essential algorithm in fringe projection profilometry (FPP), especially when measuring complex objects with discontinuities and isolated surfaces. Among others, the multi-frequency TPU has been proven to be the most reliable algorithm in the presence of noise. For a practical FPP system, in order to achieve an accurate, efficient, and reliable measurement, one needs to make wise choices about three key experimental parameters: the highest fringe frequency, the phase-shifting steps, and the fringe pattern sequence. However, there was very little research on how to optimize these parameters quantitatively, especially considering all three aspects from a theoretical and analytical perspective simultaneously. In this work, we propose a new scheme to determine simultaneously the optimal fringe frequency, phase-shifting steps and pattern sequence under multi-frequency TPU, robustly achieving high accuracy measurement by a minimum number of fringe frames. Firstly, noise models regarding phase-shifting algorithms as well as 3-D coordinates are established under a projector defocusing condition, which leads to the optimal highest fringe frequency for a FPP system. Then, a new concept termed frequency-to-frame ratio (FFR) that evaluates the magnitude of the contribution of each frame for TPU is defined, on which an optimal phase-shifting combination scheme is proposed. Finally, a judgment criterion is established, which can be used to judge whether the ratio between adjacent fringe frequencies is conducive to stably and efficiently unwrapping the phase. The proposed method provides a simple and effective theoretical framework to improve the accuracy, efficiency, and robustness of a practical FPP system in actual measurement conditions. The correctness of the derived models as well as the validity of the proposed schemes have been verified through extensive simulations and experiments. Based on a normal monocular 3-D FPP hardware system, our method enables high-precision unambiguous 3-D shape measurement with the highest fringe frequency up to 180 by using only 7 fringe patterns achieving a depth precision ∼ 38μm across a field of view of 400 × 300 × 400 mm.

Motion-compensated three-step phase-shifting profilometry

Phase-shifting profilometry (PSP) is a widely used approach to high-accuracy three-dimensional shape measurements. However, when it comes to moving objects, phase errors induced by the movement often result in severe artifacts even though a high-speed camera is in use. From our observations, there are three kinds of motion artifacts: motion ripples, motion-induced phase unwrapping errors, and motion outliers. We present a novel motion-compensated PSP to remove the artifacts for dynamic measurements of rigid objects. The phase error of motion ripples is analyzed for the phaseshifting algorithm and is compensated using the statistical nature of the fringes. The phase unwrapping errors are corrected exploiting adjacent reliable pixels, and the outliers are removed by comparing the original phase map with a smoothed phase map. Compared with the three-step PSP, our method can improve the accuracy significantly for objects in motion

Robust stereo phase unwrapping based on a quad-camera system

In this work, we demonstrate that by using a quad-camera multi-view fringe projection system and carefully arranging the relative spatial positions between the cameras and the projector, it becomes possible to completely eliminate the phase ambiguities in conventional three-step PSP patterns with high-fringe-density without projecting any additional patterns or embedding any auxiliary signals. Benefit from the position-optimized quadcamera system, stereo phase unwrapping can be efficiently and reliably performed by flexible phase consistency checks. Besides, redundant information of multiple phase consistency checks is fully used through a weighted phase difference scheme to further enhance the reliability of phase unwrapping. This paper explains the 3D measurement principle and the basic design of quad-camera system, and finally demonstrates that the resultant dynamic 3D sensing system can realize real-time 3D reconstruction with a depth precision of 50 μm.