Tianyang Tao

Real-time 3-D shape measurement with composite phase-shifting fringes and multi-view system

In recent years, fringe projection has become an established and essential method for dynamic three-dimensional (3-D) shape measurement in different fields such as online inspection and real-time quality control. Numerous high-speed 3-D shape measurement methods have been developed by either employing high-speed hardware, minimizing the number of pattern projection, or both. However, dynamic 3-D shape measurement of arbitrarily-shaped objects with full sensor resolution without the necessity of additional pattern projections is still a big challenge. In this work, we introduce a high-speed 3-D shape measurement technique based on composite phase-shifting fringes and a multi-view system. The geometry constraint is adopted to search the corresponding points independently without additional images. Meanwhile, by analysing the 3-D position and the main wrapped phase of the corresponding point, pairs with an incorrect 3-D position or a considerable phase difference are effectively rejected. All of the qualified corresponding points are then corrected, and the unique one as well as the related period order is selected through the embedded triangular wave. Finally, considering that some points can only be captured by one of the cameras due to the occlusions, these points may have different fringe orders in the two views, so a left-right consistency check is employed to eliminate those erroneous period orders in this case. Several experiments on both static and dynamic scenes are performed, verifying that our method can achieve a speed of 120 frames per second (fps) with 25-period fringe patterns for fast, dense, and accurate 3-D measurement.

Micro Fourier Transform Profilometry (μFTP): 3D shape measurement at 10,000 frames per second

Abstract Fringe projection profilometry is a well-established technique for optical 3D shape measurement. However, in many applications, it is desirable to make 3D measurements at very high speed, especially with fast moving or shape changing objects. In this work, we demonstrate a new 3D dynamic imaging technique, Micro Fourier Transform Profilometry (μFTP), which can realize an acquisition rate up to 10,000 3D frame per second (fps). The high measurement speed is achieved by the number of patterns reduction as well as high-speed fringe projection hardware. In order to capture 3D information in such a short period of time, we focus on the improvement of the phase recovery, phase unwrapping, and error compensation algorithms, allowing to reconstruct an accurate, unambiguous, and distortion-free 3D point cloud with every two projected patterns. We also develop a high-frame-rate fringe projection hardware by pairing a high-speed camera and a DLP projector, enabling binary pattern switching and precisely synchronized image capture at a frame rate up to 20,000 fps. Based on this system, we demonstrate high-quality textured 3D imaging of 4 transient scenes: vibrating cantilevers, rotating fan blades, flying bullet, and bursting balloon, which were previously difficult or even unable to be captured with conventional approaches.

Motion-oriented high speed 3-D measurements by binocular fringe projection using binary aperiodic patterns

Fringe projection is an extensively used technique for high speed three-dimensional (3-D) measurements of dynamic objects. To precisely retrieve a moving object at pixel level, researchers prefer to project a sequence of fringe images onto its surface. However, the motion often leads to artifacts in reconstructions due to the sequential recording of the set of patterns. In order to reduce the adverse impact of the movement, we present a novel high speed 3-D scanning technique combining the fringe projection and stereo. Firstly, promising measuring speed is achieved by modifying the traditional aperiodic sinusoidal patterns so that the fringe images can be cast at kilohertz with the widely used defocusing strategy. Next, a temporal intensity tracing algorithm is developed to further alleviate the influence of motion by accurately tracing the ideal intensity for stereo matching. Then, a combined cost measure is suggested to robustly estimate the cost for each pixel and lastly a three-step framework of refinement follows not only to eliminate outliers caused by the motion but also to obtain sub-pixel disparity results for 3-D reconstructions. In comparison with the traditional method where the effect of motion is not considered, experimental results show that the reconstruction accuracy for dynamic objects can be improved by an order of magnitude with the proposed method.

Robust and efficient multi-frequency temporal phase unwrapping: optimal fringe frequency and pattern sequence selection

Temporal phase unwrapping (TPU) is an essential algorithm in fringe projection profilometry (FPP), especially when measuring complex objects with discontinuities and isolated surfaces. Among others, the multi-frequency TPU has been proven to be the most reliable algorithm in the presence of noise. For a practical FPP system, in order to achieve an accurate, efficient, and reliable measurement, one needs to make wise choices about three key experimental parameters: the highest fringe frequency, the phase-shifting steps, and the fringe pattern sequence. However, there was very little research on how to optimize these parameters quantitatively, especially considering all three aspects from a theoretical and analytical perspective simultaneously. In this work, we propose a new scheme to determine simultaneously the optimal fringe frequency, phase-shifting steps and pattern sequence under multi-frequency TPU, robustly achieving high accuracy measurement by a minimum number of fringe frames. Firstly, noise models regarding phase-shifting algorithms as well as 3-D coordinates are established under a projector defocusing condition, which leads to the optimal highest fringe frequency for a FPP system. Then, a new concept termed frequency-to-frame ratio (FFR) that evaluates the magnitude of the contribution of each frame for TPU is defined, on which an optimal phase-shifting combination scheme is proposed. Finally, a judgment criterion is established, which can be used to judge whether the ratio between adjacent fringe frequencies is conducive to stably and efficiently unwrapping the phase. The proposed method provides a simple and effective theoretical framework to improve the accuracy, efficiency, and robustness of a practical FPP system in actual measurement conditions. The correctness of the derived models as well as the validity of the proposed schemes have been verified through extensive simulations and experiments. Based on a normal monocular 3-D FPP hardware system, our method enables high-precision unambiguous 3-D shape measurement with the highest fringe frequency up to 180 by using only 7 fringe patterns achieving a depth precision ∼ 38μm across a field of view of 400 × 300 × 400 mm.

Motion-compensated three-step phase-shifting profilometry

Phase-shifting profilometry (PSP) is a widely used approach to high-accuracy three-dimensional shape measurements. However, when it comes to moving objects, phase errors induced by the movement often result in severe artifacts even though a high-speed camera is in use. From our observations, there are three kinds of motion artifacts: motion ripples, motion-induced phase unwrapping errors, and motion outliers. We present a novel motion-compensated PSP to remove the artifacts for dynamic measurements of rigid objects. The phase error of motion ripples is analyzed for the phaseshifting algorithm and is compensated using the statistical nature of the fringes. The phase unwrapping errors are corrected exploiting adjacent reliable pixels, and the outliers are removed by comparing the original phase map with a smoothed phase map. Compared with the three-step PSP, our method can improve the accuracy significantly for objects in motion

Robust stereo phase unwrapping based on a quad-camera system

In this work, we demonstrate that by using a quad-camera multi-view fringe projection system and carefully arranging the relative spatial positions between the cameras and the projector, it becomes possible to completely eliminate the phase ambiguities in conventional three-step PSP patterns with high-fringe-density without projecting any additional patterns or embedding any auxiliary signals. Benefit from the position-optimized quadcamera system, stereo phase unwrapping can be efficiently and reliably performed by flexible phase consistency checks. Besides, redundant information of multiple phase consistency checks is fully used through a weighted phase difference scheme to further enhance the reliability of phase unwrapping. This paper explains the 3D measurement principle and the basic design of quad-camera system, and finally demonstrates that the resultant dynamic 3D sensing system can realize real-time 3D reconstruction with a depth precision of 50 μm.

High-speed three-dimensional shape measurement using improved bi-frequency scheme and number-theoretical phase unwrapping

In recent years, fringe projection profilometry (FPP), as a kind of three-dimensional shape measurement technology, has achieved the great breakthrough, due to the rapid development of the high-speed camera and high-speed projection equipment. The number-theoretical approach, as a classical method for the temporal phase unwrapping algorithm, is suitable for the binary defocusing FPP since it can avoid the acquiring of low frequency fringes. However, in order to ensure the stability of phase unwrapping, the period of fringe is generally around 20, which leads to the limited accuracy of 3D measurement. In this paper, we propose a bi-frequency number-theoretical phase unwrapping method with depth constraint. Using the principle of depth constraint, we will eliminate the period ambiguities of each pixel within a pixel-variant local period range so that the method only requires the coprime of two fringe frequencies within the local period range instead of the conventional global range. In this way, the requirement of stability of the traditional number-theoretical phase unwrapping can be adjusted from global range to local range. The stability is higher in the local period range due to containing less period ambiguities. As a result, we can realize phase unwrapping of higher frequency fringes with the same stability. Several experiments on various scenes are performed, verifying that our method can achieve high-speed and high-precision 3D measurement.

Calibration of telecentric cameras with distortion center estimation

A distortion-free telecentric camera dose not have an optical center because of the orthogonal projection. However, the position of optical center should be known when the lens distortion is considered. Since the full-scale parameters are derived through an iterative algorithm, critical initial values of the optical center should be provided to avoid being trapped in local minima. In this paper, we proposed a two-step algorithm to estimate the optical center as the trustworthy initial value for the subsequent iteration process. The first step is directly calculating the pixel coordinates of the lateral distortion center using the extracted control points. The second step is optimizing both lateral and tangential coefficients considering the properties of the affine transformation in the imaging process. The effectiveness of our proposed method is proven by the measurement results using a new developed microscopic telecentric stereovision system.

Three-dimensional measurement based on a Greenough-type stereomicroscope using phase-shifting projection

We propose an absolute 3D micro surface profile measurement technique based on a Greenough-type stereomicroscope. The camera and the projector are fixed on the stereomicroscope, facilitating a flexible 3D measurement of objects with different heights. Experiments of both calibration and measurements are conducted, and the results show that our proposed method works well for measuring different types of geometry like spheres, ramps and planes etc. The reconstruction accuracy can achieve 4.8 μm with a measurement depth about 3 mm.

Multi-view phase unwrapping with composite fringe patterns

We introduce a high-speed 3-D shape measurement technique based on composite phase-shifting fringes and a stereo camera system. Epipolar constraint is adopted to search the corresponding point independently without additional images. Meanwhile, by analysing the 3-D position and the main wrapped phase of the corresponding point, pairs with an incorrect 3-D position or considerable phase difference are effectively rejected. Then all the qualified corresponding points are corrected, and the unique one as well as the related period order is selected through the embedded triangular wave. Finally, considering that some points can only be captured by a single camera in some shading areas, the final period order of these points in one camera and the one of their corresponding points in another camera always have different values, so left-right consistency check is employed to eliminate those erroneous period orders in this case. Several experiments on both static and dynamic scenes are performed, verifying that our method can achieve a speed of 120 frames per second (fps) with 25-period fringe patterns for fast, dense, and accurate 3-D measurement.