Yan Hu

Real-time 3-D shape measurement with composite phase-shifting fringes and multi-view system

In recent years, fringe projection has become an established and essential method for dynamic three-dimensional (3-D) shape measurement in different fields such as online inspection and real-time quality control. Numerous high-speed 3-D shape measurement methods have been developed by either employing high-speed hardware, minimizing the number of pattern projection, or both. However, dynamic 3-D shape measurement of arbitrarily-shaped objects with full sensor resolution without the necessity of additional pattern projections is still a big challenge. In this work, we introduce a high-speed 3-D shape measurement technique based on composite phase-shifting fringes and a multi-view system. The geometry constraint is adopted to search the corresponding points independently without additional images. Meanwhile, by analysing the 3-D position and the main wrapped phase of the corresponding point, pairs with an incorrect 3-D position or a considerable phase difference are effectively rejected. All of the qualified corresponding points are then corrected, and the unique one as well as the related period order is selected through the embedded triangular wave. Finally, considering that some points can only be captured by one of the cameras due to the occlusions, these points may have different fringe orders in the two views, so a left-right consistency check is employed to eliminate those erroneous period orders in this case. Several experiments on both static and dynamic scenes are performed, verifying that our method can achieve a speed of 120 frames per second (fps) with 25-period fringe patterns for fast, dense, and accurate 3-D measurement.

Lensless phase microscopy and diffraction tomography with multi-angle and multi-wavelength illuminations using a LED matrix.

We demonstrate lensless quantitative phase microscopy and diffraction tomography based on a compact on-chip platform, using only a CMOS image sensor and a programmable color LED matrix. Based on the multi-wavelength phase retrieval and multi-angle illumination diffraction tomography, this platform offers high quality, depth resolved images with a lateral resolution of 3.72μm and an axial resolution of 5μm, across a wide field-of-view of 24mm2. We experimentally demonstrate the success of our method by imaging cheek cells, micro-beads, and fertilized eggs of Parascaris equorum. Such high-throughput and miniaturized imaging device can provide a cost-effective tool for telemedicine applications and point-of-care diagnostics in resource-limited environments.

High-precision real-time 3D shape measurement using a bi-frequency scheme and multi-view system

High-speed and high-precision 3D shape measurement plays a central role in diverse applications such as automatic online inspection, robotics control, and human-computer interaction. Conventional multi-frame phase-shifting-based fringe projection profilometry techniques face inherent trade-offs between the speed and measurement precision, which are fundamentally limited by the fringe density and extra pattern projections used for de-ambiguity of fringe orders. Increasing the frequency of the projection fringes can obviously improve the measurement precision; however, it creates difficulties in the subsequent phase unwrapping. For this reason, to date, the frequency of the fringes in typical real-time 3D shape measurement techniques is generally less than 30 to guarantee a reasonable reliability of phase unwrapping. To overcome this limitation, a bi-frequency phase-shifting technique based on a multi-view fringe projection system is proposed, which significantly enhances the measurement precision without compromising the measurement speed. Based on the geometric constraints in a multi-view system, the unwrapped phase of the low-frequency (10-period) fringes can be obtained directly, which serves as a reference to unwrap the high-frequency phase map with a total number of periods of up to 160. Besides, the proposed scheme with 10-period and 160-period fringes is suitable for slightly defocusing projection, allowing a higher projection rate and measurement speed. Experiments on both static and dynamic scenes are performed, verifying that our method can achieve high-speed and high-precision 3D measurement at 300 frames per second with a precision of about 50xa0μm.

Motion-oriented high speed 3-D measurements by binocular fringe projection using binary aperiodic patterns

Fringe projection is an extensively used technique for high speed three-dimensional (3-D) measurements of dynamic objects. To precisely retrieve a moving object at pixel level, researchers prefer to project a sequence of fringe images onto its surface. However, the motion often leads to artifacts in reconstructions due to the sequential recording of the set of patterns. In order to reduce the adverse impact of the movement, we present a novel high speed 3-D scanning technique combining the fringe projection and stereo. Firstly, promising measuring speed is achieved by modifying the traditional aperiodic sinusoidal patterns so that the fringe images can be cast at kilohertz with the widely used defocusing strategy. Next, a temporal intensity tracing algorithm is developed to further alleviate the influence of motion by accurately tracing the ideal intensity for stereo matching. Then, a combined cost measure is suggested to robustly estimate the cost for each pixel and lastly a three-step framework of refinement follows not only to eliminate outliers caused by the motion but also to obtain sub-pixel disparity results for 3-D reconstructions. In comparison with the traditional method where the effect of motion is not considered, experimental results show that the reconstruction accuracy for dynamic objects can be improved by an order of magnitude with the proposed method.

Coded multi-angular illumination for Fourier ptychography based on Hadamard codes

Fourier ptychographic microscopy (FPM) is a newly developed super-resolution technique, which employs angularly varying illumination and a phase retrieval algorithm to surpass the diffraction limit of the objective lens. Specifically, FP captures a set of low-resolution (LR) images, under angularly varying illuminations, and stitches them together in the Fourier domain. However, because the requisite large number of incident illumination angles, the long capturing process becomes an obvious limiting factor. Furthermore, in order to acquire high-dynamic range images, the time can be increased several times over. In this work, utilizing the Hadamard code principle, we propose a highly efficient method, which applies coded multi-angular illumination for FPM, to shorten the exposure time of each raw image. High acquisition efficiency is achieved by employing an optimal multi-angular illumination scheme by using two set of Hadamard coded multiplexing patterns. Both simulation and experimental results indicate that the proposed multi-angular illumination process could shorten the acquisition time of conventional FPM.

Robust and efficient multi-frequency temporal phase unwrapping: optimal fringe frequency and pattern sequence selection

Temporal phase unwrapping (TPU) is an essential algorithm in fringe projection profilometry (FPP), especially when measuring complex objects with discontinuities and isolated surfaces. Among others, the multi-frequency TPU has been proven to be the most reliable algorithm in the presence of noise. For a practical FPP system, in order to achieve an accurate, efficient, and reliable measurement, one needs to make wise choices about three key experimental parameters: the highest fringe frequency, the phase-shifting steps, and the fringe pattern sequence. However, there was very little research on how to optimize these parameters quantitatively, especially considering all three aspects from a theoretical and analytical perspective simultaneously. In this work, we propose a new scheme to determine simultaneously the optimal fringe frequency, phase-shifting steps and pattern sequence under multi-frequency TPU, robustly achieving high accuracy measurement by a minimum number of fringe frames. Firstly, noise models regarding phase-shifting algorithms as well as 3-D coordinates are established under a projector defocusing condition, which leads to the optimal highest fringe frequency for a FPP system. Then, a new concept termed frequency-to-frame ratio (FFR) that evaluates the magnitude of the contribution of each frame for TPU is defined, on which an optimal phase-shifting combination scheme is proposed. Finally, a judgment criterion is established, which can be used to judge whether the ratio between adjacent fringe frequencies is conducive to stably and efficiently unwrapping the phase. The proposed method provides a simple and effective theoretical framework to improve the accuracy, efficiency, and robustness of a practical FPP system in actual measurement conditions. The correctness of the derived models as well as the validity of the proposed schemes have been verified through extensive simulations and experiments. Based on a normal monocular 3-D FPP hardware system, our method enables high-precision unambiguous 3-D shape measurement with the highest fringe frequency up to 180 by using only 7 fringe patterns achieving a depth precision ∼ 38μm across a field of view of 400 × 300 × 400 mm.

Motion-compensated three-step phase-shifting profilometry

Phase-shifting profilometry (PSP) is a widely used approach to high-accuracy three-dimensional shape measurements. However, when it comes to moving objects, phase errors induced by the movement often result in severe artifacts even though a high-speed camera is in use. From our observations, there are three kinds of motion artifacts: motion ripples, motion-induced phase unwrapping errors, and motion outliers. We present a novel motion-compensated PSP to remove the artifacts for dynamic measurements of rigid objects. The phase error of motion ripples is analyzed for the phaseshifting algorithm and is compensated using the statistical nature of the fringes. The phase unwrapping errors are corrected exploiting adjacent reliable pixels, and the outliers are removed by comparing the original phase map with a smoothed phase map. Compared with the three-step PSP, our method can improve the accuracy significantly for objects in motion

Robust stereo phase unwrapping based on a quad-camera system

In this work, we demonstrate that by using a quad-camera multi-view fringe projection system and carefully arranging the relative spatial positions between the cameras and the projector, it becomes possible to completely eliminate the phase ambiguities in conventional three-step PSP patterns with high-fringe-density without projecting any additional patterns or embedding any auxiliary signals. Benefit from the position-optimized quadcamera system, stereo phase unwrapping can be efficiently and reliably performed by flexible phase consistency checks. Besides, redundant information of multiple phase consistency checks is fully used through a weighted phase difference scheme to further enhance the reliability of phase unwrapping. This paper explains the 3D measurement principle and the basic design of quad-camera system, and finally demonstrates that the resultant dynamic 3D sensing system can realize real-time 3D reconstruction with a depth precision of 50 μm.

Calibration of telecentric cameras with distortion center estimation

A distortion-free telecentric camera dose not have an optical center because of the orthogonal projection. However, the position of optical center should be known when the lens distortion is considered. Since the full-scale parameters are derived through an iterative algorithm, critical initial values of the optical center should be provided to avoid being trapped in local minima. In this paper, we proposed a two-step algorithm to estimate the optical center as the trustworthy initial value for the subsequent iteration process. The first step is directly calculating the pixel coordinates of the lateral distortion center using the extracted control points. The second step is optimizing both lateral and tangential coefficients considering the properties of the affine transformation in the imaging process. The effectiveness of our proposed method is proven by the measurement results using a new developed microscopic telecentric stereovision system.

Three-dimensional measurement based on a Greenough-type stereomicroscope using phase-shifting projection

We propose an absolute 3D micro surface profile measurement technique based on a Greenough-type stereomicroscope. The camera and the projector are fixed on the stereomicroscope, facilitating a flexible 3D measurement of objects with different heights. Experiments of both calibration and measurements are conducted, and the results show that our proposed method works well for measuring different types of geometry like spheres, ramps and planes etc. The reconstruction accuracy can achieve 4.8 μm with a measurement depth about 3 mm.