Laurence G. Hassebrook

Performance measures for correlation filters

Several performance criteria are described to enable a fair comparison among the various correlation filter designs: signal-to-noise ratio, peak sharpness, peak location, light efficiency, discriminability, and distortion invariance. The trade-offs resulting between some of these criteria are illustrated with the help of a new family of filters called fractional power filters (FPFs). The classical matched filter, phase-only filter (POF), and inverse filter are special cases of FPFs. Using examples, we show that the POF appears to provide a good compromise between noise tolerance and peak sharpness.

Dual-frequency pattern scheme for high-speed 3-D shape measurement

A novel dual-frequency pattern is developed which combines a high-frequency sinusoid component with a unit-frequency sinusoid component, where the high-frequency component is used to generate robust phase information, and the unit-frequency component is used to reduce phase unwrapping ambiguities. With our proposed pattern scheme, phase unwrapping can overcome the major shortcomings of conventional spatial phase unwrapping: phase jumping and discontinuities. Compared with conventional temporal phase unwrapped approaches, the proposed pattern scheme can achieve higher quality phase data using a less number of patterns. To process data in real time, we also propose and develop look-up table based fast and accurate algorithms for phase generation and 3-D reconstruction. Those fast algorithms can be applied to our pattern scheme as well as traditional phase measuring profilometry. For a 640 x 480 video stream, we can generate phase data at 1063.8 frames per second and full 3-D coordinate point clouds at 8.3 frames per second. These achievements are 25 and 10 times faster than previously reported studies.

Composite structured light pattern for three-dimensional video

Based on recent discoveries, we introduce a method to project a single structured pattern onto an object and then reconstruct the three-dimensional range from the distortions in the reflected and captured image. Traditional structured light methods require several different patterns to recover the depth, without ambiguity or albedo sensitivity, and are corrupted by object movement during the projection/ capture process. Our method efficiently combines multiple patterns into a single composite pattern projection allowing for real-time implementations. Because structured light techniques require standard image capture and projection technology, unlike time of arrival techniques, they are relatively low cost.

Optimized two-frequency phase-measuring-profilometry light-sensor temporal-noise sensitivity

Temporal frame-to-frame noise in multipattern structured light projection can significantly corrupt depth measurement repeatability. We present a rigorous stochastic analysis of phase-measuring-profilometry temporal noise as a function of the pattern parameters and the reconstruction coefficients. The analysis is used to optimize the two-frequency phase measurement technique. In phase-measuring profilometry, a sequence of phase-shifted sine-wave patterns is projected onto a surface. In two-frequency phase measurement, two sets of pattern sequences are used. The first, low-frequency set establishes a nonambiguous depth estimate, and the second, high-frequency set is unwrapped, based on the low-frequency estimate, to obtain an accurate depth estimate. If the second frequency is too low, then depth error is caused directly by temporal noise in the phase measurement. If the second frequency is too high, temporal noise triggers ambiguous unwrapping, resulting in depth measurement error. We present a solution for finding the second frequency, where intensity noise variance is at its minimum.

Gamma model and its analysis for phase measuring profilometry

Phase measuring profilometry is a method of structured light illumination whose three-dimensional reconstructions are susceptible to error from nonunitary gamma in the associated optical devices. While the effects of this distortion diminish with an increasing number of employed phase-shifted patterns, gamma distortion may be unavoidable in real-time systems where the number of projected patterns is limited by the presence of target motion. A mathematical model is developed for predicting the effects of nonunitary gamma on phase measuring profilometry, while also introducing an accurate gamma calibration method and two strategies for minimizing gammas effect on phase determination. These phase correction strategies include phase corrections with and without gamma calibration. With the reduction in noise, for three-step phase measuring profilometry, analysis of the root mean squared error of the corrected phase will show a 60x reduction in phase error when the proposed gamma calibration is performed versus 33x reduction without calibration.

A multistage, optimal active contour model

Energy-minimizing active contour models or snakes can be used in many applications such as edge detection, motion tracking, image matching, computer vision, and three-dimensional (3-D) reconstruction. We present a novel snake that is superior both in accuracy and convergence speed over previous snake algorithms. High performance is achieved by using spline representation and dividing the energy-minimization process into multiple stages. The first stage is designed to optimize the convergence speed in order to allow the snake to quickly approach the minimum-energy state. The second stage is devoted to snake refinement and to local minimization of energy, thereby driving the snake to a quasiminimum-energy state. The third stage uses the Bellman (1957) optimality principle to fine-tune the snake to the global minimum-energy state. This three-stage scheme is optimized for both accuracy and speed.

Period Coded Phase Shifting Strategy for Real–time 3-D Structured Light Illumination

Phase shifting structured light illumination for range sensing involves projecting a set of grating patterns where accuracy is determined, in part, by the number of stripes. However, high pattern frequencies introduce ambiguities during phase unwrapping. This paper proposes a process for embedding a period cue into the projected pattern set without reducing the signal-to-noise ratio. As a result, each period of the high frequency signal can be identified. The proposed method can unwrap high frequency phase and achieve high measurement precision without increasing the pattern number. Therefore, the proposed method can significantly benefit real-time applications. The method is verified by theoretical and experimental analysis using prototype system built to achieve 120 fps at 640 × 480 resolution.

Data Acquisition and Processing of 3-D Fingerprints

To solve the problems associated with conventional 2-D fingerprint scanners such as skin deformation and print smearing, in this paper we introduce a noncontact fingerprint scanner employing structured light illumination to generate high-resolution albedo images as well as 3-D ridge scans. The question to be answered in this research is whether or not ridge depth information improves the quality and matching capability of acquired fingerprints? For evaluation of this question, we use the National Institute of Standards and Technology fingerprint image quality metrics. These metrics require the 3-D prints to be flattened. We present a complete and detailed flattening algorithm based upon unfolding an elastic tube fit to the peaks and valleys of ridges identified within the scan. Further improvement of the flattened print is achieved through the incorporation of ridge information extracted from the albedo image with the depth and albedo ridge information fused together according to local scan quality. Our study compares image quality between the flattened 3-D prints and ink rolled prints. Most significantly, the matching performance of 3-D flattened to 3-D flattened prints is evaluated based on ridge depth only, albedo only, and depth with albedo fusion.

Robust Active Stereo Vision Using Kullback-Leibler Divergence

Active stereo vision is a method of 3D surface scanning involving the projecting and capturing of a series of light patterns where depth is derived from correspondences between the observed and projected patterns. In contrast, passive stereo vision reveals depth through correspondences between textured images from two or more cameras. By employing a projector, active stereo vision systems find correspondences between two or more cameras, without ambiguity, independent of object texture. In this paper, we present a hybrid 3D reconstruction framework that supplements projected pattern correspondence matching with texture information. The proposed scheme consists of using projected pattern data to derive initial correspondences across cameras and then using texture data to eliminate ambiguities. Pattern modulation data are then used to estimate error models from which Kullback-Leibler divergence refinement is applied to reduce misregistration errors. Using only a small number of patterns, the presented approach reduces measurement errors versus traditional structured light and phase matching methodologies while being insensitive to gamma distortion, projector flickering, and secondary reflections. Experimental results demonstrate these advantages in terms of enhanced 3D reconstruction performance in the presence of noise, deterministic distortions, and conditions of texture and depth contrast.

Very high resolution 3D surface scanning using multi-frequency phase measuring profilometry

Structured light projection is one of the most accurate non-contact methods for scanning surface topologies. The field of view of such a scan may range from millimeters to several meters. One of the most precise and robust methods of structured light is Phase Measuring Profilometry. This method utilizes a sinusoidal pattern that is laterally shifted across a surface. An image is captured at uniform intervals and the “phase” is recovered for each pixel position by correlating across the shifted patterns. In general, the more pattern shifts and the higher the spatial frequency, the more accurate the depth measurement becomes, at each pixel location. However, with a high frequency, ambiguity errors can occur, so a dual frequency approach is commonly used where a low frequency pattern is used for non-ambiguous depth, followed by high frequency pattern. The low frequency result is used to unwrap the high frequency to yield a non-ambiguous and precise phase. If the high frequency is too high, then ambiguity errors will occur. The solution is a multi-frequency method. We present experimental results for several variations of the multi-frequency approach yielding accuracies of 0.127mm standard deviation in depth with 0.92mm pixel spacing. With consumer camera mega pixel technology this equates to a 0.127mm deviation over a field of view of 2 to 3 meters. To achieve this level of accuracy also requires calibration for radial and perspective distortions. Applications for this technology include non-contact surface measurement and robotic and computer vision.