Hongshi Yan

Precise Subpixel Disparity Measurement From Very Narrow Baseline Stereo

To obtain depth-from-stereo imagery, it is traditionally required that the baseline separation between images (or the base-to-height ratio) be very large in order to ensure the largest image disparity range for effective measurement. Typically, a B/H ratio in the range of 0.6-1 is preferred. As a consequence, most existing stereo-matching algorithms are designed to measure disparities reliably with only integer-pixel precision. However, wide baselines may increase the possibility of occlusion occurring between highly contrasting relief, imposing a serious problem to digital elevation model (DEM) generation in urban and highly dissected mountainous areas. A narrow-baseline stereo configuration can alleviate the problem significantly but requires very precise measurements of disparity at subpixel levels. In this paper, we demonstrate a stereo-matching algorithm, based upon the robust phase correlation method, that is capable of directly measuring disparities up to 1/50th pixel accuracy and precision. The algorithm enables complete and dense surface shape information to be retrieved from images with unconventionally low B/H ratios (e.g., less than 0.01), potentially allowing DEM generation from images that would otherwise not be deemed suitable for the purpose.

Sub-Pixel Stereo-Matching for DEM Generation from Narrow Baseline Stereo Imagery

We demonstrate an automated stereo-matching algorithm capable of extracting disparity/depth information from stereo image pairs with an unconventionally narrow baseline separation i.e. low base-to-height ratio (B/H), thus potentially allowing digital elevation models (DEMs) to be derived from images that previously might not have been considered suitable for stereo applications. For very small B/H ratios the disparity magnitudes may be reduced to sub-pixel levels that conventional stereo-matching algorithms fail to measure. By utilising a new sub-pixel image matching algorithm, based upon the phase correlation (PC) method, we are able to measure the very subtle, sub-pixel, disparities that result from image pairs with B/H ratios as small as 0.06. Initial tests with this algorithm on SPOT 5 satellite image data have demonstrated that this routine is capable of generating very dense and detailed disparity/depth maps of a scene for DEM generation without prior knowledge of the satellite/sensor parameters, whilst also being exceptionally sensitive to small scale textural features, robust to noise, and efficient to implement.

The Illumination Robustness of Phase Correlation for Image Alignment

Phase correlation (PC) is a well-established image matching algorithm. It is robust to the variation of image contrast and brightness, but whether it is invariant to local illumination change, particularly sun angle change, is yet to be investigated. This paper presents our study to prove and demonstrate the robustness of illumination invariance of the PC algorithm via mathematical analysis and image matching experiments. First, a 3-D space named slope-aspect-intensity (SAI) is proposed to characterize the 3-D relationship between image intensity and terrain slope/aspect angles for a given illumination geometry. Based on the SAI space, the mathematical relationship between PC cross-power spectra and illumination direction (e.g., solar azimuth and zenith angles) is analyzed. The impact of illumination angle variation between two images for matching is then rigorously investigated via experiments using simulated terrain shading images from a digital elevation model and real optical images taken under different sun illumination conditions. Our study confirms that PC is robustly invariant to illumination and therefore can achieve reliable matching between images taken under different solar illumination conditions for various remote sensing applications.

High quality DEM generation from PCIAS

This paper presents an efficient Phase Correlation based Image Analysis System (PCIAS) for high quality DEM generation. A multi-resolution phase correlation based disparity estimation and refinement algorithm has been implemented in PCIAS. It can easily cope with the precise disparity estimation from sub-pixel to very large disparity range with varying baseline/distance ratio in vertical or slightly oblique view stereo imaging. The PCIAS is now a fully operational, professional C++ software package equipped with a robust phase correlation engine, which is among the most advanced technology for sub-pixel image feature shift analysis, and is able to achieve <;1/50th pixel accuracy in dense disparity map estimation. Our experiment indicates PCIAS can generate high quality DEM from very narrow baseline satellite image pairs with view angle difference as small as just 1 degree.

PCIAS Subpixel Technology

PCIAS represents Phase Correlation based Image Analysis System. It is a professional C++ software package to deliver advanced subpixel technology that has been developed during the last several years supported by the SEAS DTC (System Engineering for Autonomous Systems Defence Technology Centre). This paper provides and overview of the PCIAS. After a brief introduction to the principles and algorithms, the paper illustrates, with examples, the major applications based on high precision sub-pixel disparity estimation.

Three-dimensional surface displacement map of the 2008 Wenchuan earthquake derived from Phase Correlation (PC) sub-pixel offset method and Adaptive Local Kriging (alk) DInSAR data

Differential Interferometric Synthetic Aperture Radar (DInSAR) is an effective technique to measures the surface displacement caused by strong earthquakes, such as the 2008 Wenchuan earthquake in China. However, in the area subject to the most significant deformation along the Longmenshan fault zone, the coherence between pre- and after- earthquake SAR images is completely lost because of the earthquake induced violent and chaotic destruction on the land surface and consequently, no surface displacement data can be measured. The missing data were recovered using Adaptive Local Kriging (ALK) method to produce a complete Line Of Sight (LOS) displacement map. As a further step to characterize the 3D (three-dimensional) co-seismic deformation, horizontal displacement maps were generated using the Phase Correlation based Image Analysis System (PCIAS), and in combination with the ALK DInSAR data, the vertical displacement map can then be decomposed. Thus, a 3D displacement dataset is accomplished. This dataset shows the thrust and right-lateral strike slip motions along the Longmenshan fault system with two major uplift in Yingxiu and Beichuan areas respectively.

Robust sub-pixel disparity estimation and its refinement around depth discontinuity and featureless areas

This paper presents an enhanced robust phase correlation (ERPC) method for sub-pixel disparity estimation together with a package of refinement techniques for DEM generation. The multi-scale ERPC algorithm can cope with very large disparity measurement on the one hand and improve sub-pixel accuracy by entirely avoiding the ill-posed problem of 2D phase unwrapping on the other. Recognizing the inherent weakness of phase correlation as an area matching based method, we then present a refinement scheme with compound phase correlation (CPC) [1] and Median Shift Propagation (MSP) [2] filter techniques to deal with the unreliable disparity estimates around the depth discontinuities, featureless areas and other low correlation areas. Through this refinement scheme, we are able to greatly improve the accuracy of phase correlation based disparity estimation for DEM generation from stereo image pairs with versatile baseline settings (from narrow to wide baseline).

3D super resolution scene depth reconstruction based on SkySat video image sequences

Traditional DEM (Digital Elevation Model) generation from satellite imagery is based on stereo pair or triple images using wide baseline. These days, a number of low-cost microsatellites, such as SkySat, have been launched. The high resolution video image sequences they provided result in large number of image frames and more flexible selection of baseline, which allows us to design a 3D super resolution scene reconstruction approach based on multiple narrow baseline stereo pairs. The multiple disparity maps, generated using Phase Correlation (PC) based sub-pixel stereo matching algorithm, are co-registered pixel by pixel and up-sampled and then stacked to produce a super resolution scene depth map. The scene depth image constructed in this way has three advantages: i) “pixel locking” error typical for sub-pixel image matching is minimized; ii) super resolution is achieved; and iii) occlusion problem is minimized by multiple narrow baseline stereo pairs. A 3D scene super resolution reconstruction example is demonstrated using a SkySat video image of Usak, Western Turkey.

Automatic Mars rover detection from multi-temporal HiRISE imagery

In this paper, we present a Phase Correlation (PC) based change detection algorithm which is able to automatically detect very small target such as the Mars rover in multi-temporal HiRISE images under illumination variation and camera motion. The robustness of PC to illumination variation was proved by our previous work while the geometric distortion caused by camera motion can be eliminated by PC-based pixel-wise image co-registration. Experiments using three HiRISE images taken under different illumination and view angle demonstrate that our algorithm can robustly detect the Mars rover as small as 6×9 pixels in conditions of varying illumination and camera 3D status.