Habib Fathi

Automated sparse 3D point cloud generation of infrastructure using its distinctive visual features

The commercial far-range (>10m) spatial data collection methods for acquiring infrastructures geometric data are not completely automated because of the necessary manual pre- and/or post-processing work. The required amount of human intervention and, in some cases, the high equipment costs associated with these methods impede their adoption by the majority of infrastructure mapping activities. This paper presents an automated stereo vision-based method, as an alternative and inexpensive solution, to producing a sparse Euclidean 3D point cloud of an infrastructure scene utilizing two video streams captured by a set of two calibrated cameras. In this process SURF features are automatically detected and matched between each pair of stereo video frames. 3D coordinates of the matched feature points are then calculated via triangulation. The detected SURF features in two successive video frames are automatically matched and the RANSAC algorithm is used to discard mismatches. The quaternion motion estimation method is then used along with bundle adjustment optimization to register successive point clouds. The method was tested on a database of infrastructure stereo video streams. The validity and statistical significance of the results were evaluated by comparing the spatial distance of randomly selected feature points with their corresponding tape measurements.

Automated as-built 3D reconstruction of civil infrastructure using computer vision

Image-based 3D reconstruction of civil infrastructure is an emerging topic that is gaining significant interest both in the scientific and commercial sectors of the construction industry. Reliable computer vision-based algorithms have become available over the last decade and they can now be applied to solve real-life problems in uncontrolled environments. While a large number of such algorithms have been developed by the computer vision and photogrammetry communities, relatively little work has been done to study their performance in the context of infrastructure. This paper aims to analyze the state-of-the-art in image-based 3D reconstruction and categorize existing algorithms according to different metrics that are important for the given purpose. An ideal solution is portrayed to show what the ultimate goal is. This will be followed by identifying gaps in knowledge and highlighting future research topics that could contribute to the widespread adoption of this technology in the construction industry. Finally, a list of practical constraints that make the 3D reconstruction of infrastructure a challenging task is presented.

Innovative Stereo Vision-Based Approach to Generate Dense Depth Map of Transportation Infrastructure

Three-dimensional (3-D) spatial data of a transportation infrastructure contain useful information for civil engineering applications, including as-built documentation, on-site safety enhancements, and progress monitoring. Several techniques have been developed for acquiring 3-D point coordinates of infrastructure, such as laser scanning. Although the method yields accurate results, the high device costs and human effort required render the process infeasible for generic applications in the construction industry. A quick and reliable approach, which is based on the principles of stereo vision, is proposed for generating a depth map of an infrastructure. Initially, two images are captured by two similar stereo cameras at the scene of the infrastructure. A Harris feature detector is used to extract feature points from the first view, and an innovative adaptive window-matching technique is used to compute feature point correspondences in the second view. A robust algorithm computes the nonfeature point correspondences. Thus, the correspondences of all the points in the scene are obtained. After all correspondences have been obtained, the geometric principles of stereo vision are used to generate a dense depth map of the scene. The proposed algorithm has been tested on several data sets, and results illustrate its potential for stereo correspondence and depth map generation.

A videogrammetric as-built data collection method for digital fabrication of sheet metal roof panels

A roofing contractor typically needs to acquire as-built dimensions of a roof structure several times over the course of its build because a structure is never built to the exact drawing dimensions. In the construction phase and in order to digitally fabricate sheet metal roof panels, the contractor has to measure end-to-end dimensions of boundaries of every roof plane with a certain level of accuracy (i.e., errors less than ?2cm). This is necessary to be able to cut sheet metal coil such that different pieces perfectly fit together. Obtaining these measurements using the exiting roof surveying methods could be costly in terms of equipment, labor, and/or worker exposure to safety hazards. This paper presents a video-based surveying framework as an alternative method which is simple to use, automated, less expensive, and safe. When using this framework, the contractor collects video streams with a calibrated stereo camera set. The captured data is processed to automatically generate a 3D wire-diagram of the target roof. Measurements from the wire-diagram are saved in a digital file (XML format) which could be loaded into an on-site sheet metal folding and cutting machine. Experimental analyses demonstrate applicability of the proposed framework.

Automated computation of the fundamental matrix for vision based construction site applications

Estimating the fundamental matrix (F), to determine the epipolar geometry between a pair of images or video frames, is a basic step for a wide variety of vision-based functions used in construction operations, such as camera-pair calibration, automatic progress monitoring, and 3D reconstruction. Currently, robust methods (e.g., SIFT+normalized eight-point algorithm+RANSAC) are widely used in the construction community for this purpose. Although they can provide acceptable accuracy, the significant amount of required computational time impedes their adoption in real-time applications, especially video data analysis with many frames per second. Aiming to overcome this limitation, this paper presents and evaluates the accuracy of a solution to find F by combining the use of two speedy and consistent methods: SURF for the selection of a robust set of point correspondences and the normalized eight-point algorithm. This solution is tested extensively on construction site image pairs including changes in viewpoint, scale, illumination, rotation, and moving objects. The results demonstrate that this method can be used for real-time applications (5 image pairs per second with the resolution of 640x480) involving scenes of the built environment.

Machine Vision-Based Infrastructure As-Built Documentation Using Edge Points

The existing machine vision-based 3D reconstruction software programs provide a promising low-cost and in some cases automatic solution for infrastructure as-built documentation. However in several steps of the reconstruction process, they only rely on detecting and matching corner-like features in multiple views of a scene. Therefore, in infrastructure scenes which include uniform materials and poorly textured surfaces, these programs fail with high probabilities due to lack of feature points. Moreover, except few programs that generate dense 3D models through significantly time-consuming algorithms, most of them only provide a sparse reconstruction which does not necessarily include required points such as corners or edges; hence these points have to be manually matched across different views that could make the process considerably laborious. To address these limitations, this paper presents a video-based as-built documentation method that automatically builds detailed 3D maps of a scene by aligning edge points between video frames. Compared to corner-like features, edge points are far more plentiful even in untextured scenes and often carry important semantic associations. The method has been tested for poorly textured infrastructure scenes and the results indicate that a combination of edge and corner-like features would allow dealing with a broader range of scenes.

A Transformational Approach To Explicit Stereo Camera Calibration For Improved Euclidean Accuracy Of Infrastructure 3D reconstruction

The accuracy of the results in stereo image-based 3D reconstruction is very sensitive to the intrinsic and extrinsic camera parameters determined during camera calibration. The existing camera calibration algorithms induce a significant amount of error due to poor estimation accuracies in camera parameters when they are used for long-range scenarios such as mapping civil infrastructure. This leads to unusable results, and may result in the failure of the whole reconstruction process. This paper proposes a novel way to address this problem. Instead of incremental improvements to the accuracy typically induced by new calibration algorithms, the authors hypothesize that a set of multiple calibrations created by videotaping a moving calibration pattern along a specific path can increase overall calibration accuracy. This is achieved by using conventional camera calibration algorithms to perform separate estimations for some predefined distance values. The result, which is a set of camera parameters for different distances, is then uniquely input in the Structure from Motion process to improve the Euclidean accuracy of the reconstruction. The proposed method has been tested on infrastructure scenes and the experimental analyses indicate the improved performance. INTRODUCTION Automatic 3D reconstruction of infrastructure from multiple view imagery is considered as an inexpensive alternative to laser-based systems but cannot replace them without acceptable levels of geometrical accuracy (Strecha et al., 2008). Two main issues need to be studied for this purpose: camera calibration and dense multiview geometry (Strecha et al., 2008). The scope of this paper is to focus on the first issue because it is the first step in the 3D reconstruction pipeline. Camera calibration is the process of determining a set of parameters that describe the mapping between 3D world coordinates and 2D image coordinates. The existing methods are divided into two categories: a) explicit calibration (i.e., conventional approach) and b) selfcalibration. Methods in the first category estimate the calibration parameters by establishing correspondences between reference points on an object with known 3D dimensions and their projection on the image. On the other hand, self-calibration automatically provides necessary parameters through the geometrical constraints in images, but is less accurate than the explicit methods (Furukawa and Ponce, 2009). This paper aims to focus on the explicit approach because accuracy is typically the main concern in the 3D reconstruction of infrastructure.

Automated 3D Structure Inference of Civil Infrastructure Using a Stereo Camera Set

The commercial far-range (>10m) infrastructure spatial data collection methods are not completely automated. They need significant amount of manual post-processing work and in some cases, the equipment costs are significant. This paper presents a method that is the first step of a stereo videogrammetric framework and holds the promise to address these issues. Under this method, video streams are initially collected from a calibrated set of two video cameras. For each pair of simultaneous video frames, visual feature points are detected and their spatial coordinates are then computed. The result, in the form of a sparse 3D point cloud, is the basis for the next steps in the framework (i.e., camera motion estimation and dense 3D reconstruction). A set of data, collected from an ongoing infrastructure project, is used to show the merits of the method. Comparison with existing tools is also shown, to indicate the performance differences of the proposed method in the level of automation and the accuracy of results.

A Scale, Rotation, and Affine Invariant Line Detection and Matching Algorithm for 3D Reconstruction of Infrastructure

Structure from Motion (SfM) is a well-known approach for extracting the 3D geometry of a structure from 2D images. Despite being inexpensive, simple, and automatic, it has not been widely used in the construction industry mainly due to challenges in detecting and matching visual features. In built infrastructure scenes, most areas lack distinctive points due to the prevalence of poorly-textured surfaces which obstructs the successful use of SfM. In contrast, an abundant number of straight lines are visible but cannot be fully exploited because of the low performance of existing line matching algorithms. When applied to infrastructure scenes, such algorithms produce many mismatches because of inaccurate locations of line endpoints, fragmented lines, lack of geometric constraints, and lack of distinctive texture in the local neighborhood. This paper presents a novel method for tackling this problem. The key innovations of the proposed method are the joint use of local and global information in the scene, and the canonical form representation of the support region of each feature. Using the scale-space theory, lines that are stable with respect to scale variations are first extracted. A dynamic pixel support region is then assigned to each line based on the Laplacian function. Each region is described by an affineinvariant vector. Finally, the Euclidean distance between vectors and global geometric constraints in the scene are applied to determine corresponding pairs. Experimental analyses have shown performance improvement compared to the stateof-the-art algorithms.