Mohammad R. Jahanshahi

A Survey and Evaluation of Promising Approaches for Automatic Image-based Defect Detection of Bridge Structures

Automatic health monitoring and maintenance of civil infrastructure systems is a challenging area of research. Nondestructive evaluation techniques, such as digital image processing, are innovative approaches for structural health monitoring. Current structure inspection standards require an inspector to travel to the structure site and visually assess the structure conditions. A less time consuming and inexpensive alternative to current monitoring methods is to use a robotic system that could inspect structures more frequently. Among several possible techniques is the use of optical instrumentation (e.g. digital cameras) that relies on image processing. The feasibility of using image processing techniques to detect deterioration in structures has been acknowledged by leading experts in the field. A survey and evaluation of relevant studies that appear promising and practical for this purpose is presented in this study. Several image processing techniques, including enhancement, noise removal, registration, edge detection, line detection, morphological functions, colour analysis, texture detection, wavelet transform, segmentation, clustering and pattern recognition, are key pieces that could be merged to solve this problem. Missing or deformed structural members, cracks and corrosion are main deterioration measures that are found in structures, and they are the main examples of structural deterioration considered here. This paper provides a survey and an evaluation of some of the promising vision-based approaches for automatic detection of missing (deformed) structural members, cracks and corrosion in civil infrastructure systems. Several examples (based on laboratory studies by the authors) are presented in the paper to illustrate the utility, as well as the limitations, of the leading approaches.

Multi-image stitching and scene reconstruction for evaluating defect evolution in structures:

It is well-recognized that civil infrastructure monitoring approaches that rely on visual approaches will continue to be an important methodology for condition assessment of such systems. Current inspection standards for structures such as bridges require an inspector to travel to a target structure site and visually assess the structures condition. A less time-consuming and inexpensive alternative to current visual monitoring methods is to use a system that could inspect structures remotely and also more frequently. This article presents and evaluates the underlying technical elements for the development of an integrated inspection software tool that is based on the use of inexpensive digital cameras. For this purpose, digital cameras are appropriately mounted on a structure (e.g., a bridge) and can zoom or rotate in three directions (similar to traffic cameras). They are remotely controlled by an inspector, which allows the visual assessment of the structures condition by looking at images captured by the cameras. By not having to travel to the structures site, other issues related to safety considerations and traffic detouring are consequently bypassed. The proposed system gives an inspector the ability to compare the current (visual) situation of a structure with its former condition. If an inspector notices a defect in the current view, he/she can request a reconstruction of the same view using images that were previously captured and automatically stored in a database. Furthermore, by generating databases that consist of periodically captured images of a structure, the proposed system allows an inspector to evaluate the evolution of changes by simultaneously comparing the structures condition at different time periods. The essential components of the proposed virtual image reconstruction system are: keypoint detection, keypoint matching, image selection, outlier exclusion, bundle adjustment, composition, and cropping. Several illustrative examples are presented in this article to demonstrate the capabilities, as well as the limitations, of the proposed vision-based inspection procedure.

Unsupervised Approach for Autonomous Pavement-Defect Detection and Quantification Using an Inexpensive Depth Sensor

AbstractCurrent pavement condition–assessment procedures are extensively time consuming and laborious; in addition, these approaches pose safety threats to the personnel involved in the process. In this study, a RGB-D sensor is used to detect and quantify defects in pavements. This sensor system consists of a RGB color image, and an infrared projector and a camera that act as a depth sensor. An approach, which does not need any training, is proposed to interpret the data sensed by this inexpensive sensor. This system has the potential to be used for autonomous cost-effective assessment of road-surface conditions. Various road conditions including patching, cracks, and potholes are autonomously detected and, most importantly, quantified, using the proposed approach. Several field experiments have been carried out to evaluate the capabilities, as well as the limitations of the proposed system. The global positioning system information is incorporated with the proposed system to localize the detected defects...

A new methodology for non-contact accurate crack width measurement through photogrammetry for automated structural safety evaluation

In mechanical, aerospace and civil structures, cracks are important defects that can cause catastrophes if neglected. Visual inspection is currently the predominant method for crack assessment. This approach is tedious, labor-intensive, subjective and highly qualitative. An inexpensive alternative to current monitoring methods is to use a robotic system that could perform autonomous crack detection and quantification. To reach this goal, several image-based crack detection approaches have been developed; however, the crack thickness quantification, which is an essential element for a reliable structural condition assessment, has not been sufficiently investigated. In this paper, a new contact-less crack quantification methodology, based on computer vision and image processing concepts, is introduced and evaluated against a crack quantification approach which was previously developed by the authors. The proposed approach in this study utilizes depth perception to quantify crack thickness and, as opposed to most previous studies, needs no scale attachment to the region under inspection, which makes this approach ideal for incorporation with autonomous or semi-autonomous mobile inspection systems. Validation tests are performed to evaluate the performance of the proposed approach, and the results show that the new proposed approach outperforms the previously developed one.

A texture-Based Video Processing Methodology Using Bayesian Data Fusion for Autonomous Crack Detection on Metallic Surfaces

Regular inspection of the components of nuclear power plants is important to improve their resilience. However, current inspection practices are time consuming, tedious, and subjective: they involve an operator manually locating cracks in metallic surfaces in the plant by watching videos. At the same time, prevalent automatic crack detection algorithms may not detect cracks in metallic surfaces because these are typically very small and have low contrast. Moreover, the existences of scratches, welds, and grind marks lead to a large number of false positives when state-of-the-art vision-based crack detection algorithms are used. In this study, a novel crack detection approach is proposed based on local binary patterns LBP, support vector machine SVM, and Bayesian decision theory. The proposed method aggregates the information obtained from different video frames to enhance the robustness and reliability of detection. The performance of the proposed approach is assessed by using several inspection videos. The results indicate that it is accurate and robust in cases where state-of-the-art crack detection approaches fail. The experiments show that Bayesian data fusion improves the hit rate by 20% and the hit rate achieves 85% with only one false positive per frame.

NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Naïve Bayes Data Fusion

Regular inspection of nuclear power plant components is important to guarantee safe operations. However, current practice is time consuming, tedious, and subjective, which involves human technicians reviewing the inspection videos and identifying cracks on reactors. A few vision-based crack detection approaches have been developed for metallic surfaces, and they typically perform poorly when used for analyzing nuclear inspection videos. Detecting these cracks is a challenging task since they are tiny, and noisy patterns exist on the components’ surfaces. This study proposes a deep learning framework, based on a convolutional neural network (CNN) and a Naïve Bayes data fusion scheme, called NB-CNN, to analyze individual video frames for crack detection while a novel data fusion scheme is proposed to aggregate the information extracted from each video frame to enhance the overall performance and robustness of the system. To this end, a CNN is proposed to detect crack patches in each video frame, while the proposed data fusion scheme maintains the spatiotemporal coherence of cracks in videos, and the Naïve Bayes decision making discards false positives effectively. The proposed framework achieves a 98.3% hit rate against 0.1 false positives per frame that is significantly higher than state-of-the-art approaches as presented in this paper.

Parametric Performance Evaluation of Wavelet-Based Corrosion Detection Algorithms for Condition Assessment of Civil Infrastructure Systems

Corrosion is a crucial defect in structural systems that can lead to catastrophic effects if neglected. Current structure inspection standards require an inspector to visually assess the conditions of a target structure. A less time-consuming and inexpensive alternative to current monitoring methods is to use a robotic system, which can inspect structures more frequently and perform autonomous damage detection. The feasibility of using image processing techniques to detect corrosion in structures has been acknowledged by leading experts in the field; however, there has not been a systematic study to evaluate the effects of different parameters on the performance of vision-based corrosion detection systems. This study evaluates several parameters that can affect the performance of color wavelet-based texture analysis algorithms for detecting corrosion. Furthermore, an approach is proposed to utilize the depth perception for corrosion detection. The proposed approach improves the reliability of the corrosion detection algorithm. The integration of depth perception with pattern classification algorithms, which has never been reported in published studies, is part of the contribution of the current study. Several quantitative evaluations are presented to scrutinize the performance of the investigated approaches.

A Novel Crack Detection Approach for Condition Assessment of Structures

Automated health monitoring and maintenance of civil infrastructure systems is an active yet challenging area of research. Current inspection standards require an inspector to travel to a target structure site and visually assess the structures condition. If a region is inaccessible, binoculars must be used to detect and characterize defects. This approach is labor-intensive, yet highly qualitative. A less time-consuming and inexpensive alternative to current monitoring methods is to use a robotic system that could inspect structures more frequently, and perform autonomous damage detection. Among several possible techniques, the use of optical instrumentation (e.g., digital cameras), image processing and computer vision are promising approaches as nondestructive testing methods. The feasibility of using image processing techniques to detect deterioration in structures has been acknowledged by leading researches in the field. This study presents and evaluates the technical elements for the development of a novel crack detection methodology that is based on the use of inexpensive digital cameras. Guidelines are presented for optimizing the acquisition and processing of images, thereby enhancing the quality and reliability of the damage detection approach and allowing the capture of even the slightest, which are routinely encountered in realistic field applications where the camera-object distance and image contrast are incontrollable.

Inexpensive Multimodal Sensor Fusion System for Autonomous Data Acquisition of Road Surface Conditions

This paper presents the development, evaluation, calibration, and field application of a novel, relatively inexpensive, vision-based sensor system employing commercially available off-the-shelf devices, for enabling the autonomous data acquisition of road surface conditions. Detailed evaluations and enhancements of a variety of technical approaches and algorithms for overcoming vision-based measurement distortions induced by the motion of the monitoring platform were conducted. It is shown that the proposed multi-sensor system, by capitalizing on powerful data-fusion approaches of the type developed in this paper, can provide a robust cost-effective road surface monitoring system with sufficient accuracy to satisfy typical maintenance needs, in regard to the detection, localization, and quantification of potholes and similar qualitative deterioration features where the measurements are acquired via a vehicle moving at normal speeds on typical city streets. The proposed system is ideal to be used for crowdsourcing where several vehicles would be equipped with this cost-effective system for more frequent data collection of road surfaces. Suggestions for future research needs to enhance the capabilities of the proposed system are included.

A novel system for road surface monitoring using an inexpensive infrared laser sensor

In this study, an inexpensive depth sensor is used to identify defects in pavements. This depth sensor consists of an infrared projector and camera. An innovative approach is proposed to interpret the data acquired by this sensor. The proposed system in this study is a breakthrough achievement for autonomous cost-effective condition assessment of roads and transportation systems. Various road conditions including patching, cracks, and potholes can be robustly and autonomously assessed using the proposed approach. Several field experiments have been carried out to evaluate the capabilities of this system. The field tests clearly demonstrate the superior features of the developed system in this study compared to conventional approaches for pavement evaluation.