Crack Detection and Segmentation Using Deep Learning with 3D Reality Mesh Model for Quantitative Assessment and Integrated Visualization

Within the framework of intelligent bridge detection, a number of crack detection methods based on image processing techniques have been implemented. In this study, a combined novel approach with deep learning of a single shot multibox detector (SSD) and the eight neighborhood algorithm is proposed and applied to bridge crack image identification to provide an automatic method for crack detection. First, a large number of concrete crack images collected from the site were segmented and preprocessed for the establishment of a crack image dataset. Deep learning of the SSD algorithm was introduced on the training set to establish the detection model, where the model parameters were adjusted by the validation set. Sliding window technology was integrated to identify the cracks in the test set. The effects of the sliding window size and dataset size on the crack detection results were discussed. Moreover, the eight neighborhood algorithm was adopted for further crack detection correction. The results show that the configuration achieves good crack detection by the deep learning of the SSD algorithm with high precision and recall. The introduction of the eight neighborhood correction algorithm further improves the detection results by eliminating some misjudged results. Finally, the developed algorithm was placed into a portable device, with which cracks were effectively identified. The introduced method shows significantly better performance in crack detection, and the system installed on the portable device provides a way to broaden its application in the automatic crack detection of concrete bridges.

Damage due to cracking can be detected through either manual visual methods or machine vision techniques for early prevention and maintenance. In recent years, image-based deep learning methods have emerged as potent tools for automatic crack detection. In this study, five deep learning object detection algorithms—faster R-CNN, single-shot detector (SSD), You Only Look Once (YOLO) v3 and v8, and RetinaNet—were systematically compared, and the results were analyzed. Object detection involves the generation of bounding boxes of various sizes for objects of interest. Because cracks are thin and small and thus difficult to capture in a unique bounding box, redundant measurements are common, but they compromise the accuracy and consistency of the model. Therefore, an improved confusion matrix based on linear features was employed in this study to evaluate the crack detection performance of the five object detection algorithms. In evaluation experiments, the overall accuracy levels of SSD were 90.6% on visible atmospherically resistant index (VARI) images, indicating effective concrete crack detection performance. Notably, SSD excels in cases involving small cracks and data imbalance, thus demonstrating a high level of model stability. This comparative analysis of the performances of different deep learning algorithms in crack detection contributes to the formulation of methods for automatic damage detection.

Infrastructure, such as buildings, bridges, pavement, etc., needs to be examined periodically to maintain its reliability and structural health. Visual signs of cracks and depressions indicate stress and wear and tear over time, leading to failure/collapse if these cracks are located at critical locations, such as in load-bearing joints. Manual inspection is carried out by experienced inspectors who require long inspection times and rely on their empirical and subjective knowledge. This lengthy process results in delays that further compromise the infrastructure’s structural integrity. To address this limitation, this study proposes a deep learning (DL)-based autonomous crack detection method using the convolutional neural network (CNN) technique. To improve the CNN classification performance for enhanced pixel segmentation, 40,000 RGB images were processed before training a pretrained VGG16 architecture to create different CNN models. The chosen methods (grayscale, thresholding, and edge detection) have been used in image processing (IP) for crack detection, but not in DL. The study found that the grayscale models (F1 score for 10 epochs: 99.331%, 20 epochs: 99.549%) had a similar performance to the RGB models (F1 score for 10 epochs: 99.432%, 20 epochs: 99.533%), with the performance increasing at a greater rate with more training (grayscale: +2 TP, +11 TN images; RGB: +2 TP, +4 TN images). The thresholding and edge-detection models had reduced performance compared to the RGB models (20-epoch F1 score to RGB: thresholding −0.723%, edge detection −0.402%). This suggests that DL crack detection does not rely on colour. Hence, the model has implications for the automated crack detection of concrete infrastructures and the enhanced reliability of the gathered information.

In this paper, we study the application of deep learning and image segmentation techniques for crack detection in civil infrastructure. We investigate the effectiveness of some deep learning models, such as Inception, ResNet, MobileNet, VGG, and U-Net, in detecting cracks on civil infrastructure surfaces. Additionally, we implement two segmentation models, U-Net and SAM, to enhance the precision of crack extraction from images. Through simulations and comparative analysis, we evaluate the performance of the models in accurately identifying and delineating cracks in civil infrastructure. The results demonstrate the efficacy of the proposed approach in achieving accurate crack detection, which is crucial for ensuring the structural integrity and safety of civil infrastructure. The proposed model achieved an accuracy of 100% and IoU of 0.95.

A Deep Active Learning Framework for Crack Detection in Digital Images of Paintings

In the practical applications of crack detection, traditional image processing based detection methods are faced with a lot of challenges, such as low level of intelligence, poor adaptability and so on. In this paper, an automatic pixel-level crack detection method based on deep transfer learning is proposed for engineering applications. The proposed detection method is composed of two stages: crack recognition and crack semantic segmentation, which makes it easy to meet the needs of efficient and reliable detection for large-scale collected images. The fine-tuned Vgg16 model in the first stage can accurately identify the crack images and avoid the computing cost for non-crack images in the further processing. The Unet++ model in the second stage provides a pixel-level semantic segmentation for crack images. Multifold knowledge based transfer learning and weighted loss function are proposed to improve the performance and generalization ability of the models. On 5 publicly available datasets, the proposed method achieved a mIoU of 84.62%. Tested on totally 8064 images with a resolution of 224×224, the proposed method achieved a mean detection speed of 8.1ms per image. Experiment results exhibit that the proposed method can effectively detect the pixel-level cracks. It provides an end-to-end deep learning method for large-scale crack detection for civil infrastructure. For the crack detection in complex scenes and the improvement of processing speed will be our next in-depth study.

The current intelligent detection of asphalt pavement cracks relies on visible images, which are ineffective in handling temporary shadow and reflection. In contrast, infrared images reflect thermal radiation intensity, making them suitable for crack detection by mitigating environmental interference. The GSkYOLOv5 method is proposed in this study for accurate crack detection in infrared images of asphalt pavement. Firstly, an infrared dataset consisting of 2400 images of asphalt pavement cracks was captured and produced, including three types of cracks – strip cracks, cross cracks and reticular cracks. And the imbalanced samples were balanced using generative adversarial networks. Secondly, GhostNet and Selective Kernel Networks were added to enhance crack information perception, and the SPPF module was replaced to improve feature learning for different scale cracks. Lastly, GSkYOLOv5 was tested with a self-built infrared dataset of asphalt pavement cracks. GSkYOLOv5 improved detection accuracy by 4.7% and recall rate by 1.3% compared to YOLOv5s, outperforming the other tested algorithms. This method can be combined with visible image-based crack detection for comprehensive intelligent detection of asphalt pavement cracks using dual lenses for visible and infrared images.

The identification of pavement cracks is critical for ensuring road safety. Currently, manual crack detection is quite time-consuming. To address this issue, automated pavement crack-detecting technology is required. However, automatic pavement crack recognition remains challenging, owing to the intensity heterogeneity of cracks and the complexity of the backdrop, e.g., low contrast of damages and backdrop may have shadows with comparable intensity. Motivated by breakthroughs in deep learning, we present a new network architecture combining the feature pyramid with the attention mechanism (PSA-Net). In a feature pyramid, the network integrates spatial information and underlying features for crack detection. During the training process, it improves the accuracy of automatic road crack recognition by nested sample weighting to equalize the loss caused by simple and complex samples. To verify the effectiveness of the suggested technique, we used a dataset of real road cracks to test it with different crack detection methods.

Recently, the construction industry has benefited from the increased application of smart construction led by the core technologies of the fourth industrial revolution, such as BIM, AI, modular construction, and AR/VR, which enhance productivity and work efficiency. In addition, the importance of “Off-Site Construction (OSC)”, a factory-based production method, is being highlighted as modular construction increases in the domestic construction market as a means of productivity enhancement. The problem with OSC construction is that the quality inspection of Precast Concrete (PC) members produced at the factory and brought to the construction site is not carried out accurately and systematically. Due to the shortage of quality inspection manpower, a lot of time and money is wasted on inspecting PC members on-site, compromising inspection efficiency and accuracy. In this study, the major inspection items to be checked during the quality inspection are classified based on the existing PC member quality inspection checklist and PC construction specifications. Based on the major inspection items, the items to which AI technology can be applied (for automatic quality inspection) were identified. Additionally, the research was conducted focusing on the detection of cracks, which are one of the major types of defects in PC members. However, accurate detection of cracks is difficult since the inspection mostly relies on a visual check coupled with subjective experience. To automate the detection of cracks for PC members, video images of cracks and non-cracks on the surface were collected and used for image training and recognition using Convolutional Neural Network (CNN) and object detection, one of the deep learning technologies commonly applied in the field of image object recognition. Detected cracks were classified according to set thresholds (crack width and length), and finally, an automated PC member crack detection system that enables automatic crack detection based on mobile and web servers using deep learning and imaging technologies was proposed. This study is expected to enable more accurate and efficient on-site PC member quality inspection. Through the smart PC member quality inspection system proposed in this study, the time required for each phase of the existing PC member quality inspection work was reduced. This led to a reduction of 13 min of total work time, thereby improving work efficiency and convenience. Since quality inspection information can be stored and managed in the system database, human errors can be reduced while managing the quality of OSC work systematically and accurately. It is expected that through optimizing and upgrading our proposed system, quality work for the precise construction of OSC projects can be ensured. At the same time, systematic and accurate quality management of OSC projects is achievable through inspection data. In addition, the smart quality inspection system is expected to establish a smart work environment that enables efficient and accurate quality inspection practices if applied to various construction activities other than the OSC projects.

Deep learning has remarkably impacted several different scientific disciplines over the last few years. For example, in image processing and analysis, deep learning algorithms were able to outperform other cutting-edge methods. Additionally, deep learning has delivered state-of-the-art results in tasks like autonomous driving, outclassing previous attempts. There are even instances where deep learning outperformed humans, for example with object recognition and gaming. Deep learning is also showing vast potential in the medical domain. With the collection of large quantities of patient records and data, and a trend towards personalized treatments, there is a great need for automated and reliable processing and analysis of health information. Patient data is not only collected in clinical centers, like hospitals and private practices, but also by mobile healthcare apps or online websites. The abundance of collected patient data and the recent growth in the deep learning field has resulted in a large increase in research efforts. In Q2/2020, the search engine PubMed returned already over 11,000 results for the search term ‘deep learning’, and around 90% of these publications are from the last three years. However, even though PubMed represents the largest search engine in the medical field, it does not cover all medical-related publications. Hence, a complete overview of the field of ‘medical deep learning’ is almost impossible to obtain and acquiring a full overview of medical sub-fields is becoming increasingly more difficult. Nevertheless, several review and survey articles about medical deep learning have been published within the last few years. They focus, in general, on specific medical scenarios, like the analysis of medical images containing specific pathologies. With these surveys as a foundation, the aim of this article is to provide the first high-level, systematic meta-review of medical deep learning surveys.

Forecasting the behavior of the stock market is a classic but difficult topic, one that has attracted the interest of both economists and computer scientists. Over the course of the last couple of decades, researchers have investigated linear models as well as models that are based on machine learning (ML), deep learning (DL), reinforcement learning (RL), and deep reinforcement learning (DRL) in order to create an accurate predictive model. Machine learning algorithms can now extract high-level financial market data patterns. Investors are using deep learning models to anticipate and evaluate stock and foreign exchange markets due to the advantage of artificial intelligence. Recent years have seen a proliferation of the deep reinforcement learning algorithm’s application in algorithmic trading. DRL agents, which combine price prediction and trading signal production, have been used to construct several completely automated trading systems or strategies. Our objective is to enable interested researchers to stay current and easily imitate earlier findings. In this paper, we have worked to explain the utility of Machine Learning, Deep Learning, Reinforcement Learning, and Deep Reinforcement Learning in Quantitative Finance (QF) and the Stock Market. We also outline potential future study paths in this area based on the overview that was presented before.

A systematic approach to pixel-level crack detection and localization with a feature fusion attention network and 3D reconstruction

Cracks are a common type of road distress. However, the traditional manual and vehicle-borne methods of detecting road cracks are inefficient, with a high rate of missed inspections. The development of unmanned aerial vehicles (UAVs) and deep learning has led to their use in crack detection and classification becoming an increasingly popular topic. In this paper, an aerial drone is used to efficiently and safely collect road data. However, this also brings many challenges. For example, flying too high or too fast may produce poor quality images, with unclear cracks that may be ignored or misjudged as other features and increased environmental noise that may make it difficult to distinguish between cracks and other noise features. To address the above challenges, this paper proposes the CrackNet model and CrackClassification algorithm. The CrackNet network is an encoder–decoder architecture. Low- and high-level semantic information are combined through the skip feature fusion layers between the encoder and decoder to enhance the model’s expression and ability to recover image details. Additionally, the MHDC module at the bottom of the network can significantly increase the receptive field without reducing the feature map resolution. The MHSA module can simultaneously capture features from multiple subspaces. The average precision (AP) scores of the CrackNet network on three datasets, namely UAVRoadCrack, CRKWH100, and CrackLS315, were 0.665, 0.942, and 0.895, respectively. In addition, values of the other two evaluation metrics, ODS and OIS, were the highest among the compared methods. Meanwhile, the proposed CrackClassification algorithm in this paper achieves 85% classification accuracy for transverse and longitudinal cracks and 78% classification accuracy for block cracks and reticulated cracks. Overall, the CrackNet algorithm provides a new baseline model for crack detection in UAV remote sensing image scenes. The CrackClassification algorithm provides a new approach for batch classification of highway cracks. The detection and classification algorithm proposed in this paper were applied to 108 km of road sections.

Building cracks such as gaping cracks, separation, and horizontal cracks are a few types of cracks that possess a severe issue on reinforced concrete; hence, the earlier the detection the cheaper the repairs. Numerous studies about crack detection considered VGG16, and Faster R-CNN because the severity level is crucial to each construction company business owner. This paper aims to build a deep learning model using Yolov3 that can detect a crack in reinforce concrete structures and categorize a medium, severe, or very severe crack using an android application. An android application was developed instead of using an expensive Ultrasonic Pulse Velocity in the market to detect the severity of the crack on the concrete. The overall accuracy summary of the android application is 93.33%, while the kappa value is. 97. Therefore, the deep learning model and android application produced an accurate calculation in detecting the crack and determining its crack classification.

Efficient detection of pavement cracks can effectively prevent traffic accidents and reduce pavement maintenance costs. In order to overcome the complicated and uneconomical disadvantages of traditional crack detection methods, this paper introduces a pavement crack detection network based on deep learning, which can automatically detect pavement cracks and achieves excellent detection accuracy. And the network can easily use the sensors to collect data to facilitate industrial applications. In additional, considering that most cracks have slim feature, we apply the latest Transformer module in the network to improve the effect of cracks detection. Transformer has a strong ability to capture the long-range dependence of the cracks, which enables the network to learn the context information of the crack region. Furthermore, the network also utilizes some techniques to improve the ability of algorithm to detect various cracks. Our network is trained on pavement data sets containing India, the Czech Republic and Japan. It achieved F1 scores of 0.6739 and 0.6650 on two online test sets with fewer network parameters.