Research on rapid detection of cross-scale defects in surface based on deep learning

Cascaded detection method for surface defects of lead frame based on high-resolution detection images

Wind energy is a renewable resource with abundant reserves, and its sustainable development and utilization are crucial. The components of wind turbines, particularly the blades and various surfaces, require meticulous defect detection and maintenance due to their significance. The operational status of wind turbine generators directly impacts the efficiency and safe operation of wind farms. Traditional surface defect detection methods for wind turbines often involve manual operations, which suffer from issues such as high subjectivity, elevated risks, low accuracy, and inefficiency. The emergence of computer vision technologies based on deep learning has provided a novel approach to surface defect detection in wind turbines. However, existing datasets designed for wind turbine surface defects exhibit overall category scarcity and an imbalance in samples between categories. The algorithms designed face challenges, with low detection rates for small samples. Hence, this study first constructs a benchmark dataset for wind turbine surface defects comprising seven categories that encompass all common surface defects. Simultaneously, a wind turbine surface defect detection algorithm based on improved YOLOv5 is designed. Initially, a multi-scale copy-paste data augmentation method is proposed, introducing scale factors to randomly resize the bounding boxes before copy-pasting. This alleviates sample imbalances and significantly enhances the algorithm’s detection capabilities for targets of different sizes. Subsequently, a dynamic label assignment strategy based on the Hungarian algorithm is introduced that calculates the matching costs by weighing different losses, enhancing the network’s ability to learn positive and negative samples. To address overfitting and misrecognition resulting from strong data augmentation, a two-stage progressive training method is proposed, aiding the model’s natural convergence and improving generalization performance. Furthermore, a multi-scenario negative-sample-guided learning method is introduced that involves incorporating unlabeled background images from various scenarios into training, guiding the model to learn negative samples and reducing misrecognition. Finally, slicing-aided hyper inference is introduced, facilitating large-scale inference for wind turbine surface defects in actual industrial scenarios. The improved algorithm demonstrates a 3.1% increase in the mean average precision (mAP) on the custom dataset, achieving 95.7% accuracy in mAP_50 (the IoU threshold is half of the mAP). Notably, the mAPs for small, medium, and large targets increase by 18.6%, 16.4%, and 6.8%, respectively. The experimental results indicate that the enhanced algorithm exhibits high detection accuracy, providing a new and more efficient solution for the field of wind turbine surface defect detection.

To improve the accuracy of clinical diagnosis of severe patients with advanced liver cancer and enhance the effect of chemotherapy treatment, the U-Net model was optimized by introducing the batch normalization (BN) layer and the dropout layer, and the segmentation training and verification of the optimized model were realized by the magnetic resonance (MR) image data. Subsequently, HepG2 cells were taken as the research objects and treated with 0, 10, 20, 40, 60, 80, and 100 μmol/L emodin (EMO), respectively. The methyl thiazolyl tetrazolium (MTT) method was used to explore the changes in cell viability, the acridine orange (AO)/ethidium bromide (EB) and 4′,6-diamidino-2-phenylindole (DAPI) were used for staining, the Annexin V fluorescein isothiocyanate (FITC)/propidium iodide (PI) (Annexin V-FITC/PI) was adopted to detect the apoptosis after EMO treatment, and the Western blot (WB) method was used with the purpose of exploring the changes in protein expression levels of PARP, Bcl-2, and p53 in the cells after treatment. It was found that compared with the original U-Net model, the introduction of the BN layer and the dropout layer can improve the robustness of the U-Net model, and the optimized U-Net model had the highest dice similarity coefficient (DSC) (98.45%) and mean average precision (MAP) (0.88) for the liver tumor segmentation.

Surface defect detection for printed circuit board (PCB) is indispensable for managing PCB production quality. However, automatic detection of PCB surface defects is still a challenging task because, even within the same category of surface defect, defects present great differences in morphology and pattern. Although many computer vision-based detectors have been established to handle these problems, current detectors struggle to achieve high detection accuracy, fast detection speed and low memory consumption simultaneously. To address those issues, we propose a cost-effective deep learning (DL)-based detector based on the cutting-edge YOLOv4 to detect PCB surface defect quickly and efficiently. The YOLOv4 is improved upon with respect to its backbone network and the activation function in its neck/prediction network. The improved YOLOv4 is evaluated with a customized dataset, collected from a PCB factory. The experimental results show that the improved detector achieved a high performance, scoring 98.64% on mean average precision (mAP) at 56.98 frames per second (FPS), outperforming the other compared SOTA detectors. Furthermore, the improved YOLOv4 reduced the parameter space of YOLOv4 from 63.96 M to 39.59 M and the number of multiply-accumulate operations (Madds) from 59.75 G to 26.15 G.

Computer vision and deep learning techniques are the most emerging technologies in this era. Both of these can greatly raise the rate at which defects on metal surfaces are identified while performing industrial quality checks. The identification of faults over metal surfaces can be viewed as a significant challenge since they are easily impacted by ambient factors including illumination and light reflections. This paper proposes novel metal surface defect detection network called as YOLOv-5s-FRN in response to the problems of ineffective detection brought by the conventional manual inspection system. The proposed system is developed through the integration of a novel architectural module called as Feature Recalibration Network (FRN) to the You Only Look Once-version-5 small network )YOLOv-5s(. In order to extract the global feature information from the provided image, FRN is able to evaluate the interdependencies between the channels. This improves the feature discrimination capability and prediction accuracy of the defect detection system. The incorporation of FRN structure makes YOLOv-5s architecture to selectively enhance the necessary features and discard the unwanted ones. Therefore, the proposed novel method will efficiently detect and classify the metal surface defects such as crazing, patches, inclusions, scratches, pitted surfaces and rolled in scale. North Eastern University surface defect database (NEU-DET) has been used to train and test the proposed architectural model. The suggested system has been compared with alternative models based on several performance matrices such as precision, recall and Mean Average Precision (mAP). It is observed that the proposed YOLOv-5s-FRN architecture provides significant performance improvement than state-of-the-art methods. The proposed system has been provided satisfactory results by means of improvement in mAP and time consumption. The proposed model has delivered value of mAP_0.5 as 98.05% and that of mAP_0.5:0.95 as 89.03%.

Accurate identification of defect morphology and size distribution on metallic surfaces is crucial for manufacturing, particularly in the steel industry where such materials are widely used. This study proposes a novel lightweight architecture based on YOLOv7-tiny to address the challenges of real-time metallic surface defect detection. The original anchor-based detection head is replaced with an anchor-free mechanism to reduce missed detections of defects with extreme aspect ratios, while a logarithmic transformation-based enhancement is introduced to strengthen defect features. The backbone is replaced by the lightweight MobileNetV3-large network to reduce the number of parameters, and an Efficient Multi-Scale Attention (EMA) module is embedded into the bottleneck structure to increase target importance and suppress background interference, especially for small defects. In addition, a bidirectional feature pyramid network with adaptively spatial feature fusion (BAFPN) is integrated as the Head architecture to provide richer semantic information. The improved model was evaluated on the DAGM 2007, NEU-DET, and GC10-DET datasets, which encompass metal surface defects with varying degrees of complexity. The results demonstrate an average increase of 6.24% in mean average precision (mAP) and an inference speed over 90 FPS, confirming both the efficacy and real-time capability of the proposed method.

Production defects caused by irresistible factors such as process design problems or differences in steel properties in strip production affect the economic benefits of the enterprise and threaten production safety. Traditional defect detection methods are difficult to achieve real-time and high-precision detection, so developing surface defect detection methods based on deep learning is of great significance for strip production. In order to effectively improve the accuracy of the deep learning model in detecting surface defects on hot-rolled strip, in this work we propose a real-time detection model for surface defects on strip steel based on the YOLOv8n model. Firstly, the newly convolutional layer Con5v is designed to replace the original convolutional layer in the neck, and an attention mechanism is added in front of each Con5v to improve the algorithm’s ability to extract small target information. Secondly, an additional set of upsampled feature extraction units is added to the neck in order to enhance the spatial information of the feature map. Subsequently, a set of feature fusion units is incorporated and the convolutional layers in it are improved to provide better feature maps. Thirdly, the number of decoupling detection heads is increased to receive more high-quality features. The final experimental results show that YOLOv8n-GAM (YOLOv8 Nano Model with Global Attention Mechanism) achieves 81.4mAP and 82.0FPS on the NEU-DET dataset and 71.2mAP and 55.0FPS on the GC10-DET dataset, which are 5.7% and 6.9% higher than those of YOLOv8n, respectively. The model proposed in this paper achieves a comprehensive performance improvement in strip steel.

Internal defects detection and classification in hollow cylindrical surfaces using single shot detection and MobileNet

Accurate detection of surface defects for steel is essential to improve surface quality and service life. Deep learning (DL) used in steel surface defect detection can solve the problems of low efficiency and poor accuracy of traditional manual detection. The classic YOLOv5 as a DL method is used to accomplish defect detection tasks without attention mechanisms, resulting in a loss of global information. Besides, it is difficult to complete complex network detection tasks with low-configuration hardware, especially for surface defects with complex defect types and variable defect sizes. To solve these issues, this paper introduces an improved global feature reuse and hardware-aware YOLOv5 by using BoTNet, RepGhost, and EfficientRep model (BGE-YOLOv5). The multi-head self-attention layer is used to obtain global information and only part of the convolutional layers is replaced to avoid the excessive computational cost. The RepGhost model is introduced to extract the remaining feature information for feature reuse. EfficientRep is used to replace the original structure to achieve hardware-aware and to balance the detection veracity and efficiency. The distance IOU is replaced by SCYLLA-IOU to accelerate the iteration and improve stability. The results of the framework on the surface defect database (NEU-DET) show that BGE-YOLOv5 achieves a mean average precision of 79.5%, which is 10.3% greater than the baseline. The proposed BGE-YOLOv5 has a better performance in steel surface defect detection.

When using visual image technology to detect surface defects of sealing rings for aerospace, traditional image processing algorithms are constrained by surface defects types, scale differences, illumination variations, and non-uniformities of surface image. As a result, the application of each algorithm is relatively limited. To solve the problems, a general deep learning algorithm for surface defects detection of sealing rings is proposed. Texture features of defects, gradient features of edges and contour features are selected as main features of surface defects of sealing rings differentiated from the background, and a new cascade modular backbone network is design for extraction of these features. The average aspect ratio of label box for defects among dataset of defect images of sealing rings is calculated and aspect ratio distribution is counted to obtain the proportional parameter for generating prior box. Based on the calculated proportional parameters of prior box and scale features of defects, multi-scale defects detection network is designed. Meanwhile, defects classification and position regression compound loss function are defined. In the end, weighting parameters of the network are updated on the basis of end-to-end training. Experimental result shows that the accuracy of proposed algorithm for defects detection reach 94.03% under the condition that the original defects sample images is 107 and confidence is 0.6. Compared with RefineDet network, the depth of backbone network of feature extraction increases by 78% while the scale of parameters reduces by 84.82% and the accuracy of defects detection reduces by 3.01%.

Mechanical parts often have surface defects such as scratches, cracks and pits during the production process, which seriously affect product quality and service life. To address this challenge, this paper proposes a method for detecting and quantifying surface defects of mechanical parts based on image processing. First, this paper uses high-resolution image acquisition technology to obtain surface images of mechanical parts, and improves image quality through preprocessing steps. Then, this paper uses edge detection, morphological operations, and image segmentation algorithms to extract the features of surface defects, thereby realizing automatic identification of defective areas. Finally, this paper combines the Faster R-CNN (Faster Region-Based Convolutional Neural Network) algorithm to further refine the classification and quantification of defects and obtain the specific location, size and type of defects. Experimental results show that among the 500 mechanical parts samples tested, the proposed method has a mean average precision (mAP) of 0.93 in surface defect detection. Compared with other image processing methods, this method has significantly improved defect detection accuracy and speed. Especially when processing higher-resolution images, the Faster R-CNN model can achieve efficient defect classification and quantification to meet the needs of real-time detection in industrial production.

In the context of resource constrained embedded terminals, the current deep learning-based technology for detecting surface defects in strip steel faces challenges with low detection accuracy and efficiency. This study introduces a lightweight convolution neural network (CNN) model for detecting surface defects of strip steel, which is based on deep learning. Through the integration of non-parameter attention mechanism SimAM, inverse residual structure, and depth-wise convolution module, a lightweight attention mobile backbone network is developed to achieve optimal feature extraction. To address the challenge of detecting small surface defects the context augmentation module is introduced to provide more information for small defects detection by using multi-scale features. To improve the efficiency of feature fusion and reduce parameter redundancy, the GVFPN neck network is proposed. The network aims to represent and deal with multi-scale features objectively while minimizing costs. The experimental results indicate that on the NEU-DET dataset, the proposed network attains an mAP of 78.8% and FPS at 97 frame/s. Moreover, the model requires only 2.39 M parameters and 3.1 G FLOPs. Compared to YOLOv5s, the proposed network reduces parameters by 65.9% and FLOPs by 80.7%, while achieving a 1.3% higher mAP and a 30 frame/s increase in detection speed. These results effectively demonstrate the effectiveness and superiority of the proposed method.

Deep learning now plays an important role in solving complex problems in computer vision fields. The highly challenging high-resolution remote sensing image scene classification problem can also be solved using deep learning methods. The most commonly used method of deep learning is the convolutional neural network model. In this letter, based on deep learning, a combined model named Inception-long short-term memory (LSTM) is proposed. First, we combine the deep learning feature extracted from the pretrained Inception-V3 model with a hand-crafted feature: the GIST feature. The different features are then combined and input into the batch normalization (BN) layer. Second, the BN layer plays the role of the bridge to combine the InceptionV3 model with the LSTM model, which features a softmax classifier. The LSTM model is used to analyze the features and classify the different high-resolution remote sensing scene images. The proposed model, as a whole, can be uniformly trained. Three different datasets-the NWPU-RESISC45 dataset, the UC Merced dataset, and the SIRI-WHU dataset-were used to verify the effectiveness of the proposed model. The results show that the proposed Inception-LSTM model shows an outstanding performance in the scene classification task.

Nondestructive detection of surface defects of curved mosaic ceramics based on deep learning

Electrolytic capacitor surface defect detection based on deep convolution neural network