Automatic Industry PCB Board DIP Process Defect Detection System Based on Deep Ensemble Self-Adaption Method

The conventional PCB (Printed Circuit Board) DIP (Dual Inline Package) process solder defect detection was done by labor inspection, which is not only time-intensive but also labor-intensive. This paper proposes a deep ensemble method to inspect the PCB solder defects to replace the labor inspection. To achieve a high detection rate and a low false alarm rate, two distinct detection models, a hybrid YOLOv2 (YOLOv2 as a foreground detector and ResNet-101 as a classifier) and Faster RCNN with ResNet-101 and FPN are separately trained to obtain a high detection rate result. The final ensemble model aggregates the result from the two detection models. That achieves a 96.73% detection rate and a 19.73% false alarm rate in real tests. The detection time is less than 15 seconds for inferencing a PCB image with a resolution of 7296*6000. The proposed method has been proven efficient in terms of guiding operators to identify and fix PCB solder defects [1] and thus is able to reduce 33% of labor demand for each PCB production line at our real test site. [1].

The traditional global template matching is time consuming, has low accuracy, and cannot be adapted to rotation and scale change. The template matching technique proposed in this study improves the time, accuracy and robustness for printed circuit boards (PCB). In order to shorten the image positioning time, the image preprocessing is implemented on PCB image and the image blocks are labeled to obtain the tagged image, and the feature vector is extracted and the marking point region image is selected. The feature vector with rotation change and scale change robustness is extracted from the tagged image after labeling in the PCB image by using artificial neural network, combined with image moments for training. The marking point region image in the PCB image is selected. The scale value of the marking point region image is estimated by parametric template vector matching. The deflection angle of marking point region image is calculated by Hough transform. The obtained scale value and deflection angle value are used for fast template matching to determine the marking point positioning. The three-dimensional (3D) parabolic curve fitting is implemented in marking point positioning and adjacent pixel position to reach the sub-pixel level accuracy. The experiment showed that the proposed template matching technique for the PCB image with or without noise or angle rotation, the average position accuracy error of each translated image is lower than 7 \(\upmu \)m, and the error standard deviation is lower than 5 \(\upmu \)m. The rotation angle error average and standard deviation of angular error of Hough transform are lower than 0.2\(^{\circ }\), more accurate than orientation code (OC) method. The scale value estimation, relative error average and error standard deviation are lower than 0.004 and 0.006 for the image with or without noise. The average complete positioning time of PCB image at resolution of \(2500\times 2500\) is only 0.55 s, which is better than the 3.97 s of traditional global template matching. The results prove that the template matching technique of this study not only has sub-pixel level high accuracy and short computing time, but also has the robustness of rotation change and scale change interference. It can implement rapid, efficient and accurate positioning.

Replacing time-consuming and costly manual inspections on production lines with efficient and accurate defect detection algorithms for Printed Circuit Boards (PCBs) remains a significant challenge. Current PCB defect detection methods are typically optimized using existing models such as YOLO and Faster R-CNN to enhance detection accuracy. In this study, we analyse a PCB defect dataset characterized by small targets and a concentrated size distribution. SEPDNet (Simple and Effective PCB Defect Detection Network) is designed for the characteristics of the dataset, only one detection head is used, which reduces the number of parameters and improves the detection performance at the same time. SEPDNet uses RepConv (Re-parameterizable Convolution) to improve the backbone representation ability, and FPN (Feature Pyramid Network) is used in the neck part to simplify the model. SEPDNet has fewer than 30% of the parameters of YOLOv9u-s, yet achieves an improvement of 0.025 in the F1 score, 2.7% in mAP50, and 3.8% in mAP50:95 compared to YOLOv9u-s. We propose the method of designing the model according to the characteristics of the dataset. Our experiments show that customizing the model design according to dataset characteristics can achieve strong performance with a simplified structure and fewer parameters.

Electronic components (EC) detection is relevant for Printed Circuit Board (PCB) manufacturing, quality inspection, and assurance (cyber-security). Moreover, at end-of-life electronic devices, the PCBs become Waste PCBs (WPCBs), the primary source of high-value materials in electronic waste. However, the recycling process is challenging because of the WPCB’s high composition diversity. EC detection in WPCBs can reduce the uncertainty about WPCBs composition and help select a better recycling process. Nevertheless, large fully-annotated datasets for the PCB domain are available at a high cost. For WPCBs, there is only one publicly available dataset, the PCB-DSLR, and it is partially labeled. As for PCBs, the FICS-PCB dataset is available. However, it contains images of modern and clean PCBs, all in good condition, while the first has broken and dirty parts. Thus, we propose a self-training strategy for object detection algorithms without needing a fully labeled dataset. In the strategy, the YOLOv5 model is used to detect the eight types of electronic components present in FICS-PCB in PCB-DSLR, which initially has labels only for integrated circuits. The process occurs through the interaction between a teacher model, trained with the FICSPCB data, and a student model, which is trained by joining the FICS-PCB data and the teacher-generated labels for PCB-DSLR. The results show that the student model performs better in the electronic components detection task, obtaining greater precision and sensitivity when compared to the teacher model. Since we are dealing with partially labeled data, we provide a low-dimensional representation of the detections of both models using t-SNE.

Given the increase in digital product waste, demand for recycling of printed circuit boards (PCBs) is increasing. Precious and minor metals are often well concentrated in integrated circuit (IC) chips, especially in PCBs; hence, IC chips are primary targets for recycling. The technology for the non-destructive detachment of IC chips from PCBs is increasing in sophistication; however, the effectiveness of IC chip detachment is currently assessed manually and visually. In the present study, an automated IC chip detachment assessment method was developed, which combines multistep binarization and template matching of X-ray transmission images of crushed PCBs. To validate the method, five types of mechanically crushed PCBs from mobile phones were examined, and the developed method successfully assessed the IC chip detachment rate, with an average error rate of only 2.2% compared to visual assessment.

The inspection machine in PCB (printed circuit board) assembly line checks assembly errors by inspecting the images inside of the component inspection region. The component inspection region consists of region of component package and region of soldering. It is necessary to extract the regions automatically for auto-teaching system of the inspection machine. We propose an image segmentation method to extract the component inspection regions automatically from images of PCB. The acquired image is transformed to HSI color model, and then segmented by several regions by clustering method. We develop a modified K-means algorithm to increase the accuracy of extraction. The heuristics generating the initial clusters and merging the final clusters are newly proposed. The vertical and horizontal projection is also developed to distinguish the region of component package and region of soldering. The experimental results are presented to verify the usefulness of the proposed method.

FPN (Feature Pyramid Network) and transformer-based target detectors are commonly employed in target detection tasks. However, these approaches suffer from design flaws that restrict their performance. To overcome these limitations, we proposed TIG-DETR (Texturized Instance Guidance DETR), a novel target detection model. TIG-DETR comprises a backbone network, TE-FPN (Texture-Enhanced FPN), and an enhanced DETR detector. TE-FPN addresses the issue of texture information loss in FPN by utilizing a bottom-up architecture, Lightweight Feature-wise Attention, and Feature-wise Attention. These components effectively compensate for texture information loss, mitigate the confounding effect of cross-scale fusion, and enhance the final output features. Additionally, we introduced the Instance Based Advanced Guidance Module in the DETR-based detector to tackle the weak detection of larger objects caused by the limitations of window interactions in Shifted Window-based Self-Attention. By incorporating TE-FPN instead of FPN in Faster RCNN and employing ResNet-50 as the backbone network, we observed an improvement of 1.9 AP in average accuracy. By introducing the Instance-Based Advanced Guidance Module, the average accuracy of the DETR-based target detector has been improved by 0.4 AP. TIG-DETR achieves an impressive average accuracy of 44.1% with ResNet-50 as the backbone network.

A variety of ways has been established to detect defects found on printed circuit boards (PCB). In previous studies, defects are categories into seven groups with a minimum of one defect and up to a maximum of 4 defects in each group. Using Matlab image processing tools this research separates two of the existing groups containing two defects each into four new groups containing one defect each by processing synthetic images of bare through-hole single layer PCBs.

Through the use of automatic inspection systems is possible to improve the Printed Circuits Board (PCB) manufacturing and reduce the fail rate in this process. In this paper an automatic inspection system for detecting small PCB defects is presented, the system uses a 2-D high precision laser sensor; this kind of laser sensor allows getting high precision measurements with an error of units of micrometers. The sensor is able to detect Surface-Mount Devices (SMD) onto PCB in order to perform a defect-detection process. The system uses an open, modular and scalable architecture for controlling a robotic motion in two axes, also a graphic application coded in Java for data processing and unify all the components is developed. Finally a judgment algorithm for defect-detection evaluation using binary matrices is added to the system. The aim of this system is that it can detect small PCB defects using a 2-D laser sensor and that is possible to implement it in a production line. Design, tests and results gotten are shown.

Timely fault diagnosis and localization of transmission lines is crucial for ensuring the reliable operation of increasingly complex power systems. This study proposes an optimized R-Net algorithm based on a feature pyramid network (FPN) and densely connected convolutional network (D-Net) for transmission line fault diagnosis and localization. The R-Net network is enhanced by reshaping the anchor points using an improved K-means algorithm and incorporating an FPN for multi-scale feature extraction. The backbone network is further optimized using D-Net to strengthen inter-layer connections and improve feature reuse. Experimental results demonstrate that the optimized R-Net achieves an overall average accuracy of 0.64, outperforming the original network by 1.30%. The accuracy improvement is particularly significant for ground wire defects (2.40%). The D-Net-based R-Net, despite having fewer parameters, maintains high accuracy (0.6502). Compared to other object detection algorithms, such as YOLO-v3 and Faster R-CNN, the optimized R-Net exhibits superior performance in terms of mean average precision (15.58% and 2.45% higher, respectively) and parameter efficiency (17M vs. 38M and 81M). Considering both performance and speed, the optimized R-Net achieves a processing rate of 10.5 frames per second. This study provides an efficient and accurate tool for transmission line fault diagnosis and localization, with significant practical implications for power system operation and maintenance.

Feature pyramid network (FPN) has dominated the task of object detection for many years with its congenital capability of semantic information interaction between different feature levels. Satisfactory performance can be easily achieved via the heavy backbones or tricky designs of anchors. However, the heavy structure brings about a huge increase of parameters and calculation. To address this challenge, this paper proposes a novel Hint Pyramid Learning(HPL) strategy to optimize a lightweight FPN-based detector such as one based on ResNet18. In HPL, the hint feature representation is learnt via salient semantic mining based on the teacher-student learning mechanism and self learning mechanism. Besides minimizing of the discrepancy of features belonging to the same feature pyramid layer, HPL also focuses on the relationships between feature pyramid layers. Unlike the popular mimicking algorithms such as hint learning and knowledge distillation which achieve comparable performance on classification tasks but failed to generalize to the detection tasks with FPN, HPL can be easily plugged into different FPN-based pipelines such as one-stage RetinaNet or two-stage Faster R-CNN and improves the performance straightforwardly. Without any meticulous designing, when the backbone of students is ResNet18 and the backbone of teachers is ResNet50, HPL effectively improves the performance by 2.1 AP for RetinaNet and 1.0 AP for Faster R-CNN on COCO benchmark.

Given the wide diffusion of deep neural network architectures for computer vision tasks, several new applications are nowadays more and more feasible. Among them, a particular attention has been recently given to instance segmentation, by exploiting the results achievable by two-stage networks (such as Mask R-CNN or Faster R-CNN), derived from R-CNN. In these complex architectures, a crucial role is played by the Region of Interest (RoI) extraction layer, devoted to extracting a coherent subset of features from a single Feature Pyramid Network (FPN) layer attached on top of a backbone. This paper is motivated by the need to overcome the limitations of existing RoI extractors which select only one (the best) layer from FPN. Our intuition is that all the layers of FPN retain useful information. Therefore, the proposed layer (called Generic RoI Extractor - GRoIE) introduces non-local building blocks and attention mechanisms to boost the performance. A comprehensive ablation study at component level is conducted to find the best set of algorithms and parameters for the GRoIE layer. Moreover, GRoIE can be integrated seamlessly with every two-stage architecture for both object detection and instance segmentation tasks. Therefore, the improvements brought about by the use of GRoIE in different state-of-the-art architectures are also evaluated. The proposed layer leads up to gain a 1.1% AP improvement on bounding box detection and 1.7% AP improvement on instance segmentation. The code is publicly available on GitHub repository at https://github.com/IMPLabUniPr/mmdetection/tree/groie_dev

Industry 4.0 is currently the goal of many factories, promoting manufacturing factories and sustainable operation. Automated Optical Inspection (AOI) is a part of automation. Products in the production line are usually inspected visually by operators. Due to human fatigue and inconsistent standards, product inspections still have defects. In this study, the sample component assembly printed circuit board (PCB), PCB provided by the company was tested for surface components. The types of defects on the surface of the PCB include missing parts, multiple parts, and wrong parts. At present, the company is still using visual inspection by operators, the PCB surface components are more complex. In order to reduce labor costs and save the development time required for different printed circuit boards. In the proposed method, we use digital image processing, positioning correction algorithm, and deep learning YOLO for identification, and use 450 images and 10500 components of the PCB samples. The result and contribution of this paper shows the total image recognition rate is 92% and the total component recognition rate reaches 99%, and they are effective. It could use on PCB for different light, different color backplanes, and different material numbers, and the detection compatibility reaches 98%.

The application of base on deep learning remote sensing image landslide detection has become a trend. In view of the, large difference in the scale of landslide objects and the very important characteristics of the shallow layer of the landslide object, we modified the original Faster RCNN. First compare the average accuracy values of different network models, and select the network ResNet50 with deeper layers, and then integrated into Feature Pyramid Network (FPN) on the basis of ResNet50, Fusion of FPN structure with RPN and Fast RCNN. Finally, it is verified on the public landslide dataset in BiJie City, Guizhou Province, China, and the average accuracy of the obtained model is 87.10%, which is 16.9% higher than the 70.20% of VGG16. Contrast experiments with different anchor ratios and different hyperparameters in the FPN part are carried out, and the generated results are analyzed and compared. The experimental results show that the proposed method has good robustness and high accuracy for the detection of landslide objects, in remote sensing images, which can provide a technical reference for further research on landslide detection in remote sensing images.

Printed circuit boards (PCBs) are ubiquitous and essential electronic components. Tiny targets and high precision are the focus of PCB defect detection. This paper proposes an improved model focusing on tiny defect detection and model compression to achieve better performance in PCB defect detection. The improved model has a compact structure based on MobileNet v3 Small-CA and an image-cutting layer. Moreover, it applies an improved multi-scale fusion step with a location weighted mechanism to enhance representation performance. The proposed model outperforms state-of-the-art detection algorithms such as Faster R-CNN, EfficientDet, SSD, and YOLO v7, based on experimental results on a public synthetic PCB dataset. The proposed tiny object detection model has better performances on speed and detection accuracy, thereby benefitting the manufacturing industries associated with PCBs.