A Defect Detection Method Based on Improved Mask R-CNN for Wafer Maps

The non-maximum suppression (NMS) algorithm, which merges neighboring bounding boxes as the detection result, is widely utilized in object detection methods. However, the traditional NMS algorithm is not suitable for text detection in natural scenes, especially for long texts and dense texts. In this paper, we observe that the coordinates of candidate bounding boxes are presented in skew distribution. On this observation, an improved NMS algorithm called SD-NMS (Skew Distribution NMS) is designed. First, the mode and the median of the coordinate set are counted to filter out the redundant bounding boxes. Then the left bounding boxes are merged for the location of the text regions. The SD-NMS method improves the detection ability of the model without extra model training and can be easily embedded in text detection methods. The experimental results show that our method obtains F-measure more than other NMS methods in public data sets ICDAR2015 and MSRATD500.

Non-maximum suppression (NMS) is an indispensable post-processing step in object detection. After a picture is detected by the target, a large number of redundant anchors will be obtained, so we need to go through NMS to process these repeated anchors. Its essence is to extract the target detection frame with high confidence and suppress the false detection frame with low confidence. The biggest problem in the NMS algorithm is that it completely removes adjacent low-confidence detection frames. In some images with dense targets, this is very likely to cause missed detection and false detection. Therefore, we proposed the Simple-NMS algorithm, which adds two new thresholds to the original NMS, and sets a constraint condition different from the original based on the new threshold, and does not change the complexity of the traditional NMS algorithm, which can be very good Improve the effectiveness of NMS missed targets. The new NMS algorithm is improved on the standard data sets PASAL VOC2007 and MS COCO. In addition, the algorithm is simple and efficient, and can be easily integrated into any other object detection process.

High-resolution UAV imagery paired with a convolutional neural network approach offers significant advantages in accurately measuring forestry ecosystems. Despite numerous studies existing for individual tree crown delineation, species classification, and quantity detection, the comprehensive situation in performing the above tasks simultaneously has rarely been explored, especially in mixed forests. In this study, we propose a new method for individual tree segmentation and identification based on the improved Mask R-CNN. For the optimized network, the fusion type in the feature pyramid network is modified from down-top to top-down to shorten the feature acquisition path among the different levels. Meanwhile, a boundary-weighted loss module is introduced to the cross-entropy loss function Lmask to refine the target loss. All geometric parameters (contour, the center of gravity and area) associated with canopies ultimately are extracted from the mask by a boundary segmentation algorithm. The results showed that F1-score and mAP for coniferous species were higher than 90%, and that of broadleaf species were located between 75–85.44%. The producer’s accuracy of coniferous forests was distributed between 0.8–0.95 and that of broadleaf ranged in 0.87–0.93; user’s accuracy of coniferous was distributed between 0.81–0.84 and that of broadleaf ranged in 0.71–0.76. The total number of trees predicted was 50,041 for the entire study area, with an overall error of 5.11%. The method under study is compared with other networks including U-net and YOLOv3. Results in this study show that the improved Mask R-CNN has more advantages in broadleaf canopy segmentation and number detection.

PT.XYZ is a manufacturer of body bus which produce four type of bus that is big bus, medium bus, small bus, and mini bus. There is ten process to produce 1 unit big bus that is machine work, sub assembly, pra production, body assembly, metal finish, epoxy, putty, painting, trimming, and PDI (Pre Delivery Inspection). Based on the check sheet data summary from January to August of 2013, most major defect are in the putty process and based on the observation and interviews, there is product defects that pass inspection and entry process of painting. So, this study aims to identify the root cause of the defect and the escape of product defect from inspection. Root Cause Analysis method are used to identify the root cause of this problem. Using Root Cause Analysis method, RCA diagram showed that the root causes of defects are SOP that cannot be applied, the color of paint (epoxy) is less clear for inspection, poor lighting, lack of machinery, the wages of workers, and porous of fiber, while the root cause of the escape of product defect from inspection are SOP that cannot be applied, the color of paint (epoxy) is less clear for inspection, poor lighting, lack of machinery, the wages of workers, and lack of quality standards.

This paper reports on a new type of through-silicon via (TSV) defect, silicon fin defect, which was found after TSV deep-reactive-ion-etching (DRIE) process for TSV integration with front-end-of-line (FEOL) devices. One possible root cause for this defect is that the bulk micro defect (BMD) in silicon substrate serves as a micro-mask during etching and results in silicon fin defects at TSV bottom. These defects have to be eliminated as they are killer TSV defects for several reasons: (1) could serve as a weak point for isolation liner deposition; (2) could be a weak point for barrier/seed layer deposition; and (3) may cause mechanical failures during TSV backside reveal. Previously, silicon fin defects were removed by switching to a non-BMD silicon substrate for interposer application. However, for TSV integration with FEOL devices, the BMD layer serves as an intrinsic gettering layer for devices, therefore, it cannot be removed from the silicon substrate, which makes it challenging to get rid of silicon fin defects. In order to establish a non-destructive in-line detection method of the fin defects, scanning electron microscope (SEM) automatic process inspection (API) was set up to image the fin defects at the bottom of the trench. A special working point with high depth of focus (DoF) and contrast was created to obtain good top-down SEM imaging of the defects at the bottom of this high-aspect-ratio (HAR) structure. Three types of silicon substrates (A, B, and C) were used for this study to investigate the potential root cause. SEM API results show defect rates of 20%, 3.3% and 0% for substrates A, B, and C, respectively. This is in good agreement with both BMD simulation results and benchmarking data in which substrates A, B, and C had normalized BMD densities of 11.7, 5.74, and 1 cm-3, respectively, with a comparable BMD size of 80~90 nm and a denuded zone (DNZ) depth of 10~15 μm. The correlation between BMD density in a silicon substrate and silicon fin defect rate indicates that BMD is a key root cause for silicon fin defects. To eliminate silicon fin defects, an optimized DRIE process has been developed. On the same type of substrate, the DRIE process with a typical voltage bias results in a defect rate of 6.7%, while no silicon fin defect was detected out of 200 TSVs with a polynomial bias ramp to relatively higher final voltage bias during the last 15 μm etch. The hypothesis is that higher voltage bias is able to sputter away BMD and shows potential to get rid of the silicon fin defects at the TSV bottom. In summary, a capable inspection method, a preferred silicon substrate with BMD spec range, and a promising way for DRIE process optimization to eliminate the silicon fin defect at the TSV bottom have been identified and developed in this work. Detailed results and analysis, particularly the fin defect images, statistical inspection results, BMD benchmarking data, simulation results, and TSV profile with optimized process will be discussed in the paper.

The wide variety of crops in the image of agricultural products and the confusion with the surrounding environment information makes it difficult for traditional methods to extract crops accurately and efficiently. In this paper, an automatic extraction algorithm is proposed for crop images based on Mask RCNN. First, the Fruits 360 Dataset label is set with Labelme. Then, the Fruits 360 Dataset is preprocessed. Next, the data are divided into a training set and a test set. Additionally, an improved Mask RCNN network model structure is established using the PyTorch 1.8.1 deep learning framework, and path aggregation and features are added to the network design enhanced functions, optimized region extraction network, and feature pyramid network. The spatial information of the feature map is saved by the bilinear interpolation method in ROIAlign. Finally, the edge accuracy of the segmentation mask is further improved by adding a micro-fully connected layer to the mask branch of the ROI output, employing the Sobel operator to predict the target edge, and adding the edge loss to the loss function. Compared with FCN and Mask RCNN and other image extraction algorithms, the experimental results demonstrate that the improved Mask RCNN algorithm proposed in this paper is better in the precision, Recall, Average precision, Mean Average Precision, and F1 scores of crop image extraction results.

There are many types of large-scale complicated equipments in process industry. Accidents caused by faults of these equipments always lead to serious consequences. Fault diagnosis and root cause tracing based on FMEA and FTA are main ways to prevent abnormal events occurring on these equipments. Different from small equipment, large-scale complicated equipment always has numerous components and complex internal structure. FMEA has weak ability to represent the complex relation between faults of different component. Also, FTA modeling is time intensive and unreliable when equipment structure becomes complicated. According to these problems, this paper proposed a fault root cause tracing method of large-scale complicated equipment based on fault graph (FG). FG is a special kind of fault-oriented directed graph expanded from fault tree, which takes advantage of graph rather than tree to describe complex relation between fault modes of components, and add logic “AND” and “OR” connectives. FG is a useful tool to model the complex structure and fault affecting relation between components of large-scale complicated equipment in process industry, and can be used to trace fault root causes effectively in system failure of equipment. In this method, FMEA is used firstly to extract fault modes and fault affecting relation between components of equipment. The fault modes and their affecting relation are stored in polychromatic matrix. Then the polychromatic matrix of fault modes and their affecting relation is converted into FG automatically by algorithm with several rules. The FG can represent complex fault affecting relation between components of the equipment conveniently. The reachability from the failure nodes of FG is then calculated to obtain those possible fault transfer reachable components. The fault transfer pathways from root causes to failure component is searched based on graph-theoretic methods. An example of FG modeling and fault analysis of turbine which has complex internal structure and is widely used in process industry is introduced. The example proved that the proposed method is effective for fault root cause tracing of large-scale complicated equipment.

In the infrared small target detection, the clutter formed by buildings, trees and protruding clouds is densely distributed and difficult to filter out. The hysteresis threshold detection algorithm utilizes the geometric features of small target to reduce false alarms. Images are filtered in multiple scales, the location and scale of the points of interest are extracted by non-maximum suppression. To determine the connection state of the focus and clutter, local gradient second-order origin moment is proposed to eliminate strong edges. The hysteresis threshold segmentation is performed to exclude stubborn false alarms and detect small targets. Experiments show that the proposed algorithm has a significant effect in removing false alarms, and achieves both the high detection probability and low false alarm probability.

In the complex and changeable actual scene, the identification of rice diseases generally has the problem of small and dense targets. The current method mainly introduces the spatial regularization loss into the Fast R-CNN framework, and integrates the detection and technical tasks into a model. Due to the limitations of this method in the actual scene, a rice disease recognition method based on improved Mask R-CNN is proposed in this paper. This proposed method mainly improves the feature fusion process of the feature pyramid, changes the original top-down path to bottom-up to retain more spatial position information in the shallow feature map, and adds multi-scale dilation convolution in the feature fusion process to increase the receptive field of the feature map and keep the resolution unchanged, which can avoid the loss of features in the down-sampling process; the dataset was established and labeled by using the photographs of rice bacterial blight. The proposed algorithm is compared with the Mask R-CNN algorithm. When the IOU is 0.6, 0.7 and 0.8, the accuracy is increased by 0.46, 0.66 and 0.48 respectively. Experimental results show that the algorithm can get more accurate results on the established dataset.

Overhead transmission lines are prone to various natural disaster and phenomena which lead to system fault. System fault causes tripping which can be due to various root causes such as lightning strikes, tree encroachment, crane encroachment and defect equipment. In this paper, fault characteristics from different root causes are studied and analyzed based on field data extracted from the Digital Fault Recorder (DFR). It is found that different fault types of overhead transmission line have identical fault signatures in terms of current and voltage waveform. Moreover, to quantify the fault signatures, several indices are extracted to identify the root cause of fault. The indices are based on pre-fault and fault event current and voltage waveforms. These findings can be beneficial to enhance the decision making on identifying the root cause of fault.

Multispectral image fusion based on fractal features

Small and dim target detection is a critical challenge in hyperspectral remote sensing, particularly in complex, large-scale scenes where spectral variability across diverse land cover types complicates the detection process. In this paper, we propose a novel target reasoning algorithm named Adaptive Global Dense Nested Reasoning Network (AGDNR). This algorithm integrates spatial, spectral, and domain knowledge to enhance the detection accuracy of small and dim targets in large-scale environments and simultaneously enables reasoning about target categories. The proposed method involves three key innovations. Firstly, we develop a high-dimensional, multi-layer nested U-Net that facilitates cross-layer feature transfer, preserving high-level features of small and dim targets throughout the network. Secondly, we present a novel approach for computing physicochemical parameters, which enhances the spectral characteristics of targets while minimizing environmental interference. Thirdly, we construct a geographic knowledge graph that incorporates both target and environmental information, enabling global target reasoning and more effective detection of small targets across large-scale scenes. Experimental results on three challenging datasets show that our method outperforms state-of-the-art approaches in detection accuracy and achieves successful classification of different small targets. Consequently, the proposed method offers a robust solution for the precise detection of hyperspectral small targets in large-scale scenarios.

Deep learning methods are widely used in the field of rolling bearing fault diagnosis and produce good results when faced with datasets with roughly equal numbers of normal and faulty samples. However, real-world data often has a serious imbalance, with the number of fault samples being significantly less than the number of normal samples. This dataset imbalance challenges the performance of traditional deep learning methods. To address this problem, this paper proposes an efficient imbalanced data rolling bearing fault diagnosis method. The method consists of two parts: a deep learning network based on a multi-scale self-attention mechanism and a novel loss function. In terms of the deep learning network, firstly, the one-dimensional vibration signal is converted into a two-dimensional image through the Gramian angular field. This conversion maximises the inherent feature extraction capability of the network. Subsequently, the multi-scale learning capability of the network is enhanced by implementing different expansion rates for the head of the multi-scale self-attention mechanism. This nuanced approach allows the network to capture the underlying information more efficiently. Finally, the inclusion of Ghost bottlenecks and feature pyramid networks (FPNs) helps to optimise network efficiency and improve generalisation performance. A novel loss function is also proposed to make the method more suitable for imbalanced data. During the training process, the classification of samples in each class is analysed using the recall metric of imbalanced classification and the real-time recall is used as a weight to weaken the dominance of the majority class. This weighting ensures the adaptability of the method to imbalanced datasets. The proposed method is evaluated using rolling bearing datasets from Case Western Reserve University, USA, and Southeast University, China. Comparison results with other state-of-the-art deep learning methods show that the proposed method has a robust performance when dealing with imbalanced data.

A deep autoencoder feature learning method for process pattern recognition

Airport runways, as the core part of airports, belong to vital national infrastructure, and the target detection and segmentation of airport runways in remote sensing images using deep learning methods have significant research value. Most of the existing airport target detection methods based on deep learning rely on horizontal bounding boxes for localization, which often contain irrelevant background information. Moreover, when detecting multiple intersecting airport runways in a single remote sensing image, issues such as false positives and false negatives are apt to occur. To address these challenges, this study proposes an end-to-end remote sensing image airport runway detection and segmentation method based on an improved Mask RCNN (CR-Mask RCNN). The proposed method uses a rotated region generation network instead of a non-rotated region generation network, allowing it to generate rotated bounding boxes that fit the shape of the airport runway more closely, thus avoiding the interference of a large amount of invalid background information brought about by horizontal bounding boxes. Furthermore, the method incorporates an attention mechanism into the backbone feature extraction network to allocate attention to different airport runway feature map scales, which enhances the extraction of local feature information, captures detailed information more effectively, and reduces issues of false positives and false negatives when detecting airport runway targets. The results indicate that, when comparing horizontal bounding boxes with rotated bounding boxes for detecting and segmenting airport runways, the latter are more precise for complex backgrounds. Furthermore, incorporating an attention mechanism enhances the accuracy of airport runway recognition, making it highly effective and practical.