Edge-Aware Multi-Level Interactive Network for Salient Object Detection of Strip Steel Surface Defects

With the development of productivity, people set higher demands on the quality of steel. In recent years, artificial intelligence, especially deep learning-based computer vision technology has attracted great attention, which can be used for detecting steel surface defects such as scratches, patches, and rust spots. However, due to the complexity of the strip steel surface, it is still a challenge to accurately and effectively detect defect regions by the existing defects detection methods. Therefore, we propose a unique saliency model, i.e., deeper feature integration network to highlight the defect regions on the strip steel surface. To be specific, after each encoder stage, we introduce the multiscale global feature extraction module to elevate the multiscale deep features from the encoder. Meanwhile, we deploy the deeper feature extraction module, which contains a bidirectional feature extraction unit, to dig the effective representation for defects. Particularly, the forward one is equipped with a channel-space weighted module and the backward one is equipped with a split attention module. After that, the features from the two branches are progressively integrated by the decoder, yielding the final high-quality saliency maps that give a good depiction of the strip steel surface defects. Extensive experiments are executed on the public dataset, and the comparison results show that our model performs better than the 15 state-of-the-art methods, which proves the effectiveness and superiority of the proposed model.

Surface defect detection is the task of identifying and localizing defects on the surface of an object, which is a widely applied task in various industries. In the logistics industry, logistics companies need to monitor the condition of goods for potential defects throughout the entire logistics process for effective logistics quality control. However, effective defect detection methods are still lacking for courier packages using corrugated cardboard boxes, which rely on judging whether deformation and leakage have occurred by examining areas on their surface with abundant texture. Specifically, the defect rate and supporting structure of the packages are influenced by temperature and humidity, and the openings and bends of defects are inconsistent. This results in defective packages having rich and non-uniform texture features. Moreover, convolutional neural networks struggle to effectively extract low-level semantic texture features of defects and perceive multi-level image features of packages. Considering the above challenges, we propose a novel texture prior-aware multi-level feature fusion network (TPMN). We first introduce prior knowledge and attention mechanisms to enable the neural network to focus on extracting low-level texture features from the image in the early stages. We also design a multi-level feature fusion method to integrate features from different levels, avoiding the gradual loss of low-level semantic information in CNN and enabling comprehensive perception of multi-level image features. To support further research, we contribute the cardboard-boxes-dataset, comprising 1210 images of packages. Experiments on this dataset showcase the superior performance of TPMN, even in few-shot learning scenarios, demonstrating its effectiveness in surface defect detection within the logistics and supply chain domains.

Many existing salient object detection methods are dedicated to fusing features from different levels of a pre-trained convolutional neural network (CNN). However, these methods can easily lead to internal discontinuities within the salient objects because of unreasonable feature fusion strategies and short-range dependencies resulting from common convolution and pooling operations. In this paper, we propose a novel non-local duplicate pooling (NLDP) network to overcome these internal discontinuities. NLDP begins by removing the first few convolutional layers of a classic CNN, which have small receptive fields and require large amounts of calculation. A novel duplicate pooling module (DPM) is then used to generate richer and more detailed saliency maps. This is achieved by constructing a double-pathway that can integrating partial feature maps. Within the DPM, a non-local module (NLM) is used to obtain long-range dependencies. This enhances the internal continuities between the saliency maps. Comprehensive experiments conducted on six benchmark datasets have confirmed the increased effectiveness and detection speed of our method in relation to other salient object detection methods.

Detection of surface defects in strip steel is crucial in industrial automation to ensure steel quality. Recently, learning-based detection methods have gained prominence among researchers for their robust capabilities. However, accurately delineating clear area boundaries remains a challenge, particularly in the complex backgrounds of steel surfaces. In this paper, we propose the edge-aware Interactive Refinement Network (EAIRNet) to address this challenge. EAIRNet comprises three core modules: a Multiscale Feature Extractor, a Sal-Edge Interactive Module, and a regional attention guided module (RAGM). Firstly, our network extracts multiscale features using ResNet from the steel surface environment. Subsequently, features from encoder’s stage 3–5 are integrated to derive edge features. The SEIM facilitates interaction between salient and edge features in the decoder stage, thereby enhancing the network’s focus on defect area boundaries and refining detection precision. Additionally, we introduce the RAGM to improve the focus of shallow decoder layers on defect areas. Our EAIRNet’s effectiveness is proven through comprehensive experimental results on the public dataset named SD-Saliency-900, in comparison to state-of-the-art methods.

DFN-PSAN: Multi-level deep information feature fusion extraction network for interpretable plant disease classification

Event Abstract Back to Event Can state-of-the-art saliency systems model infant gazing behavior in tutoring situations? Vikram Tadmeri Narayan1*, Katrin S. Lohan1, Marko Tscherepanow1, Katharina Rohlfing1 and Britta Wrede1 1 Bielefeld University, Applied Informatics Group, CoR-Lab, Germany I. I NTRODUCTION The behavior for a humanoid robot is often modeled in accordance with human behavior. Current research suggests that analyzing infant behavior as a basis for designing the robot behavior can guide us to a natural robot interface. Based on this idea many researchers support saliency systems as a bottom-up inspired way to simulate infant-like gazing behavior. In the field of saliency systems many different approaches have proposed and quantified in terms of speed, quality and other technical issues. But so far, no one compared and quantified them in terms of natural infant tutor interaction. The question we would like to address in this paper is: Can state-of-the-art saliency systems model infant gazing behavior in tutoring situations? By addressing these issues we want to take a step towards an autonomous robot system, which could be used more natural interaction experiments in future. II. M ETHODS FOR A NALYZING We compared 7 different saliency systems with the gazing behavior of infants between the age of 8-11 months. The infant’s gazing behavior and the interaction structure of the tutoring situation was manually annotated. Based on this annotation we selected images as input for the saliency systems. The interaction data used in the course of this research is described in the following subsections. The employed saliency systems are briefly described in subsection II-B. A. Parent-child interaction Out of the well-known Bielefeld-motionese corpus [8], we focus on the youngest participants comprising 12 families of 8 to 11 months old children to analyze their gazing behavior, as the main feedback and controlling capabilities of these infants is based on the gazing behavior [13]. We selected the stacking cups task, because it was often analyzed [7],[12] and the interaction is very detailed analysed. We selected several pictures based on an action segmentation. The pictures represent the beginning and ending point of the stacking cups task. We have chosen this points because at these points the scene is changing. Setting: Parents were instructed to demonstrate a stacking cups task to an interaction partner. The first interaction partner was their infant, and another adult constituted the second. Fig. 1 illustrates the top-view of the experimental setup, and shows sample image frames of cameras which were set behind the parent and the interaction partner and focused on each of them. The stacking cups task was to sequentially pick up the green (a1), the yellow (a2), and the red (a3) cup and put them into the blue one on the white tray. B. Saliency Systems Frequency tuned saliency model [1] is perhaps the simplest of all the existing saliency systems. The absolute difference of each pixel to the image mean is accounted as its saliency value. They recommend to decompose a given input image into L*, a*, b* color space and perform the aforementioned operation on each of the color plane and fuse the results to compute the final saliency map. Graph-based visual saliency models [4] envisages the input image as a complete graph with each pixel as its nodes. A Weber’s [3] law based dissimilarity measure is employed to compute the dissimilarity between any two given pixels. A normalizing function is further employed to compute the final saliency map. Experiments carried out in [4], [10] have shown that the method has high performance in correlating with human eye gaze and also for object segmentation. The Multi resolution saliency map [5] is perhaps the most cited saliency system to date. The authors propose a computational model for the theoretical framework presented by Koch and Ulman [6] to model visual attention. An input image is analyzed with opponent color maps, pixels gradient and orientation at different scale spaces. A winner take all (WTA) network fuses these multiple maps into a single saliency map. Local steering kernel-based saliency [9] is a based on the center-surround paradigm, which is employed to compute the saliency of a selected region. They propose a novel local steering kernel which is employed to compute the similarity between a center region and the surrounding patches. The methodology is inherently robust to image brightness and contrast changes, has very few parameters that requires to be fined-tuned. Symmetry based Saliency [2] is compute rectangular regions of interest centered on a pixel and compute first order moments of features to calculate the saliency of the given pixel. The methodology is among the most best in terms of programming simplicity. The authors also claim the biological plausibility of the model. Random center-surround pattern based Saliency [11] is a methodology that employs a biologically plausible dissimilarity metric which is employed to compute the contrast between any two random pixels on the input image. The contrasts are updated and normalized to generate the final saliency map. The said methodology is shown to have state-of the-art performance in the task of salient region detection. For the Random rectangular regions of interest based saliency [10], the same authors as in [11] propose to compute frequent- tuned salient region detection [1], on random rectangular regions of an image for a large number of times and then sum them to compute the final saliency map. Such a formulation is shown to have excellent correlation with human eye-gaze and has good performance for the task of salient regions as shown in their experiments. The methodology also has only two parameters which requires fine tuning, which reduces the implementation complexity. III. DATA The beginning and concluding snapshots of 12 motionese videos were cropped, and were parsed into the seven aforementioned saliency systems. A 15 x 15 region centered on the maximally salient point was chosen as the focus of attention. see Fig. 1. Results: Results are summarized in Fig. 2. The percentages describes the matching accuracy of a given saliency system to the child’s eye gazing behavior. It seems that the methodology of Seo and Milanfar [9] has the best matching with the gazing behavior of a 8-11 month old child. Figure 1 Figure 2 Acknowledgements Tadmeri Narayan Vikram gratefully acknowledges the financial support from the EU FP7 Marie Curie ITN RobotDoc. Contract No. 235065. Katrin S. Lohan, Katharina Rohlfing, and Britta Wrede gratefully acknowledge the financial support from the FP7 European Project ITALK.

This article proposes a bottom-up visual saliency model that uses the wavelet transform to conduct multiscale analysis and computation in the frequency domain. First, we compute the multiscale magnitude spectra by performing a wavelet transform to decompose the magnitude spectrum of the discrete cosine coefficients of an input image. Next, we obtain multiple saliency maps of different spatial scales through an inverse transformation from the frequency domain to the spatial domain, which utilizes the discrete cosine magnitude spectra after multiscale wavelet decomposition. Then, we employ an evaluation function to automatically select the two best multiscale saliency maps. A final saliency map is generated via an adaptive integration of the two selected multiscale saliency maps. The proposed model is fast, efficient, and can simultaneously detect salient regions or objects of different sizes. It outperforms state-of-the-art bottom-up saliency approaches in the experiments of psychophysical consistency, eye fixation prediction, and saliency detection for natural images. In addition, the proposed model is applied to automatic ship detection in optical satellite images. Ship detection tests on satellite data of visual optical spectrum not only demonstrate our saliency model's effectiveness in detecting small and large salient targets but also verify its robustness against various sea background disturbances.

Given that existing salient object detection methods cannot effectively predict the fine contours of salient objects when extracting local or global contexts and features, we propose a novel contour self-compensated network (CSCNet) to generate a more accurate saliency map with complete contour. Unlike the common binary saliency detection, we reconstruct the salient object detection problem into a multi-classification problem of the background, the salient object, and the salient contour, where the salient contour is used as the third label for ground truth. Meanwhile, the image and its superpixel map are concatenated as the input of our network to add more edge information. Also, a penalty loss is proposed to restrict the spatial relationship between the background, objects, and their contours. Experimentally, we evaluate the proposed CSCNet on six benchmark datasets in both accuracy and efficiency and evaluate the attribute-based performance on the SOC dataset. Compared with 13 state-of-the-art algorithms, our CSCNet can detect salient objects more accurately and completely without adding too many convolutional layers and parameters.

Internal defects detection and classification in hollow cylindrical surfaces using single shot detection and MobileNet

For the detection of wood surface defects, a convolutional neural network has a low detection efficiency and insufficient generalization ability, so it does not meet the requirements of online detection. Aiming to solve the above problems, the YOLOv3 baseline model, which has the advantage of multi-objective dynamic detection, was improved and applied to the online detection of wood surface defects. To solve the problem of the poor generalization ability of the network, GridMask was used to enhance the data and improve the robustness of the network. In order to solve the problem of the considerable amount of network parameter calculations and insufficient real-time performance, the residual block of the backbone network was changed to a Ghost block structure to achieve a lightweight model. Finally, the confidence loss function of the network was improved to reduce the influence of simple samples and negative samples on model convergence. The experimental results showed that, compared with the original network, the improved algorithm increased the mean average precision by 5.73% and the detection speed was increased to 28 frames per second (an increase of 11), which met the requirements for real-time industrial detection.

Surface-mounted technology (SMT) welding defect detection plays a key role in the printed circuit board assembly (PCBA) production process, which affects the use of electronic products and cost. Previous works tend to realize defect detection with only defect samples and they assume that there are sufficient defect samples. However, defect samples are usually difficult to collect in real-life scenarios while enough template samples can be easily obtained. In addition, most existing works carry out defect detection based on benchmarks with simple backgrounds of PCBA, which is not suitable for PCBA with complex structures in modern electronic product manufacturing. To address the above issues, we propose a template-guidance Siamese network based on YOLO for SMT welding defect detection (TGSYOLO), which is deployed on a real SMT automatic optical inspection (AOI) system. First, the two-stream structure is introduced to extract deep features in defect images and template images, in which template features serve as guidance knowledge. Then, a template fusion Transformer (TFT) is proposed to model global features between detect and template features in the low-level stage, which could acquire long-range correlations to force the network to focus on potential defect regions. Next, to avoid the disappearance of tiny defect features during deep feature fusion, a multiscale attention feature pyramid network (MAFPN) is proposed to directly fuse defect semantic information from low-level features, which retains detailed expressions of defects through skip connection and obtains compact fusion features. Furthermore, we collect limited welding defect samples based on more complex PCBA backgrounds than previous works through a real SMT AOI system. Experiments on the limited dataset show that TGSYOLO could reach 0.985 of mAP@0.5, 0.885 of mAP@0.75, and 0.984 of F1, which is 0.008, 0.054, and 0.025 higher than other SOTA methods. Also, generalization experiments on the public DeepPCB show that TGSYOLO could still reach the best with 0.991 of mAP@0.5 and 0.89 of mAP@0.75, which proves that TGSYOLO has good generalization performance.

Sanitary ceramic products, such as toilet and wash basin, are widely used in our daily life. Sanitary ceramics are expected to have some excellent physical properties, such as corrosion resistance, easy cleaning, and low water absorption. However, surface defects in sanitary ceramics are inevitable due to complex production processes and changing production environment. Therefore, surface defect detection must be performed in the manufacturing process of sanitary ceramics. There are many types of surface defects in sanitary ceramics, and different types of defects have large differences in characteristics and scales. Traditional detection methods with artificially designed features and classifiers are difficult to apply in this context. In addition, there are few studies on surface defect detection methods of sanitary ceramics based on deep neural networks. In this article, a lightweight real-time defect detection network based on the lightweight backbone MobileNetV3 is presented. The proposed network achieves multi-scale detection of surface defects in sanitary ceramics with a multi-layer feature pyramid. Combining region proposal network and anchor-free method, real-time defect detection is achieved. Finally, a detection head with channel attention structure and a low-level mixed feature classification strategy is used to perform defect classification with higher accuracy. Experimental results show that the proposed approach achieves at least 22.9% detection speed improvement and 35.0% average precision improvement while reducing memory consumption by at least 8.4% compared with the classic one-stage SSD, YOLO V3 and two-stage Faster R-CNN methods.

A pooling-based feature pyramid network for salient object detection

In this work, the creation of a dataset labeled in a pixel-wise manner for the uncommon domain of stain detection on patterned laundry is described. The unique properties of images in this dataset—stains are small and sometimes occur in large amounts—led to the creation of noisy labels. Indeed, the training of a fully convolutional neural network for salient object detection with this dataset revealed that the model predicts stains missed by human labelers. Thus, the reduction in label noise by adding overlooked regions with the help of the model’s predictions is examined in two different experiments. In the model-assisted labeling experiment, a simulation is ran where a human selects correct regions from the predictions. In the self-training experiment, regions of high certainty are automatically selected from the predictions. Re-training the model with the revised labels shows that model-assisted labeling leads to an average improvement in performance by 8.52%. In contrast, with self-training, the performance increase is generally lower (2.58% on average) and a decrease is even possible since regions of high certainty are often false positives.

In this paper, we propose a detection method for salient objects whose eyes are focused on gaze tracking; this method does not require a device in a single image. A network was constructed using Neg-Region Attention (NRA), which predicts objects with a concentrated line of sight using deep learning techniques. The existing deep learning-based method has an autoencoder structure, which causes feature loss during the encoding process of compressing and extracting features from the image and the decoding process of expanding and restoring. As a result, a feature loss occurs in the area of the object from the detection results, or another area is detected as an object. The proposed method, that is, NRA, can be used for reducing feature loss and emphasizing object areas with encoders. After separating positive and negative regions using the exponential linear unit activation function, converted attention was performed for each region. The attention method provided without using the backbone network emphasized the object area and suppressed the background area. In the experimental results, the proposed method showed higher detection results than the conventional methods.