Surface Defects Classification Using Transfer Learning and Deep Sparse Coding

Microscopic laser engraving surface defect classification plays an important role in the industrial quality inspection field. The key challenges of accurate surface defect classification are the complete description of the defect and the correct distinction into categories in the feature space. Traditional classification methods focus on the terms of feature extraction and independent classification; therefore, feed handcrafted features may result in useful feature loss. In recent years, convolutional neural networks (CNNs) have achieved excellent results in image classification tasks with the development of deep learning. Deep convolutional networks integrate feature extraction and classification into self-learning, but require large datasets. The training datasets for microscopic laser engraving image classification are small; therefore, we used pre-trained CNN models and applied two fine-tuning strategies. Transfer learning proved to perform well even on small future datasets. The proposed method was evaluated on the datasets consisting of 1986 laser engraving images captured by a metallographic microscope and annotated by experienced staff. Because handcrafted features were not used, our method is more robust and achieves better results than traditional classification methods. Under five-fold-validation, the average accuracy of the best model based on DenseNet121 is 96.72%.

Quality control in metal product manufacturing relies heavily on accurately detecting and classifying surface defects through visual inspection. Recently, convolutional neural networks (CNNs) have shown promising results in automating this process with high accuracy. This research paper proposes a new (experimental version) Lite Convolutional Neural Network (LCNN) designed to analyze image data to detect and classify surface defects on metallic surfaces. Our model was trained on a metal surface defects dataset comprising 1800 images of six different types of surface defects. Despite using relatively small datasets, the proposed LCNN version achieves a classification accuracy of 91.67%, highlighting its effectiveness in real-world defect detection scenarios.

Defect recognition plays an important part in the health monitoring of in-service equipment. Surface defects and sub-surface defects of key components have different effects on the safety of the equipment. Sub-surface defects are difficult to be identified, and it is more likely to cause unpredictable damage than surface defects. However, the existing characteristic cannot accurately identify the defect type. Therefore, it is critical to find features that are useful for defect classification. The fusion of multiple feature values facilitates defect classification. For obtaining multi-feature fusion eigenvalues, this article introduces a fusion matrix based on Fisher's discriminant criterion and correlation analysis, which realizes the fusion of two dimensions of the whole and the local and different aspects. The BP neural network classifier is then used to discriminate and classify surface defects and sub-surface defects. The recognition accuracy of surface defects and sub-surface defects can reach 98.1%. The classification accuracy of this approach is significantly improved compared with the method based on a single feature value and other state-of-the-art methods. The approach can also be used to identify defects with different cross-sectional shapes, indicating that this approach is suited for nature defects.

The quality of smooth technical surfaces is characterized by the surface roughness and by surface defects. For a stable quality control, objective measuring conditions are necessary. The objective measurement of surface defects especially is an unsolved problem. For the investigation of the scattering characteristics of different surface defects, a measuring device was developed. With this measuring device, the type of defect and the angle dependent scattering intensity were measured and compared with theoretical calculations.

Application of Shearlet transform to classification of surface defects for metals

FaNet: Feature-aware network for few shot classification of strip steel surface defects

Surface defect identification is a crucial task in many manufacturing systems, including automotive, aircraft, steel rolling, and precast concrete. Although image-based surface defect identification methods have been proposed, these methods usually have two limitations: images may lose partial information, such as depths of surface defects, and their precision is vulnerable to many factors, such as the inspection angle, light, color, noise, etc. Given that a three-dimensional (3D) point cloud can precisely represent the multidimensional structure of surface defects, we aim to detect and classify surface defects using a 3D point cloud. This has two major challenges: (i) the defects are often sparsely distributed over the surface, which makes their features prone to be hidden by the normal surface and (ii) different permutations and transformations of 3D point cloud may represent the same surface, so the proposed model needs to be permutation and transformation invariant. In this paper, a two-step surface defect identification approach is developed to investigate the defects’ patterns in 3D point cloud data. The proposed approach consists of an unsupervised method for defect detection and a multi-view deep learning model for defect classification, which can keep track of the features from both defective and non-defective regions. We prove that the proposed approach is invariant to different permutations and transformations. Two case studies are conducted for defect identification on the surfaces of synthetic aircraft fuselage and the real precast concrete specimen, respectively. The results show that our approach receives the best defect detection and classification accuracy compared with other benchmark methods.

Real-time quality inspection of capsules manufacturing in pharmaceutical applications is an important issue from the point of view of industry productivity, competitiveness and quality aspect of the product. Pharmaceutical products are susceptible to several common flows like incorrect size or color, surface defect, missing, broken capsules. To guarantee every capsule is free of defects, each capsule must be inspected individually. In this paper we have compared different approaches of image processing for detection of defective capsule and presence of category of defects. All (sectorization of DFT transformed images for feature extraction, object counting, gray intensity distribution, color intensity distribution, GLCM, area counting and individual area calculation) methods proposed and compared in this paper are applied over database of 39 images of pharmaceutical capsule strips speeded over 3 different classes (single color, double color and Multi color). The experiment has been carried out to detect 5 different possible types of defects i.e. missing, broken, missing and broken, improper alignment and surface defect. The attempt has also been made to indicate the count of number of missing tablets/capsules in the strip. Overall Average and sum of area is calculated for the performance evaluation to detect the presence of defects. The range of differences obtained among the defective capsule images and defect free image has been analyzed to classify the category of defects present in the image. It has been observed that individual object area calculation, gray intensity density calculation and GLCM works better compared to all other approaches.

Cylinder liner is an internal part of the automobile engine, which plays an important role in the automobile internal combustion engine. Therefore, it is a top priority to accurately and quickly detect the cylinder liner surface defects. In order to effectively achieve the classification and localization of surface defects on the cylinder liner, this paper establishes a dataset for surface defects on cylinder liner and proposes a based on improved YOLOv5 algorithm for detecting surface defects on cylinder liner. Firstly, a machine vision system is established to acquire on-site images and perform manual annotation to build the dataset of surface defects on cylinder liner. Secondly, the GSConv SlimNeck mechanism is introduced to reduce the model complexity; the Bi-directional Feature Pyramid Network (BiFPN) is used to fuse the feature information at different scales to enhance the detection accuracy of small surface defects on cylinder liner; and embedding the SimAM attention mechanism to focus on the object region of interest and improve the accuracy and robustness of the model. The final improved YOLOv5 model reduces the number of model parameters by 15.8% compared to the non-improved YOLOv5. And the experimental results on our self-built dataset for cylinder liner defects show that the mAP0.5 is improved by 0.4%. This means that the accuracy of model detection was not compromised. This method can be applied to actual production processes.

Surface defects on metallic and non-metallic components can have serious effects on the reliability of the complete system. Typical examples are pistons with surface defects that cause scratching of the cylinder surface. For this reason it is of utmost importance to detect such defects as early as possible in the production process. So far, no real time solutions exist that fully satisfy industrial requirements in terms of speed, accuracy and reliability. Hence it is common to use visual inspection by humans which is error prone and causes additional personal costs. In this paper a new approach to obtain real time behaviour of industrial image processing systems by using multivariate techniques is presented. This methodology is originally used in chemometrics for statistical evaluation of measurement data and is now applied to image processing to take advantage of the high numerical efficiency of the underlying mathematics. Multivariate techniques can be applied to both the problem of automatic identification and classification of surface defects with digital images. The key to the envisaged real time ability is the high numerical efficiency of the proposed multivariate method. It manages defect detection with vector/matrix multiplication only - no calculation of powers or exponential functions is required. This enables efficient real time implementations on DSP platforms which are profiled for this type of calculations.

Sparse coding-based transfer learning for energy disaggregation

In this study, using various text prompts that combine objects and defect types, we establish a semantic space linking images and texts, enabling explainable defect predictions using natural language. We introduce contrastive language–image pre-training-based multimodal defect classification (CLIP-MDC), a framework designed for multimodal defect detection and classification in smart manufacturing. The model integrates a lightweight backbone network with contrastive language–image pre-training (CLIP) encoders to perform both pixel-level anomaly segmentation and image-level defect classification effectively in supervised and weakly supervised settings. Additionally, we incorporate a Perlin noise-based synthetic anomaly generation technique to facilitate learning in environments with limited labeled data, and the dual prediction architecture enables accurate simultaneous inference of defect location and type. In experiments on the MVTec AD and KSDD2 datasets, the model achieved outstanding performance with an area under the receiver operating characteristic curve (AUROC) of 99.9%, an area under the per-region overlap curve (AUPRO) of 98.6%, a pixel-level AUROC (P-AUROC) of 99.9%, and an average precision for localization ( $$AP_{loc}$$ ) of 87.6%. It also demonstrated real-time capability, registering an average inference speed of 6.6ms on an A100 GPU. CLIP–MDC uses a semantic-based multimodal learning framework that combines visual and linguistic information to deliver accuracy, explainability, generalization, and real-time efficiency in defect detection, making it a practical and scalable solution for industrial defect analysis in real-world manufacturing environments.

For the problem of classification and identification of defects in polyethylene (PE) gas pipelines, this paper firstly performs preliminary screening of the acquired images and acquisition efficiency of defective image acquisition was improved. Images of defective PE gas pipelines were pre-processed. Then, edge detection of the defective images was performed using the improved Sobel algorithm and an adaptive threshold segmentation method was applied to segment the defects in the pipeline images. Finally, the defect images were morphologically processed to obtain binary images. The obtained binary images were applied with VGG16 to complete the training of the defect classifier. The experimental findings show that in the TensorFlow API environment, the test set’s highest accuracy reached 97%, which can achieve the identification of defect types of underground PE gas transmission pipelines.

Sanitary ceramic products, such as toilet and wash basin, are widely used in our daily life. Sanitary ceramics are expected to have some excellent physical properties, such as corrosion resistance, easy cleaning, and low water absorption. However, surface defects in sanitary ceramics are inevitable due to complex production processes and changing production environment. Therefore, surface defect detection must be performed in the manufacturing process of sanitary ceramics. There are many types of surface defects in sanitary ceramics, and different types of defects have large differences in characteristics and scales. Traditional detection methods with artificially designed features and classifiers are difficult to apply in this context. In addition, there are few studies on surface defect detection methods of sanitary ceramics based on deep neural networks. In this article, a lightweight real-time defect detection network based on the lightweight backbone MobileNetV3 is presented. The proposed network achieves multi-scale detection of surface defects in sanitary ceramics with a multi-layer feature pyramid. Combining region proposal network and anchor-free method, real-time defect detection is achieved. Finally, a detection head with channel attention structure and a low-level mixed feature classification strategy is used to perform defect classification with higher accuracy. Experimental results show that the proposed approach achieves at least 22.9% detection speed improvement and 35.0% average precision improvement while reducing memory consumption by at least 8.4% compared with the classic one-stage SSD, YOLO V3 and two-stage Faster R-CNN methods.

This paper addresses the problem of energy disaggregation/non-intrusive load monitoring. It introduces a new method based on the transform learning formulation. Several recent techniques, such as discriminative sparse coding, powerlet disaggregation, and deep sparse coding, are based on the synthesis dictionary learning/sparse coding approach; ours is based on its analysis equivalent. The theoretical advantage of analysis dictionary compared to its synthesis counterpart is that the former can learn from fewer training s--this has implications in reducing the cost of energy disaggregation. Experiments have been carried out on two benchmark data sets-REDD and Dataport (Pecan Street). Comparison has been done with factorial HMM, discriminative sparse coding, powerlet disaggregation, and deep sparse coding. In the low training data regime, our method always excels over the others.