Detection and classification of metal surface defects using lite convolutional neural network (LCNN)

Steel is one of the most important building materials of our time, and the process of producing flat plates is complex. Before steel is shipped or delivered, the sheets must undergo a thorough inspection procedure to avoid defects. Identification and classification of rolled metal surface defects are one of the main tasks for the correct evaluation of product quality. This work aims to develop a method for recognizing and classifying defects of metal surfaces by their images in real-time. The algorithm is aimed at improving production standards and process efficiency. In this paper, Deep Learning (DL) and Computer Vision (CV) techniques are used to solve the problem of defect detection on the steel surface sheets. Convolutional Neural Network (CNN) architectures are compared, and various steel defects are detected and recognized. The result of this work is a comparative analysis of DL models and the choice of an algorithm designed to search and classify defects in real-time. The use of one CNN model can make it possible to create a tool that greatly facilitates the work of a person.

Defect classification is an important part of automated surface defect detection systems. Generally, the types of surface defects of industrial products are complex and diverse, and the correct defect type classification can help to subsequently extract the characteristic of defects. Moreover, defect classification can help to count the number of defect modes automatically and provide data support for the precise maintenance of product production. Due to the large number of surface defects and the small difference between defect types, traditional classification methods are difficult to classify defect accurately. Therefore, in order to improve the accuracy of defect image classification, this paper proposes a defect image classification method based on transfer learning and sparse coding. Firstly, a deep CNN feature extraction algorithm for defect images is proposed in combination with transfer learning. Then, the deep CNN features of the defect image are dimension-reduced and sparsely optimized using the sparse coding techniques, and the sparse CNN features are obtained. Finally, the sparse CNN features are classified to realize the defect type determination using the linear SVM. The accuracy of the proposed method is verified by using a steel surface defect image benchmark database, and the effectiveness of the proposed method is proved.

Sanitary ceramic products, such as toilet and wash basin, are widely used in our daily life. Sanitary ceramics are expected to have some excellent physical properties, such as corrosion resistance, easy cleaning, and low water absorption. However, surface defects in sanitary ceramics are inevitable due to complex production processes and changing production environment. Therefore, surface defect detection must be performed in the manufacturing process of sanitary ceramics. There are many types of surface defects in sanitary ceramics, and different types of defects have large differences in characteristics and scales. Traditional detection methods with artificially designed features and classifiers are difficult to apply in this context. In addition, there are few studies on surface defect detection methods of sanitary ceramics based on deep neural networks. In this article, a lightweight real-time defect detection network based on the lightweight backbone MobileNetV3 is presented. The proposed network achieves multi-scale detection of surface defects in sanitary ceramics with a multi-layer feature pyramid. Combining region proposal network and anchor-free method, real-time defect detection is achieved. Finally, a detection head with channel attention structure and a low-level mixed feature classification strategy is used to perform defect classification with higher accuracy. Experimental results show that the proposed approach achieves at least 22.9% detection speed improvement and 35.0% average precision improvement while reducing memory consumption by at least 8.4% compared with the classic one-stage SSD, YOLO V3 and two-stage Faster R-CNN methods.

A nondestructive testing method for detecting surface defects of Si3N4-Bearing cylindrical rollers based on an optimized convolutional neural network

Microscopic laser engraving surface defect classification plays an important role in the industrial quality inspection field. The key challenges of accurate surface defect classification are the complete description of the defect and the correct distinction into categories in the feature space. Traditional classification methods focus on the terms of feature extraction and independent classification; therefore, feed handcrafted features may result in useful feature loss. In recent years, convolutional neural networks (CNNs) have achieved excellent results in image classification tasks with the development of deep learning. Deep convolutional networks integrate feature extraction and classification into self-learning, but require large datasets. The training datasets for microscopic laser engraving image classification are small; therefore, we used pre-trained CNN models and applied two fine-tuning strategies. Transfer learning proved to perform well even on small future datasets. The proposed method was evaluated on the datasets consisting of 1986 laser engraving images captured by a metallographic microscope and annotated by experienced staff. Because handcrafted features were not used, our method is more robust and achieves better results than traditional classification methods. Under five-fold-validation, the average accuracy of the best model based on DenseNet121 is 96.72%.

Using deep learning to identify the depth of metal surface defects with narrowband SAW signals

Metal surface defect segmentation can play an important role in dealing with the issue of quality control during the production and manufacturing stages. There are still two major challenges in industrial applications. One is the case that the number of metal surface defect samples is severely insufficient, and the other is that the most existing algorithms can only be used for specific surface defects and it is difficult to generalize to other metal surfaces. In this work, a theory of few-shot metal generic surface defect segmentation is introduced to solve these challenges. Simultaneously, the Triplet-Graph Reasoning Network (TGRNet) and a novel dataset Surface Defects- 4 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sup> are proposed to achieve this theory. In our TGRNet, the surface defect triplet (including triplet encoder and trip loss) is proposed and is used to segment background and defect area, respectively. Through triplet, the few-shot metal surface defect segmentation problem is transformed into few-shot semantic segmentation problem of defect area and background area. For few-shot semantic segmentation, we propose a method of multi-graph reasoning to explore the similarity relationship between different images. And to improve segmentation performance in the industrial scene, an adaptive auxiliary prediction module is proposed. For Surface Defects- 4 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sup> , it includes multiple categories of metal surface defect images to verify the generalization performance of our TGRNet and adds the nonmetal categories (leather and tile) as extensions. Through extensive comparative experiments and ablation experiments, it is proved that our architecture can achieve state-of-the-art results.

Surface defects on metallic and non-metallic components can have serious effects on the reliability of the complete system. Typical examples are pistons with surface defects that cause scratching of the cylinder surface. For this reason it is of utmost importance to detect such defects as early as possible in the production process. So far, no real time solutions exist that fully satisfy industrial requirements in terms of speed, accuracy and reliability. Hence it is common to use visual inspection by humans which is error prone and causes additional personal costs. In this paper a new approach to obtain real time behaviour of industrial image processing systems by using multivariate techniques is presented. This methodology is originally used in chemometrics for statistical evaluation of measurement data and is now applied to image processing to take advantage of the high numerical efficiency of the underlying mathematics. Multivariate techniques can be applied to both the problem of automatic identification and classification of surface defects with digital images. The key to the envisaged real time ability is the high numerical efficiency of the proposed multivariate method. It manages defect detection with vector/matrix multiplication only - no calculation of powers or exponential functions is required. This enables efficient real time implementations on DSP platforms which are profiled for this type of calculations.

The quality of smooth technical surfaces is characterized by the surface roughness and by surface defects. For a stable quality control, objective measuring conditions are necessary. The objective measurement of surface defects especially is an unsolved problem. For the investigation of the scattering characteristics of different surface defects, a measuring device was developed. With this measuring device, the type of defect and the angle dependent scattering intensity were measured and compared with theoretical calculations.

Machine vision systems with effective image processing methods are used in quality grading of agricultural products. A pattern recognition technique was developed to detect and classify surface defects such as pitting, splitting and stem-end rot found in images of mandarin fruits. The developed technique employs fuzzy thresholding for image segmentation, binary wavelet transform (BWT) for feature extraction and a rule based linear classifier model for detection and classification of the defects. The moment invariants computed from the detail subimage of BWT were taken as feature values. This paper in detail describes about the pattern recognition algorithm and its implementation. The detection and classification results obtained from the algorithm are reported and discussed.

Internal defects detection and classification in hollow cylindrical surfaces using single shot detection and MobileNet

Surface defect identification is a crucial task in many manufacturing systems, including automotive, aircraft, steel rolling, and precast concrete. Although image-based surface defect identification methods have been proposed, these methods usually have two limitations: images may lose partial information, such as depths of surface defects, and their precision is vulnerable to many factors, such as the inspection angle, light, color, noise, etc. Given that a three-dimensional (3D) point cloud can precisely represent the multidimensional structure of surface defects, we aim to detect and classify surface defects using a 3D point cloud. This has two major challenges: (i) the defects are often sparsely distributed over the surface, which makes their features prone to be hidden by the normal surface and (ii) different permutations and transformations of 3D point cloud may represent the same surface, so the proposed model needs to be permutation and transformation invariant. In this paper, a two-step surface defect identification approach is developed to investigate the defects’ patterns in 3D point cloud data. The proposed approach consists of an unsupervised method for defect detection and a multi-view deep learning model for defect classification, which can keep track of the features from both defective and non-defective regions. We prove that the proposed approach is invariant to different permutations and transformations. Two case studies are conducted for defect identification on the surfaces of synthetic aircraft fuselage and the real precast concrete specimen, respectively. The results show that our approach receives the best defect detection and classification accuracy compared with other benchmark methods.

Defect recognition plays an important part in the health monitoring of in-service equipment. Surface defects and sub-surface defects of key components have different effects on the safety of the equipment. Sub-surface defects are difficult to be identified, and it is more likely to cause unpredictable damage than surface defects. However, the existing characteristic cannot accurately identify the defect type. Therefore, it is critical to find features that are useful for defect classification. The fusion of multiple feature values facilitates defect classification. For obtaining multi-feature fusion eigenvalues, this article introduces a fusion matrix based on Fisher's discriminant criterion and correlation analysis, which realizes the fusion of two dimensions of the whole and the local and different aspects. The BP neural network classifier is then used to discriminate and classify surface defects and sub-surface defects. The recognition accuracy of surface defects and sub-surface defects can reach 98.1%. The classification accuracy of this approach is significantly improved compared with the method based on a single feature value and other state-of-the-art methods. The approach can also be used to identify defects with different cross-sectional shapes, indicating that this approach is suited for nature defects.

This study was carried out with the aim of detecting the defects on the surfaces of metallic products, which are frequently encountered in our daily lives and widely used in industry, with deep learning. Many metal products in the industry undergo different processes during the production phase. As a result of these processes, the detection of defects such as breakage, cracking, etc. on the surfaces of metal products is carried out by quality control personnel or production personnel. However, this error detection made by manpower both slows down production and causes overlooked errors. In order to easily detect these defects, in our study, we used a dataset consisting of 285 images in total with images of two different surface defects and images containing flawless metal parts and a CNN architecture, ResNet50 architecture, to defects. There are three classes in total in the dataset. Two of these classes consist of images of different types of defects on the surface of the metal piston part used in air conditioners, and one of them consists of images of perfect metal piston parts. Convolutional Neural Network (CNN) method was used to determine the features of the images. Precision, recall, F1-Score and accuracy metrics were used to measure the performance of the model. With the ResNet50 architecture, defects on the surfaces of metal piston parts are detected quickly and with high accuracy. As a result of the study, it was suggested that the proposed model can detect surface defects that occur in the usage areas of metal products in various sectors more quickly and accurately using deep learning. This shows that the quality control problems experienced in the industry can be reduced by using deep learning, saving time and manpower.

In this paper, we systematically review recent advances in surface inspection using computer vision andimage processing techniques, particularly those based on texture analysis methods. The aim is to reviewthe state-of-the-art techniques for the purposes of visual inspection and decision making schemes that areable to discriminate the features extracted from normal and defective regions. This field is so vast that itis impossible to cover all the aspects of visual inspection. This paper focuses on a particular but importantsubset which generally treats visual surface inspection as texture analysis problems. Other topics related tovisual inspection such as imaging system and data acquisition are out of the scope of this survey.The surface defects are loosely separated into two types. One is local textural irregularities which is themain concern for most visual surface inspection applications. The other is global deviation of colour and/ortexture, where local pattern or texture does not exhibit abnormalities. We refer this type of defects as shadeor tonality problem. The second type of defects have been largely neglected until recently, particularly whencolour imaging system has been widely used in visual inspection and where chromatic consistency plays animportant role in quality control. The emphasis of this survey though is still on detecting local abnormalities,given the fact that majority of the reported works are dealing with the first type of defects.The techniques used to inspect textural abnormalities are discussed in four categories, statistical approaches,structural approaches, filter based methods, and model based approaches, with a comprehensivelist of references to some recent works. Due to rising demand and practice of colour texture analysis inapplication to visual inspection, those works that are dealing with colour texture analysis are discussedseparately. It is also worth noting that processing vector-valued data has its unique challenges, which conventionalsurface inspection methods have often ignored or do not encounter.We also compare classification approaches with novelty detection approaches at the decision makingstage. Classification approaches often require supervised training and usually provide better performancethan novelty detection based approaches where training is only carried out on defect-free samples. However,novelty detection is relatively easier to adapt and is particularly desirable when training samples areincomplete