Insulator Fault Diagnosis Based on Improved Transfer Learning from UAV Images

Deep neural networks (DNNs), e.g., convolutional neural network (CNN), are able to learn effective features from wafer maps for dimensional reduction and feature extraction. However, very large image data are needed to train DNNs to obtain high generalization performance. It is still a difficult task due to the lack of sufficient labeled images with various defects. This article proposes a semisupervised deep transfer learning algorithm called joint feature and label adversarial network (JFLAN). JFLAN uses CNNs to extract transferable features of wafer maps and then introduces a multilayer domain adaptation and pseudolabel learning block based on the generative adversarial network (GAN). This effectively reduces the distribution discrepancy and the among-class distance of the transferable features. Finally, JFLAN transfers knowledge from wafer image source data collected offline and then achieves the goal of significantly improved accuracy of wafer defect recognition and realizes online adaptive defect recognition. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The defect recognition on wafer maps plays a key role to recognize fault sources of the semiconductor manufacturing processes. Transfer learning is able to use the existing labeled data to assist in the classification of unlabeled data and is very effective to solve the problem of small samples and nonstationary generalization errors. In particular, the infusion of adversarial learning in transfer learning will provide a new idea for deep feature learning. This article provides a novel method based on transfer learning to implement wafer map defect recognition (WMDR) to quickly identify defect root causes for yield enhancement. This article provides a novel way for quality control of semiconductor manufacturing processes based on transfer and adversarial learning.

The success of Convolutional Neural Network (CNN) is attributed to their ability to learn rich midlevel image representations as opposed to hand-crafted low-level features used in many natural image classification methods. Learning CNN, however, amounts to estimating millions of parameters and requires a very large number of annotated image samples. In this paper, we explored transfer learning for gastrointestinal bleeding detection on small-size imbalanced endoscopy images, and showed how image representations learned with CNN on large-scale annotated datasets can be efficiently transferred to other tasks with limited amount of training data. We first transferred pre-trained Inception V3 model trained on the ImageNet dataset to compute mid-level image representation, and then fine-tuned the trained model with labeled endoscopy images, and resumed training from already learned weights. Additionally, we introduce both data augmentation and image resampling to increase the size of the training database and the positive sample rate to perform the Transfer Learning. Our results showed that our transfer learning method produces the best performance on AUC (the area under the receiver operating curve), Precision, Recall and Accuracy as compared to both the hand-crafted feature based method and training CNN model from-scratch method.

We developed a computer-aided diagnosis (CADx) method for classification between benign nodule, primary lung cancer, and metastatic lung cancer and evaluated the following: (i) the usefulness of the deep convolutional neural network (DCNN) for CADx of the ternary classification, compared with a conventional method (hand-crafted imaging feature plus machine learning), (ii) the effectiveness of transfer learning, and (iii) the effect of image size as the DCNN input. Among 1240 patients of previously-built database, computed tomography images and clinical information of 1236 patients were included. For the conventional method, CADx was performed by using rotation-invariant uniform-pattern local binary pattern on three orthogonal planes with a support vector machine. For the DCNN method, CADx was evaluated using the VGG-16 convolutional neural network with and without transfer learning, and hyperparameter optimization of the DCNN method was performed by random search. The best averaged validation accuracies of CADx were 55.9%, 68.0%, and 62.4% for the conventional method, the DCNN method with transfer learning, and the DCNN method without transfer learning, respectively. For image size of 56, 112, and 224, the best averaged validation accuracy for the DCNN with transfer learning were 60.7%, 64.7%, and 68.0%, respectively. DCNN was better than the conventional method for CADx, and the accuracy of DCNN improved when using transfer learning. Also, we found that larger image sizes as inputs to DCNN improved the accuracy of lung nodule classification.

Breast tumor is a fatal threat to the health of women. Ultrasound (US) is a common and economical method for the diagnosis of breast cancer. Breast imaging reporting and data system (BI-RADS) category 4 has the highest false-positive value of about 30% among five categories. The classification task in BI-RADS category 4 is challenging and has not been fully studied. This work aimed to use convolutional neural networks (CNNs) for breast tumor classification using B-mode images in category 4 to overcome the dependence on operator and artifacts. Additionally, this work intends to take full advantage of morphological and textural features in breast tumor US images to improve classification accuracy. First, original US images coming directly from the hospital were cropped and resized. In 1385 B-mode US BI-RADS category 4 images, the biopsy eliminated 503 samples of benign tumor and left 882 of malignant. Then, K-means clustering algorithm and entropy of sliding windows of US images were conducted. Considering the diversity of different characteristic information of malignant and benign represented by original B-mode images, K-means clustering images and entropy images, they are fused in a three-channel form multi-feature fusion images dataset. The training, validation, and test sets are 969, 277, and 139. With transfer learning, 11 CNN models including DenseNet and ResNet were investigated. Finally, by comparing accuracy, precision, recall, F1-score, and area under curve (AUC) of the results, models which had better performance were selected. The normality of data was assessed by Shapiro-Wilk test. DeLong test and independent t-test were used to evaluate the significant difference of AUC and other values. False discovery rate was utilized to ultimately evaluate the advantages of CNN with highest evaluation metrics. In addition, the study of anti-log compression was conducted but no improvement has shown in CNNs classification results. With multi-feature fusion images, DenseNet121 has highest accuracy of 80.22±1.45% compared to other CNNs, precision of 77.97±2.89% and AUC of 0.82±0.01. Multi-feature fusion improved accuracy of DenseNet121 by 1.87% from classification of original B-mode images (p<0.05). The CNNs with multi-feature fusion show a good potential of reducing the false-positive rate within category 4. The work illustrated that CNNs and fusion images have the potential to reduce false-positive rate in breast tumor within US BI-RADS category 4, and make the diagnosis of category 4 breast tumors to be more accurate and precise.

Quality assurance of data prior to use in automated pipelines and image analysis would assist in safeguarding against biases and incorrect interpretation of results. Automation of quality assurance steps would further improve robustness and efficiency of these methods, motivating widespread adoption of techniques. Previous work by our group demonstrated the ability of convolutional neural networks (CNN) to efficiently classify head and neck (H&N) computed-tomography (CT) images for the presence of dental artifacts (DA) that obscure visualization of structures and the accuracy of Hounsfield units. In this work we demonstrate the generalizability of our previous methodology by validating CNNs on six external datasets, and the potential benefits of transfer learning with fine-tuning on CNN performance. 2112 H&N CT images from seven institutions were scored as DA positive or negative. 1538 images from a single institution were used to train three CNNs with resampling grid sizes of 643, 1283 and 2563. The remaining six external datasets were used in five-fold cross-validation with a data split of 20% training/fine-tuning and 80% validation. The three pre-trained models were each validated using the five-folds of the six external datasets. The pre-trained models also underwent transfer learning with fine-tuning using the 20% training/fine-tuning data, and validated using the corresponding validation datasets. The highest micro-averaged AUC for our pre-trained models across all external datasets occurred with a resampling grid of 2563 (AUC = 0.91 ± 0.01). Transfer learning with fine-tuning improved generalizability when utilizing a resampling grid of 2563 to a micro-averaged AUC of 0.92 ± 0.01. Despite these promising results, transfer learning did not improve AUC when utilizing small resampling grids or small datasets. Our work demonstrates the potential of our previously developed automated quality assurance methods to generalize to external datasets. Additionally, we showed that transfer learning with fine-tuning using small portions of external datasets can be used to fine-tune models for improved performance when large variations in images are present.

EEG based multi-class seizure type classification using convolutional neural network and transfer learning

The usage of Artificial intelligence in medical arena has proved to be a game changer in the detection and diagnosis of several medical conditions. In the current digital era, children with stressful medical issues are suffering from Deep Obsessive-Compulsive Disorder (DOCD). This kind of mental stress occurs in children because of the continuous usage of gadgets such as mobile phone, playing games using play stations, watching videos on tablets, etc. In most of the possibilities, single children are the ones affected with several obsessions such as stubborn activities, fighting for selfish priorities and so on. In medical terms, these kinds of complex behavioral changes are identified as DOCD. Genetic behaviors sometimes in a few group of children are also noticed as a modality difference. As symptoms are psychiatric impairment, such a child remains isolated, abnormal silence, being obsessive and repeating irrelevant words, high stress or anxiety. All medical challenges could be treated as healthcare research metrics and the gradual increase in DOCD disorder among children of this generation can be considered too. Early detection of DOCD is essential as it can help in early diagnosis but techniques to do so is unavailable currently. Deep learning-an artificial intelligence method can be utilized to detect DOCD, diagnose and treat it and bring about a positive character in children. Behavior changes in children can be classified and detected using transfer learning algorithms. In COVID-19 pandemic situation, 3% of DOCD has increased to 10-15% as a disorder. This information is retrieved from children by monitoring negative activities, unusual behavior such as nail biting, removing spectacles and placing them in the wrong place, watching tablets, mobile phones and television for more hours. Using Convolutional Neural Networks (CNN), input such as MRI (Magnetic resonance Imaging) is used for experimenting the variations in behavior with the high dimension that are analyzed from the image dataset. Using Transfer Learning with Inception V3-, CNN generalization of misophonia level can be statistically analyzed to avoid overfitting problems. By employing AI techniques, the aggression level can be predicted using data augmentation method with better accuracy and a low error rate than the existing systems. In the research it is observed that using the model employing Inception-V3 transfer learning CNN a better prediction of aggression levels can be achieved in comparison to the existing CNN model used.

Feature fusion methods have been demonstrated to be effective for many computer vision based applications. These methods generally use multiple hand-crafted features. However, in recent days, features extracted through transfer leaning procedures have been proved to be robust than the hand-crafted features in myriad applications, such as object classification and recognition. The transfer learning is a highly appreciated strategy in deep convolutional neural networks (DCNNs) due to its multifaceted benefits. It heartens us to explore the effect of fusing multiple transfer learning features of different DCNN architectures. Thus, in this work, we extract features of image statistics by exploiting three different pre-trained DCNNs through transfer learning. Then, we transform the features into a generalized subspace through a recently introduced Autoencoder network and fuse them to form intra-class invariant feature vector that is used to train a multi-class Support Vector Machine (SVM). The experimental results on various datasets, including object and action image statistics show that the fusion of multiple transfer learning features improves classification accuracy as compared to fusion of multiple hand-crafted features and usage of single component transfer learning features.

Although convolutional neural networks have achieved success in the field of image classification, there are still challenges in the field of agricultural product quality sorting such as machine vision-based jujube defects detection. The performance of jujube defect detection mainly depends on the feature extraction and the classifier used. Due to the diversity of the jujube materials and the variability of the testing environment, the traditional method of manually extracting the features often fails to meet the requirements of practical application. In this paper, a jujube sorting model in small data sets based on convolutional neural network and transfer learning is proposed to meet the actual demand of jujube defects detection. Firstly, the original images collected from the actual jujube sorting production line were pre-processed, and the data were augmented to establish a data set of five categories of jujube defects. The original CNN model is then improved by embedding the SE module and using the triplet loss function and the center loss function to replace the softmax loss function. Finally, the depth pre-training model on the ImageNet image data set was used to conduct training on the jujube defects data set, so that the parameters of the pre-training model could fit the parameter distribution of the jujube defects image, and the parameter distribution was transferred to the jujube defects data set to complete the transfer of the model and realize the detection and classification of the jujube defects. The classification results are visualized by heatmap through the analysis of classification accuracy and confusion matrix compared with the comparison models. The experimental results show that the SE-ResNet50-CL model optimizes the fine-grained classification problem of jujube defect recognition, and the test accuracy reaches 94.15%. The model has good stability and high recognition accuracy in complex environments.

In recent years, Convolutional Neural Networks (CNN) have revealed noteworthy enactment in classification of image jobs. Because of limited availability of annotated brain tumor MRI data, overfitting and lack of generalization are common challenges faced by traditional CNN models. To deal with these challenges, our studies explore the practice of Pre-Trained model of CNN and Transfer learning practices to expand the correctness of cataloguing the brain tumor through the data of MRI. Our experimental results demonstrate that the incorporation of Pre-Trained model of CNN with Transfer learning significantly improves the rain tumor’s classification accuracy on MRI data compared to traditional CNN models trained from scratch. Moreover, the research outcomes highlight potential of transfer learning as an operative approach for improving generalization capabilities of CNN models in medical imaging tasks with limited annotated data. By incorporating pre-trained CNN replicas and transfer erudition techniques, the objective of this studies aims to elevate brain tumor taxonomy accurateness with the assistance of MRI data.

With the continuous development of deep learning technology, neural networks such as convolutional neural network (CNN) have shown good performance in many fields, such as image processing. Meanwhile, the relevant algorithm has made great progress. But the experiment results show that the deeper the network layers, the more the number of parameters that need to be trained in neural network, and the massive computing resources will be consumed to reconstruct and train the deep convolutional neural network (DCNN) model. These parameters often need to be trained in large dataset. But in many practical applications, the effective sample dataset that can be collected are usually small and lack of annotated samples. It is a pity that models that perform well on large datasets often have overfitting problems when applied to small datasets. And transfer learning can recognize and apply knowledge and skills learned in previous domains/tasks to novel domains/tasks. Combining deep convolutional neural network learning with transfer learning can make full use of existing models with good performance to solve problems in new fields, so has received considerable attentions due to its high research value and wide application prospect. This paper focuses on the combination of CNN and transfer learning, analyzes their characteristics, summarizes the relevant models, methods and applications, so as to promote their effective fusion in image classification.

With the improvement of living standards, the types and quantities of garbage produced by residents are increasing, and the garbage disposal methods needs to be further improved. Standardize garbage disposal, garbage classification and garbage recycling will contribute to construction a green city. Garbage classification technologies also needs innovation. The intelligent garbage classification which depends on the Internet and intelligent technologies will overcome the disadvantages of traditional garbage classification. To this end, this paper proposes a smart trash can which based on convolutional neural network and transfer learning technologies. The smart trash can in this paper builds a smart trash monitoring network through NB-IOT, and automatically extract garbage features based on convolutional neural network, then trains the network through transfer learning technologies. The experimental results shows that the smart trash can designed in this paper has fast response speed and high accuracy.

This paper was aimed to address the problem of image-based human age estimation. It has the following main contributions. First, we provide a comparison of three hand-crafted image features and five deep convolutional neural networks (DCNNs). Secondly, we show that the use of pre-trained DCNNs as feature extractors can transfer the knowledge of DCNNs to new datasets and domains that were not necessarily addressed in the training phase. This is achieved by only retraining a shallow regressor over the deep features. Thirdly, we provide a cross-database evaluation involving biological and apparent ages. The paper shows that transfer learning allows the use of pre-trained DCNNs regardless of the type of ages (apparent or biological) that is adopted in DCNN training. The experiments are carried out on three public databases: MORPH, PAL, and Chalearn2016.

The classification process is also used in artificial intelligence (AI), which is intelligence created by a computer, so that it can mimic actions like humans in general and can capture events that occur in the surrounding environment. Seeing the very high development of the international coffee trade, it can be concluded that if there is a type of coffee that has the best quality, it will be sought after by coffee importing countries. Batik is cloth that is painted using canting and liquid wax to form motifs or patterns that have high artistic value. Batik has a variety of unique and distinctive patterns that reflect the region where the batik motif originates. One area that has batik motifs in Indonesia is batik in the North Coastal Region of Java Island. In this area, various batiks are produced according to the characteristics of the area itself. Regarding information related to the introduction of types of batik motifs, perhaps it comes from people who have batik skills and batik craftsmen who best understand batik as a whole, while the general public does not really know about batik motifs. Because batik has different motifs and batik motifs in some areas have motifs that are almost uniform but not the same. One of the technologies that can be used is deep learning. The purpose of this study is to propose a Convolutional Neural Network (CNN) and Transfer Learning (TL) model to be implemented in an intelligent system for the process of image classification of coffee bean types. The method used in this study is the PSOCNN transfer learning model VGG16 From the results of tests carried out on 2 models, namely the CNN model, the PSOCNN transfer learning model VGG16. it was found that the highest accuracy was obtained when classifying batik motives images using the PSOCNN-transfer learning model VGG16, which was 83%. The level of accuracy that is increased when compared to the usual CNN model indicates that the use of transfer learning has a good effect on the level of accuracy obtained. Although an increase of 6% is significant, but with this increase it opens up opportunities to increase even higher by using other transfer learning models

As the failure of power line insulators leads to the failure of power transmission systems, an insulator inspection system based on an aerial platform is widely used. Insulator defect detection is performed against complex backgrounds in aerial images, presenting an interesting but challenging problem. Traditional methods, based on handcrafted features or shallow-learning techniques, can only localize insulators and detect faults under specific detection conditions, such as when sufficient prior knowledge is available, with low background interference, at certain object scales, or under specific illumination conditions. This paper discusses the automatic detection of insulator defects using aerial images, accurately localizing insulator defects appearing in input images captured from real inspection environments. We propose a novel deep convolutional neural network (CNN) cascading architecture for performing localization and detecting defects in insulators. The cascading network uses a CNN based on a region proposal network to transform defect inspection into a two-level object detection problem. To address the scarcity of defect images in a real inspection environment, a data augmentation method is also proposed that includes four operations: 1) affine transformation; 2) insulator segmentation and background fusion; 3) Gaussian blur; and 4) brightness transformation. Defect detection precision and recall of the proposed method are 0.91 and 0.96 using a standard insulator dataset, and insulator defects under various conditions can be successfully detected. Experimental results demonstrate that this method meets the robustness and accuracy requirements for insulator defect detection.