A negative correlation ensemble transfer learning method for fault diagnosis based on convolutional neural network.

With the development of hardware, deep learning, which based on high computation complexity, has become a popular tool for intelligent fault diagnosis. Compared with traditional methods, fault diagnosis method based on deep learning could approximate complex non-linear relationship and achieve very high precision under the same load. However, the performance of these deep learning-based fault diagnosis methods will be greatly reduced when the load changes, which are significant in fault diagnosis issues since the working load is changing all the time in real world industrial applications. Thus, a fault diagnosis method based on multi-scale convolutional neural network (CNN) is proposed to solve this problem. CNN is a type of neural network which can extract feature from the fault signal directly. In order to obtain sufficient fault information, the proposed method uses multi-scale convolution kernels to extract features of different scales. In addition, dropout is also added to the network to enhance the robustness and generalization capabilities of the model. The experimental results show that the proposed model not only performs well under the same load, but also achieves high fault diagnosis accuracy when the load changes.

The inverter is the most vulnerable module of photovoltaic (PV) systems. The insulated gate bipolar transistor (IGBT) is the core part of inverters and the root source of PV inverter failures. How to effectively diagnose the IGBT faults is critical for reliability, high efficiency, and safety of PV systems. Recently, deep learning (DL) methods are widely used for fault detection and diagnosis. Different from traditional diagnosis methods, DL methods use deep neural networks which can automatically extract the useful representative features from raw data. However, DL methods require large amounts of data, which leads to the high cost of communication, storage, and computation. To tackle these issues, a data-driven fault detection and diagnosis method for IGBT open-circuit faults based on compressed sensing (CS) and convolutional neural networks (CNN) is proposed in this paper. CS is adopted to compress raw signals, and the optimal value of compression ratio (CR) is determined by considering the trade-off between classification accuracy and model training time. The overlap sampling method is adopted for data segmentation. Meanwhile, overlap sampling can also increase the number of training samples and improve the sample correlation. The compressed signals are segmented and reconstructed into two-dimensional feature maps for model training. Finally, compared with CNN of the same structure, the developed CS-CNN model can compress 85% of data without accuracy loss. The performance comparison with the state-of-the-art networks demonstrates that the test accuracy is 98.68% and the model training time is much shorter than other methods.

Sentiment analysis is a subfield of Natural Language Processing (NLP) which tries to process a text to extract opinions or attitudes towards topics or entities. Recently, the use of deep learning methods for sentiment analysis has received noticeable attention from researchers. Generally, different deep learning methods have shown superb performance in sentiment analysis problem. However, deep learning models are different in nature and have different strengths and limitations. For example, convolutional neural networks are useful for extracting local structures from data, while recurrent models are able to learn order dependence in sequential data. In order to combine the advantages of different deep models, in this paper we have proposed a novel approach for aspect-based sentiment analysis which utilizes deep ensemble learning. In the proposed method, we first build four deep learning models, namely CNN, LSTM, BiLSTM and GRU. Then the outputs of these models are combined using stacking ensemble approach where we have used logistic regression as meta-learner. The results of applying the proposed method on the real datasets show that our method has increased the accuracy of aspect-based prediction by 5% to 20% compared to the basic deep learning methods.

In previous deep learning-based fault diagnosis methods for rotating machinery, the method of directly feeding one-dimensional data into convolutional neural networks can lead to the loss of important fault features. To address the problem, a novel rotating machinery fault diagnosis model based on a rainbow recursive plot (RRP) is proposed. Our main innovation and contributions are: First, a RRP is proposed to convert the one-dimensional vibration signal from the rotating machinery into a two-dimensional color image, facilitating the capturing of more significant fault information. Second, a new CNN based on LeNet-5 is devised, which extracts a feature that describes substantial fault information from the converted two-dimensional color image, thus performing fault diagnosis recognition accurately. The public rolling bearing datasets and the online fault diagnosis platform are adopted to verify proposed method performance. Experiments on public datasets show that the proposed method can improve the accurate rate of recognition to 97.86%. More importantly, online experiment on the self-made fault diagnosis platform demonstrates that our approach achieves the best comprehensive performance in terms of recognition speed and accuracy compared to mainstream algorithms.

The online spread of fake news on various platforms has emerged as a significant concern, posing threats to public opinion, political stability, and the dissemination of reliable information. Researchers have turned to advanced technologies, including machine learning (ML) and deep learning (DL) techniques, to detect and classify fake news to address this issue. This research study explores fake news classification using diverse ML and DL approaches. We utilized a well-known “Fake News” dataset sourced from Kaggle, encompassing a labelled news collection. We implemented diverse ML models, including multinomial naïve bayes (MNB), gaussian naïve bayes (GNB), Bernoulli naïve Bayes (BNB), logistic regression (LR), and passive aggressive classifier (PAC). Additionally, we explored DL models, such as long short-term memory (LSTM), convolutional neural networks (CNN), and CNN-LSTM. We compared the performance of these models based on key evaluation metrics, such as accuracy, precision, recall, and the F1 score. Additionally, we conducted cross-validation and hyperparameter tuning to ensure optimal performance. The results provide valuable insights into the strengths and weaknesses of each model in classifying fake news. We observed that DL models, particularly LSTM and CNN-LSTM, showed better performance compared to traditional ML models. These models achieved higher accuracy and demonstrated robustness in classification tasks. These findings emphasize the potential of DL models to tackle the spread of fake news effectively and highlight the importance of utilizing advanced techniques to address this challenging problem.

Artificial intelligence (AI)-based approaches are crucial in computer-aided diagnosis (CAD) for various medical applications. Their ability to quickly and accurately learn from complex data is remarkable. Deep learning (DL) models have shown promising results in accurately classifying Alzheimer's disease (AD) and its related cognitive states, Early Mild Cognitive Impairment (EMCI) and Late Mild Cognitive Impairment (LMCI), along with the healthy conditions known as Cognitively Normal (CN). This offers valuable insights into disease progression and diagnosis. However, certain traditional machine learning (ML) classifiers perform equally well or even better than DL models, requiring less training data. This is particularly valuable in CAD in situations with limited labeled datasets. In this paper, we propose an ensemble classifier based on ML models for magnetic resonance imaging (MRI) data, which achieved an impressive accuracy of 96.52%. This represents a 3-5% improvement over the best individual classifier. We evaluated popular ML classifiers for AD classification under both data-scarce and data-rich conditions using the Alzheimer's Disease Neuroimaging Initiative and Open Access Series of Imaging Studies datasets. By comparing the results to state-of-the-art CNN-centric DL algorithms, we gain insights into the strengths and weaknesses of each approach. This work will help users to select the most suitable algorithm for AD classification based on data availability.

With the huge success of applying deep learning (DL) methodologies to image recognition and natural language processing in recent years, researchers are now keen to use them in the machine condition monitoring (MCM) context. There are numerous papers in applying various DL techniques, such as auto-encoder, restricted Boltzmann machine, convolutional neural network and recurrent neural network, etc., to MCM problems ranging from component-level condition monitoring (machine tool wear prediction, bearing fault diagnosis and classification and hydraulic pump fault diagnosis) to system-level health management (aircraft and spacecraft diagnosis). In this paper, we give a brief overview in the area of DL for MCM with a focus on reviewing the most recent papers published since 2019. In Part 1, we present some critical views regarding whether any breakthrough has been achieved from an MCM domain expert perspective, with the main conclusion that DL has great potential for MCM applications, and a major breakthrough could come soon since the shortfalls lie more in data than in the DL methodologies. Our overall impression is that (a) DL models are not really showing their great potentials with only a small training data; (b) faulty-condition data is hard to come by for training DL, but normal condition data is abundant, so anomaly detection makes more sense; (c) applying DL only to the Case Western Reserve University (CWRU) bearing fault dataset is not sufficient for real world industrial applications as it was from a very simple test rig, and applying DL to data from complex systems like helicopter gearbox data may deliver much more convincing results. In Part 2, we enhance the main conclusion of the critical review with supplement views and a case study on analysing Bell-206B helicopter main gearbox planet bearing failure data using some traditional MCM techniques in contrast to applying the long short-term memory (LSTM) DL method. We can conclude from the case study that the DL-based methods are not necessarily always superior to the traditional MCM techniques for dataset from moderately complex machinery.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Fault diagnosis is very important for condition based maintenance. Recently, deep learning models are introduced to learn hierarchical representations from raw data instead of using hand-crafted features, which exhibit excellent performance. The success of current deep learning lies in: 1) the training (source domain) and testing (target domain) datasets are from the same feature distribution; 2) Enough labeled data with fault information exist. However, because the machine operates under a non-stationary working condition, the trained model built on the source domain can not be directly applied on the target domain. Moreover, since no sufficient labeled or even unlabeled data are available in target domain, collecting the labeled data and building the model from scratch is time-consuming and expensive. Motivated by transfer learning (TL), we present a new fault diagnosis method, which generalizes convolutional neural network (CNN) to TL scenario. Two layers with regard to task-specific features are adapted in a layer-wise way to regularize the parameters of CNN. What's more, the domain loss is calculated by a linear combination of multiple Gaussian kernels so that the ability of adaptation is enhanced compared to single kernel. Through these two means, the distribution discrepancy is reduced and the transferable features are learned. The proposed method is validated by transfer fault diagnosis experiments. Compared to CNN without domain adaptation and shallow transfer learning methods, the proposed method gets the best performance for fault classification.

Deep learning methods have become popular among researchers in the field of fault detection. However, their performance depends on the availability of big datasets. To overcome this problem researchers started applying transfer learning to achieve good performance from small available datasets, by leveraging multiple prediction models over similar machines and working conditions. However, the influence of negative transfer limits their application. Negative transfer among prediction models increases when the environment and working conditions are changing continuously. To overcome the effect of negative transfer, we propose a novel deep transfer learning method, coined deep boosted transfer learning, for wind turbine gearbox fault detection that prevents negative transfer and only focuses on relevant information from the source machine. The proposed method is an instance-based deep transfer learning method that updates the weights of the source and the target machine training samples separately. The weights of different source training samples are gradually decreased to reduce the impact on the final model. The proposed method is verified by the Case Western Reserve University bearing and real field wind farm datasets. The results show that the proposed method ignores negative transfer and achieves higher accuracy compared to standard deep learning and deep transfer learning methods.

Deep learning (DL) has been widely applied in the fault diagnosis field. However, the depth of DL models in fault diagnosis is very shallow compared with benchmark convolutional neural network (CNN) models for ImageNet. But it is hard to train a very deep CNN model without the large amount well-organized datasets like ImageNet. In this research, a new transfer learning based on pre-trained VGG-19 (TranVGG-19) is proposed for fault diagnosis. Firstly, a time-domain signals to RGB images conversion method is proposed. Then, the pre-trained VGG-19 is applied as feature extractor to obtained the features of converted images. Finally, a softmax classifier is trained on the features. The proposed TranVGG-19is tested on the famous motor bearing dataset from Case Western Reserve University. The final prediction accuracy of TCNN is 99.175% and the training time of TranVGG-19is only near 200 seconds. These results outperform many DL and machining learning methods.

Deep learning algorithms can be used to design systems that report data on patients and deliver warnings to medical applications or electronic health information if there are changes in the patient's health. These systems could be created using deep Learning. This helps verify that patients get the proper effective care at the proper time for each specific patient. A healthcare decision support system was presented using the Internet of Things and deep learning methods. In the proposed system, we examined the capability of integrating deep learning technology into automatic diagnosis and IoT capabilities for faster message exchange over the Internet. We have selected the suitable Neural Network structure (number of best-hidden layers and activation function classes) to construct the e-health system. In addition, the e-health system relied on data from doctors to understand the Neural Network. In the validation method, the total evaluation of the proposed healthcare system for diagnostics provides dependability under various patient conditions. Based on evaluation and simulation findings, a dual hidden layer of feed-forward NN and its neurons store the tanh function more effectively than other NN. To overcome challenges, this study will integrate artificial intelligence with IoT. This study aims to determine the NN's optimal layer counts and activation function variations.

Fault diagnosis plays a very important role in ensuring the safe and reliable operations of machines. Currently, the deep learning-based fault diagnosis is attracting increasing attention. However, fault diagnosis under variable working conditions has been a significant challenge due to the domain discrepancy problem. This problem is also unavoidable in deep learning-based fault diagnosis methods. This paper contributes to the ongoing investigation by proposing a new approach for the fault diagnosis under variable working conditions based on STFT and transfer deep residual network (TDRN). The STFT was employed to convert vibration signal to time-frequency image as the input of the TDRN. To address the domain discrepancy problem, the TDRN was developed in this paper. Unlike traditional deep convolutional neural network (DCNN) methods, by combining with transfer learning, the TDRN can make a bridge between two different working conditions, thereby using the knowledge learned from a working condition to achieve a high classification accuracy in another working condition. Moreover, since the residual learning is introducing, the TDRN can overcome the problems of training difficulty and performance degradation existing in traditional DCNN methods, thus further improving the classification accuracy. Experiments are conducted on the popular CWRU bearing dataset to validate the effectiveness and superiority of the proposed approach. The results show that the developed TDRN outperforms those methods without transfer learning and/or residual learning in terms of the accuracy and feature learning ability for the fault diagnosis under variable working conditions.

Electricity load forecasting is a crucial undertaking within all the deregulated markets globally. Among the research challenges on a global scale, the investigation of deep transfer learning (DTL) in the field of electricity load forecasting represents a fundamental effort that can inform artificial intelligence applications in general. In this paper, a comprehensive study is reported regarding day-ahead electricity load forecasting. For this purpose, three sequence-to-sequence (Seq2seq) deep learning (DL) models are used, namely the multilayer perceptron (MLP), the convolutional neural network (CNN) and the ensemble learning model (ELM), which consists of the weighted combination of the outputs of MLP and CNN models. Also, the study focuses on the development of different forecasting strategies based on DTL, emphasizing the way the datasets are trained and fine-tuned for higher forecasting accuracy. In order to implement the forecasting strategies using deep learning models, load datasets from three Greek islands, Rhodes, Lesvos, and Chios, are used. The main purpose is to apply DTL for day-ahead predictions (1–24 h) for each month of the year for the Chios dataset after training and fine-tuning the models using the datasets of the three islands in various combinations. Four DTL strategies are illustrated. In the first strategy (DTL Case 1), each of the three DL models is trained using only the Lesvos dataset, while fine-tuning is performed on the dataset of Chios island, in order to create day-ahead predictions for the Chios load. In the second strategy (DTL Case 2), data from both Lesvos and Rhodes concurrently are used for the DL model training period, and fine-tuning is performed on the data from Chios. The third DTL strategy (DTL Case 3) involves the training of the DL models using the Lesvos dataset, and the testing period is performed directly on the Chios dataset without fine-tuning. The fourth strategy is a multi-task deep learning (MTDL) approach, which has been extensively studied in recent years. In MTDL, the three DL models are trained simultaneously on all three datasets and the final predictions are made on the unknown part of the dataset of Chios. The results obtained demonstrate that DTL can be applied with high efficiency for day-ahead load forecasting. Specifically, DTL Case 1 and 2 outperformed MTDL in terms of load prediction accuracy. Regarding the DL models, all three exhibit very high prediction accuracy, especially in the two cases with fine-tuning. The ELM excels compared to the single models. More specifically, for conducting day-ahead predictions, it is concluded that the MLP model presents the best monthly forecasts with MAPE values of 6.24% and 6.01% for the first two cases, the CNN model presents the best monthly forecasts with MAPE values of 5.57% and 5.60%, respectively, and the ELM model achieves the best monthly forecasts with MAPE values of 5.29% and 5.31%, respectively, indicating the very high accuracy it can achieve.

Deep learning (DL) has become a popular option for data-driven fault diagnosis, because it can avert the influence of subjective factors in an artificial feature extraction process. However, it also suffers from the adverse effects accompanied with small fault sample and unbalanced data, resulting in limited accuracy improvement. For the aforementioned problem, this paper introduces a variational auto-encoder (VAE) into a fault diagnosis framework to realize data amplification by vibration signal generation, then an enhanced fault diagnosis approach is proposed combining with a convolution neural network. The well-trained VAE can realize the infinite generation of vibration signal by using the hidden variables sampled from Gaussian distribution, and then the generated artificial signals are mixed with the real original signals to form an enhanced training set, which can be utilized for classifier training to realize fault identification. Experimental results show that the generated artificial signals have similar time-frequency characteristics compared with the original real ones, and the enhanced fault diagnosis method holds a higher and more stable recognition accuracy than the unenhanced version and other typical methods.