An Enhanced Fault Diagnosis Method with Uncertainty Quantification Using Bayesian Convolutional Neural Network

Concrete cracks significantly impact structural durability by allowing harmful elements such as carbon dioxide and chloride ions to penetrate. Thus, the cracks should be prevented and treated to maintain the lifespan of concrete structures. Convolutional neural network (CNN) has been widely applied in crack detection. This study investigated the use of a Bayesian convolutional neural network (BCNN) for concrete crack classification and compared the performance of BCNN with UNet, a deep CNN-based model, and CNN. BCNN weight uncertainty through a Bayesian method, improved prediction reliability and prevented overfitting. BCNN demonstrated superior expected calibration error (0.0030) and Brier score (0.0025) compared to CNN, highlighting its robustness in prediction reliability. Furthermore, Bayesian entropy analysis confirmed the ability of BCNN to quantify uncertainty, leading to more precise crack classification. These results imply that BCNN provides a balance between computational efficiency and performance, offering strong potential for smart construction applications. Validation results indicated that BCNN achieved a validation accuracy of 92.22%, higher than CNN (85.78%) and slightly lower than UNet (94.34%). However, BCNN demonstrated a superior intersection over union (IoU) of 0.8049, outperforming CNN (0.6685) and approaching UNet (0.9135). These results implied that BCNN provides a balance between computational efficiency and performance, offering strong potential for smart construction applications.

To find out the hidden danger in the industrial production process in time, it is necessary to monitor the health condition of the key components of the mechanical system in operation. However, traditional fault diagnosis methods usually adopt manual feature extraction, which not only costs expensively and depends on prior knowledge. Therefore, it is of great significance to study the application of automatic fault identification based on the original vibration signals. Recently, existing studies have shown that most of fault diagnoses are implemented by using deep neural network. Although these methods have achieved satisfactory performances, there are obvious limitations in real applications, that is, the complexity of deep neural network requires a lot of hardware computing resources. This hinders the development of online fault diagnosis tools. To solve this problem, this paper proposes a fault diagnosis model based on lightweight convolutional neural network MobileNet, and realizes an end-to-end intelligent fault classification and diagnosis application. We evaluated the proposed method with the rolling bearing dataset from Western Reserve University. The best average precision, recall and F1 score of ten different bearing health conditions are about 96%, 82% and 88%, respectively. In addition, we also compare the accuracy of the rolling bearing fault diagnosis classification model under the standard ReLU and the improved ReLU. Experimental results show that both obtain good performances, but the improved ReLU reaches the over 96% accuracy more rapidly.

<div class="section abstract"><div class="htmlview paragraph">Self-piercing riveting (SPR) is a key joining method in multi/thin-material automotive structures, yet accurately predicting the mechanical strength of SPR joints remains challenging due to numerous influencing factors. Empirical engineering equations [<span class="xref">1</span>] provide a foundation for estimating lap-shear and cross-tension strength but require several geometric parameters that are often unavailable in the design phase.</div><div class="htmlview paragraph">To address this limitation, we extract and leverage the core physical relationships embedded in these formulas. By reformulating the dependence of joint strength on the yield strength and total thickness of the sheet stack as practical regression models, we enable strength prediction using only commonly available material properties.</div><div class="htmlview paragraph">Furthermore, a Bayesian convolutional neural network (BCNN) model is developed to incorporate additional material features, offering improved prediction accuracy and uncertainty quantification.</div></div>

Probabilistic SAR-based water segmentation with adapted Bayesian convolutional neural network

El Niño Southern Oscillation (ENSO) significantly influences Earth's system and regional climate patterns. Despite dedicated efforts spanning decades, the precise prediction of ENSO events through numerical modelling beyond  one-year lead time remains a formidable challenge. The advent of deep learning-based approaches marks a transformative era in climate and weather prediction. However, many machine learning-based studies attempting ENSO prediction are confined to singular estimates, lacking adequate quantification of uncertainty in learned parameters and overlooking the crucial need for a nuanced understanding of ENSO prediction confidence. Here, we introduce a deep learning-based Bayesian Convolutional Neural Network (BCNN) model that provides probabilistic predictions for ENSO  with a lead time of up to 24 months. The Bayesian layers within the CNN maintain the capability to predict a distribution of learned parameters. The inherent capacity for uncertainty modeling enhances the reliability of BNNs, making them particularly valuable in operational services. The model was initially trained using globally gridded monthly sea surface temperature and upper 300 m integrated ocean potential temperature predictors from 25 Climate Model Intercomparison Project (CMIP) models, including 11 CMIP5 and 14 CMIP6. To address systematic errors in the BCNN, reflecting those of the CMIP samples, a learning transfer technique was applied using Simple Ocean Data Assimilation (SODA) reanalysis predictors spanning from 1871 to 1980. Validation of the all-season correlation skill of the Nino3.4 index in the BCNN model demonstrates significantly higher accuracy compared to current state-of-the-art dynamical forecast systems. This research holds substantial socio-economic implications as it enhances our forecasting capabilities and rigorously quantifies forecast uncertainties, providing valuable insights for planning and policy-making.

Industrial fault diagnosis (FD) is crucial for ensuring a manufacturing process’s safety, reliability, and efficiency. However, significant challenges arise from small fault sample sizes and the inherent variability of operating conditions. To tackle the above challenges, this article presents a novel small-sample-oriented FD framework, CF-DDIM-MCCL, leveraging the classifier-free denoising diffusion implicit model combined with multiclass contrastive learning. Additionally, CF-DDIM-MCCL incorporates a dynamic training termination strategy to enhance training efficiency by optimizing the learning process. It also employs a K-nearest neighbor (KNN) classifier instead of the traditional Softmax classifier, resulting in notable improvements in fault identification stability and accuracy. Extensive confirmatory and comparative experiments were conducted on two benchmark datasets, the Case Western Reserve University (CWRU) and Spectra Quest (SQ) bearing datasets, and in a real industrial application scenario, axial flow fan FD from a steel-making plant. Experimental results demonstrate the fault classification accuracies of 99.361%, 98.140%, and 98.542% across the three test cases. The outstanding performance of the proposed approach across diverse benchmark datasets and practical applications underscores its potential to revolutionize FD practices in natural industrial environments.

Recently, tremendous progress has been achieved in applying deep learning schemes (DLSs) to solve inverse scattering problems (ISPs), where state-of-the-art performance has been attained. However, little attention has been paid to the uncertainty quantification of these deep-learning methods in solving ISPs. In other words, the error of the prediction is not known since the ground truth is not available in practice. In this article, a Bayesian convolutional neural network (BCNN) is used to quantify the uncertainties in solving ISPs. With Monte Carlo dropout, the proposed BCNN is able to directly predict the pixel-based uncertainties of the widely used DLSs in ISPs. To quantitatively evaluate the performance of uncertainty predictions, both correlation coefficient and nonlinear correlation distribution between the predicted uncertainty and true absolute error are calculated. The tests on both synthetic and experimental data show that the predictive uncertainty is highly correlated with true absolute error calculated from the ground truth. Besides, when DLSs fail to solve an ISP, the predicted uncertainty increases significantly, which offers a pixel-based “confidence level” in solving ISPs.

When establishing a safety case for a high-level nuclear waste repository, coupled thermal, hydraulic, mechanical and chemical (THMC) processes are modelled to assess radionuclide transport and the integrity of barriers. Modelling these processes requires numerous parameters, which are all subject to uncertainty. However, complex and extensive uncertainty quantification (UQ) methods can come at great computational cost, especially when performed on multi-parametric problems. Therefore, UQ methods are sought with manageable computational effort while still capturing the relevant information to allow a meaningful interpretation of the results. The aim of this study is the comparison of three different UQ methods with different levels of complexity and computational cost in terms of their results. Radionuclide transport and THM-calculations are performed by employing these methods to the French Callovo-Oxfordian claystone as a reference material. The three methods include: (1) a UQ method based on the complete sampling of input parameter distributions by a Monte Carlo approach, (2) a UQ method based on a minimal number of data points, by sampling quantiles of the input parameter distributions, as well as the bounds of the distributions’ intervals, and (3) a UQ method that involves a first-order second-moment reliability approach. The results depict the different limitations and benefits of the UQ methods analysed while highlighting that a comprehensive understanding of parameter sensitivity and modelling approach are key to choosing the appropriate UQ method. • Applicability of UQ methods depends on complexity of problem. • FOSM and quantile method can be appropriate for linear problems and non-coupled processes. • MC approach remains method of choice for complex non-linear analyses.

Accurate fault classification and detection for the microgrid (MG) becomes a concern among the researchers from the state-of-art of fault diagnosis as it increases the chance to increase the transient response. The MG frequently experiences a number of shunt faults during the distribution of power from the generation end to user premises, which affects the system reliability, damages the load, and increases the fault line restoration cost. Therefore, a noise-immune and precise fault diagnosis model is required to perform the fast recovery of the unhealthy phases. This paper presents a review on the MG fault diagnosis techniques with their limitations and proposes a novel discrete-wavelet transform (DWT) based probabilistic generative model to explore the precise solution for fault diagnosis of MG. The proposed model is made of multiple layers with a restricted Boltzmann machine (RBM), which enables the model to make the probability reconstruction over its inputs. The individual RBM layer is trained with an unsupervised learning approach where an artificial neural network (ANN) algorithm tunes the model for minimizing the error between the true and predicted class. The effectiveness of the proposed model is studied by varying the input signal and sampling frequencies. A level of considered noise is added with the sample data to test the robustness of the studied model. Results prove that the proposed fault detection and classification model has the ability to perform the precise diagnosis of MG faults. A comparative study among the proposed, kernel extreme learning machine (KELM), multi KELM, and support vector machine (SVM) approaches is studied to confirm the robust superior performance of the proposed model.

Recent studies have been shown the vulnerability of Deep neural networks (DNNs) by a small adversarial perturbation in images that humans cannot distinguish and a well-trained neural network misclassifies. There are many defense methods, e.g. randomness, ensemble, and adversarial training that can improve the robustness of neural networks with respect to adversaries. Among them, adversarial training has been being the most outstanding defense. However, the training cost becomes a big challenge to adversarial learning. In this paper, we propose a new defense algorithm called Bayes without Bayesian Learning, which does not add the training phase in resisting adversarial attacks. Our method has based on the stochastic components of Bayesian Convolutional Neural Network (BCNN) to prevent the forceful gradient based attacks and generate the ensemble model to enhance the performance of models. In order to avoid adversarial training, we utilize pretrained CNN models on both natural and perturbed data. Therefore, this approach can be applied to any model with learned parameters to accelerate the computation of the defensive method. Our defensive algorithm has significantly improved the accuracy of pretrained CNN models under the high level of attacks. For instance, on ImageNet with Resnet50 network, we achieve 57% prediction accuracy in the top5, under strong PGD attack. We also confirm 3% improvement when combining our method with adversarial training. Experimental results prove the efficiency of our BCNN model that does not need Bayesian learning but still resists adversarial perturbation on both small and large scale datasets as CIFAR10, CIFAR100, and ImageNet.

Remaining useful life (RUL) is one of the essential ingredients in the battery management system. However, due to the characteristic of the dynamic and time-varying electrochemical system with nonlinear and complicated internal mechanisms, the uncertainty of RUL estimation has been expanded, and it is difficult to give an accurate time to reach the end of life. This article proposes the Bayesian mixture neural network (BMNN), a probabilistic deep learning method, to obtain more accurate RUL prediction and provide uncertainty estimation, while the quasi-Gramian angular field (Q-GAF) beneficial to identify prior distribution is utilized to transform time-series sequence into temporal images. BMNN consists of the Bayesian convolutional neural network (BCNN) extracting features in temporal images and Bayesian long short-term memory (B-LSTM) learning correlation between retention capacity and other degradation inducements. After concatenating two terms, the variational Bayesian neural network outputs the distribution of prediction results. In the experimental stage, the performance of the proposed method is validated on four different lithium-ion battery datasets and demonstrates higher stability, lower uncertainty, and more accuracy than other methods.

With the huge success of applying deep learning (DL) methodologies to image recognition and natural language processing in recent years, researchers are now keen to use them in the machine condition monitoring (MCM) context. There are numerous papers in applying various DL techniques, such as auto-encoder, restricted Boltzmann machine, convolutional neural network and recurrent neural network, etc., to MCM problems ranging from component-level condition monitoring (machine tool wear prediction, bearing fault diagnosis and classification and hydraulic pump fault diagnosis) to system-level health management (aircraft and spacecraft diagnosis). In this paper, we give a brief overview in the area of DL for MCM with a focus on reviewing the most recent papers published since 2019. In Part 1, we present some critical views regarding whether any breakthrough has been achieved from an MCM domain expert perspective, with the main conclusion that DL has great potential for MCM applications, and a major breakthrough could come soon since the shortfalls lie more in data than in the DL methodologies. Our overall impression is that (a) DL models are not really showing their great potentials with only a small training data; (b) faulty-condition data is hard to come by for training DL, but normal condition data is abundant, so anomaly detection makes more sense; (c) applying DL only to the Case Western Reserve University (CWRU) bearing fault dataset is not sufficient for real world industrial applications as it was from a very simple test rig, and applying DL to data from complex systems like helicopter gearbox data may deliver much more convincing results. In Part 2, we enhance the main conclusion of the critical review with supplement views and a case study on analysing Bell-206B helicopter main gearbox planet bearing failure data using some traditional MCM techniques in contrast to applying the long short-term memory (LSTM) DL method. We can conclude from the case study that the DL-based methods are not necessarily always superior to the traditional MCM techniques for dataset from moderately complex machinery.

Accurate and robust bearing fault diagnosis is crucial for the reliability of rotating machinery. To improve the precision of bearing fault classification, this study introduces a novel methodology that integrates the Adaptive t-distribution Slime Mold Algorithm (AtSMA) with the Least Squares Support Vector Machine (LSSVM). During the signal processing phase, Local Mean Decomposition (LMD) is employed to extract intrinsic mode functions from bearing vibration signals, which are subsequently reconstructed using the Pearson correlation coefficient method. Key features, such as sample entropy, permutation entropy, and energy entropy, are calculated to create a comprehensive feature vector for fault diagnosis. To enhance the convergence stability and global exploration capabilities of the Slime Mold Algorithm (SMA), an adaptive t-distribution mutation mechanism is incorporated to increase population diversity. Additionally, an improved step size strategy is implemented to prevent premature convergence and to expedite optimization speed. AtSMA is utilized to optimize the kernel parameters and penalty factor of LSSVM, thereby enhancing fault classification accuracy. Experimental evaluations conducted on two benchmark bearing datasets reveal that the proposed method achieves an average diagnostic accuracy of 96% on the Case Western Reserve University (CWRU) dataset and 93.25% on the Xi’an Jiaotong University dataset, surpassing conventional optimization algorithms and diagnostic techniques. These findings substantiate the superior diagnostic precision and robustness of the proposed approach under various fault scenarios and dynamic operating conditions.

In this article, an improved negative selection algorithm (INSA) has been proposed to identify faulty sensor nodes in wireless sensor network (WSN) and then the faults are classified into soft permanent, soft intermittent, and soft transient fault using the support vector machine technique. The performance metrics such as fault detection accuracy, false alarm rate, false positive rate, diagnosis latency (DL), energy consumption, fault classification accuracy (FCA), and false classification rate (FCR) are used to evaluate the performance of the proposed INSA. The simulation result shows that the INSA gives better result as compared to the existing algorithms in terms of performance metrics. The fault classification performance is measured by FCA and FCR. It has also seen that the proposed algorithm gives less DL and consumes less energy than that of existing algorithms proposed by Mohapatra et al, Zhang et al, and Panda et al for WSN.

Where to model the epistemic uncertainty of Bayesian convolutional neural networks for classification