Bayesian networks of customer satisfaction survey data

Fault diagnosis is useful in helping technicians detect, isolate, and identify faults, and troubleshoot. Bayesian network (BN) is a probabilistic graphical model that effectively deals with various uncertainty problems. This model is increasingly utilized in fault diagnosis. This paper presents bibliographical review on use of BNs in fault diagnosis in the last decades with focus on engineering systems. This work also presents general procedure of fault diagnosis modeling with BNs; processes include BN structure modeling, BN parameter modeling, BN inference, fault identification, validation, and verification. The paper provides series of classification schemes for BNs for fault diagnosis, BNs combined with other techniques, and domain of fault diagnosis with BN. This study finally explores current gaps and challenges and several directions for future research.

Bayesian network (BN) inference is an important statistical tool with additive symmetry. However, BN inference cannot deal with the uncertain, fuzzy, random, and conflicting information from experts’ knowledge in the process of conducting a risk assessment. To tackle this issue, a new fuzzy BN inference approach for risk assessments was proposed based on cloud model (CM), interval type-2 fuzzy set (IT2 FS), interval type-2 fuzzy logic system (IT2 FLS), modified Dempster–Shafer (D-S) evidence theory (ET), and Latin hypercube sampling (LHS) methods along the following lines. Firstly, CM was integrated into IT2 FS, and CM-based IT2 FS (CM-IT2 FS) was defined in the IT2 FLS. Secondly, modified D-S ET was utilized to determine the CM-IT2 FS-based a priori probabilities, and the CM-IT2 FS-based BN model was established. Thirdly, the CM-IT2 FS-based a priori probabilities were reduced to the CM-IT1 FS-based ones using a type reducer in the IT2 FLS, LHS was applied to propose a new fuzzy BN inference algorithm, and then, the new algorithm was used in a typical case to perform the fuzzy BN positive inference for risk prediction and the fuzzy BN reverse inference for risk sensitivity analysis. Finally, the BN inference results were analyzed using the proposed algorithm and the two common BN inference algorithms, and the effectiveness of the proposed approach was validated. It can be concluded that the proposed approach was both accurate and promising.

Sophisticated course projects in senior computer science classes have raised great challenges to educators. Students now have stronger needs for instant and individualized guidance when exposed to the complexity of software development. In this paper, we describe the application of Bayesian Network, a probabilistic graphic model, to automated project failure analysis. Students can make multiple project submissions to an automated grader before the project deadline. Each submission will be graded immediately, by running a collection of test cases prepared by instructors in advance. Then failure behaviors of test cases are piped to a Bayesian Network (BN) inference engine. The BN engine generates a report on the most probable failure causes, and helps students in removing software bugs. The analysis of the BN engine can be improved by parameter learning. A case study is presented to illustrate the effectiveness of the approach.

Inference in hybrid Bayesian networks with large discrete and continuous domains

The intermittence and fluctuation of renewable energy bring significant uncertainty to the power system, which enormously increases the operational risks of the power system. The development of efficient interval prediction models can provide data support for decision making and help improve the economy and reliability of energy interconnection operation. The performance of Bayesian deep learning models and Bayesian shallow neural networks in short-term interval prediction of photovoltaic power is compared in this study. Specifically, an LSTM Approximate Bayesian Neural Network model (ABNN-I) is built on the basis of the deep learning and Monte Carlo Dropout method. Meanwhile, a Feedforward Bayesian Neural Network (ABNN-II) model is introduced by Feedforward Neural Network and the Markov Chain Monte Carlo method. To better compare and verify the interval prediction capability of the ABNN models, a novel clustering method with three-dimensional features which include the number of peaks and valleys, the average power value, and the non-stationary measurement coefficient is proposed for generating sunny and non-sunny clustering sets, respectively. Results show that the ABNN-I model has an excellent performance in the field of photovoltaic short-term interval forecasting. At a 95% confidence level, the interval coverage from ABNN-I to ABNN-II can be increased by up to 3.1% and the average width of the interval can be reduced by 56%. Therefore, with the help of the high computational capacity of deep learning and the inherent ability to quantify uncertainty of the interval forecast from Bayesian methods, this research provides high-quality interval prediction results for photovoltaic power prediction and solves the problem of difficult modeling for over-fitting that exists in the training process, especially on the non-sunny clustering sets.

Traditionally, a single junction tree is used as the secondary structure for inference in a Bayesian network. However, its applicability and efficiency are restricted by the size of the junction tree. In this paper, we demonstrate that using a hierarchy of junction trees (HJT) as the secondary structure instead will greatly alleviate this restriction and improve the performance. We also compare the proposed HJT with other similar schemes for inference in Bayesian networks.

Non-parametric Bayesian networks: Improving theory and reviewing applications

Recently, there has been much attention in the use of machine learning methods, particularly deep learning for stock price prediction. A major limitation of conventional deep learning is uncertainty quantification in predictions which affect investor confidence. Bayesian neural networks feature Bayesian inference for providing inference (training) of model parameters that provides a rigorous methodology for uncertainty quantification in predictions. Markov Chain Monte Carlo (MCMC) sampling methods have been prominent in implementing inference of Bayesian neural networks; however certain limitations existed due to a large number of parameters and the need for better computational resources. Recently, there has been much progress in the area of Bayesian neural networks given the use of Langevin gradients with parallel tempering MCMC that can be implemented in a parallel computing environment. The COVID-19 pandemic had a drastic impact in the world economy and stock markets given different levels of lockdowns due to rise and fall of daily infections. It is important to investigate the performance of related forecasting models during the COVID-19 pandemic given the volatility in stock markets. In this paper, we use novel Bayesian neural networks for multi-step-ahead stock price forecasting before and during COVID-19. We also investigate if the pre-COVID-19 datasets are useful of modelling stock price forecasting during COVID-19. Our results indicate due to high volatility in the stock-price during COVID-19, it is more challenging to provide forecasting. However, we found that Bayesian neural networks could provide reasonable predictions with uncertainty quantification despite high market volatility during the first peak of the COVID-19 pandemic.

A Bayesian Network is a probabilistic graphical model that encodes probabilistic dependencies between a set of random variables. We introduce bnstruct, an open source R package to (i) learn the structure and the parameters of a Bayesian Network from data in the presence of missing values and (ii) perform reasoning and inference on the learned Bayesian Networks. To the best of our knowledge, there is no other open source software that provides methods for all of these tasks, particularly the manipulation of missing data, which is a common situation in practice. The software is implemented in R and C and is available on CRAN under a GPL licence. francesco.sambo@unipd.it. Supplementary data are available at Bioinformatics online.

PyBNesian: An extensible python package for Bayesian networks

We propose a new method of sensor planning for mobile robot localization using Bayesian network inference. Since we can model causal relations between situations of the robot's behavior and sensing events as nodes of a Bayesian network, we can use the inference of the network for dealing with uncertainty in sensor planning and thus derive appropriate sensing actions. In this system we employ a multi-layered-behavior architecture for navigation and localization. This architecture effectively combines mapping of local sensor information and the inference via a Bayesian network for sensor planning. The mobile robot recognizes the local sensor patterns for localization and navigation using a learned regression function. Since the environment may change during the navigation and the sensor capability has limitations in the real world, the mobile robot actively gathers sensor information to construct and reconstruct a Bayesian network, and then derives an appropriate sensing action which maximizes a utility function based on inference of the reconstructed network. The utility function takes into account belief of the localization and the sensing cost. We have conducted some simulation and real robot experiments to validate the sensor planning system.

Multivariate density estimation is a fundamental problem in Applied Statistics and Machine Learning. Given a collection of data sampled from an unknown distribution, the task is to approximately reconstruct the generative distribution. There are two different approaches to the problem, the parametric approach and the non-parametric approach. In the parametric approach, the approximate distribution is represented by a model from a predetermined family. In this thesis, we adopt the parametric approach and investigate the use of a model family called latent tree models for the task of density estimation. Latent tree models are tree-structured Bayesian networks in which leaf nodes represent observed variables, while internal nodes represent hidden variables. Such models can represent complex relationships among observed variables, and in the meantime, admit efficient inference among them. Consequently, they are a desirable tool for density estimation. While latent tree models are studied for the first time in this thesis for the purpose of density estimation, they have been investigated earlier for clustering and latent structure discovery. Several algorithms for learning latent tree models have been proposed. The state-of-the-art is an algorithm called EAST. EAST determines model structures through principled and systematic search, and determines model parameters using the EM algorithm. It has been shown to be capable of achieving good trade-off between fit to data and model complexity. It is also capable of discovering latent structures behind data. Unfortunately, it has a high computational complexity, which limits its applicability to density estimation problems. In this thesis, we propose two latent tree model learning algorithms specifically for density estimation. The two algorithms have distinct characteristics and are suitable for different applications. The first algorithm is called HCL. HCL assumes a predetermined bound on model complexity and restricts to binary model structures. It first builds a binary tree structure based on mutual information and then runs the EM algorithm once on the resulting structure to determine the parameters. As such, it is efficient and can deal with large applications. The second algorithm is called Pyramid. Pyramid does not assume predetermined bounds on model complexity and does not restrict to binary tree structures. It builds model structures using heuristics based on mutual information and local search. It is slower than HCL. However, it is faster than EAST and is only slightly inferior to EAST in terms of the quality of the resulting models. In this thesis, we also study two applications of the density estimation techniques that we develop. The first application is to approximate probabilistic inference in Bayesian networks. A Bayesian network represents a joint distribution over a set of random variables. It often happens that the network structure is very complex and making inference directly on the network is computational intractable. We propose to approximate the joint distribution using a latent tree model and exploit the latent tree model for faster inference. The idea is to sample data from the Bayesian network, learn a latent tree model from the data offline, and when online, make inference with the latent tree model instead of the original Bayesian network. HCL is used here because the sample size needs to be large to produce accurate approximation and it is possible to predetermine a bound on the online running. Empirical evidence shows that this method can achieve good approximation accuracy at low online computational cost. The second application is classification. A common approach to this task is to formulate it as a density estimation problem: One constructs the class-conditional density for each class and then uses the Bayes rule to make classification. We propose to estimate those class-conditional densities using either EAST or Pyramid. Empiricalevidence shows that this method yields good classification performances. Moreover, the latent tree models built for the class-conditional densities are often meaningful, which is conducive to user confidence. A comparison between EAST and Pyramid reveals that Pyramid is significantly more efficient than EAST, while it results in more or less the same classification performance as the latter.

Bayesian network (BN) is a probabilistic graphical model which describes the joint probability distribution over a set of random variables. Finding an optimal network structure based on an available training dataset is one of the most important challenges in the field of BNs. Since the problem of searching the optimal BN structure belongs to the class of NP-hard problems, typically greedy algorithms are used to solve it. In this paper two novel learning automata-based algorithms are proposed to solve the BNs’ structure learning problem. In both, there is a learning automaton corresponding with each possible edge to determine the appearance and the direction of that edge in the constructed network; therefore, we have a game of learning automata, at each stage of the proposed algorithms. Two special cases of the game of the learning automata have been discussed, namely, the game with a common payoff and the competitive game. In the former, all the automata in the game receive a unique payoff from the environment, but in the latter, each automaton receives its own payoff. As the algorithms proceed, the learning processes focus on the BN structures with higher scores. The use of learning automata has led to design the algorithms with a guided search scheme, which can avoid getting stuck in local maxima. Experimental results show that the proposed algorithms are capable of finding the optimal structure of BN in an acceptable execution time; and compared with other search-based methods, they outperform them.

Graphical modelling is an important tool for the efficient representation and analysis of uncertain information in knowledge-based systems. While Bayesian networks and Markov networks from probabilistic graphical modelling are well-known for a couple of years, the field of possibilistic graphical modelling occurs as a new promising area of research. Possibilistic networks provide an alternative approach compared to probabilistic networks, whenever it is necessary to model uncertainty and imprecision as two different kinds of imperfect information. Imprecision in the sense of set-valued data has often to be considered in situations where data are obtained from human observations or non-precise measurement units. In this contribution we present a comparison of the background and perspectives of probabilistic and possibilistic graphical models, and give an overview on the current state of the art of possibilistic networks with respect to propagation and learning algorithms, applicable to data mining and data fusion problems.

This paper reviews the use of Bayesian networks (BNs) in predicting software defects and software reliability. The approach allows analysts to incorporate causal process factors as well as combine qualitative and quantitative measures, hence overcoming some of the well-known limitations of traditional software metrics methods. The approach has been used and reported on by organizations such as Motorola, Siemens, and Philips. However, one of the impediments to more widespread use of BNs for this type of application was that, traditionally, BN tools and algorithms suffered from an obvious ‘Achilles’ heel’ – they were not able to handle continuous nodes properly, if at all. This forced modellers to have to predefine discretization intervals in advance and resulted in inaccurate predictions where the range, for example, of defect counts was large. Fortunately, recent advances in BN algorithms now make it possible to perform inference in BNs with continuous nodes, without the need to prespecify discretization levels. Using such ‘dynamic discretization’ algorithms results in significantly improved accuracy for defects and reliability prediction type models.