An Algorithm for Bayesian Network Structure Learning Based on Simulated Annealing with Adaptive Selection Operator

Bayesian networks are useful analytical models for designing the structure of knowledge in machine learning. Bayesian networks can represent probabilistic dependency relationships among the variables. One strategy of Bayesian Networks structure learning is the score and search technique. The authors present the Elephant Swarm Water Search Algorithm (ESWSA) as a novel approach to Bayesian network structure learning. In the algorithm; Deleting, Reversing, Inserting, and Moving are used to make the ESWSA for reaching the optimal structure solution. Mainly, water search strategy of elephants during drought periods is used in the ESWSA algorithm. The proposed method is compared with simulated annealing and greedy search using BDe score function. The authors have also investigated the confusion matrix performances of these techniques utilizing various benchmark data sets. As presented by the results of the evaluations, the proposed algorithm has better performance than the other algorithms and produces better scores and accuracy values.

We present a hybrid algorithm called parallel simulated annealing with a greedy algorithm (PSAGA) to learn Bayesian network structures. This work focuses on simulated annealing and its parallelization with memoization to accelerate the search process. At each step of the local search, a hybrid search method combining simulated annealing with a greedy algorithm was adopted. The proposed PSAGA aims to achieve both the efficiency of parallel search and the effectiveness of a more exhaustive search. The Bayesian Dirichlet equivalence metric was used to determine an optimal structure for PSAGA. The proposed PSAGA was evaluated on seven well-known Bayesian network benchmarks generated at random. We first conducted experiments to evaluate the computational time performance of the proposed parallel search. We then compared PSAGA with existing variants of simulated annealing-based algorithms to evaluate the quality of the learned structure. Overall, the experimental results demonstrate that the proposed PSAGA shows better performance than the alternatives in terms of computational time and accuracy.

In this work we applied Variable Mesh Optimization population metaheuristc (VMO) for Bayesian network (BN)structure learning as score-and-search method. Our idea was to represent each node of the Mesh as a Bayesian network through a set of arcs. Then new BNs are created using among sets (union and difference) operations. For this process, three types of BNs are identified, local optima (BNs with the best score in each neighborhood), global optima (BN with the best score among local optima), and frontier solutions (more and less different in structure BNs). Finally the clearing process is applied to select the most representative BNs in the Mesh (score and structure). For determining the global score, each BN is used as a Bayesian classifier and classification accuracy is obtained using cross validation over dataset. The Proposal is compared with other classifiers using UCI repository data set. Results show that our proposal obtains the best score, that proves to be a very competitive algorithm for supervised classification.

Bayesian networks have become a popular knowledge representation scheme for probabilistic knowledge. One of the main challenges in Bayesian networks structure learning is the development of inductive learning techniques that scale up to large and possibly physically distributed data sets. We consider an approach to learning the structure of BN from distributed data. This is based on the collective learning strategy, where a local structure is obtained at each site and the global structure is obtained by cross learning and combining. Local learning is used to identify the local structure of local nodes and local links of cross nodes from local sample data sets. Cross learning can find the entire cross-links of cross nodes. Combined together, the local BNs and BN learnt from cross learning and removes any extra local links. We use the collective learning algorithm in distributed flood decision supporting system. The result shows that the Bayesian network structure is the same compare the collective learning algorithm from distributed data sets with learning from concentrated data sets. And meanwhile the collective learning algorithm is more efficient than transmitting all samples to a central site.

We present an improved method for learning Bayesian network (BN) structures. The new approach incorporate the idea of simulated annealing algorithm (SA) into the selection operator of genetic algorithm(GA). The BN structure with the highest score is given high priority in selection, as well as the structure with lower score could also be given opportunity to be selected. That is high score prior genetic-simulated annealing algorithm to Bayesian network structure learning(GSA_BNSL).This strategy will reserve optimal gene while avoiding the premature caused by the misleading from high score individual in the population. Experiments on comparison of several BN learning algorithms are carried out using typical data set of data mining. The result indicates that the GSA_BNSL is able to obtain an optimized BN structure with higher accuracy rate.

When the number of samples is large, the scale of the Bayesian network (BN) structure search space increases exponentially with the number of nodes, resulting in a sharp increase in the difficulty of learning the BN structure. Aiming at this problem, this paper proposes a fault diagnosis model construction method combining association rules and a BN network. The Euclidean distance under the Symbolic Aggregation Approximation (SAX) algorithm is utilized to compute and average the distance between the standard and faulty samples and filter the candidate nodes by the average value, which in turn reduces the search sample space. The method of combining Association Rules algorithm with traditional BN structure learning results is used to solve the problem of wrong edges in structure learning. Finally, the maximum likelihood estimation method is used for parameter learning to complete the construction of the diagnostic network. The experimental results show that the running time of the Bayesian Network based on the Association Rules (AR-BN) model proposed in this paper is short and that the Hamming distance with the original structure is small, so this model can effectively reduce the search space and solve the problem of wrong edges, and it also has a good performance in fault diagnosis.

Learning Bayesian network structure with immune algorithm

The type of constraint-based methods is one of the most important approaches to learn Bayesian network (BN) structures from observational data with conditional independence (CI) tests. In this paper, we find that existing constraint-based methods often perform many redundant CI tests, which significantly reduces the learning efficiency of those algorithms. To tackle this issue, we propose a novel framework to accelerate BN structure learning by reducing redundant CI tests without sacrificing accuracy. Specifically, we first design a CI test cache table to store CI tests. If a CI test has been computed before, the result of the CI test is obtained from the table instead of computing the CI test again. If not, the CI test is computed and stored in the table. Then based on the table, we propose two CI test cache table based PC (CTPC) learning frameworks for reducing redundant CI tests for BN structure learning. Finally, we instantiate the proposed frameworks with existing well-established local and global BN structure learning algorithms. Using twelve benchmark BNs, the extensive experiments have demonstrated that the proposed frameworks can significantly accelerate existing BN structure learning algorithms without sacrificing accuracy.

In recent years there has been a growing interest in Bayesian Network learning from uncertain data. While many researchers focus on Bayesian Network learning from data with tuple uncertainty, Bayesian Network structure learning from data with attribute uncertainty gets little attention. In this paper we make a clear definition of attribute uncertain data and Bayesian Network Learning problem from such data. We propose a structure learning method named DTAU based on information theory. The algorithm assumes that the structure of a Bayesian network is a tree. It avoids enumerating all possible worlds. The dependency tree is computed with polynomial time complexity. We conduct experiments to demonstrate the effectiveness and efficiency of our method. The experiments show the clustering results on uncertain dataset by our dependency tree are acceptable.KeywordsuncertaintyBayesian Network structuredependency

After the fault is injected, different failure models are presented inside the processor. This paper proposes an automatic method that can inject fault in the processor and collect time-series data on the Simics simulation platform. At the same time, combined with relevant standards, each sample is labeled by data cleaning method. Finally, the Bayesian network structure and parameter learning are used to take a cascading analysis of the key registers inside of the processor, and detect the fault type at the real time.

Bayesian network is an important model for reasoning in an uncertain environment. A reliable node rank is required by K2 algorithm to learn Bayesian network structure better. To provide a high-quality node rank tailored for K2 algorithm, we propose a node priority-based sorting algorithm. Given observable data only, our algorithm can be employed to learn a node rank without expert knowledge. Specifically, MCMC algorithm is first utilized to yield some Bayesian network structures that can sufficiently fit the observed data. We then define the priority of each node in these network structures. Node rank is finally obtained through weighted scoring based on the priority. The empirical results show that our sorting algorithm performs significantly better than commonly used methods, e.g., randomly sorting and MCMC algorithm, on an Asia network-learning dataset.

Several hardness results are presented for the parent assignment problem: Given m observations of n attributes x 1, ..., x n , find the best parents for x n , that is, a subset of the preceding attributes so as to minimize a fixed cost function. This attribute or feature selection task plays an important role, e.g., in structure learning in Bayesian networks, yet little is known about its computational complexity. In this paper we prove that, under the commonly adopted full-multinomial likelihood model, the MDL, BIC, or AIC cost cannot be approximated in polynomial time to a ratio less than 2 unless there exists a polynomial-time algorithm for determining whether a directed graph with n nodes has a dominating set of size logn, a LOGSNP-complete problem for which no polynomial-time algorithm is known; as we also show, it is unlikely that these penalized maximum likelihood costs can be approximated to within any constant ratio. For the NML (normalized maximum likelihood) cost we prove an NP-completeness result. These results both justify the application of existing methods and motivate research on heuristic and super-polynomial-time algorithms.KeywordsPolynomial TimeBayesian NetworkDirected GraphPositive InstanceParent AssignmentThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Estimation of conditional dependencies over a joint multivariate probability distribution is a difficult task for big data, e.g. –omics datasets, and it becomes quickly intractable when the number of variables involved grows large. For instance, structure learning in Bayesian networks has super-exponential complexity. Dimension reduction techniques such as principal component analysis can be useful but often transform the original space and can still pose problems with scalability. This substantially limits characterization of joint probability: in general, only pairwise or k-level correlations can be analyzed efficiently. We introduce the Multivariate Independence Set Search via Progressive Addition for Conditional Markov Acyclic Networks (MISS-PACMAN), which operates a greedy selection of jointly independent feature sets from a larger set of covariates, given a variable ordering. The method is non-parametric and can be used with any kernel function. MISS-PACMAN is therefore suitable for heterogeneous big data, as it combines flexibility and scalability. In our tests on both simulated and real-world data, using random forests as kernel, MISS-PACMAN was able to select independence feature sets linearly with the number of features. Further, by combining multiple independence sets, MISS-PACMAN well-approximates the underlying conditional structure of data variables (according to the generating Bayesian network), and compares favorably with other network structure discovery algorithms, such as the Peter-Clark and the fast incremental association Markov blanket.

A new structure learning approach for Bayesian networks based on asexual reproduction optimization (ARO) is proposed in this paper. ARO can be considered an evolutionary-based algorithm that mathematically models the budding mechanism of asexual reproduction. In ARO, a parent produces a bud through a reproduction operator; thereafter, the parent and its bud compete to survive according to a performance index obtained from the underlying objective function of the optimization problem: This leads to the fitter individual. The convergence measure of ARO is analyzed. The proposed method is applied to real-world and benchmark applications, while its effectiveness is demonstrated through computer simulations. Results of simulations show that ARO outperforms genetic algorithm (GA) because ARO results in a good structure and fast convergence rate in comparison with GA.

A hybrid algorithm for Bayesian network structure learning with application to multi-label learning