Driving rule extraction based on cognitive behavior analysis

Artificial Neural Network (ANN) is one of the powerful techniques of machine learning. It has shown its effectiveness in both prediction and classification problems. However, in some fields there is still some reticence towards their use mainly the fact that they do not justify their answers. The lack of transparency on how ANN makes decisions motivated us to develop our rule extraction algorithm that extracts comprehensible rules with high accuracy and high fidelity. The aim is to generate a set of rules that mimic the decision of ANN and cover a larger set of patterns. The obtained rule sets should satisfy a well-balanced trade-off between the fidelity, the accuracy and the comprehensibility. The proposed algorithm consists of a three steps: ANN learning phase, rule extraction phase and rule simplification phase. The rule extraction phase is based on the extraction of the association rules while the rules simplification procedure is based on the laws of Boolean algebra. To evaluate the performance of our algorithm, the system has been studied using four datasets, and then compared with other rule extraction methods. The results show that our proposal offers a small set of rules having the highest accuracy and fidelity values.

When performing predictive modeling, the key criterion is always accuracy. With this in mind, complex techniques like neural networks or ensembles are normally used, resulting in opaque models impossible to interpret. When models need to be comprehensible, accuracy is often sacrificed by using simpler techniques directly producing transparent models; a tradeoff termed the accuracy vs. comprehensibility tradeoff. In order to reduce this tradeoff, the opaque model can be transformed into another, interpretable, model; an activity termed rule extraction. In this paper, it is argued that rule extraction algorithms should gain from using oracle data; i.e. test set instances, together with corresponding predictions from the opaque model. The experiments, using 17 publicly available data sets, clearly show that rules extracted using only oracle data were significantly more accurate than both rules extracted by the same algorithm, using training data, and standard decision tree algorithms. In addition, the same rules were also significantly more compact; thus providing better comprehensibility. The overall implication is that rules extracted in this fashion explain the predictions made on novel data better than rules extracted in the standard way; i.e. using training data only

How to extract rules from trained SVMs has become an important preprocessing technique for data mining, pattern classification, and so on. There are two key problems required to be solved in the SVM based classification rule extraction, i.e. the attribute selection and the discretization to continuous attributes. In this paper, the differential characteristic of SVR (Support vector regression) is discussed. A new measure for determining the importance level of the attributes based on the trained SVR classifiers is proposed. Based on this new measure, a new approach for rule extraction from trained SVR classifiers is proposed. A new algorithm for rule extraction is given. The performance of the new approach is demonstrated by several computing cases. Experiment results show that the proposed approach can improve the validity of the extracted rules remarkably compared to other rule extracting approaches, especially for complicated classification problems.

This paper presents a hybrid data mining mechanism to extract expert knowledge from historical data and extend expert systems' reasoning capabilities by using fuzzy neural network (FNN)-based learning & rule extraction algorithm. Our hybrid data mining mechanism is based on association rule extraction mechanism, FNN learning and fuzzy rule extraction algorithm. Most of traditional data mining mechanisms are depended ()n association rule extraction algorithm. However, the basic association rule-based data mining systems has not the learning ability. Therefore, there is a problem to extend the knowledge base adaptively. In addition, sequential patterns of association rules can`t represent the complicate fuzzy logic in real-world. To resolve these problems, we suggest the hybrid data mining mechanism based on association rule-based data mining, FNN learning and fuzzy rule extraction algorithm. Our hybrid data mining mechanism is consisted of four phases. First, we use general association rule mining mechanism to develop an initial rule base. Then, in the second phase, we adopt the FNN learning algorithm to extract the hidden relationships or patterns embedded in the historical data. Third, after the learning of FNN, the fuzzy rule extraction algorithm will be used to extract the implicit knowledge from the FNN. Fourth, we will combine the association rules (initial rule base) and fuzzy rules. Implementation results show that the hybrid data mining mechanism can reflect both association rule-based knowledge extraction and FNN-based knowledge extension.

In this chapter, FMM is endowed with a rule extraction algorithm. With the rule extraction algorithm, FMM is able to explain its predictions using fuzzy if-then rules, thus overcoming the “black-box” phenomenon as suffered by most NN models. Applicability of FMM to fault diagnosis tasks in a power generation plant has been examined. The potential of FMM in learning and predicting faults in complex processes as well as in providing a comprehensible explanation for its predictions has been demonstrated in two experiments. The proposed rule extraction algorithm is able to yield a comprehensible rule set. The extracted rules have been verified as meaningful and are in line with the domain knowledge as well as experts’ opinions. Further research work will concentrate on the aspects of implementation, validation, and verification of FMM as a useful, robust, and intelligent fault diagnosis tool in a variety of application domains.

Advances in data mining have led to algorithms that produce accurate regression models for large and difficult to approximate data. Many of these use non-linear models to handle complex data-relationships in the input data. Their lack of transparency, however, is problematic since comprehensibility is a key requirement in many potential application domains. Rule-extraction algorithms have been proposed to solve this problem for classification by extracting comprehensible rule sets from the often better performing, complex models. We present a new pedagogical rule extraction algorithm for regression, based on active learning, which can be combined with any existing rule induction technique. Empirical results show that the proposed ALPA-R rule extraction method improves on classical rule induction techniques, both in accuracy and fidelity.

Efficient Distributed Genetic Algorithm for Rule extraction

The main challenge in using supervised neural networks in data mining applications is to get explicit knowledge from these models. For this purpose, a study on knowledge acquisition from supervised neural networks employed for classification problems is presented. An algorithm for rule extraction from neural networks, based on the RX algorithm is developed. This algorithm, named modified RX, is experimentally evaluated in two different domains: Iris Plants Database and Pima Indians Diabetes Database. The results are compared to those obtained by classification trees. As far as the efficacy is concerned, one observes that the successful application of the algorithm mainly depends on the knowledge representation acquired by the connectionist model, whereas the efficiency only depends on the neural network training time.

Parallel extraction of association rules from genomics data

Neural networks have been applied in various domain including science, commerce, medicine, and industry. However, The knowledge learned by a trained neural network is difficult to understand. This paper proposes a Boolean algebra based algorithm to extract comprehensible Boolean rules from supervised feed-forward neural networks to uncover the black-boxed knowledge. This algorithm is called the BAB-BB rule extraction algorithm, which stands for a Boolean algebra based rule extraction algorithm for neural networks with binary and bipolar inputs. Decomposition techniques and interval arithmetic are used in the algorithm. First, each neuron associated with its inputs is analyzed and a Boolean function, describing the activation rule from its inputs to the neuron, is derived. These Boolean functions are merged into an aggregated Boolean rule according to the network topology. The Boolean rule is then further simplified by Boolean algebra operations. During the rule extraction procedure, redundant hidden neurons can be detected and removed without affecting the original function of the neural network. Examples of unipolar and bipolar inputs are presented to demonstrate the use of our algorithm. Finally, the Exclusive OR problem is presented and solved by our algorithm. Results show that our BAB-BB algorithm is practicable and of high efficiency.

A modified fuzzy min–max neural network with rule extraction and its application to fault detection and classification

Rule extraction based on PROMETHEE-assisted multi-objective genetic algorithm for generating interpretable neural networks

Mortality rate due to fatal heart disease (HD) or cardiovascular disease (CVD) has increased drastically over the world in recent decades. HD is a very hazardous problem prevailing among people which is treatable if detected early. But in most of the cases, the disease is not diagnosed until it becomes severe. Hence, it is requisite to develop an effective system which can accurately diagnosis HD and provide a concise description for the underlying causes [risk factors (RFs)] of the disease, so that in future HD can be controlled only by managing the primary RFs. Recently, researchers are using various machine learning algorithms for HD diagnosis, and neural network (NN) is one among them which has attracted tons of people because of its high performance. But the main obstacle with a NN is its black-box nature, i.e., its incapability in explaining the decisions. So, as a solution to this pitfall, the rule extraction algorithms can be very effective as they can extract explainable decision rules from NNs with high prediction accuracies. Many neural-based rule extraction algorithms have been applied successfully in various medical diagnosis problems. This study assesses the performance of rule extraction algorithms for HD diagnosis, particularly those that construct rules recursively from NNs. Because they subdivide a rule’s subspace until the accuracy improves, recursive algorithms are known for delivering interpretable decisions with high accuracy. The recursive rule extraction algorithms’ efficacy in HD diagnosis is demonstrated by the results. Along with the significant data ranges for the primary RFs, a maximum accuracy of 82.59% is attained.

A rule extraction algorithm based on K-means clustering with interval-valued intuitionistic fuzzy sets (IVIFS) information, which is the combination of K-means clustering and IVIFS, is proposed in this paper. First, we introduce IVIFS and its distance. Second, we introduce IVIFS method and its application to the classification of software project. Finally, we present a rule extraction model according to IVIFS fuzziness and its K-means clustering, and apply them to pattern classification of outsourced software project risk to demonstrate the advantages of this model. The experimental results show that the rules from IVIFS model are better than that from the conventional K-means clustering model in rule extraction, and the prediction effect from the former is more effective than that from the latter. According to this combinational rule extraction method, based on database from a special and professional investigation for Chinese small and medium software-outsourced enterprise, we obtain some valuable, realistic and available project development risks decision rules.

This paper introduces the Fast Iterative Rule-based Linguistic Classifier (FaIRLiC), a Genetic Fuzzy Rule-Based Classification System (GFRBCS) which targets at reducing the structural complexity of the resulting rule base, as well as its learning algorithm's computational requirements, especially when dealing with high-dimensional feature spaces. The proposed methodology follows the principles of the iterative rule learning (IRL) approach, whereby a rule extraction algorithm (REA) is invoked in an iterative fashion, producing one fuzzy rule at a time. The REA is performed in two successive steps: the first one selects the relevant features of the currently extracted rule, whereas the second one decides the antecedent part of the fuzzy rule, using the previously selected subset of features. The performance of the classifier is finally optimized through a genetic tuning post-processing stage. Comparative results in a hyperspectral remote sensing classification as well as in 12 real-world classification datasets indicate the effectiveness of the proposed methodology in generating high-performing and compact fuzzy rule-based classifiers, even for very high-dimensional feature spaces.