Rule Extraction from Data

Association Rule Mining is one of the important data mining activities and has received substantial attention in the literature. Association rule mining is a computationally and I/O intensive task. In this paper, we propose a solution approach for mining optimized fuzzy association rules of different orders. We also propose an approach to define membership functions for all the continuous attributes in a database by using clustering techniques. Although single objective genetic algorithms are used extensively, they degenerate the solution. In our approach, extraction and optimization of fuzzy association rules are done together using multi-objective genetic algorithm by considering the objectives such as fuzzy support, fuzzy confidence and rule length. The effectiveness of the proposed approach is tested using computer activity dataset to analyze the performance of a multi processor system and network audit data to detect anomaly based intrusions. Experiments show that the proposed method is efficient in many scenarios.

Proposes a fuzzy neural expert system (FNES) which has a feedforward fuzzy neural network whose input layer consists of fuzzy cell groups and crisp (non-fuzzy) cell groups. Here, the truthfulness of fuzzy information and crisp information of training data is represented by fuzzy cell groups and crisp cell groups, respectively. The expert system has the following two functions: generalization of the information derived from the training data and embodiment of knowledge in the form of the fuzzy neural network; and extraction of fuzzy If-Then rules with linguistic relative importance of each proposition in an antecedent (If-part) from a trained fuzzy neural network. The paper also gives a method to extract automatically fuzzy If-Then rules from the trained neural network. To prove the effectiveness and validity of the proposed fuzzy neural expert system, a fuzzy neural expert system for medical diagnosis has been developed. >

Modeling of friction forces has been a challenging task in mechanical engineering. Parameterized approaches for modeling friction find it difficult to achieve satisfactory performance due to the presence of nonlinearity and uncertainties in dynamical systems. This paper aims to develop adaptive fuzzy friction models by the use of data-mining techniques and system theory. Our main technical contributions are twofold: extraction of fuzzy rules and formulation of a static fuzzy friction model and adaptation of the fuzzy friction model by the use of the Lyapunov stability theory, which is associated with a control compensation of a typical motion dynamics. The proposed framework in this paper shows a successful application of adaptive data-mining techniques in engineering. A single-degree-of-freedom mechanical system is employed as an experimental model in simulation studies. Results demonstrate that our proposed fuzzy friction model has promise in the design of uncertain mechanical control systems.

This paper proposes a new method for fuzzy rule extraction from trained support vector machines (SVMs) for multi-class problems, named FREx_SVM. SVMs have been used in a variety of applications. However, they are considered “black box models,” where no interpretation about the input–output mapping is provided. Some methods to reduce this limitation have already been proposed, but they are restricted to binary classification problems and to the extraction of symbolic rules with intervals or functions in their antecedents. In order to improve the interpretability of the generated rules, this paper presents a new model for extracting fuzzy rules from a trained SVM. The proposed model is suited for classification in multi-class problems and includes a wrapper feature selection algorithm. It is evaluated in four benchmark databases, and results obtained demonstrate its capacity to generate a reduced set of interpretable fuzzy rules that explains both the classification database and the influence of each input variable on the determination of the final class.

A genetic algorithm for generating fuzzy classification rules

This paper focuses on one half of the knowledge acquisition problem for fuzzy systems, namely the acquisition of a fuzzy rule base from a set of input/output data, and in particular on the extraction of a set of fuzzy rules from a trained neural network. Some limitations with previously reported work in this area are first identified. Two simple rule extraction techniques are then described and tested on a well known classification problem. The performance of the resultant rule bases compares more favourably than those reported using alternative techniques.

We examine the performance of compact fuzzy rule-based classification systems that consist of a small number of simple fuzzy rules with high comprehensibility. Those fuzzy systems are designed in a heuristic manner using rule selection criteria. We first describe fuzzy rule-based classification. Next we describe heuristic rule selection criteria using the terminology in data mining: confidence and support. A small number of fuzzy rules are extracted from numerical data based on each rule selection criterion. Then we examine the classification performance of extracted fuzzy rules through computational experiments on a number of benchmark data sets from the UCI ML Repository. Our results should be viewed as the lowest benchmark performance of fuzzy rule-based classification systems because fuzzy rules are extracted using a simple heuristic method with no optimization or tuning procedures. Nevertheless our results on some data sets are comparable to reported results by the C4.5 algorithm in the literature.

Fuzzy rules automatically derived from a set of training examples quite often produce better classification results than fuzzy rules translated from medical knowledge. This study aims to investigate the difference in domain representation between a knowledge-based and a data-driven fuzzy system applied to an electrocardiography classification problem. For a three-class electrocardiographic arrhythmia classification task a set of fifteen fuzzy rules is derived from medical expertise on the basis of twelve electrocardiographic measures. A second set of fuzzy rules is automatically constructed on thirty-nine MIT-BIH database's records. The performances of the two classifiers on thirteen different records are comparable and up to a certain extent complementary. The two fuzzy models are then analyzed, by using the concept of information gain to estimate the impact of each ECG measure on each fuzzy decision process. Both systems rely on the beat prematurity degree and the QRS complex width and neglect the P wave existence and the ST segment features. The PR interval is not well characterized across the fuzzy medical rules while it plays an important role in the data-driven fuzzy system. The T wave area shows a higher information gain in the knowledge based decision process, and is not very much exploited by the data-driven system. The main difference between a human designed and a data driven ECG arrhythmia classifier is found about the PR interval and the T wave.

<abstract><p>The training of artificial neural networks (ANNs) with rectified linear unit (ReLU) activation via gradient descent (GD) type optimization schemes is nowadays a common industrially relevant procedure. GD type optimization schemes can be regarded as temporal discretization methods for the gradient flow (GF) differential equations associated to the considered optimization problem and, in view of this, it seems to be a natural direction of research to <italic>first aim to develop a mathematical convergence theory for time-continuous GF differential equations</italic> and, thereafter, to aim to extend such a time-continuous convergence theory to implementable time-discrete GD type optimization methods. In this article we establish two basic results for GF differential equations in the training of fully-connected feedforward ANNs with one hidden layer and ReLU activation. In the first main result of this article we establish in the training of such ANNs under the assumption that the probability distribution of the input data of the considered supervised learning problem is absolutely continuous with a bounded density function that every GF differential equation admits for every initial value a solution which is also unique among a suitable class of solutions. In the second main result of this article we prove in the training of such ANNs under the assumption that the target function and the density function of the probability distribution of the input data are piecewise polynomial that every non-divergent GF trajectory converges with an appropriate rate of convergence to a critical point and that the risk of the non-divergent GF trajectory converges with rate 1 to the risk of the critical point. We establish this result by proving that the considered risk function is <italic>semialgebraic</italic> and, consequently, satisfies the <italic>Kurdyka-Łojasiewicz inequality</italic>, which allows us to show convergence of every non-divergent GF trajectory.</p></abstract>

Although hierarchical fuzzy neural networks (FNNs) perform with high accuracy in medical diagnosis systems, their popularity is held back from well-known disadvantage of not providing explanation. This paper presents a novel rule extraction approach to extract accurate and comprehensible fuzzy IF-THEN rules via genetic algorithm (GA) from hierarchical heterogeneous FNNs (HHFNNs). When each sub-FNNs is constructed and trained, entire HHFNNs are constructed and trained jointly through integrating all trained sub-FNNs. The proposed rule extraction approach is used to extract rule set from each concerned sub-FNNs, all extracted rule sets are then combined as one set to provide automatically exclusive explanation to diagnostic conclusion when IF part contains input features and THEN part contains diagnostic conclusions. Experimental study on diagnosing three most common and important cardiovascular diseases using hospital site-measured data demonstrates that such proposed approach exhibits satisfactory explanation capability without concerning inner structures of HHFNNs.Keywordshierarchical heterogeneous fuzzy neural networksfuzzy logicrule extractiongenetic algorithmfuzzy IF-THEN rules

Extracting useful patterns is an important theme in data mining. Fuzzy logic and Rough sets are the most common techniques applied in data mining problems where rule sets are used to classify new cases. In order to extract minimum rules with high data coverage and fast processing time based on rule induction. This paper presents a hybrid method called, Updating Induction of Decision Rules based on Fuzzy Rough Set (UIDR-FRS). The UIDR-FRS method is based on the integration of fuzzy logic and rough sets and hence constructs a novel rule extraction model. The attribute reduction algorithm using fuzzy logic and rules extraction method based on rough sets are proposed. The attributes of decision table are analyzed using fuzzy logic and subsequently a rule extraction model based on these attributes is built through integration of rough sets and fuzzy logic model. Experiments on real-life data sets are conducted to test and verify the validity of the proposed measure. Applications of the proposed rule induction in rule extraction and accuracy are also studied with experiments. Extensive experimental results verify the effectiveness and efficiency of the attribute reduction algorithms for a compacted decision table.

The author previously (1990, 1991) proposed a fuzzy neural expert system and provided a method to extract automatically fuzzy IF-THEN rules from a trained neural network. The previous work is extended and a neural expert system is proposed using fuzzy teaching input. The neural expert system can perform generalization of the information derived from training data with fuzzy teaching input and embodiment of knowledge in the form of a fuzzy neural network, where the fuzzy teaching input is subjectively given by domain experts: and extraction of fuzzy IF-THEN rules with linguistic relative importance of each proposition in an antecedent (IF-part) from a trained neural network. A method is proposed to extract automatically fuzzy IF-THEN rules from the trained neural network generated by training data with fuzzy teaching input. >

We have been using fuzzy data mining techniques to extract patterns that represent behavior for intrusion detection. We describe a variety of modifications that we have made to the data mining algorithms in order to improve accuracy and efficiency. We use sets of fuzzy association rules that are mined from network audit data as models of behavior. To detect anomalous behavior, we generate fuzzy association rules from new audit data and compute the similarity with sets mined from normal data. If the similarity values are below a threshold value, an alarm is issued. We describe an algorithm for computing fuzzy association rules based on Borgelt's (2001) prefix trees, modifications to the computation of support and confidence of fuzzy rules, a new method for computing the similarity of two fuzzy rule sets, and feature selection and optimization with genetic algorithms. Experimental results demonstrate that we can achieve better running time and accuracy with these modifications.

The generation of fuzzy rule base and rule extraction are efficient approaches to enhance the performance of fuzzy rule system. Here, we propose a novel fuzzy neural network structure that can be used for rule extraction. First of all, inspired by the compactly combined fuzzy rule base (CoCo-FRB) and fully combined fuzzy rule base (FuCo-FRB), we develop a new fuzzy rule base, compromise fuzzy rule base (CmPm-FRB), which generates rules by cutting off the long rules and compensating the short ones. In addition, Group Lasso penalty is utilized in the objective function with the rule threshold to produce sparsity in rules in a grouped manner for rule extraction. However, as the Group Lasso penalty is not differentiable at the origin, a tiny bias term is added to the gradient formula to achieve the smoothness. In order to verify the effectiveness of the proposed model, extensive experiments are conducted on ten classification data sets. The empirical results explicitly demonstrate the effectiveness of the proposed model for classification problems.

Rule extraction based on PROMETHEE-assisted multi-objective genetic algorithm for generating interpretable neural networks