Genetic Rule Selection as a Postprocessing Procedure in Fuzzy Data Mining

In this paper, we demonstrate that multiobjective genetic rule selection can significantly improve the accuracy-complexity tradeoff curve of fuzzy rule-based classification systems generated by a heuristic rule extraction procedure for classification problems with many continuous attributes. First a prespecifled number of fuzzy rules are extracted in a heuristic manner based on a rule evaluation criterion. This step can be viewed as fuzzy data mining. Then multiobjective genetic rule selection is applied to the extracted rules to find a number of non-dominated rule sets with respect to accuracy maximization and complexity minimization. This step can be viewed as a postprocessing procedure in fuzzy data mining. Experimental results show that multiobjective genetic rule selection finds a number of smaller rule sets with higher classification accuracy than heuristically extracted rule sets. That is, the accuracy-complexity tradeoff curve of heuristically extracted rule sets in fuzzy data mining is improved by multiobjective genetic rule selection. This observation suggests that multiobjective genetic rule selection plays an important role in fuzzy data mining as a postprocessing procedure.

In this paper, we examine the effectiveness of genetic rule selection with a multi-classifier coding scheme for ensemble classifier design. Genetic rule selection is a two-stage method. The first stage is rule extraction from numerical data using a d

One advantage of evolutionary multiobjective optimization (EMO) algorithms over classical approaches is that many non-dominated solutions can be simultaneously obtained by their single run. This paper shows how this advantage can be utilized in genetic rule selection for the design of fuzzy rule-based classification systems. Our genetic rule selection is a two-stage approach. In the first stage, a pre-specified number of candidate rules are extracted from numerical data using a data mining technique. In the second stage, an EMO algorithm is used for finding non-dominated rule sets with respect to three objectives: to maximize the number of correctly classified training patterns, to minimize the number of rules, and to minimize the total rule length. Since the first objective is measured on training patterns, the evolution of rule sets tends to overfit to training patterns. The question is whether the other two objectives work as a safeguard against the overfitting. In this paper, we examine the effect of the three-objective formulation on the generalization ability (i.e., classification rates on test patterns) of obtained rule sets through computer simulations where many non-dominated rule sets are generated using an EMO algorithm for a number of high-dimensional pattern classification problems.

We examine the performance of compact fuzzy rule-based classification systems that consist of a small number of simple fuzzy rules with high comprehensibility. Those fuzzy systems are designed in a heuristic manner using rule selection criteria. We first describe fuzzy rule-based classification. Next we describe heuristic rule selection criteria using the terminology in data mining: confidence and support. A small number of fuzzy rules are extracted from numerical data based on each rule selection criterion. Then we examine the classification performance of extracted fuzzy rules through computational experiments on a number of benchmark data sets from the UCI ML Repository. Our results should be viewed as the lowest benchmark performance of fuzzy rule-based classification systems because fuzzy rules are extracted using a simple heuristic method with no optimization or tuning procedures. Nevertheless our results on some data sets are comparable to reported results by the C4.5 algorithm in the literature.

This paper compares heuristic criteria used for extracting a pre-specified number of fuzzy classification rules from numerical data. We examine the performance of each heuristic criterion through computational experiments on well-known test problems. Experimental results show that better results are obtained from composite criteria of confidence and support measures than their individual use. It is also shown that genetic algorithm-based rule selection can improve the classification ability of extracted fuzzy rules by searching for good rule combinations. This observation suggests the importance of taking into account the combinatorial effect of fuzzy rules (i.e., the interaction among them).

We have been using fuzzy data mining techniques to extract patterns that represent behavior for intrusion detection. We describe a variety of modifications that we have made to the data mining algorithms in order to improve accuracy and efficiency. We use sets of fuzzy association rules that are mined from network audit data as models of behavior. To detect anomalous behavior, we generate fuzzy association rules from new audit data and compute the similarity with sets mined from normal data. If the similarity values are below a threshold value, an alarm is issued. We describe an algorithm for computing fuzzy association rules based on Borgelt's (2001) prefix trees, modifications to the computation of support and confidence of fuzzy rules, a new method for computing the similarity of two fuzzy rule sets, and feature selection and optimization with genetic algorithms. Experimental results demonstrate that we can achieve better running time and accuracy with these modifications.

This contribution is focused on the enhancement of the precision for Fuzzy Rule Based Classification Systems by the refinement of the Knowledge Base. Specifically, we make use of a Hierarchical Fuzzy Rule Based Classification System, which consists in the application of a thicker granularity in order to generate the initial Rule Base, and to reinforce those problem subspaces that are specially difficult by means of the application of rules with a higher granularity. Furthermore, we will perform a genetic rule selection process in order to obtain a compact and accurate model. Our experimental results show the goodness of this approach, especially when the number of classes is high, which usually implies a higher difficulty in the separability of the examples. Our conclusions are supported by means of the corresponding statistical tests.

Rule extraction from data is one of the key technologies for solving the bottlenecks in artificial intelligence. Artificial neural networks are well suited for representing any knowledge in given data. Extraction of logical/fuzzy rules from the trained artificial neural network is of great importance to researchers in the fields of artificial intelligence and soft computing. Fuzzy rule sets are capable of approximating any nonlinear mapping relationships. Extraction of rules from data has been discussed in terms of fuzzy modeling, fuzzy clustering, and classification with fuzzy rule sets. This special issue entitled"Rule Extraction from Data" is aimed at providing the readers with good insights into the advanced studies in the field of rule extraction from data using neural networks/fuzzy rule sets. I invited seven research papers best suited for the theme of this special issue. All the papers were reviewed rigorously by two reviewers each. The first paper proposes an interesting rule extraction method from data using neural networks. Ishikawa presents a combination of learning with an immediate critic and a structural learning with forgetting. This method is capable of generating skeletal networks for logical rule extraction from data with correct and wrong answers. The proposed method is applied to rule extraction from lense data. The second paper presents a new methodology for logical rule extraction based on transformation of MLP (multilayered perceptron) to a logical network. Duck et al. applied their C-MLP2LN to the Iris benchmark classification problem as well as real-world medical data with very good results. In the third paper, Geczy and Usui propose fuzzy rule extraction from trained artificial neural networks. The proposed algorithm is implied from their theoretical study, not from heuristics. Their study enables to initially consider derivation of crisp rules from trained artificial neural network, and in case of conflict, application of fuzzy rules. The proposed algorithm is experimentally demonstrated with the Iris benchmark classification problem. The fourth paper presents a new framework for fuzzy modeling using genetic algorithm. The authors have broken new ground of fuzzy rule extraction from neural networks. For the fuzzy modeling, they have proposed a particular type of neural networks containing nodes representing membership functions. In this fourth paper, the authors discuss input variable selection for the fuzzy modeling under multiple criteria with different importance. A target system with a strong nonlinearity is used for demonstrating the proposed method. Kasabov, et al. present, in the fifth paper, a method for extraction of fuzzy rules that have different level of abstraction depending on several modifiable thresholds. Explanation quality becomes better with higher threshold values. They apply the proposed method to the Iris benchmark classification problem and to a real world problem. J. Yen and W. Gillespie address interpretability issue of Takagi-Sugeno-Kang model, one of the most popular fuzzy mdoels, in the fifth paper. They propose a new approach of fuzzy modeling that ensures not only a high approximation of the input-output relationship in the data, but also good insights about the local behavior of the model. The proposed method is applied to fuzzy modeling of sinc function and Mackey-Glass chaotic time series data. The last paper discusses fuzzy rule extraction from numerical data for high-dimensional classification problems. H.Ishibuchi, et al. have been pioneering methods for classification of data using fuzzy rules and genetic algorithm. In this last paper, they introduced a new criterion, simplicity of each rule, together with the conventional ones, compactness of rule base and classification ability, for high-dimensional problem. The Iris data is used for demonstrating their new classification method. They applied it also to wine data and credit data. I hope that the readers will be encouraged to explore the frontier to establish a new paradigm in the field of knowledge representation and rule extraction.

Genetic rule selection with data partition for large-scale data sets

تصمیم گیری یکی از مهمترین موضوعات مورد بررسی در تحقیقات نظامی بشمار می رود. یکی از چالش های موجود دراین بحث وجود عدم قطعیت در محیط های جنگی می باشد که تاثیرات مخربی بر کیفیت و دقت تصمیم گیری می گذارد.در مقاله هروی و همکارانش، چاپ شده در سال 2193 ، بکارگیری ترکیبی از دو موضوع عامل های شناختی و دسته بندیبراساس قوانین وابستگی فازی به عنوان زمینه های موثر و پرکاربرد، توانسته بود تا حدودی این مسئله را کمرنگ کرده وسعی در کاهش عدم قطعیت داشته باشد. ولی هم چنان در شرایط حساس و بحرانی، نیاز به سرعت عمل بیشتر با حذفقوانین نامعتبر و ناکارای استخراج شده در اتخاذ تصمیمهای موثرتر قابل انکار نیست.هدف این مقاله، بهره گیری از ظرفیت های الگوریتم ژنتیک در انتخاب قوانین واقعبینانهتر به عنوان یک روش فراابتکاریدر تکمیل روش قبلی بصورت ترکیبی، برای کاهش هرچه بیشتر عدم قطعیت در تصمیم گیری ها می باشد. نتایج تجربیبدست آمده در مقایسه با روش پیشین، به روشنی نشان می دهد که این ترکیب علاوهبر مزیتهای روش قبل، بدلیل کاهشهرچه بیشتر قوانین تولید شده برای اتخاذ تصمیم، قابل فهم تر، دقیقتر و ریسک پذیری عاقلانهتری دارد.

Modeling of friction forces has been a challenging task in mechanical engineering. Parameterized approaches for modeling friction find it difficult to achieve satisfactory performance due to the presence of nonlinearity and uncertainties in dynamical systems. This paper aims to develop adaptive fuzzy friction models by the use of data-mining techniques and system theory. Our main technical contributions are twofold: extraction of fuzzy rules and formulation of a static fuzzy friction model and adaptation of the fuzzy friction model by the use of the Lyapunov stability theory, which is associated with a control compensation of a typical motion dynamics. The proposed framework in this paper shows a successful application of adaptive data-mining techniques in engineering. A single-degree-of-freedom mechanical system is employed as an experimental model in simulation studies. Results demonstrate that our proposed fuzzy friction model has promise in the design of uncertain mechanical control systems.

A fuzzy rules extraction based-integration (FREI) method is proposed to integrate numerical data from measuring instruments and linguistic rules from human experts. At first, the fuzzy IF-THEN rules are extracted from numerical data and acquired from experts individually. Based on a simple and intuitive integration scheme, a uniform fuzzy rules base is constructed through integrating above two kinds of information. The integrated information can be used for hybrid intelligent systems (HIS) design. With inverse learning, the proposed FREI method is able to integrate linguistic and numerical information with various forms, such as direct and indirect, and thus improve the performance of HIS using a priori knowledge where it is available. The validity of the proposed methods are verified through the fuzzy rules extraction-based hybrid intelligent control of a biped walking robot.KeywordsFuzzy RuleBiped RobotLinguistic InformationNumerical InformationFuzzy Rule BaseThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The process of monitoring the events occurring in a computer system or network and analyzing them for sign of intrusions is known as intrusion detection system (IDS). The objective of this paper is to extract fuzzy classification rules for intrusion detection in computer networks. The proposed method is based on the iterative rule learning approach (IRL) to fuzzy rule base system design. The fuzzy rule base is generated in an incremental fashion, in that the evolutionary algorithm optimizes one fuzzy classifier rule at a time. The performance of final fuzzy classification system has been investigated using intrusion detection problem as a high-dimensional classification problem. Results show that the presented algorithm produces fuzzy rules, which can be used to construct a reliable intrusion detection system.

A method for extraction of fuzzy rules from numerical input-output data is proposed. The main idea consists in separation of the input space into activation rectangles, corresponding to different output intervals. The generation of fuzzy rules and membership functions is based on these activation rectangles, whereas an appropriate fuzzy rules inference mechanism is proposed. The method formalises the synthesis of the fuzzy system and could be used for function approximation, classification and control purposes. An illustrative example for implementation of the method for synthesis of traffic fuzzy control is given.

Inferring descriptive and approximate fuzzy rules for credit scoring using evolutionary algorithms