40 - Techniques and Applications of Genetic Algorithm-Based Methods for Designing Compact Fuzzy Classification Systems

In this chapter, we explain how multi-objective genetic algorithms can be applied to the design of fuzzy rule-based systems for pattern classification problems. For a multi-class pattern classification problem with many continuous attributes (e.g., wine classification with 13 continuous attributes [1]), a fuzzy rule-based classification system is designed by a multi-objective genetic algorithm [2, 3] with two objectives: to minimize the size of the fuzzy rule-based classification system and to maximize its performance [4, 5]. The size of the fuzzy rule-based classification system is expressed by the number of fuzzy if-then rules, and its performance is measured by the number of correctly classified training patterns. In our approach to the design of the fuzzy rule-based classification system, first a large number of fuzzy if-then rules are generated from training patterns as candidate rules for the rule selection. Then a small number of fuzzy if-then rules are selected from the candidate rules by the two-objective genetic algorithm. The two-objective genetic algorithm tries to find all the non-dominated solutions (i.e., non-dominated rule sets) of the rule selection problem with the above-mentioned objectives. One difficulty of the rule selection method for high-dimensional pattern classification problems with many continuous attributes is that the number of candidate fuzzy if-then rules becomes intractably large. For example, the number of candidate rules is 613 ≅ 1.3 × 1010 for the wine classification problem with 13 continuous attributes if we have six fuzzy sets (terms) on each axis of the 13-dimensional pattern space.

Fuzzy systems based on fuzzy if-then rules have been applied to various problems. The main application area has been fuzzy control problems. In many cases, such fuzzy systems can handle only a few input variables. This is because the number of fuzzy if-then rules exponentially increases as the number of input variables increases. In this paper, we try to design fuzzy classification systems based on fuzzy if-then rules for multidimensional pattern classification problems with many attributes. For designing such fuzzy classification systems, we compare two frameworks in the area of genetics-based machine learning: the Michigan approach and the Pittsburgh approach. The performance of fuzzy rule-based classification systems is also compared with that of various pattern classification methods. In computer simulations, we use a wine classification problem with 13 attributes, a cancer diagnosis problem with 9 attributes, and a credit approval problem with 14 attributes. © 1997 Scripta Technica, Inc. Electron Comm Jpn Pt 3, 80(12): 10–19, 1997

This paper proposes a cost-based fuzzy classification system for pattern classification problems with misclassification costs. The task is to minimize the total misclassification cost incurred by a fuzzy classification system consisting of a number of fuzzy if-then rules where the number of generated fuzzy if-then rules depends on the specification of fuzzy partitions for each axis. In the proposed fuzzy classification system the consequent class of a fuzzy if-then rule is determined so that the misclassification cost is minimal over the covered training patterns by the antecedent part of the rule. On the other hand, conventional fuzzy classification systems are compatibility-based. That is, the consequent class of a fuzzy if-then rule is determined from the compatibility of training patterns covered by the antecedent part of the rule. The grade of certainty of the fuzzy if-then rules in both classification systems is calculated by using the compatibility of training patterns from each class. In a series of computational experiments, we compare the performance of the proposed cost-based fuzzy classification systems with that of the conventional compatibility-based systems. The performance of both classifiers is measured for three real-world pattern classification problems.

This paper proposes a genetic-algorithm-based approach to the construction of fuzzy classification systems with rectangular fuzzy rules. In the proposed approach, compact fuzzy classification systems are automatically constructed from numerical data by selecting a small number of significant fuzzy rules using genetic algorithms. Since significant fuzzy rules are selected and unnecessary fuzzy rules are removed, the proposed approach can be viewed as a knowledge acquisition tool for classification problems. In this paper, first we describe a generation method of rectangular fuzzy rules from numerical data for classification problems. Next, we formulate a rule selection problem for constructing a compact fuzzy classification system as a combinatorial optimization problem. Then we show how genetic algorithms are applied to the rule selection problem.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A genetic algorithm for generating fuzzy classification rules

This paper tries to apply a genetic algorithm-based method to fuzzy rule-base system for fuzzy classification with minimum fuzzy rules, which simultaneously enhances or maintain the performance of the fuzzy classification system with fuzzy rule-base. That is, the optimization is included with the minimization of the number of the extracted fuzzy rules and the maximization of the performance of the fuzzy classification system, i.e., the number of correctly classified training patterns with the results fuzzy rules. In optimization process, we also try to apply some experiments in order to employ more suitable reasoning method of the selected fuzzy rules. Finally, we demonstrate with numerical experiments. From the results, we can see that our method is effective and efficient with respect to the number of the correctly classified patterns and the number of the used fuzzy rules in the fuzzy classification systems.

We examine the performance of compact fuzzy rule-based classification systems that consist of a small number of simple fuzzy rules with high comprehensibility. Those fuzzy systems are designed in a heuristic manner using rule selection criteria. We first describe fuzzy rule-based classification. Next we describe heuristic rule selection criteria using the terminology in data mining: confidence and support. A small number of fuzzy rules are extracted from numerical data based on each rule selection criterion. Then we examine the classification performance of extracted fuzzy rules through computational experiments on a number of benchmark data sets from the UCI ML Repository. Our results should be viewed as the lowest benchmark performance of fuzzy rule-based classification systems because fuzzy rules are extracted using a simple heuristic method with no optimization or tuning procedures. Nevertheless our results on some data sets are comparable to reported results by the C4.5 algorithm in the literature.

We propose a hybrid algorithm of two fuzzy genetics-based machine learning approaches (i.e., Michigan and Pittsburgh) for designing fuzzy rule-based classification systems. First, we examine the search ability of each approach to efficiently find fuzzy rule-based systems with high classification accuracy. It is clearly demonstrated that each approach has its own advantages and disadvantages. Next, we combine these two approaches into a single hybrid algorithm. Our hybrid algorithm is based on the Pittsburgh approach where a set of fuzzy rules is handled as an individual. Genetic operations for generating new fuzzy rules in the Michigan approach are utilized as a kind of heuristic mutation for partially modifying each rule set. Then, we compare our hybrid algorithm with the Michigan and Pittsburgh approaches. Experimental results show that our hybrid algorithm has higher search ability. The necessity of a heuristic specification method of antecedent fuzzy sets is also demonstrated by computational experiments on high-dimensional problems. Finally, we examine the generalization ability of fuzzy rule-based classification systems designed by our hybrid algorithm.

We propose a hybrid algorithm of fuzzy versions of two genetics-based machine learning approaches: Michigan and Pittsburgh approaches. First, we examine the performance of each approach by computer simulations on commonly used data sets. Simulation results clearly demonstrate that each approach has its own advantages and disadvantages. While the Michigan approach has high search ability to efficiently find good fuzzy rules in large search spaces for high-dimensional pattern classification problems, it can not directly optimize fuzzy rule-based systems. On the other hand, the Pittsburgh approach can directly optimize fuzzy rule-based systems while its search ability to find good fuzzy rules is not high. Then we combine these two approaches into a single hybrid algorithm. Our hybrid algorithm is based on the Pittsburgh approach where a set of fuzzy rules is coded as a string. The Michigan approach is used as a mutation operation in our hybrid algorithm for partially modifying each string by generating new rules from existing good rules. In this manner, our hybrid algorithm utilizes the advantages of the two approaches.

In this paper, an interval type-2 Takagi-Sugeno fuzzy classification system (IT2T-SFCS) learned by particle swarm optimization (PSO) and support vector machine (SVM) for antecedent and consequent parameters optimization is proposed. The IT2T-SFCS is constructed by fuzzy if-then rules whose antecedents are interval type-2 fuzzy sets and consequents are linear state equations. The antecedents of IT2T-SFCS use the fuzzy iterative self-organizing data analysis technique (ISODATA) and PSO to learn and calculate the optimal centers and the uncertain widths of the Gaussian membership functions. Consequent parameters in IT2T-SFCS are learned through SVM for the purpose of achieving higher generalization ability. The proposed IT2T-SFCS is able to directly handle uncertainties, minimize the effects of uncertainties and get the better generalization performance, which inherits the benefits of interval type-2 T-S fuzzy system and SVM. For demonstration, IT2T-SFCS is used as a classifier in gender recognition. The experimental results show that the performance of the proposed IT2T-SFCS is superior to that of the previous mainstream classifiers.

A Fuzzy Classifier System for evolutionary learning of robot behaviors

This research work presents a new approach for fuzzy system building taking into account the accuracy and interpretability of the system. One difficulty in the handling of high-dimensional problems by fuzzy rule-based systems is the exponential increase in the number of rules and in the number of conditions in the antecedent part of the rule. Thus, as first step of the proposed approach we apply a feature selection process in order to exclude irrelevant variables. Besides that, dimensionality reduction generally promotes the accuracy and comprehensibility of the system. After that, a genetic algorithm is used for deriving short and comprehensible fuzzy rules. Finally another genetic algorithm is used for optimizing the rule base obtained in the last step, excluding unnecessary and redundant rules. The fitness function of the algorithms consider both accuracy and interpretability of the fuzzy model and the use of don't care condition allows to generate more comprehensible with high generalization capacity. The application domain is multidimensional fuzzy pattern classification. By computational simulation in some well-know datasets, we can see that the proposed approach is able to generate compact fuzzy rule bases with high classification ability. When compared to other fuzzy building method reported in the literature, our proposed method presented a good performance

In this paper, an intelligent Automatic Surveillance System is proposed using fuzzy rule base system and genetic algorithms. The aim of our Automatic Surveillance System is to detect a moving object and make a decision on whether it is human or not. Various object features such as the ratio of the width and the length of the moving object, the distance dispersion between the principal axis and the object contour, the eigenvectors, the symmetric axes, and the areas of the segmented regions are used in this paper. These features are not unique and decisive characteristics for representing human. Also, due to the outdoor image property, the object feature information is unavoidably vague and inaccurate. In order to make an efficient decision from that information, we use a fuzzy rule base system as an approximate reasoning method The fuzzy rules, combining various object features, are able to describe the conditions for making an intelligent decision. The fuzzy rule base system is initially constructed by heuristic information and then, trained and tested with input/output data. For the effective training, the well-known Genetic Algorithms (GA) are used. Experimental results are shown, demonstrating the validity of our system.

Fuzzy controller design of micro-unmanned helicopter relying on improved genetic optimization algorithm

Learning fuzzy rules from iterative execution of games