Support vector learning for fuzzy rule-based classification systems

We examine the performance of compact fuzzy rule-based classification systems that consist of a small number of simple fuzzy rules with high comprehensibility. Those fuzzy systems are designed in a heuristic manner using rule selection criteria. We first describe fuzzy rule-based classification. Next we describe heuristic rule selection criteria using the terminology in data mining: confidence and support. A small number of fuzzy rules are extracted from numerical data based on each rule selection criterion. Then we examine the classification performance of extracted fuzzy rules through computational experiments on a number of benchmark data sets from the UCI ML Repository. Our results should be viewed as the lowest benchmark performance of fuzzy rule-based classification systems because fuzzy rules are extracted using a simple heuristic method with no optimization or tuning procedures. Nevertheless our results on some data sets are comparable to reported results by the C4.5 algorithm in the literature.

In this chapter, we explain how multi-objective genetic algorithms can be applied to the design of fuzzy rule-based systems for pattern classification problems. For a multi-class pattern classification problem with many continuous attributes (e.g., wine classification with 13 continuous attributes [1]), a fuzzy rule-based classification system is designed by a multi-objective genetic algorithm [2, 3] with two objectives: to minimize the size of the fuzzy rule-based classification system and to maximize its performance [4, 5]. The size of the fuzzy rule-based classification system is expressed by the number of fuzzy if-then rules, and its performance is measured by the number of correctly classified training patterns. In our approach to the design of the fuzzy rule-based classification system, first a large number of fuzzy if-then rules are generated from training patterns as candidate rules for the rule selection. Then a small number of fuzzy if-then rules are selected from the candidate rules by the two-objective genetic algorithm. The two-objective genetic algorithm tries to find all the non-dominated solutions (i.e., non-dominated rule sets) of the rule selection problem with the above-mentioned objectives. One difficulty of the rule selection method for high-dimensional pattern classification problems with many continuous attributes is that the number of candidate fuzzy if-then rules becomes intractably large. For example, the number of candidate rules is 613 ≅ 1.3 × 1010 for the wine classification problem with 13 continuous attributes if we have six fuzzy sets (terms) on each axis of the 13-dimensional pattern space.

Genetic fuzzy rule based classification systems for coronary plaque characterization based on intravascular ultrasound images

This paper presents a new fuzzy inference system for modeling of nonlinear dynamic systems based on input and output data with measurement noise. The proposed fuzzy system has a number of fuzzy rules and parameter values of membership functions which are automatically generated using the extended relevance vector machine (RVM). The RVM has a probabilistic Bayesian learning framework and has good generalization capability. The RVM consists of the sum of product of weight and kernel function which projects input space into high dimensional feature space. The structure of proposed fuzzy system is same as that of the Takagi-Sugeno fuzzy model. However, in the proposed method, the number of fuzzy rules can be reduced under the process of optimizing a marginal likelihood by adjusting parameter values of kernel functions using the gradient ascent method. After a fuzzy system is determined, coefficients in consequent part are found by the least square method. Examples illustrate effectiveness of the proposed new fuzzy inference system.

RST-BatMiner: A fuzzy rule miner integrating rough set feature selection and Bat optimization for detection of diabetes disease

A study of the behaviour of linguistic fuzzy rule based classification systems in the framework of imbalanced data-sets

In this paper, a new scheme for constructing parsimonious fuzzy classifiers is proposed based on the L2-support vector machine (L2-SVM) technique with model selection and feature ranking performed simultaneously in an integrated manner, in which fuzzy rules are optimally generated from data by L2-SVM learning. In order to identify the most influential fuzzy rules induced from the SVM learning, two novel indexes for fuzzy rule ranking are proposed and named as alpha-values and omega-values of fuzzy rules in this paper. The alpha-values are defined as the Lagrangian multipliers of the L2-SVM and adopted to evaluate the output contribution of fuzzy rules, while the omega-values are developed by considering both the rule base structure and the output contribution of fuzzy rules. As a prototype-based classifier, the L2-SVM-based fuzzy classifier evades the curse of dimensionality in high-dimensional space in the sense that the number of support vectors, which equals the number of induced fuzzy rules, is not related to the dimensionality. Experimental results on high-dimensional benchmark problems have shown that by using the proposed scheme the most influential fuzzy rules can be effectively induced and selected, and at the same time feature ranking results can also be obtained to construct parsimonious fuzzy classifiers with better generalization performance than the well-known algorithms in literature.

In several studies, fuzzy classifiers (FCs) have been shown to achieve higher generalization ability when learned through support vector machines (SVMs) compared to learning through neural networks that aim to minimize only the training error. This chapter introduces the learning of FCs using linear SVMs. Two types of FCs with zero-order and high-order Takagi-Sugeno (TS)-type consequents are considered. Given a number of rules, the antecedent parameters in the two FCs are determined using a self-splitting clustering (SSC) algorithm. Regarding the consequent parameter optimization, this chapter first describes the basic concept of linear SVMs followed by its application to the learning of the consequent parameters to endow the FCs with high generalization ability. These two types of FCs are subsequently applied to object detection and shape extraction using a red-green-blue-depth (RGB-D) camera. In this application, after the detection of an object using a color-feature-based FC, the depth information from the camera is used to extract the shape of an object. A histogram-based shape feature is proposed for improving the object detection performance. The performance of the proposed classification approach is evaluated through the detection of different objects and comparisons with various object detection approaches.

In this paper, we propose a particle swarm optimization method incorporating quantum qubit operation to construct and optimize fuzzy rule-based classifiers. The proposed algorithm, denoted as QiQPSO, is inspired by the quantum computing principles. It employs quantum rotation gates to update the probability of each qubit with the corresponding quantum angle updating according to the update equation of the quantum-behaved particle swarm optimization (QPSO). After description of the principle of QiQPSO, we show how to apply QiQPSO to establish a fuzzy classifier through two procedures. The QiQPSO algorithm is first used to construct the initial fuzzy classification system based on the sample data and the grid method of partitioning the feature space, and then the fuzzy rule base of the initial fuzzy classifier is optimized further by QiQPSO in order to reduce the number of the fuzzy rules and thus improve its interpretability. In order to verify the effectiveness of the proposed method, QiQPSO is tested on various real-world classification problems. The experimental results show that the QiQPSO is able to effectively select feature variables and fuzzy rules of the fuzzy classifiers with high classification accuracies. The performance comparison with other methods also shows that the fuzzy classifier optimized by QiQPSO has higher interpretability as well as comparable or even better classification accuracies.

In this paper we propose a fuzzy rule-generation method for pattern classification problems with classification priority. The assumption in this paper is that a classification priority is given a priori in relation to other classes. Our fuzzy rule-based classification system consists of a set of fuzzy if-then rules that are automatically generated from a set of given training patterns. The proposed method decides the consequent class of fuzzy if-then rules based on the number of covered training patterns for each class. In computational experiments we first show the effect of introducing classification priority for a synthetic two-dimensional problem. Then we show the effectiveness of the proposed method for several real-world pattern classification problems.

Learning fuzzy rules from iterative execution of games

We propose a hybrid algorithm of two fuzzy genetics-based machine learning approaches (i.e., Michigan and Pittsburgh) for designing fuzzy rule-based classification systems. First, we examine the search ability of each approach to efficiently find fuzzy rule-based systems with high classification accuracy. It is clearly demonstrated that each approach has its own advantages and disadvantages. Next, we combine these two approaches into a single hybrid algorithm. Our hybrid algorithm is based on the Pittsburgh approach where a set of fuzzy rules is handled as an individual. Genetic operations for generating new fuzzy rules in the Michigan approach are utilized as a kind of heuristic mutation for partially modifying each rule set. Then, we compare our hybrid algorithm with the Michigan and Pittsburgh approaches. Experimental results show that our hybrid algorithm has higher search ability. The necessity of a heuristic specification method of antecedent fuzzy sets is also demonstrated by computational experiments on high-dimensional problems. Finally, we examine the generalization ability of fuzzy rule-based classification systems designed by our hybrid algorithm.

There is a trend in recent machine learning community to construct a nonlinear version of a linear algorithm using the 'kernel method', e.g. Support Vector Machines (SVMs), kernel principal component analysis, kernel fisher discriminant analysis and the recent kernel clustering algorithms. In unsupervised clustering algorithms using kernel method, typically, a nonlinear mapping is used first to map the data into a potentially much higher feature space, where clustering is then performed. A drawback of these kernel clustering algorithms is thatthe clustering prototypes lie in high dimensional feature space and hence lack clear and intuitive descriptions unless using additional projection approximation from the feature to the data space as done in the existing literatures. In this paper, a novel clustering algorithm using the 'kernel method' based on the classical fuzzy clustering algorithm (FCM) is proposed and called as kernel fuzzy c-means algorithm (KFCM). KFCM adopts a new kernel-induced metric in the data space to replace the original Euclidean norm metric in FCM and the clustered prototypes still lie in the data space so that the clustering results can be reformulated and interpreted in the original space. Our analysis shows that KFCM is robust to noise and outliers and also tolerates unequal sized clusters. And finally this property is utilized to cluster incomplete data. Experiments on two artificial and one real datasets show that KFCM hasbetter clustering performance and more robust than several modifications of FCM for incomplete data clustering.

Interpretability improvements to find the balance interpretability-accuracy in fuzzy modeling: an overview.- Regaining comprehensibility of approximative fuzzy models via the use of linguistic hedges.- Identifying flexible structured premises for mining concise fuzzy knowledge.- A multiobjective genetic learning process for joint feature selection and granularity and contexts learning in fuzzy rule-based classification systems.- Extracting linguistic fuzzy models from numerical data-AFRELI algorithm.- Constrained optimization of fuzzy decision trees.- A new method for inducing a set of interpretable fuzzy partitions and fuzzy inference systems from data.- A Feature Ranking Algorithm for Fuzzy Modelling Problems.- Interpretability in multidimensional classification.- Interpretable semi-mechanistic fuzzy models by clustering, OLS and FIS model reduction.- Trade-off between approximation accuracy and complexity: TS controller design via HOSVD based complexity minimization.- Simplification and reduction of fuzzy rules.- Effect of rule representation in rule base reduction.- Singular value-based fuzzy reduction with relaxed normalization condition.- Interpretability, complexity, and modular structure of fuzzy systems.- Hierarchical genetic fuzzy systems: accuracy, interpretability and design autonomy.- About the trade-off between accuracy and interpretability of Takagi-Sugeno models in the context of nonlinear time series forecasting.- Accurate, transparent and compact fuzzy models by multi-objective evolutionary algorithms.- Transparent fuzzy systems in modeling and control.- Uniform fuzzy partitions with cardinal splines and wavelets: getting interpretable linguistic fuzzy models.- Relating the theory of partitions in MV-logic to the design of interpretable fuzzy systems.- A formal model of interpretability of linguistic variables.- Expressing relevance and interpretability of rule-based systems.- Conciseness of fuzzy models.- Exact trade-off between approximation accuracy and interpretability: solving the saturation problem for certain FRBSs.- Interpretability improvement of RBF-based neurofuzzy systems using regularized learning.- Extracting fuzzy classification rules from fuzzy clusters on the basis of separating hyperplanes.

Fuzzy systems based on fuzzy if-then rules have been applied to various problems. The main application area has been fuzzy control problems. In many cases, such fuzzy systems can handle only a few input variables. This is because the number of fuzzy if-then rules exponentially increases as the number of input variables increases. In this paper, we try to design fuzzy classification systems based on fuzzy if-then rules for multidimensional pattern classification problems with many attributes. For designing such fuzzy classification systems, we compare two frameworks in the area of genetics-based machine learning: the Michigan approach and the Pittsburgh approach. The performance of fuzzy rule-based classification systems is also compared with that of various pattern classification methods. In computer simulations, we use a wine classification problem with 13 attributes, a cancer diagnosis problem with 9 attributes, and a credit approval problem with 14 attributes. © 1997 Scripta Technica, Inc. Electron Comm Jpn Pt 3, 80(12): 10–19, 1997