A Novel Fuzzy Rule Based Neuro-system with Sparse Rule Extraction for Classification Problems

Automatic generation of fuzzy rule base and membership functions from an input-output data set, for reliable construction of an adaptive fuzzy inference system, has become an important area of research interest. We propose a new robust, fast acting adaptive fuzzy pattern classification scheme, named influential rule search scheme (IRSS). In IRSS, rules which are most influential in contributing to the error produced by the adaptive fuzzy system are identified at the end of each epoch and subsequently modified for satisfactory performance. This fuzzy rule base adjustment scheme is accompanied by an output membership function adaptation scheme for fine tuning the fuzzy system architecture. This iterative method has shown a relatively high speed of convergence. Performance of the proposed IRSS is compared with other existing pattern classification schemes by implementing it for Fisher's iris data problem and Wisconsin breast cancer data problems.

We examine the performance of compact fuzzy rule-based classification systems that consist of a small number of simple fuzzy rules with high comprehensibility. Those fuzzy systems are designed in a heuristic manner using rule selection criteria. We first describe fuzzy rule-based classification. Next we describe heuristic rule selection criteria using the terminology in data mining: confidence and support. A small number of fuzzy rules are extracted from numerical data based on each rule selection criterion. Then we examine the classification performance of extracted fuzzy rules through computational experiments on a number of benchmark data sets from the UCI ML Repository. Our results should be viewed as the lowest benchmark performance of fuzzy rule-based classification systems because fuzzy rules are extracted using a simple heuristic method with no optimization or tuning procedures. Nevertheless our results on some data sets are comparable to reported results by the C4.5 algorithm in the literature.

The process of monitoring the events occurring in a computer system or network and analyzing them for sign of intrusions is known as intrusion detection system (IDS). The objective of this paper is to extract fuzzy classification rules for intrusion detection in computer networks. The proposed method is based on the iterative rule learning approach (IRL) to fuzzy rule base system design. The fuzzy rule base is generated in an incremental fashion, in that the evolutionary algorithm optimizes one fuzzy classifier rule at a time. The performance of final fuzzy classification system has been investigated using intrusion detection problem as a high-dimensional classification problem. Results show that the presented algorithm produces fuzzy rules, which can be used to construct a reliable intrusion detection system.

Rule extraction from data is one of the key technologies for solving the bottlenecks in artificial intelligence. Artificial neural networks are well suited for representing any knowledge in given data. Extraction of logical/fuzzy rules from the trained artificial neural network is of great importance to researchers in the fields of artificial intelligence and soft computing. Fuzzy rule sets are capable of approximating any nonlinear mapping relationships. Extraction of rules from data has been discussed in terms of fuzzy modeling, fuzzy clustering, and classification with fuzzy rule sets. This special issue entitled"Rule Extraction from Data" is aimed at providing the readers with good insights into the advanced studies in the field of rule extraction from data using neural networks/fuzzy rule sets. I invited seven research papers best suited for the theme of this special issue. All the papers were reviewed rigorously by two reviewers each. The first paper proposes an interesting rule extraction method from data using neural networks. Ishikawa presents a combination of learning with an immediate critic and a structural learning with forgetting. This method is capable of generating skeletal networks for logical rule extraction from data with correct and wrong answers. The proposed method is applied to rule extraction from lense data. The second paper presents a new methodology for logical rule extraction based on transformation of MLP (multilayered perceptron) to a logical network. Duck et al. applied their C-MLP2LN to the Iris benchmark classification problem as well as real-world medical data with very good results. In the third paper, Geczy and Usui propose fuzzy rule extraction from trained artificial neural networks. The proposed algorithm is implied from their theoretical study, not from heuristics. Their study enables to initially consider derivation of crisp rules from trained artificial neural network, and in case of conflict, application of fuzzy rules. The proposed algorithm is experimentally demonstrated with the Iris benchmark classification problem. The fourth paper presents a new framework for fuzzy modeling using genetic algorithm. The authors have broken new ground of fuzzy rule extraction from neural networks. For the fuzzy modeling, they have proposed a particular type of neural networks containing nodes representing membership functions. In this fourth paper, the authors discuss input variable selection for the fuzzy modeling under multiple criteria with different importance. A target system with a strong nonlinearity is used for demonstrating the proposed method. Kasabov, et al. present, in the fifth paper, a method for extraction of fuzzy rules that have different level of abstraction depending on several modifiable thresholds. Explanation quality becomes better with higher threshold values. They apply the proposed method to the Iris benchmark classification problem and to a real world problem. J. Yen and W. Gillespie address interpretability issue of Takagi-Sugeno-Kang model, one of the most popular fuzzy mdoels, in the fifth paper. They propose a new approach of fuzzy modeling that ensures not only a high approximation of the input-output relationship in the data, but also good insights about the local behavior of the model. The proposed method is applied to fuzzy modeling of sinc function and Mackey-Glass chaotic time series data. The last paper discusses fuzzy rule extraction from numerical data for high-dimensional classification problems. H.Ishibuchi, et al. have been pioneering methods for classification of data using fuzzy rules and genetic algorithm. In this last paper, they introduced a new criterion, simplicity of each rule, together with the conventional ones, compactness of rule base and classification ability, for high-dimensional problem. The Iris data is used for demonstrating their new classification method. They applied it also to wine data and credit data. I hope that the readers will be encouraged to explore the frontier to establish a new paradigm in the field of knowledge representation and rule extraction.

The central point in designing a fuzzy system for modeling and control is the generation of a set of efficient and effective fuzzy rules. Generally, fuzzy rules can be generated either from heuristics or from experimental data. Heuristic fuzzy rule generation, as discussed in Chapter 1, consists of the following main steps: Determination of the structure of the fuzzy systems. This includes mainly the selection of the inputs and outputs. Usually, a multi-input multi-output fuzzy systems can be decomposed into a number of multi-input single-output fuzzy systems. Definition of linguistic terms and their fuzzy membership function for each fuzzy variables. If a Mamdani fuzzy system is designed, each input and output can be seen as a linguistic variable. Therefore, it is necessary to define the linguistic terms (e.g., Small, Medium, Large etc.) and their fuzzy membership function. Details about the definition of a linguistic variable is presented in Section 1.2.1. Determination of fuzzy inference method. For example, for fuzzy mapping rules, the fuzzy Cartesian product can be used to define the fuzzy relation. Refer to Section 1.2.2. Defuzzification. If the Mamdani type fuzzy rules are used, the fuzzy output derived from the fuzzy rule base should be defuzzified. Refer also to Section 1.2.2 for different defuzzification methods. If the Takagi-Sugeno fuzzy rules are used, the defuzzification is included in the fuzzy inference and a crisp output is obtained directly.

Over the last years fuzzy control has become a very popular and successful control paradigm. The basic idea of fuzzy control is to incorporate human expert knowledge. This expert knowledge is specified in a rule based manner on a high and granular level of abstraction. By using vague predicates a fuzzy rule base neglects useless details and concentrates on important relations. Following L.A. Zadeh’s famous principle of incompatibility, this technique is most promising when applied to large and complex problems. Nevertheless, nowadays most fuzzy rule bases are small and represent simple knowledge. From our point of view this surprising and somewhat disappointing observation is due to a major lack of understanding how to handle a fuzzy rule base. In this paper we present a new theory for fuzzy reasoning. This theory is twofold. In general, a fuzzy rule base is both partially inconsistent and partially incomplete. This is the price to pay for abstraction and granularization. We show that if a fuzzy rule base maximizes consistency at the cost of completeness, the well-known possibilistic approach to fuzzy inference is the right choice. For a fuzzy rule base that maximizes completeness at the cost of consistency, we derive a new type of inference called σ-reasoning. Together, both mechanisms form an embracing theory for fuzzy reasoning in general. We propose a combined approach to be applied in order to manage complex rule bases.

In Fuzzy Rule Based Systems, the process of extracting rule bases from data can lead to problems related to scalability and computational cost. These situations usually occur when considering contexts where data sets have large volume, such as in Big Data problems. Parallel computing approaches with evolutionary computing techniques such as Multi-objective Genetic Algorithms can be used together to enable scalability and reduce the computational cost involved in the extraction process of Rules Bases of the Fuzzy Systems used in Big Data problems. This paper proposes a parallel modeling strategy for the generation of Fuzzy Rules Bases in Big Data problems using the NSGA-DO Multi-objective Genetic Algorithm. The aim of the proposal is to allow the increase of scalability and the reduction of computational cost when maximizing the accuracy (increase the precision) and minimizing the complexity (decrease number of rules and antecedents of the rules) of Fuzzy Systems generate in the context of Big data problems.

One direction of measured data-set based modeling applies fuzzy logic identification tools and results in a fuzzy rule-base model. A typical problem of fuzzy identification methods is that the complexity of the resulting fuzzy rule-base, namely the number of rules in the rule-base, explodes with the modeling accuracy. As a result, the topic of fuzzy rule-base complexity reduction techniques emerged in the last decade. A common disadvantage of fuzzy rule-base complexity reduction methods is that the resulting complexity minimized fuzzy-rule bases cannot be simply adapted to new information. If we have new information that cannot be described by the fuzzy rules of the complexity minimized fuzzy rule-base, then we have two choices. The first choice is to add new fuzzy rules to the fuzzy rule-base until the new information can be described. The second choice is to modify the new information until it can be described by the fuzzy rule-base without using additional fuzzy rules. This second case has the prominent role if the number of fuzzy rules in the fuzzy rule-base is limited. This paper proposes a method for the second choice. The proposed method minimizes the necessary modification of the new information. This paper focuses attention on a recent complexity reduction method, termed Higher Order Singular Value Decomposition (HOSVD)-based complexity reduction, and Takagi-Sugeno (TS) inference operator-based fuzzy rule-bases. An example is used to provide the validation of the proposed method. In order to demonstrate the effectiveness of the proposed method, a control system of a differential-steered automatic guided vehicle is modeled in the paper.

In most fuzzy systems, the completeness of the fuzzy rule base is required to generate meaningful output when classical fuzzy reasoning methods are applied. This means, in other words, that the fuzzy rule base has to cover all possible inputs. Regardless of the way of rule base construction, be it created by human experts or by an automated manner, often incomplete rule bases are generated. One simple solution to handle sparse fuzzy rule bases and to make infer reasonable output is the application of fuzzy rule interpolation (FRI) methods. In this paper, we present a Fuzzy Rule Interpolation Matlab Toolbox, which is freely available. With the introduction of this Matlab Toolbox, different FRI methods can be used for different real time applications, which have sparse or incomplete fuzzy rule base.

Proposes a fuzzy neural expert system (FNES) which has a feedforward fuzzy neural network whose input layer consists of fuzzy cell groups and crisp (non-fuzzy) cell groups. Here, the truthfulness of fuzzy information and crisp information of training data is represented by fuzzy cell groups and crisp cell groups, respectively. The expert system has the following two functions: generalization of the information derived from the training data and embodiment of knowledge in the form of the fuzzy neural network; and extraction of fuzzy If-Then rules with linguistic relative importance of each proposition in an antecedent (If-part) from a trained fuzzy neural network. The paper also gives a method to extract automatically fuzzy If-Then rules from the trained neural network. To prove the effectiveness and validity of the proposed fuzzy neural expert system, a fuzzy neural expert system for medical diagnosis has been developed. >

Fuzzy signature sets (FSigSets) are extensions of the original fuzzy set concept, and also of the Vector Valued Fuzzy Set notion. In a FSigSet rule base the (input) universe of discourse X is mapped into a set of hierarchically grouped fuzzy sets, and each element of X has a “membership degree” consisting of a rooted tree with membership degrees at each leaf and aggregations at the intermediate vertices. The structure of the tree is identical for each element in the case of homogenous FSigSets, and so are the aggregations, depending only on the position of the vertex. Interpolation in fuzzy rule bases allows the calculation of a conclusion in the output universe Y belonging to an observation even if there are gaps in the rule base and the observation does not intersect with any of the antecedent sets. The key question here is how to determine the degree of similarity, or inversely, the distance, of any observation from the surrounding antecedents of the rules in the base, so that the distance incorporates the information involved with the close connection of the features in the sub-groups, and the aggregations expressing the form of this connection. A solution is proposed, and a pair of numerical examples is presented.

A new approach to design of a fuzzy-rule-based classifier that is capable of selecting informative features is discussed. Three basic stages of the classifier construction—feature selection, generation of fuzzy rule base, and optimization of the parameters of rule antecedents—are identified. At the first stage, several feature subsets on the basis of discrete harmonic search are generated by using the wrapper scheme. The classifier structure is formed by the rule base generation algorithm by using extreme feature values. The optimal parameters of the fuzzy classifier are extracted from the training data using continuous harmonic search. Akaike information criterion is deployed to identify the best performing classifiers. The performance of the classifier was tested on real-world KEEL and KDD Cup 1999 datasets. The proposed algorithms were compared with other fuzzy classifiers tested on the same datasets. Experimental results show efficiency of the proposed approach and demonstrate that highly accurate classifiers can be constructed by using relatively few features.

A framework for modelling the safety of an engineering system using a fuzzy rule-based evidential reasoning (FURBER) approach has been recently proposed, where a fuzzy rule-base designed on the basis of a belief structure (called a belief rule base) forms a basis in the inference mechanism of FURBER. However, it is difficult to accurately determine the parameters of a fuzzy belief rule base (FBRB) entirely subjectively, in particular for complex systems. As such, there is a need to develop a supporting mechanism that can be used to train in a locally optimal way a FBRB initially built using expert knowledge. In this paper, the methods for self-tuning a FBRB for engineering system safety analysis are investigated on the basis of a previous study. The method consists of a number of single and multiple objective nonlinear optimization models. The above framework is applied to model the system safety of a marine engineering system and the case study is used to demonstrate how the methods can be implemented.

Combining boosting and evolutionary algorithms for learning of fuzzy classification rules

Introduces a singular value-based method for reducing a given fuzzy rule set. The method conducts singular value decomposition of the rule consequents and generates certain linear combinations of the original membership functions to form new ones for the reduced set. The present work characterizes membership functions by the conditions of sum normalization (SN), nonnegativeness (NN), and normality (NO). Algorithms to preserve the SN and NN conditions in the new membership functions are presented. Preservation of the NO condition relates to a high-dimensional convex hull problem and is not always feasible in which case a closed-to-NO solution may be sought. The proposed method is applicable regardless of the adopted inference paradigms. With product-sum-gravity inference and singleton support fuzzy rule base, output errors between the full and reduced fuzzy set are bounded by the sum of the discarded singular values. The work discusses three specific applications of fuzzy reduction: fuzzy rule base with singleton support, fuzzy rule base with nonsingleton support (which includes the case of missing rules), and the Takagi-Sugeno-Kang (TSK) model. Numerical examples are presented to illustrate the reduction process.