Incorporating Perception-Based Information in Reinforcement Learning Using Computing with Words

The authors recently suggested a new approach to the design of fuzzy rules. The method, referred to as fuzzy Lyapunov synthesis, extends classical Lyapunov synthesis to the domain of computing with words, and allows the systematic, instead of heuristic, design and analysis of fuzzy controllers given linguistic information about the plant. In this paper we use the fuzzy Lyapunov synthesis method to design and analyze the rule-base of a fuzzy scheduler. We show that fuzzy rules, previously suggested based on heuristics, can be derived systematically and, therefore, that the entire process can be automated. This may lead to a novel computing with words algorithm: the input is linguistic information concerning the plant and the control objective, and the output is a suitable fuzzy rule-base.

The central point in designing a fuzzy system for modeling and control is the generation of a set of efficient and effective fuzzy rules. Generally, fuzzy rules can be generated either from heuristics or from experimental data. Heuristic fuzzy rule generation, as discussed in Chapter 1, consists of the following main steps: Determination of the structure of the fuzzy systems. This includes mainly the selection of the inputs and outputs. Usually, a multi-input multi-output fuzzy systems can be decomposed into a number of multi-input single-output fuzzy systems. Definition of linguistic terms and their fuzzy membership function for each fuzzy variables. If a Mamdani fuzzy system is designed, each input and output can be seen as a linguistic variable. Therefore, it is necessary to define the linguistic terms (e.g., Small, Medium, Large etc.) and their fuzzy membership function. Details about the definition of a linguistic variable is presented in Section 1.2.1. Determination of fuzzy inference method. For example, for fuzzy mapping rules, the fuzzy Cartesian product can be used to define the fuzzy relation. Refer to Section 1.2.2. Defuzzification. If the Mamdani type fuzzy rules are used, the fuzzy output derived from the fuzzy rule base should be defuzzified. Refer also to Section 1.2.2 for different defuzzification methods. If the Takagi-Sugeno fuzzy rules are used, the defuzzification is included in the fuzzy inference and a crisp output is obtained directly.

Fuzzy rules automatically derived from a set of training examples quite often produce better classification results than fuzzy rules translated from medical knowledge. This study aims to investigate the difference in domain representation between a knowledge-based and a data-driven fuzzy system applied to an electrocardiography classification problem. For a three-class electrocardiographic arrhythmia classification task a set of fifteen fuzzy rules is derived from medical expertise on the basis of twelve electrocardiographic measures. A second set of fuzzy rules is automatically constructed on thirty-nine MIT-BIH database's records. The performances of the two classifiers on thirteen different records are comparable and up to a certain extent complementary. The two fuzzy models are then analyzed, by using the concept of information gain to estimate the impact of each ECG measure on each fuzzy decision process. Both systems rely on the beat prematurity degree and the QRS complex width and neglect the P wave existence and the ST segment features. The PR interval is not well characterized across the fuzzy medical rules while it plays an important role in the data-driven fuzzy system. The T wave area shows a higher information gain in the knowledge based decision process, and is not very much exploited by the data-driven system. The main difference between a human designed and a data driven ECG arrhythmia classifier is found about the PR interval and the T wave.

Robot learning with GA-based fuzzy reinforcement learning agents

We have been using fuzzy data mining techniques to extract patterns that represent behavior for intrusion detection. We describe a variety of modifications that we have made to the data mining algorithms in order to improve accuracy and efficiency. We use sets of fuzzy association rules that are mined from network audit data as models of behavior. To detect anomalous behavior, we generate fuzzy association rules from new audit data and compute the similarity with sets mined from normal data. If the similarity values are below a threshold value, an alarm is issued. We describe an algorithm for computing fuzzy association rules based on Borgelt's (2001) prefix trees, modifications to the computation of support and confidence of fuzzy rules, a new method for computing the similarity of two fuzzy rule sets, and feature selection and optimization with genetic algorithms. Experimental results demonstrate that we can achieve better running time and accuracy with these modifications.

In this paper, we formulate a pattern classification problem as a reinforcement learning problem. The problem is realized with a temporal difference method in a fuzzy adaptive learning control network (FALCON-R). FALCON-R is constructed by integrating two basic FALCON-ART networks as function approximators, where one acts as a critic network (fuzzy predictor) and the other as an action network (fuzzy controller). Thorough performance evaluation using Fisher's Iris data is presented and compared against a novel FALCON-ART network. We show that the system can converge faster, is able to escape from local minima, and has excellent disturbance rejection capability and has strengths as a pattern classification technique.

Rule extraction from data is one of the key technologies for solving the bottlenecks in artificial intelligence. Artificial neural networks are well suited for representing any knowledge in given data. Extraction of logical/fuzzy rules from the trained artificial neural network is of great importance to researchers in the fields of artificial intelligence and soft computing. Fuzzy rule sets are capable of approximating any nonlinear mapping relationships. Extraction of rules from data has been discussed in terms of fuzzy modeling, fuzzy clustering, and classification with fuzzy rule sets. This special issue entitled"Rule Extraction from Data" is aimed at providing the readers with good insights into the advanced studies in the field of rule extraction from data using neural networks/fuzzy rule sets. I invited seven research papers best suited for the theme of this special issue. All the papers were reviewed rigorously by two reviewers each. The first paper proposes an interesting rule extraction method from data using neural networks. Ishikawa presents a combination of learning with an immediate critic and a structural learning with forgetting. This method is capable of generating skeletal networks for logical rule extraction from data with correct and wrong answers. The proposed method is applied to rule extraction from lense data. The second paper presents a new methodology for logical rule extraction based on transformation of MLP (multilayered perceptron) to a logical network. Duck et al. applied their C-MLP2LN to the Iris benchmark classification problem as well as real-world medical data with very good results. In the third paper, Geczy and Usui propose fuzzy rule extraction from trained artificial neural networks. The proposed algorithm is implied from their theoretical study, not from heuristics. Their study enables to initially consider derivation of crisp rules from trained artificial neural network, and in case of conflict, application of fuzzy rules. The proposed algorithm is experimentally demonstrated with the Iris benchmark classification problem. The fourth paper presents a new framework for fuzzy modeling using genetic algorithm. The authors have broken new ground of fuzzy rule extraction from neural networks. For the fuzzy modeling, they have proposed a particular type of neural networks containing nodes representing membership functions. In this fourth paper, the authors discuss input variable selection for the fuzzy modeling under multiple criteria with different importance. A target system with a strong nonlinearity is used for demonstrating the proposed method. Kasabov, et al. present, in the fifth paper, a method for extraction of fuzzy rules that have different level of abstraction depending on several modifiable thresholds. Explanation quality becomes better with higher threshold values. They apply the proposed method to the Iris benchmark classification problem and to a real world problem. J. Yen and W. Gillespie address interpretability issue of Takagi-Sugeno-Kang model, one of the most popular fuzzy mdoels, in the fifth paper. They propose a new approach of fuzzy modeling that ensures not only a high approximation of the input-output relationship in the data, but also good insights about the local behavior of the model. The proposed method is applied to fuzzy modeling of sinc function and Mackey-Glass chaotic time series data. The last paper discusses fuzzy rule extraction from numerical data for high-dimensional classification problems. H.Ishibuchi, et al. have been pioneering methods for classification of data using fuzzy rules and genetic algorithm. In this last paper, they introduced a new criterion, simplicity of each rule, together with the conventional ones, compactness of rule base and classification ability, for high-dimensional problem. The Iris data is used for demonstrating their new classification method. They applied it also to wine data and credit data. I hope that the readers will be encouraged to explore the frontier to establish a new paradigm in the field of knowledge representation and rule extraction.

This paper describes a method to build a learning system for mobile robots based on the fuzzy control technique. Range sensors are analyzed and used. The learning system generates a set of fuzzy rules based on the robot's behaviors. Two examples illustrate the learning capabilities: 1) learn-to-avoid-obstacle, and 2) learn-to-detect-landmark behaviors. Reinforcement learning is applied to correct the fuzzy rules. Fuzzy controllers are used to control the robot using a set of fuzzy rules. The results of learn-to-avoid behavior indicate a good learning performance in dynamic environments. The results of learn-to-detect-landmark behavior show the robot can detect landmarks correctly from 68.43% to 90.09%.

In this work, fuzzy logic-based navigation control for a 2-wheel mobile robot is proposed. An autonomous mobile robot must be able to move safely in an environment, to reach its target (or goal). The work began with the mathematical model of the robot that involved the kinematic model. After which the Simulink model was developed and then fuzzy logic controller was designed for goal seeking. The problem with the differential drive model is that separate fuzzy rules need to be developed for both left and right wheels of the robot and the steering angle cannot be controlled directly from the fuzzy rules. However, these limitations have been taken care of in this work. The linear velocity and steering angle of the mobile robot were directly controlled from a single set of fuzzy rules and this has helped to reduce the time of navigation of the robot. The performance was satisfactory and the results of the simulation showed that the robot could reach its goal during navigation in shorter time than if differential drive model alone was used.

RST-BatMiner: A fuzzy rule miner integrating rough set feature selection and Bat optimization for detection of diabetes disease

An autonomous mobile robot (AMR) has to cope with uncertain, incomplete or approximate information. Moreover it has to identify sudden perceptual situations to manoeuvre in real time. This paper describes a fuzzy rule based system (FRBS) approach controlling the movement of an autonomous mobile robot (MORIA). Difficult guiding and controlling properties of the robot are achieved by combining local actions and global strategies within the fuzzy controller. Different behaviors and perceptions are detected with the help of fuzzy rules and stored in fuzzy state variables (FSV). These state variables activate different fuzzy rule sets which in turn change the behavior of the fuzzy controller.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

This paper presents Cellular Fuzzy Oriented Classifier Evolution (CFORCE), a generic method for constructing fuzzy rules to divide an image into two segments: object and background. In CFORCE, a pair of fuzzy classification rule sets for object and background is defined as a processing unit, and the identical units are allocated in each pixel of an input image. Each unit computes matching degree of each pixel with object and background class iteratively with considering the matching degree of neighbor units. The algorithm has mainly two features: 1) designing the fuzzy rules using Fuzzy Oriented Classifier Evolution (FORCE) which develops fuzzy rules represented as directed graphs flexibly and automatically by Genetic Algorithm, and 2) performing iterative segmentation with considering spatial relationship between pixels besides local features. In natural image segmentation, many pixels are overlapped between different clusters. Therefore, considering the spatial relationship is important to classify the overlapped pixels correctly. We applied CFORCE to three different object segmentation, and showed that CFORCE extracted object regions successfully.

This paper proposes a fuzzy model of emotion and behaviours selection for an autonomous mobile robot. Conventional behaviour-based robots can only exhibit a single behaviour or action after sensing an external event. In order to make a robot to adapt to its environment and to exhibit more complex behaviours, we try to model animal emotions and behaviours by a set of Fril's support logic fuzzy rules. This set of fuzzy rules will allow the robot to control and to select different behaviours depending on the environment and its experience. An summary of the calculus of support logic is given in this paper.

Detection of abnormal traffic and network intrusions based on multiple fuzzy rules

In this paper, we introduce a new fuzzy reinforcement learning method to quality of service (QoS) provisioning cognitive transmission in cognitive radio networks. The cognitive transmissions under QoS constraints are treated here as the data sending at two different average power levels depending on the activity of the primary (licensed) users, which is determined by the secondary (unlicensed) users. For this transmission, the model is defined a state-transition model. The maximum throughput under these statistical QoS constraints is determined by using fuzzy QoS reinforcement learning techniques. The performance effectiveness of the proposed method is obtained in situations and comparison with the numerical method based on the effective capacity of the cognitive radio channel under various QoS constraints. It is shown that the hybrid AI method used outperforms comparable results obtained by the classical numerical method, including various situations with different QoS limitations.