Data Mining based on applying genetic algorithm for extracting rules from a BDB Neural Network

The Sixth International Symposium on Neural Networks (ISNN 2009)

این مطالعه به منظور بررسی پیش بینی شاخص های عملکرد تولیدی در مرغان تخم گذار با استفاده از شبکه های عصبی مصنوعی و رگرسیون غیرخطی چندگانه انجام شد. بررسی بر روی اطلاعات چهار دوره متوالی پرورش در یک واحد پرورش مرغ تخم گذار صورت گرفت. روش های داده-کاوی شامل رگرسیون خطی و غیر خطی، شبکه عصبی پرسپترون سه لایه، شبکه عصبی پرسپترون چهار لایه و شبکه عصبی با تابع پایه ای شعاعی بود. در این مدل ها از متغیرهای سن گله، میزان خوراک مصرفی و فصل تولید به عنوان متغیر پیشگو و شاخص های عملکرد تولیدی شامل درصد تخم-گذاری، وزن توده ای تخم مرغ تولیدی و ضریب تبدیل غذایی به عنوان متغیر پاسخ استفاده شد. نتایج نهایی رگرسیون های خطی نشان داد که برای تمامی متغیرهای وابسته مورد مطالعه متغیر مستقل سن گله معنی دار می باشد. بنابراین رگرسیون غیر خطی شاخص های عملکرد تولیدی در مقابل سن برای مقایسه با شبکه های عصبی مختلف مورد بررسی قرار گرفت و برای مقایسه کلیه مدل ها از ضریب تعیین (R2) و میانگین قدر مطلق خطا (MAE) استفاده شد. نتایج نشان داد بین شبکه های عصبی مصنوعی مختلف مورد مطالعه، شبکه با تابع پایه ای شعاعی بهتر از سایر مدل های در پیش‌بینی شاخص های عملکرد تولیدی مرغان تخم گذار عمل می کند.

Data Mining is a powerful technology to help organization to concentrate on most important data by extracting useful information from large database. One of the most commonly used techniques in data mining is Artificial Neural Network due to its high performance in many application domains. Despite many advantages of Artificial Neural Network, one of its main drawbacks is its inherent black box nature which is the main problem of using Artificial Neural Network in data mining. Therefore, this paper proposes a rule extraction algorithm from neural network using classified and misclassified data to convert the black box nature of Artificial Neural Network into a white box. The proposed algorithm is a modification of the existing algorithm, Rule Extraction by Reverse Engineering (RxREN). The proposed algorithm extracts rules from trained neural network for datasets with mixed mode attributes using pedagogical approach. The proposed algorithm uses both classified as well as misclassified data to find out the data ranges of significant attributes in respective classes, which is the innovation of the proposed algorithm. The experimental results clearly show that the performance of the proposed algorithm is superior to existing algorithms.

Rule extraction based on PROMETHEE-assisted multi-objective genetic algorithm for generating interpretable neural networks

The Brain Emotional Learning (BEL) is a novel bio-inspired machine learning approach mentioned as a new class of artificial neural network (ANN). The artificial emotional neural network (AENN) is one of the BEL methods which used the genetic algorithm (GA) to compute proper weights, weights of the amygdala (AMYG), orbitofrontal cortex (OFC) weights and a bias value. AENN trained by GA has been reported that it could produce low error rates. However, AENN still has more rooms to enhance its prediction of performance, especially generalization and the prediction of performance. Therefore, this paper aims to propose a new training method for AENN. The JAYA optimization algorithm optimized the weights and the biases of AENN. Two new proposed models are named as AENN-Max-JAYA and AENN-Mean-JAYA. Their names are according to the way of selecting the additional expanded feature which obtained through either the max or the average of input patterns, respectively. From the experimental results for wind power forecasting dataset, the proposed methods proved that the results are better in generalization performance and give lower error rates which compared to the comparative AENN models and traditional ANNs.

NOVEL STAGING TOOL FOR LOCALIZED PROSTATE CANCER: A PILOT STUDY USING GENETIC ADAPTIVE NEURAL NETWORKS

Data mining (DM) is not a well-understood subject. Data mining is perceived as being a mystical, not mathematically rigorous methodology that often produces results without any validation or verification. Some of that is a correct perception. Indeed, few mathematical theorems demonstrate the rigorous validity of Data Mining. We need to interest more mathematicians in this field—a field that was born from Biology and not from Mathematics. Then, why use it if it is not mathematically fully validated? Simply, it is the best strategy we have knowledge and data are sparse, incomplete, or imprecise. We do not use this technique in a well-defined and well-understood problem where models have been proven and validated. Instead, we use DM when those mechanistic/deterministic models fail, knowledge is insufficient, but data is abundant. Data Mining comprises several mathematical techniques. They fulfill different needs: Expert System is a reasoning-based technique that is useful when knowledge is available and rules on a system’s behavior can be derived. One part of an Expert System is a technique now known as Block-Chain (originally as Black-Board). Information is shared securely. A sharing partner can acquire someone else’s information as long they add information to the blackboard. This technique was popularized by the Bitcoin industry and is now being rapidly accepted by Banks. Expert Systems are used mainly when knowledge is available. Additional information can be obtained by forming knowledge trees that are concatenated. Artificial Neural Networks (ANN) are techniques that can be used when only data instead of knowledge is available. The approach "mimics" the brain's "Pattern-Recognition" capabilities. It is mainly used to extract knowledge from data. There are two main types of ANN: Supervised and unsupervised learning. As an example of Supervise d learning are those ANN tools used to discover relationships between the dependent (output) and independent (input) variables. The non-supervised ANN are frequently used as excellent tools for Data Compression and for estimating the most probable values in incomplete data. Those main techniques get help from Fuzzy Logic, which enables us to deal with the data's uncertainties or with incomplete knowledge that is available. Search Techniques, such as Genetic algorithms (an exciting and fun option instead of Optimization techniques). Traditional Optimization techniques and Statistics are commonly used too as part of Data Mining. In this paper, I will address each technique's fundamentals, discuss the validity of the results, and guide the reader at the best choices for a given problem type. We will close the paper with examples on using the techniques for alloy design and stress corrosion cracking in Nuclear reactors, time permitting.

A new classification framework for a Clinical Decision Support System, utilizing a Genetic algorithm and an Artificial Flora Optimized Neural Network is presented in this paper. GAFON is an artificial neural network whose topology is optimized with Genetic Algorithm and the learnable parameters are optimized with Artificial Flora Optimization algorithm. Drop out technique is used in the topology optimization phase and weight regularization is used in the parameter optimization phase. The proposed method minimizes the co-adaptation problem, reduces over-fitting of training data and improves the generalization of a feed forward neural network. The classification framework developed has been tested for classifying both multi class and binary class clinical datasets. The proposed method attained accuracy values of 86.82% for Hepatitis C Virus (HCV) for Egyptian patients, 84.91% for Vertebral Column 95.65% for Statlog Heart Disease (SHD), SHD and 93.79% for Early Stage Diabetes Risk Prediction (ESDRP), all datasets obtained from UCI repository

Using artificial neural network and genetics algorithm to estimate the resilient modulus for stabilized subgrade and propose new empirical formula

Data Mining refers to a set of methods applicable to large and complex databases to eliminate the randomness and discover the hidden pattern. Datamining (DM), also known as knowledge discovery from databases (KDD), is the extraction of new knowledge from huge databases. Data mining involves the use of sophisticated data analysis tools to discover previously unknown, valid patterns and relationships in large data sets. Data mining tools can forecast the future trends and activities to support the decision of people. The scope of datamining is associated with Uncovering trends and patterns are a great power for the businesses of all sectors and industries. Modern intrusion detection applications are confronted with a variety of issues. These applications must be reliable, extensible, manageable, and minimal in maintenance costs. Data mining-based intrusion detection systems (IDSs) have shown high accuracy, good generalisation to novel types of intrusion, and stable behaviour in a changing environment in recent years. The number of hidden layers in various neural network topologies is evaluated in order to discover the best neural network. The technique of attempting to discover instances of network attacks by comparing current behaviour to the expected actions of an intruder is known as misuse detection. Artificial neural networks have the ability to detect and classify network activity using data that is limited, incomplete, and nonlinear. The main purpose of this work is to identify privacy and security concerns among cloud computing participants and consumers in a distributed environment. Techniques like Machine Learning, Natural Language Processing (NLP), and Data Mining are combined to automatically identify and uncover patterns from many sorts of materials. Predictive analytics is capable of dealing with both continuous and discontinuous changes. Classification, prediction, and to some extent, affinity analysis constitute the analytical methods employed in predictive analytics. The contemporary study in text or document mining is focusing on syntactic components and the semantic environment. In order to accomplish this, and with the motivation gained from our previous research contributions, we investigated a mining model to classify documents based on the Order of Context, Concept, and Semantic Relations (OCCSR). The use of data mining techniques based on Cloud computing will enable users to retrieve meaningful information from virtually integrated data warehouses, lowering infrastructure and storage costs. Data mining can extract useful and potentially useful information from the cloud. Big Data is typically defined by three characteristics known as the 3Vs (Volume, Velocity and Variety). The surveys approaches, environments, and technologies in key areas for Big Data analytics capabilities and discusses how they aid in the development of analytics solutions for Clouds. The clustering technique belongs to an unsupervised learning and it is used to discover a new set of categories. Grid-based clustering has the shortest processing time, which is typically determined by the size of the grid rather than the data. We compare the performance of three clustering algorithms: hierarchical clustering, density-based clustering, and K Means clustering. The majority of current approaches to detecting misuse involve the use of rule-based expert systems to identify indicators of known attacks. We provide a brief overview of the use of various Artificial Intelligence techniques and their advancements in the design, development, and application of Intrusion Detection Systems (IDS) for protecting computer and communication networks from intruders. The goal of Knowledge Discovery in Data (KDD) is to extract information that is not obvious by using careful and detailed analysis and interpretation. To drive decisions and actions, analytics employs KDD, data mining, text mining, statistical and quantitative analysis, explanatory and predictive models, and advanced and interactive visualisation techniques.

This paper presents an investigation of an artificial neural network (ANN) to predict the flutter characteristics of a simple wing that under went multiple weight variation changes. The weight variations were used to simulate the addition of non-structural weight or structural repair weight. The test case wing had 24 designated repair areas where repair weights were added as lumped masses. Weight and location of the repairs were randomly generated to increase the sample size and help the generalization of the ANN. The ANN was evaluated for both the flutter speed and flutter frequency for multiple levels of structural damping. The number of test cases required to train the neural network was also evaluated for both the error between the input and output results as well as the generalization of the neural networks.

In recent times, research efforts have been directed towards hybridization of various intelligent methodologies in order to solve complex industrial problems. At the same time these research efforts have been undertaken to develop a better understanding of the human information processing system. Today, one can find a number of applications involving hybridization of intelligent methodologies like knowledge based systems, fuzzy systems, genetic algorithms, and case based reasoning with artificial neural networks. The central methodology of many hybrid systems has been artificial neural networks. In fact, 2/3rd of the applications involving intelligent hybrid systems use neural networks. Broadly, in these applications neural networks are either used as the primary problem solving entity, or are used in conjunction with other intelligent methodology/ies which have a distinct and separate role to play in the problem solving process. The former are categorized as fusion and/or transformation based approaches, and the later are categorized as combination approaches. However, fusion and/or transformation based approaches center around not only neural networks but also center around the other other methodology, namely, genetic algorithms. Similarly, the combination approaches can involve intelligent methodologies other than neural networks also. The goal of this chapter is to provide an overview to the reader about fusion and transformation based synergies with neural network and genetic algorithms as the primary problem solving entities. In this direction, this chapter looks at the neuro-symbolic systems, neuro-fuzzy systems, genetic-neuro systems, and genetic-fuzzy systems.

Rule extraction from data is one of the key technologies for solving the bottlenecks in artificial intelligence. Artificial neural networks are well suited for representing any knowledge in given data. Extraction of logical/fuzzy rules from the trained artificial neural network is of great importance to researchers in the fields of artificial intelligence and soft computing. Fuzzy rule sets are capable of approximating any nonlinear mapping relationships. Extraction of rules from data has been discussed in terms of fuzzy modeling, fuzzy clustering, and classification with fuzzy rule sets. This special issue entitled"Rule Extraction from Data" is aimed at providing the readers with good insights into the advanced studies in the field of rule extraction from data using neural networks/fuzzy rule sets. I invited seven research papers best suited for the theme of this special issue. All the papers were reviewed rigorously by two reviewers each. The first paper proposes an interesting rule extraction method from data using neural networks. Ishikawa presents a combination of learning with an immediate critic and a structural learning with forgetting. This method is capable of generating skeletal networks for logical rule extraction from data with correct and wrong answers. The proposed method is applied to rule extraction from lense data. The second paper presents a new methodology for logical rule extraction based on transformation of MLP (multilayered perceptron) to a logical network. Duck et al. applied their C-MLP2LN to the Iris benchmark classification problem as well as real-world medical data with very good results. In the third paper, Geczy and Usui propose fuzzy rule extraction from trained artificial neural networks. The proposed algorithm is implied from their theoretical study, not from heuristics. Their study enables to initially consider derivation of crisp rules from trained artificial neural network, and in case of conflict, application of fuzzy rules. The proposed algorithm is experimentally demonstrated with the Iris benchmark classification problem. The fourth paper presents a new framework for fuzzy modeling using genetic algorithm. The authors have broken new ground of fuzzy rule extraction from neural networks. For the fuzzy modeling, they have proposed a particular type of neural networks containing nodes representing membership functions. In this fourth paper, the authors discuss input variable selection for the fuzzy modeling under multiple criteria with different importance. A target system with a strong nonlinearity is used for demonstrating the proposed method. Kasabov, et al. present, in the fifth paper, a method for extraction of fuzzy rules that have different level of abstraction depending on several modifiable thresholds. Explanation quality becomes better with higher threshold values. They apply the proposed method to the Iris benchmark classification problem and to a real world problem. J. Yen and W. Gillespie address interpretability issue of Takagi-Sugeno-Kang model, one of the most popular fuzzy mdoels, in the fifth paper. They propose a new approach of fuzzy modeling that ensures not only a high approximation of the input-output relationship in the data, but also good insights about the local behavior of the model. The proposed method is applied to fuzzy modeling of sinc function and Mackey-Glass chaotic time series data. The last paper discusses fuzzy rule extraction from numerical data for high-dimensional classification problems. H.Ishibuchi, et al. have been pioneering methods for classification of data using fuzzy rules and genetic algorithm. In this last paper, they introduced a new criterion, simplicity of each rule, together with the conventional ones, compactness of rule base and classification ability, for high-dimensional problem. The Iris data is used for demonstrating their new classification method. They applied it also to wine data and credit data. I hope that the readers will be encouraged to explore the frontier to establish a new paradigm in the field of knowledge representation and rule extraction.

This paper focuses on the Genetic Algorithm learning paradigm applied to train the ANNs for balancing the cart-pole balancing system. The studied system is a classic control problem namely “cart-pole” problem. We will apply the unconventional techniques Artificial Neural Network, Genetic Algorithm and Fuzzy Logic to a classic control problem "cart-pole”. In this paper we have tried to train the Artificial Neural Network (ANN) with using Genetic Algorithms (program is written in MATLAB) which is compared with the output obtained using the Artificial Neural Network Toolbox provided in MATLAB. In proposed approach we have used both ANNs and Genetic Algorithm to get more optimal solution. Here we applied the approach for the Fuzzy logic technique to design a Fuzzy Logic Controllers (FLC) using ANNs and Genetic Algorithm (GA). The Fuzzy rules which are needed to control the problem will be framed with the combination of Artificial Neural Networks and Genetic Algorithm. It has been found that such a searching technique converges intelligently and much faster than conventional learning means. Performance of the presented neural network training using the genetic algorihtms is much better and providing more accurate results.

Feature subset selection is of immense importance in the field of data mining. The increased dimensionality of data makes testing and training of general classification method difficult. This paper presents the development of a model for classifying Pima Indian diabetic database (PIDD). The model consists of two stages. In the first stage, genetic algorithm (GA) and Correlation based feature selection have been used in a cascaded fashion. GA rendered global search of attributes with fitness evaluation effected by CFS. The second stage a fine tuned classification is done using artificial neural networks by making the feature subset elicited in the first stage as inputs for the network. Experimental results signify that the feature subset identified by the proposed filter when given as input to Back propagation neural network classifier, lead to enhanced classification accuracy.