Closed-Label Concept Lattice Based Rule Extraction Approach

Convolution neural networks (CNNs) have proven effectiveness, but they are not applicable to all datasets, such as those with heterogeneous attributes, which are often used in the finance and banking industries. Such datasets are difficult to classify, and to date, existing high-accuracy classifiers and rule-extraction methods have not been able to achieve sufficiently high classification accuracies or concise classification rules. This study aims to provide a new approach for achieving transparency and conciseness in credit scoring datasets with heterogeneous attributes by using a one-dimensional (1D) fully-connected layer first CNN combined with the Recursive-Rule Extraction (Re-RX) algorithm with a J48graft decision tree (hereafter 1D FCLF-CNN). Based on a comparison between the proposed 1D FCLF-CNN and existing rule extraction methods, our architecture enabled the extraction of the most concise rules (6.2) and achieved the best accuracy (73.10%), i.e., the highest interpretability–priority rule extraction. These results suggest that the 1D FCLF-CNN with Re-RX with J48graft is very effective for extracting highly concise rules for heterogeneous credit scoring datasets. Although it does not completely overcome the accuracy–interpretability dilemma for deep learning, it does appear to resolve this issue for credit scoring datasets with heterogeneous attributes, and thus, could lead to a new era in the financial industry.

Synergy effects between grafting and subdivision in Re-RX with J48graft for the diagnosis of thyroid disease

FCANN: A new approach for extraction and representation of knowledge from ANN trained via Formal Concept Analysis

Recently, incomplete formal contexts have received more and more attention from the communities of formal concept analysis. Different from a complete context where the binary relations between all the objects and attribute are known, an incomplete formal context has at least a pair of object and attribute with a completely unknown binary relation. Partially known formal concepts use interval sets to indicate the incompleteness. Three-way formal concept analysis is capable of characterizing a target set by combining positive and negative attributes. However, how to describe target set, by pointing out what attributes it has with certainty and what attributes it has with possibility and what attributes it does not has with certainty and what attributes it does not has with possibility, is still an open problem. This paper combines the ideas of three-way formal concept analysis and partially known formal concepts and presents a framework of approximate three-way concept analysis. At first, approximate object-induced and attribute-induced three-way concept lattices are introduced, respectively. And then, the relationship between approximate three-way concept lattice and classical three-way concept lattice are investigated. Finally, examples are presented to demonstrate and verify the obtained results.

Rule extraction from data is one of the key technologies for solving the bottlenecks in artificial intelligence. Artificial neural networks are well suited for representing any knowledge in given data. Extraction of logical/fuzzy rules from the trained artificial neural network is of great importance to researchers in the fields of artificial intelligence and soft computing. Fuzzy rule sets are capable of approximating any nonlinear mapping relationships. Extraction of rules from data has been discussed in terms of fuzzy modeling, fuzzy clustering, and classification with fuzzy rule sets. This special issue entitled"Rule Extraction from Data" is aimed at providing the readers with good insights into the advanced studies in the field of rule extraction from data using neural networks/fuzzy rule sets. I invited seven research papers best suited for the theme of this special issue. All the papers were reviewed rigorously by two reviewers each. The first paper proposes an interesting rule extraction method from data using neural networks. Ishikawa presents a combination of learning with an immediate critic and a structural learning with forgetting. This method is capable of generating skeletal networks for logical rule extraction from data with correct and wrong answers. The proposed method is applied to rule extraction from lense data. The second paper presents a new methodology for logical rule extraction based on transformation of MLP (multilayered perceptron) to a logical network. Duck et al. applied their C-MLP2LN to the Iris benchmark classification problem as well as real-world medical data with very good results. In the third paper, Geczy and Usui propose fuzzy rule extraction from trained artificial neural networks. The proposed algorithm is implied from their theoretical study, not from heuristics. Their study enables to initially consider derivation of crisp rules from trained artificial neural network, and in case of conflict, application of fuzzy rules. The proposed algorithm is experimentally demonstrated with the Iris benchmark classification problem. The fourth paper presents a new framework for fuzzy modeling using genetic algorithm. The authors have broken new ground of fuzzy rule extraction from neural networks. For the fuzzy modeling, they have proposed a particular type of neural networks containing nodes representing membership functions. In this fourth paper, the authors discuss input variable selection for the fuzzy modeling under multiple criteria with different importance. A target system with a strong nonlinearity is used for demonstrating the proposed method. Kasabov, et al. present, in the fifth paper, a method for extraction of fuzzy rules that have different level of abstraction depending on several modifiable thresholds. Explanation quality becomes better with higher threshold values. They apply the proposed method to the Iris benchmark classification problem and to a real world problem. J. Yen and W. Gillespie address interpretability issue of Takagi-Sugeno-Kang model, one of the most popular fuzzy mdoels, in the fifth paper. They propose a new approach of fuzzy modeling that ensures not only a high approximation of the input-output relationship in the data, but also good insights about the local behavior of the model. The proposed method is applied to fuzzy modeling of sinc function and Mackey-Glass chaotic time series data. The last paper discusses fuzzy rule extraction from numerical data for high-dimensional classification problems. H.Ishibuchi, et al. have been pioneering methods for classification of data using fuzzy rules and genetic algorithm. In this last paper, they introduced a new criterion, simplicity of each rule, together with the conventional ones, compactness of rule base and classification ability, for high-dimensional problem. The Iris data is used for demonstrating their new classification method. They applied it also to wine data and credit data. I hope that the readers will be encouraged to explore the frontier to establish a new paradigm in the field of knowledge representation and rule extraction.

Dual multi-adjoint concept lattices

Classifier building form distributed data sources has been a fundamental computational problem to realise distributed knowledge discovery. The classification rule extraction from distributed databases suffers from the problems of high communication cost, lack of interpretability of rules and poor performance in handling high categorical data. The aim of this paper is to extend fuzzy generalised association rule extraction technique which is well proved in handling such issues to extract classification rules from distributed datasets. This paper presents a distributed data driven fuzzy generalised associative classifier D3FGAC framework for distributed knowledge discovery which extracts data driven fuzzy generalisation rules from horizontally fragmented datasets with minimum communication cost and builds global compact classifier using extracted rules. The experiments conducted on UCI datasets and their comparisons to other existing model shown in article to prove the efficiency of proposed framework.

How to extract rules from trained SVMs has become an important preprocessing technique for data mining, pattern classification, and so on. There are two key problems required to be solved in the SVM based classification rule extraction, i.e. the attribute selection and the discretization to continuous attributes. In this paper, the differential characteristic of SVR (Support vector regression) is discussed. A new measure for determining the importance level of the attributes based on the trained SVR classifiers is proposed. Based on this new measure, a new approach for rule extraction from trained SVR classifiers is proposed. A new algorithm for rule extraction is given. The performance of the new approach is demonstrated by several computing cases. Experiment results show that the proposed approach can improve the validity of the extracted rules remarkably compared to other rule extracting approaches, especially for complicated classification problems.

In order to solve the problems that how to mine and express classification knowledge and rules in compatibility of prescription, this paper introduces a new theory of formal concept analysis (FCA), and realizes the compatibility of prescription knowledge mining. Meanwhile, the prescription drugs of reverting yin disease treatment in Treatise on febrile diseases are selected to apply the FCA theory to mine the compatibility of prescription. The result shows that complete knowledge which is related to prescription can be obtained from the attribute partial order chart, therefore it has been proved that the theory of FCA provides a new method to discovery the new knowledge of prescription of TCP.

Extraction and optimization of classification rules for temporal sequences: Application to hospital data

The high dimensional data are frequently met when we apply Web text classification. Mining in high dimensional data is extraordinarily difficult because of the curse of dimensionality. We must adopt feature dimensionality reduction to solve these problems. A attribute reduction algorithm based on rough set theory is given in this paper to reduce the text feature term and extract rule. First, the weight of feature term is made discrete. Then, the decision table is made with weight as the condition attributes and classes of texts as the decision attributes. Finally, the classification rules are extracted by attribute reduction. The method is simple and feasible. It is advantageous in improving the efficiency of the selected feature subset and suitable for high-volume text classification. The extracted rules are easy understand. The accuracy is higher and the speed of classification is faster than the classification based on vector space comparison. This paper describes the proposed technique and provides experimental results.

Formal concept analysis based on fuzzy granularity base for different granulations

Two-stage rule extraction method based on tree ensemble model for interpretable loan evaluation

In this paper we propose a knowledge discovery and visualisation method for unearthing emotional states from physiological data typically available from wearable devices. In addition we investigate the viability of using a limited set of wearable sensors to extract decision tree rules which are representative of physiological changes taking place during emotional changes. Our method utilised a fusion of pre-processing and classification techniques using decision trees to discover logic rules relating to the valence and arousal emotional dimensions. This approach normalised the signal data in a manner that enabled accurate classification and generated logic rules for knowledge discovery. Furthermore, the use of three target classes for the emotional dimensions was effective at denoising the data and further enhancing classification and useful rule extraction. There are three key contributions in this work, firstly an exploration and validation of our knowledge discovery methodology, secondly successful extraction of high accuracy rules derived from physiological data and thirdly knowledge discovery and visualisation of relationships within-participant physiological data that can be inferred relating to emotions. Additionally, this work may be utilised in areas such as the medical sciences where interpretable rules are required for knowledge discovery.

Given a class model built from a dataset including labeled data, classification assigns a new data object to the appropriate class. In associative classification the class model (i.e., the classifier) is a set of association rules. Associative classification is a promising technique for the generation of highly accurate classifiers. In this article, we present a compact form which encodes without information loss the classification knowledge available in a classification rule set. This form includes the rules that are essential for classification purposes, and thus it can replace the complete rule set. The proposed form is particularly effective in dense datasets, where traditional extraction techniques may generate huge rule sets. The reduction in size of the rule set allows decreasing the complexity of both the rule generation step and the rule pruning step. Hence, classification rule extraction can be performed also with low support, in order to extract more, possibly useful, rules.