Extraction and optimization of classification rules for temporal sequences: Application to hospital data

Concept lattice is an effective tool for data analysis and extracting classification rules. However, the classical concept lattice often produce a lot of redundant rules. Closed-label concept lattice realizes reduction of concept intention, which can be used to extract fewer rules than the classical concept lattice. This paper presents a method for classification rules extraction based on the closed-label concept lattice. Examples show that the proposed method is effective for extracting more concise classification rules.

Classifier building form distributed data sources has been a fundamental computational problem to realise distributed knowledge discovery. The classification rule extraction from distributed databases suffers from the problems of high communication cost, lack of interpretability of rules and poor performance in handling high categorical data. The aim of this paper is to extend fuzzy generalised association rule extraction technique which is well proved in handling such issues to extract classification rules from distributed datasets. This paper presents a distributed data driven fuzzy generalised associative classifier D3FGAC framework for distributed knowledge discovery which extracts data driven fuzzy generalisation rules from horizontally fragmented datasets with minimum communication cost and builds global compact classifier using extracted rules. The experiments conducted on UCI datasets and their comparisons to other existing model shown in article to prove the efficiency of proposed framework.

A comparison between several classification algorithms with feature extraction on real dataset is presented. Principal Component Analysis (PCA) has been used for feature extraction with different values of the ratio R, evaluated and compared using four different types of classifiers on two real benchmark data sets. Accuracy of the classifiers is influenced by the choice of different values of the ratio R. There is no best value of the ratio R, for different datasets and different classifiers accuracy curves as a function of the number of features used may significantly differ. In our cases feature extraction is especially effective for classification algorithms that do not have any inherent feature selections or feature extraction build in, such as the nearest neighbour methods or some types of neural networks.

Extensive Copy-Number Variation of the Human Olfactory Receptor Gene Family

Among the methods proposed for identifying the number of latent traits in multidimensional IRT models, DETECT has attracted the attention of both methodologists and applied researchers as a nonparametric counterpart to other procedures. The current study investigated the overall performance of the DETECT procedure and its outcomes using a real-data sampling design recommended by MacCallum (2003) and compared the results from a purely simulated data set that was generated with a well-specified “perfect” model. The comparison revealed that the sampling behavior of the maximized DETECT value and R-ratio statistics was quite robust to minor factors and other model misspecifications that potentially exist in the real data set, as there were negligible differences between the results of the real and simulated data sets. Item classification accuracy was also nearly identical for the real and simulated data sets. The accuracy of the identified number of dimensions reported by DETECT was the only outcome with an obvious difference between the purely simulated data set and the real data set. While the difference was small for smaller sample sizes, the identified number of dimensions was more accurate for larger sample sizes when the population data set was purely simulated. In many instances, exploratory DETECT analysis outperformed the cross-validated DETECT analysis in terms of overall accuracy.

This paper presents a fraud detection model using data mining techniques such as neural networks and symbolic extraction of classification rules from trained neural network. The neural network is first trained to achieve an accuracy rate, the activation of the values in the hidden layers of the neural network is analyzed and from this analysis are generated classification rules. The proposed approach was tested on a set of data from a Colombian organization for the sending and payment of remittances, in order to identify patterns associated with fraud detection. Similarly the results of the techniques used in the model were compared with other mining techniques such as Decision Trees and Naive Bayes. A prototype software was developed to test the model, which was integrated into RapidMiner tool, which can be used as a tool for academic software.KeywordsData miningneural networksrule extractiondetection of fraudexperts

We examine the performance of compact fuzzy rule-based classification systems that consist of a small number of simple fuzzy rules with high comprehensibility. Those fuzzy systems are designed in a heuristic manner using rule selection criteria. We first describe fuzzy rule-based classification. Next we describe heuristic rule selection criteria using the terminology in data mining: confidence and support. A small number of fuzzy rules are extracted from numerical data based on each rule selection criterion. Then we examine the classification performance of extracted fuzzy rules through computational experiments on a number of benchmark data sets from the UCI ML Repository. Our results should be viewed as the lowest benchmark performance of fuzzy rule-based classification systems because fuzzy rules are extracted using a simple heuristic method with no optimization or tuning procedures. Nevertheless our results on some data sets are comparable to reported results by the C4.5 algorithm in the literature.

The high dimensional data are frequently met when we apply Web text classification. Mining in high dimensional data is extraordinarily difficult because of the curse of dimensionality. We must adopt feature dimensionality reduction to solve these problems. A attribute reduction algorithm based on rough set theory is given in this paper to reduce the text feature term and extract rule. First, the weight of feature term is made discrete. Then, the decision table is made with weight as the condition attributes and classes of texts as the decision attributes. Finally, the classification rules are extracted by attribute reduction. The method is simple and feasible. It is advantageous in improving the efficiency of the selected feature subset and suitable for high-volume text classification. The extracted rules are easy understand. The accuracy is higher and the speed of classification is faster than the classification based on vector space comparison. This paper describes the proposed technique and provides experimental results.

The matrix factorization (MF) technique has been widely adopted for solving the rating prediction problem in recommender systems. The MF technique utilizes the latent factor model to obtain static user preferences (user latent vectors) and item characteristics (item latent vectors) based on historical rating data. However, in the real world, user preferences are not static but full of dynamics. Though there are several previous works that addressed this time-varying issue of user preferences, it seems (to the best of our knowledge) that none of them are specifically designed for tracking concept drift in individual user preferences . Motivated by this, we develop a temporal MF approach for tracking concept drift in each individual user latent vector. There are two key innovative steps in our approach: 1) we develop a modified stochastic gradient descent method to learn an individual user latent vector at each time step and 2) by Lasso regression, we learn a linear model for the transition of the individual user latent vectors. We test our method on a synthetic data set and several real data sets. In comparison with the original MF, our experimental results show that our temporal method is able to achieve lower root mean square errors (RMSEs) for both the synthetic and real data sets. One interesting finding is that the performance gain in RMSE is mostly from those users who indeed have concept drift in their user latent vectors at the time of prediction. In particular, for the synthetic data set and the Ciao data set, there are quite a few users with that property and the performance gains for these two data sets are roughly 20% and 5%, respectively.

In this paper we propose a knowledge discovery and visualisation method for unearthing emotional states from physiological data typically available from wearable devices. In addition we investigate the viability of using a limited set of wearable sensors to extract decision tree rules which are representative of physiological changes taking place during emotional changes. Our method utilised a fusion of pre-processing and classification techniques using decision trees to discover logic rules relating to the valence and arousal emotional dimensions. This approach normalised the signal data in a manner that enabled accurate classification and generated logic rules for knowledge discovery. Furthermore, the use of three target classes for the emotional dimensions was effective at denoising the data and further enhancing classification and useful rule extraction. There are three key contributions in this work, firstly an exploration and validation of our knowledge discovery methodology, secondly successful extraction of high accuracy rules derived from physiological data and thirdly knowledge discovery and visualisation of relationships within-participant physiological data that can be inferred relating to emotions. Additionally, this work may be utilised in areas such as the medical sciences where interpretable rules are required for knowledge discovery.

Given a class model built from a dataset including labeled data, classification assigns a new data object to the appropriate class. In associative classification the class model (i.e., the classifier) is a set of association rules. Associative classification is a promising technique for the generation of highly accurate classifiers. In this article, we present a compact form which encodes without information loss the classification knowledge available in a classification rule set. This form includes the rules that are essential for classification purposes, and thus it can replace the complete rule set. The proposed form is particularly effective in dense datasets, where traditional extraction techniques may generate huge rule sets. The reduction in size of the rule set allows decreasing the complexity of both the rule generation step and the rule pruning step. Hence, classification rule extraction can be performed also with low support, in order to extract more, possibly useful, rules.

Extraction of Classification Rules in Floor Plan Databases using Answer Set Programming

Rule extraction from data is one of the key technologies for solving the bottlenecks in artificial intelligence. Artificial neural networks are well suited for representing any knowledge in given data. Extraction of logical/fuzzy rules from the trained artificial neural network is of great importance to researchers in the fields of artificial intelligence and soft computing. Fuzzy rule sets are capable of approximating any nonlinear mapping relationships. Extraction of rules from data has been discussed in terms of fuzzy modeling, fuzzy clustering, and classification with fuzzy rule sets. This special issue entitled"Rule Extraction from Data" is aimed at providing the readers with good insights into the advanced studies in the field of rule extraction from data using neural networks/fuzzy rule sets. I invited seven research papers best suited for the theme of this special issue. All the papers were reviewed rigorously by two reviewers each. The first paper proposes an interesting rule extraction method from data using neural networks. Ishikawa presents a combination of learning with an immediate critic and a structural learning with forgetting. This method is capable of generating skeletal networks for logical rule extraction from data with correct and wrong answers. The proposed method is applied to rule extraction from lense data. The second paper presents a new methodology for logical rule extraction based on transformation of MLP (multilayered perceptron) to a logical network. Duck et al. applied their C-MLP2LN to the Iris benchmark classification problem as well as real-world medical data with very good results. In the third paper, Geczy and Usui propose fuzzy rule extraction from trained artificial neural networks. The proposed algorithm is implied from their theoretical study, not from heuristics. Their study enables to initially consider derivation of crisp rules from trained artificial neural network, and in case of conflict, application of fuzzy rules. The proposed algorithm is experimentally demonstrated with the Iris benchmark classification problem. The fourth paper presents a new framework for fuzzy modeling using genetic algorithm. The authors have broken new ground of fuzzy rule extraction from neural networks. For the fuzzy modeling, they have proposed a particular type of neural networks containing nodes representing membership functions. In this fourth paper, the authors discuss input variable selection for the fuzzy modeling under multiple criteria with different importance. A target system with a strong nonlinearity is used for demonstrating the proposed method. Kasabov, et al. present, in the fifth paper, a method for extraction of fuzzy rules that have different level of abstraction depending on several modifiable thresholds. Explanation quality becomes better with higher threshold values. They apply the proposed method to the Iris benchmark classification problem and to a real world problem. J. Yen and W. Gillespie address interpretability issue of Takagi-Sugeno-Kang model, one of the most popular fuzzy mdoels, in the fifth paper. They propose a new approach of fuzzy modeling that ensures not only a high approximation of the input-output relationship in the data, but also good insights about the local behavior of the model. The proposed method is applied to fuzzy modeling of sinc function and Mackey-Glass chaotic time series data. The last paper discusses fuzzy rule extraction from numerical data for high-dimensional classification problems. H.Ishibuchi, et al. have been pioneering methods for classification of data using fuzzy rules and genetic algorithm. In this last paper, they introduced a new criterion, simplicity of each rule, together with the conventional ones, compactness of rule base and classification ability, for high-dimensional problem. The Iris data is used for demonstrating their new classification method. They applied it also to wine data and credit data. I hope that the readers will be encouraged to explore the frontier to establish a new paradigm in the field of knowledge representation and rule extraction.

How to extract rules from trained SVMs has become an important preprocessing technique for data mining, pattern classification, and so on. There are two key problems required to be solved in the SVM based classification rule extraction, i.e. the attribute selection and the discretization to continuous attributes. In this paper, the differential characteristic of SVR (Support vector regression) is discussed. A new measure for determining the importance level of the attributes based on the trained SVR classifiers is proposed. Based on this new measure, a new approach for rule extraction from trained SVR classifiers is proposed. A new algorithm for rule extraction is given. The performance of the new approach is demonstrated by several computing cases. Experiment results show that the proposed approach can improve the validity of the extracted rules remarkably compared to other rule extracting approaches, especially for complicated classification problems.

In the case of two small toy data sets, we found out that subsampling has a much weaker behavior in the finding of the true number of clusters K than bootstrapping (Mucha and Bartel, Soft bootstrapping in cluster analysis and its comparison with other resampling methods. In: M. Spiliopoulou, L. Schmidt-Thieme, R. Janning (eds.) Data analysis, machine learning and knowledge discovery. Springer, Cham, 2014). In contradiction, Möller and Dörte (Intell Data Anal 10:139–162, 2006) pointed out that “subsampling … clearly outperformed the bootstrapping technique in the detection of correct clustering consensus results.” Obviously, there is a need for further investigations. Therefore here we compare these two resampling techniques based on real and artificial data sets by means of different indices: ARI or Jaccard. We consider hierarchical cluster analysis methods because they find all partitions into K = 2, 3, … clusters in one run only, and, moreover, these results are (usually) unique (Spaeth, Cluster analysis algorithms for data reduction and classification of objects. Ellis Horwood, Chichester, 1982). The methods are tested on two synthetic data sets and two real data sets. Obviously, bootstrapping is better than subsampling in finding the true number of clusters.