On learning dual classifiers for better data classification

Instance selection in medical datasets: A divide-and-conquer framework

Cluster-oriented instance selection for classification problems

Large scale instance selection by means of federal instance selection

Data mining techniques have been employed for solving many medical problems, especially for disease prediction. For instance, given a dataset containing normal and cancerous patients, the goal is to develop a model to predict whether a new (unknown) patient belongs to the normal or cancerous class. In general, the model is constructed based on some machine learning technique over a collected training set. However, the quality of the training set can affect the final prediction performance of the model. That is, if the training set contains some certain amount of noisy data (or outliers), then the model's performance could be degraded. In literature, instance selection is performed over a given training set in order to filter out some noisy data and the reduced training set containing non-noisy data is used for developing the prediction model. In this paper, we present a novel approach where instance selection is performed to divide a given training set into noisy and non-noisy subsets. Then, they are used to train two models respectively. During prediction, the instance selection step is also executed over the testing set, in which the noisy and non-noisy subsets are used to test their corresponding models respectively. The experimental results based on various medical domain datasets show that our proposed approach performs better than the baseline, which is based on the conventional instance selection approach.

The traditional evolutionary instance selection algorithm has the risk of redundant and noise training samples in the training set selection, which affects the classification effect. In this paper, instance selection, instance weighting and feature weighting are integrated into the cooperative coevolution framework, and a cooperative coevolutionary algorithm for KNN training set optimization selection is proposed. The CHC algorithm based on multi-point crossover strategy is used to further improve the accuracy of instance selection. The SSGA algorithm based on fast mutation strategy of instance weighting and feature weighting is synergistic with the instance selection, and it helps to remove noise and irrelevant data in the process of instance selection. The proposed method can speed up the convergence of the population, it can also improve the efficiency of the algorithm, and improve the KNN classification performance. The experimental results show that this method has advantages in classification accuracy and efficiency compared with some current evolutionary instance selection algorithms.

SVOIS: Support Vector Oriented Instance Selection for text classification

The paper presents bagging ensembles of instance selection algorithms. We use bagging to improve instance selection. The improvement comprises data compression and prediction accuracy. The examined instance selection algorithms for classification are ENN, CNN, RNG and GE and for regression are the developed by us Generalized CNN and Generalized ENN algorithms. Results of the comparative experimental study performed using different configurations on several datasets shows that the approachbased on bagging allowed for significant improvement, especially in terms of data compression.

Performing instance selection prior to the classifier training is always beneficial in terms of computational complexity reduction of the classifier training and sometimes also beneficial in terms of improving prediction accuracy. Removing the noisy instances improves the prediction accuracy and removing redundant and irrelevant instances does not negatively effect it. However, in practice the instance selection methods usually also remove some instances, which should not be removed from the training dataset, what results in decreasing the prediction accuracy. We discuss two methods to deal with the problem. The first method is the parameterization of instance selection algorithms, which allows to choose how aggressively the instances are removed and the second one is to embed the instance selection directly into the prediction model, which in our case is an MLP neural network.

Instance selection is a popular preprocessing task in knowledge discovery and data mining. Its purpose is to reduce the size of data sets maintaining their predictive capabilities. The usual emerging problem at this point is that these methods quite often suffer of high computational complexity, which becomes highly inconvenient for processing huge data sets. In this paper, a parallel implementation for the instance selection algorithm Democratic Instance Selection (DIS) is presented. The main advantages of the DIS algorithm turn out to be its computational complexity, linear in the number of instances, as well as its internal structure, intuitively parallelizable. The purpose of this paper is threefold: firstly, the design of the DIS algorithm by following the MapReduce model; secondly, its implementation in the popular big data framework Spark; and finally, its empirical comparison over large-scale data sets. The results show that the processing time is reduced in a linear manner as the number of Spark executors increases, what makes it suitable for big data applications. In addition, the algorithm is publicly accessible to the scientific community.

The k-Nearest Neighbor (k-NN) rule is widely used for classification tasks because of its simplicity and efficiency. However, a well-known drawback of k-NN is its dependence on the quality of the training set, since the k-NN makes no assumption about the importance of each instance. In fact, the existence of noisy and superfluous instances in the training set tends to increase the classification error rate. Thus, instance selection methods are useful to identify which instances belonging to the training set will be considered in the k-NN classifier. Our proposal shows a simple and effective way to improve instance selection methods using metric learning. The idea of our proposal relies on a pure geometric intuition that metric learning transforms the input space where points in the same class are simultaneously near each other and far from points in the other classes. In a more “organised” space, we show that instance selection methods can benefit from this transformed space. We carried out an experimental evaluation to compare the instance selection with and without metric learning on UCI benchmark data sets. The results reveals that the combination of metric and instance selection is very welcome. All tested instance selection methods improved significantly.

Instance selection in the supervised machine learning, often referred to as the data reduction, aims at deciding which instances from the training set should be retained for further use during the learning process. Instance selection can result in increased capabilities and generalization properties of the learning model, shorter time of the learning process, or it can help in scaling up to large data sources. The paper proposes a cluster-based instance selection approach with the learning process executed by the team of agents and discusses its four variants. The basic assumption is that instance selection is carried out after the training data have been grouped into clusters. To validate the proposed approach and to investigate the influence of the clustering method used on the quality of the classification, the computational experiment has been carried out.

Generally, desired training set is used to make a neural network learn nonlinear relations properly. The training set consists of multiple pairs of an input vector and output one. Each input vector is given to the input layer for forward calculation, and the corresponding output vector is compared with the vector yielded from the output layer. Weights between neurons are updated based on the errors using a back propagation algorithm in backward calculation. The time required for the learning process of a neural network depends on the number of total weights and that of the input-output pairs in the training set. In the proposed learning process, after the learning is progressed, e.g., 1000 iterations, input-output pairs having had worse errors are extracted from the original training set and form a new temporary set. From the next iteration, the temporary set is applied instead of the original set. This means that only pairs with worse errors are used for updating the weights until the mean value of errors decreases to a level. After the learning using the temporary set, the original set is applied again instead of the temporary set. By alternately applying the two types of sets for learning that the calculation load for convergence can be efficiently reduced. The effectiveness of the proposed method is proved by applying to an inverse kinematics problem of an industrial robot. Generally, desired training set is used to make a neural network learn nonlinear relations properly. The training set consists of multiple pairs of an input vector and output one. Each input vector is given to the input layer for forward calculation, and the corresponding output vector is compared with the vector yielded from the output layer. Weights between neurons are updated based on the errors using a back propagation algorithm in backward calculation. The time required for the learning process of a neural network depends on the number of total weights and that of the input-output pairs in the training set. In the proposed learning process, after the learning is progressed, e.g., 1000 iterations, input-output pairs having had worse errors are extracted from the original training set and form a new temporary set. From the next iteration, the temporary set is applied instead of the original set. This means that only pairs with worse errors are used for updating the weights until the mean value of errors decreases to a level. After the learning using the temporary set, the original set is applied again instead of the temporary set. By alternately applying the two types of sets for learning that the calculation load for convergence can be efficiently reduced. The effectiveness of the proposed method is proved by applying to an inverse kinematics problem of an industrial robot.

In supervised learning, a training set providing previously known information is used to classify new instances. Commonly, several instances are stored in the training set but some of them are not useful for classifying therefore it is possible to get acceptable classification rates ignoring non useful cases; this process is known as instance selection. Through instance selection the training set is reduced which allows reducing runtimes in the classification and/or training stages of classifiers. This work is focused on presenting a survey of the main instance selection methods reported in the literature.

This paper is an continuation of the accompanying paper with the same main title. The first paper reviewed instance selection algorithms, here results of empirical comparison and comments are presented. Several test were performed mostly on benchmark data sets from the machine learning repository at UCI. Instance selection algorithms were tested with neural networks and machine learning algorithms.

IFSB-ReliefF: A New Instance and Feature Selection Algorithm Based on ReliefF