Features Selection for Supervised Learning Using Centrality Measures

Effective pixel classification of Mars images based on ant colony optimization feature selection and extreme learning machine

The article explores data mining methods that allow us to get helpful information from the data. The possibility of using these methods in practice in the financial sector was considered. Since financial activity is closely related to our social life, the use of data mining methods plays an essential role in the analysis and forecasting of the financial market in the modern era of big data. However, due to differences in the experience of researchers in different disciplines, it is not easy to use data mining methods when analyzing financial data. Therefore, creating a methodological basis for the practical application of data mining methods in the analysis of financial data is an urgent issue. The purpose of this article is to create a methodological basis for using data mining methods for efficient trading. When processing product data, a priori methods and visualization methods were used, and their implementation in practice was described. As a result, scenarios of computer applications were created as a sample of the practical implementation of the algorithms of these methods. Building a quantitative trading strategy requires first statistical analysis of the information in the market and then testing the quantitative model on the collected data. This study developed a quantitative trading system based on data mining methods. The primary development tool used is the Jupyter web platform, and three cores have been developed: quantitative data selection, strategy testing on data, time series analysis, and visualization. The developed system supports modules for making simple trading decisions.

BackgroundRecently, successful applications of the discrete wavelet transform have been reported in brain interface (BI) systems with one or two EEG channels. For a multi-channel BI system, however, the high dimensionality of the generated wavelet features space poses a challenging problem.MethodsIn this paper, a feature selection method that effectively reduces the dimensionality of the feature space of a multi-channel, self-paced BI system is proposed. The proposed method uses a two-stage feature selection scheme to select the most suitable movement-related potential features from the feature space. The first stage employs mutual information to filter out the least discriminant features, resulting in a reduced feature space. Then a genetic algorithm is applied to the reduced feature space to further reduce its dimensionality and select the best set of features.ResultsAn offline analysis of the EEG signals (18 bipolar EEG channels) of four able-bodied subjects showed that the proposed method acquires low false positive rates at a reasonably high true positive rate. The results also show that features selected from different channels varied considerably from one subject to another.ConclusionThe proposed hybrid method effectively reduces the high dimensionality of the feature space. The variability in features among subjects indicates that a user-customized BI system needs to be developed for individual users.

<p>Classification problems often have a large number of features, but not all of them are useful for classification. Irrelevant and redundant features may even reduce the classification accuracy. Feature selection is a process of selecting a subset of relevant features, which can decrease the dimensionality, shorten the running time, and/or improve the classification accuracy. There are two types of feature selection approaches, i.e. wrapper and filter approaches. Their main difference is that wrappers use a classification algorithm to evaluate the goodness of the features during the feature selection process while filters are independent of any classification algorithm. Feature selection is a difficult task because of feature interactions and the large search space. Existing feature selection methods suffer from different problems, such as stagnation in local optima and high computational cost. Evolutionary computation (EC) techniques are well-known global search algorithms. Particle swarm optimisation (PSO) is an EC technique that is computationally less expensive and can converge faster than other methods. PSO has been successfully applied to many areas, but its potential for feature selection has not been fully investigated. The overall goal of this thesis is to investigate and improve the capability of PSO for feature selection to select a smaller number of features and achieve similar or better classification performance than using all features. This thesis investigates the use of PSO for both wrapper and filter, and for both single objective and multi-objective feature selection, and also investigates the differences between wrappers and filters. This thesis proposes a new PSO based wrapper, single objective feature selection approach by developing new initialisation and updating mechanisms. The results show that by considering the number of features in the initialisation and updating procedures, the new algorithm can improve the classification performance, reduce the number of features and decrease computational time. This thesis develops the first PSO based wrapper multi-objective feature selection approach, which aims to maximise the classification accuracy and simultaneously minimise the number of features. The results show that the proposed multi-objective algorithm can obtain more and better feature subsets than single objective algorithms, and outperform other well-known EC based multi-objective feature selection algorithms. This thesis develops a filter, single objective feature selection approach based on PSO and information theory. Two measures are proposed to evaluate the relevance of the selected features based on each pair of features and a group of features, respectively. The results show that PSO and information based algorithms can successfully address feature selection tasks. The group based method achieves higher classification accuracies, but the pair based method is faster and selects smaller feature subsets. This thesis proposes the first PSO based multi-objective filter feature selection approach using information based measures. This work is also the first work using other two well-known multi-objective EC algorithms in filter feature selection, which are also used to compare the performance of the PSO based approach. The results show that the PSO based multiobjective filter approach can successfully address feature selection problems, outperform single objective filter algorithms and achieve better classification performance than other multi-objective algorithms. This thesis investigates the difference between wrapper and filter approaches in terms of the classification performance and computational time, and also examines the generality of wrappers. The results show that wrappers generally achieve better or similar classification performance than filters, but do not always need longer computational time than filters. The results also show that wrappers built with simple classification algorithms can be general to other classification algorithms.</p>

Computing significant factors quantitatively is an imperative task in understanding the underlying reasons that contribute to a final outcome. In this paper, a case of e-Government ranking is studied by attempting to find the significance of each KPIs which leads to resultant rank of a country. Significant factors in this context are inferred as some degrees of relations between the input variables (which are the KPIs in this case) and the final outcome (the rank). In the past, significant factors were either acquired as first-hand information via direct questioning from users’ satisfaction survey or qualitative inference; typical question is ‘You are satisfied with a particular e-Government service’ by applying a multi-level Likert scale. Respondents answered by choosing one of the following: Strongly agree, Agree, Neutral, Disagree, and Strongly Disagree. The replies are then counted and studied using traditional statistical methods. In this paper, an alternative method by feature selection in data mining is proposed which computes quantitatively the relative importance of each KPI with respective to the predicted class, the rank. The main advantage of feature selection by data mining (FSDM) method is that it considers the cross-dependencies of the variables and how they contribute as a whole predictive model to a particular predicted outcome. In contrast, classical significant factor analysis such as correlogram tells only the strength of correlation between an individual pair of factor and outcome. Another advantage of using data mining method over simple statistic is that the inferred predictive model could be used as a predictor and/or what-if decision simulator; given some values of KPIs a corresponding rank could be guesstimated. A case study of computing significant factors in terms of KPIs that lead to the world rank in from the data of UN e-Government Survey 2010, is presented.

As a preprocessing method of data mining with SVM, feature selection can eliminate irrelevant or redundant attributes and increase the density of samples in feature spaces that can improve classification performance. In the field of financial time series pattern recognition, the study of feature selection has been receiving increasing attention. Different from other studies, this work use two new coefficients: neighborhood dependence (ND) and neighborhood decision error (NDEM) to measure the classification complexity as a method of feature selection. In the real example, we use ASH, ASNN, ND and NDEM to measure the classification complexity of technical indicators data set of Shanghai stock exchange (SSE). Then we take the reduced sets which selected by the above four coefficients as input data for SVM. Compared with the forecasting results of SVM for the other three coefficients, the SVM with NDEM reduced data set has higher classification accuracy.

The generation of an isotopic map of the reference region featuring local stable isotopic fingerprints requires the application of data mining methods due to the heterogeneity and complexity of the generated data. In this chapter, we explore new techniques to process isotopic data with the ultimate goal of constructing a map of locally characteristic isotopic fingerprints that help predict the places of origin of particular findings and, thus, supports archaeologists in deriving and testing hypotheses. In particular, we propose a new method for feature selection and apply it to a sample dataset of animal bones from the reference region. This application confirms that a multivariate fingerprint is clearly superior over univariate analysis and that the impact of oxygen on the reliability of the fingerprints is not very prominent. These findings confirm that it is possible to explore cremated human material for provenance analysis. Based on these results, we also propose a new spatial clustering method for detecting spatially consistent areas of homogeneous isotopic fingerprints resulting in an isotopic map of the reference region.

Non-intrusive load monitoring (NILM) leverages disaggregation algorithms to obtain appliance-level power consumption from the mains measurement. Coupled with edge computing, NILM can realize real-time load disaggregation, as well as energy conservation and privacy protection. However, the high feature dimensionality and complex disaggregation algorithm of NILM are both memory-demanding in terms of the edge devices. In this paper, we propose an edge-based NILM system, with a Raspberry-based edge device as validation. To address the memory demand problem, we propose a novel feature selection method based on max-discrimination and min-redundancy (MDMR) filter, which not only reduces the feature space, but also improves the classification performance. Meanwhile, several typical supervised algorithms are trained and compared to determine an optimal solution with the best disaggregation performance and least algorithm complexity. The evaluation results reveal that the MDMR filter-based feature selection enables our e-NILM system with 64.29% feature space reduction and 97.12% accuracy.

We show that the relevant information about a classification problem in feature space is contained up to negligible error in a finite number of leading kernel PCA components if the kernel matches the underlying learning problem. Thus, kernels not only transform data sets such that good generalization can be achieved even by linear discriminant functions, but this transformation is also performed in a manner which makes economic use of feature space dimensions. In the best case, kernels provide efficient implicit representations of the data to perform classification. Practically, we propose an algorithm which enables us to recover the subspace and dimensionality relevant for good classification. Our algorithm can therefore be applied (1) to analyze the interplay of data set and kernel in a geometric fashion, (2) to help in model selection, and to (3) de-noise in feature space in order to yield better classification results.

Today, diagnosis of attention deficit hyperactivity disorder (ADHD) still primarily relies on a series of subjective evaluations that highly rely on a doctor’s experiences and intuitions from diagnostic interviews and observed behavior measures. An accurate and objective diagnosis of ADHD is still a challenge and leaves much to be desired. Many children and adults are inappropriately labeled with ADHD conditions, whereas many are left undiagnosed and untreated. Recent advances in neuroimaging studies have enabled us to search for both structural (e.g., cortical thickness, brain volume) and functional (functional connectivity) abnormalities that can potentially be used as new biomarkers of ADHD. However, structural and functional characteristics of neuroimaging data, especially magnetic resonance imaging (MRI), usually generate a large number of features. With a limited sample size, traditional machine learning techniques can be problematic to discover the true characteristic features of ADHD due to the significant issues of overfitting, computational burden, and interpretability of the model. There is an urgent need of efficient approaches to identify meaningful discriminative variables from a higher dimensional feature space when sample size is small compared with the number of features. To tackle this problem, this paper proposes a novel integrated feature ranking and selection framework that utilizes normalized brain cortical thickness features extracted from MRI data to discriminate ADHD subjects against healthy controls. The proposed framework combines information theoretic criteria and the least absolute shrinkage and selection operator (Lasso) method into a two-step feature selection process which is capable of selecting a sparse model while preserving the most informative features. The experimental results showed that the proposed framework generated the highest/comparable ADHD prediction accuracy compared with the state-of-the-art feature selection approaches with minimum number of features in the final model. The selected regions of interest in our model were consistent with recent brain–behavior studies of ADHD development, and thus confirmed the validity of the selected features by the proposed approach.

© 2017 Elsevier Inc. Feature selection has two major conflicting aims, i.e., to maximize the classification performance and to minimize the number of selected features to overcome the curse of dimensionality. To balance their trade-off, feature selection can be handled as a multi-objective problem. In this paper, a feature selection approach is proposed based on a new multi-objective artificial bee colony algorithm integrated with non-dominated sorting procedure and genetic operators. Two different implementations of the proposed approach are developed: ABC with binary representation and ABC with continuous representation. Their performance are examined on 12 benchmark datasets and the results are compared with those of linear forward selection, greedy stepwise backward selection, two single objective ABC algorithms and three well-known multi-objective evolutionary computation algorithms. The results show that the proposed approach with the binary representation outperformed the other methods in terms of both the dimensionality reduction and the classification accuracy.

Background: Feature selection (FS), a crucial preprocessing step in machine learning, greatly reduces the dimension of data and improves model performance. This paper focuses on selecting features for medical data classification. Methods: In this work, a new form of ensemble FS method called weighted rank difference ensemble (WRD-Ensemble) has been put forth. It combines three FS methods to produce a stable and diverse subset of features. The three base FS approaches are Pearson’s correlation coefficient (PCC), reliefF, and gain ratio (GR). These three FS approaches produce three distinct lists of features, and then they order each feature by importance or weight. The final subset of features in this study is chosen using the average weight of each feature and the rank difference of a feature across three ranked lists. Using the average weight and rank difference of each feature, unstable and less significant features are eliminated from the feature space. The WRD-Ensemble method is applied to three medical datasets: chronic kidney disease (CKD), lung cancer, and heart disease. These data samples are classified using logistic regression (LR). Results: The experimental results show that compared to the base FS methods and other ensemble FS methods, the proposed WRD-Ensemble method leads to obtaining the highest accuracy value of 98.97% for CKD, 93.24% for lung cancer, and 83.84% for heart disease. Conclusion: The results indicate that the proposed WRD-Ensemble method can potentially improve the accuracy of disease diagnosis models, contributing to advances in clinical decision-making.

The way we learn has changed as a result of advances in science and technology and the growing influence of information technology on daily life. E-learning is a relatively new concept in educational settings, but it has progressively earned a spot in modern training techniques. Data mining methods are used in many applications, particularly in elearning. Comparing direct communications difficult to track the student involvement in online courses. To overcome the above mentioned issues the existing methods are analyzed various techniques of data mining and optimization methods which can be applied to bring out hidden knowledge from the educational data? In the e-learning work, preprocessing, feature selection and classification process are performed to improve the Kalboard 360 educational dataset accuracy significantly. This study focused on educational data mining (EDM) and employed decision trees, support vector machines (SVM), multilayer perceptrons, and the naive bayesian C4.5 algorithm among other classification techniques to assess the features for the prediction of students’ behaviour and academic achievement. This paper analyzes the advantages and shortcomings of each algorithm applied to data set on the basis of accuracy for various classification techniques like Multilayer Perceptron based ANN algorithm (MP-ANN), Decision Tree (DT) C4.5 and Hybrid Regression model with Multi Label Classification (HR-MLC) approach. The experimental result shows that the HR-MLC algorithm provides better performance in terms of higher accuracy, precision, recall and f-measure rather than the other existing methods.

An urgent problem in the field of information security – the provision of human resources in the field of information security in the Russian Federation is presented. One of the factors influencing this is the lack of the possibility of determining at the initial stage the optimal choice of the future direction of activity of graduates of higher educational institutions with specialization in this field. To solve this problem, it is proposed to use methods of data mining, machine learning and algorithms based on artificial neural networks. For the successful application of data mining methods, several factors should be taken into account: the data set must be large enough. Secondly; the data must be structured and analyzed to extract the most important information; the data analysis methods must be adequate and accurate in order to determine the purpose of the graduate with the highest accuracy. Taking into account these factors, the article suggests a way to form the initial data set and prepare it using statistical methods to predict the success of the professional activity of an information security specialist. The results of this work can be useful for higher educational institutions, specialists in the field of information security, heads of personnel departments who are interested in optimizing the process of selecting qualified employees in this field.

Hyperspectral image classification by means of Deep Learning techniques is now a widespread practice. Its success comes from the abstract features learned by the deep architecture that are ultimately well separated in the feature space. The great amount of parameters to be learned requires the training data set to be very large, otherwise the risk of overfitting appears. Alternatively, one can resort to features selection in order to decrease the architecture's number of parameters to be learnt. For that purpose, this work proposes a simple feature selection method, based on single-layer neural networks, which select the most distinguishing features for each class. Then, the data will be classified by a deep neural network. The accuracy results for the testing data are higher for the lower dimensional data set when compared to the full data set, indicating less over-fitting for the reduced data. Besides, a metric based on scatter matrices shows that the classes are better separated in the reduced feature space.