Performance Evaluation of a Proposed Machine Learning Model for Chronic Disease Datasets Using an Integrated Attribute Evaluator and an Improved Decision Tree Classifier

Classification problem in biological Omics data has gained in popularity in recent years. In consideration of the high dimension of the Omics dataset, the importance of feature selection technology becomes apparent. Feature selection can not only improve the classification accuracy but also avoid overfitting which is a common issue in machine learning. There are three main categories of feature selection methods: filter methods, wrapper methods, and hybrid methods. Yet it is difficult for researchers to choose among them. In this study, we conducted a comprehensive comparison for filter methods, wrapper methods, and hybrid methods. Specifically, we selected information gain as the filter method, genetic algorithm, and binary particle swarms optimization as the wrapper methods, IG-GA and IG-BPSO as the hybrid methods for comparison. The experimental results show that the IG-BPSO, a hybrid method, has the highest classification accuracy among these feature selection methods on three Omics datasets.

Life expectancy prediction is important for decision-making to facilitate advanced care planning processes. This is possible by applying the needed treatment to foresee the procurement of health care services. Machine learning techniques are used to predict the life expectancy values for different people using regression algorithms. Feature selection methods are known for their power to enhance the prediction quality results by selecting the best features that affect the prediction of the results. In this paper, different regression techniques are applied to a life expectancy dataset provided by the World Health Organization. These algorithms include Linear Regression, K-Nearest Neighbor Regression, and Decision Tree Regression which are combined with two different feature selection methods which are the Correlation Feature Selection and The Mutual Information Feature Selection methods. Moreover, different k values are experimented with for the feature selection methods to generate the best results in terms of R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> and RMSE evaluation measures. The results show that the Decision Tree Regression method produced the best results for the two evaluation measures with a combination of the Mutual Information Feature Selection method with a k value of 15 and the Correlation Feature Selection method with a k value of 10 for the R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> and RMSE, respectively.

A new hybrid filter/wrapper algorithm for feature selection in classification

Hybrid filter–wrapper feature selection for short-term load forecasting

In the field of data processing and analysis, the dataset may be a large set of features that restrict data usability and applicability, and thus the dimensions of data sets need to be reduced. Feature selection is the process of removing as much of the redundant and irrelevant features as possible from the original dataset to improve the mining process efficiency. This paper presented a study to evaluate and compare the effect of filter and wrapper methods as feature selection approaches in terms of classification accuracy and time complexity. The Naive Bayes Classifier and three classification datasets from the UCI repository are utilizing in the classification procedure. To investigate the effect of feature selection methods, they are applied to the different characteristics datasets to obtain the selected feature vectors which are then classified according to each dataset category. The datasets used in this paper are the Iris, Ionosphere, and Ovarian Cancer dataset. Experimental results indicate that the filter and wrapper methods provide approximately equal classification accuracy where the average accuracy value of the Ionosphere and Ovarian Cancer dataset is 0.78 and 0.91 for the same selected feature vectors respectively. For Iris dataset, the filter method outperforms the wrapper method by achieving the same accuracy value using only half number of selected features. The results also show that the filter method surpasses when considering the execution time.

BackgroundEarly‐stage dementia, Mild Cognitive Impairment (MCI), is challenging to diagnose since it is a transient condition distinct from complete cognitive collapse. Recent clinical research studies have identified that balance impairments can be a significant indicator for predicting dementia in older adults. Accordingly, we aimed to identify key balance biomarkers using wearable inertial sensors for early detection of dementia/MCI.MethodAt National Research Center for Dementia (South Korea), 60 participants (30 Cognitively Normal‐CN, 30 Mild Cognitive Impairment‐MCI) were selected based on assessments conducted by medical professionals at Chosun University Hospital / Chonnam National University Hospital (Gwangju). Shimmer‐3 inertial sensor was placed on lower back of participants, with a tri‐axial accelerometer and tri‐axial gyroscope. The data was collected under four different conditions: eyes‐open (EO), eyes‐closed (EC), right‐leg lift (RL), left‐leg lift (LL). A set of 76 postural sway measures were considered with 43 time‐domain and 33 frequency‐domain features.Feature ranking was executed through the utilization of "Leave‐One‐Subject‐Out (LOSO)" cross‐validation technique for each condition. Filter methods: one‐way‐ANOVA (Analysis of Variance), Mutual Information and Wrapper methods: Random Forest, Support Vector Machine (SVM) were utilized to evaluate feature subsets. The proportional value of each feature was calculated by the number of times it showed in the top feature list. The final score was obtained by combining scores from all the techniques, which indicated the relative weight of characteristics across the various approaches. Further feature analysis was conducted using SHAP (Shapley‐Additive‐exPlanations).ResultThe multi‐step methodology, for feature selection, adopted in our research ensured a rigorous process, combining the strengths of Filter and Wrapper methods. In time‐domain: root‐mean‐square, mean‐distance, 95%‐confidence circle sway‐area, path‐length, mean sway‐velocity were the key features identified. While in frequency‐domain: spectral edge‐frequency, mean‐frequency, peak‐frequency, centroid‐frequency, spectral‐entropy were found to be the principal features of relevance. These features were also included in the top list of features based on SHAP impact value, hence proving the effectiveness of our method in identifying important biomarkers for detection of MCI.ConclusionThe inertial sensors based postural balance assessment provides useful biomarkers that can facilitate early screening of MCI / dementia in home settings.

We proposed a Hybrid Feature selection method, the combination of Mutual Information (MI) a filter method, and Recursive Feature Elimination (RFE) a wrapper method. The methodology combines the strengths of both filter and Wrapper method. Performance of the proposed method is measured on three benchmark datasets (Ionosphere, Libras Movement, and Clean) from the UCI Repository. We compared the classification accuracy of the proposed Hybrid method with MI, RFE, Original Features by using random forest classifier. The performance are compared using four classification measures i.e. 1. F1-Score 2. Recall 3. Precession 4. Accuracy. It is evidence from the result analysis that the proposed hybrid method has out performed other methods.

Breast cancer is the most life-threatening disease among women worldwide. Nowadays, clinical and genomic data in breast cancer datasets are analyzed to identify the incidence and impact of the disease. This paper proposes an analysis on the clinical breast cancer datasets, METABRIC and SEER for the prediction of survivability of breast cancer patients through ensemble classifiers. It attempts to identify informative independent features for those classifiers using filter methods, Mutual Information and Chi-Square and wrapper methods, Sequential Forward Selection, Sequential Backward Selection and Recursive Feature Elimination in the feature selection sequence. An analysis on the performance of ensemble techniques namely, bagging, boosting and stacking shows that the ensemble techniques produce higher accuracy when compared to the accuracies produced by the base classifiers. Also, the stacking ensemble model produces higher accuracies of 99.2% and 99.8% than other ensemble techniques on METABRIC and SEER datasets respectively. The SBS wrapper method aids higher accuracy to all classifiers than SFS and RFE.

Identification of combined power quality disturbances in the modern power systems by considering the development of different types of loads and distribution generations has become increasingly important. The novelty of this paper comes from the accurate and fast identification of the combined power quality disturbances in the presence of different distributed generations and loads such as photovoltaic cell, wind turbine with doubly fed induction generators, diesel engines, electric arc furnace, DC machine, 6-pulse and 12-pulse rectifier loads. In this paper, the features are extracted using variational mode decomposition, just from voltage waveforms. To reduce the redundant data, dimension of features vector, and time, the Relief-F method and correlation feature selection method are applied on the extracted features and these two methods are compared together. In this paper, the K-nearest neighbors classifier is used to classify the multiple power quality disturbances. To verify the effectiveness of the proposed method, different scenarios such as misfiring, variation of sun radiation and wind speed, entrance and exit of loads, capacitors and distributed generators, different fault at the grid in half-load to full-load were simulated. This method can be used as an added algorithm for smart metering in modern and smart power systems.

Bridge the semantic gap between pop music acoustic feature and emotion: Build an interpretable model

BackgroundFeature selection, as a preprocessing stage, is a challenging problem in various sciences such as biology, engineering, computer science, and other fields. For this purpose, some studies have introduced tools and softwares such as WEKA. Meanwhile, these tools or softwares are based on filter methods which have lower performance relative to wrapper methods. In this paper, we address this limitation and introduce a software application called FeatureSelect. In addition to filter methods, FeatureSelect consists of optimisation algorithms and three types of learners. It provides a user-friendly and straightforward method of feature selection for use in any kind of research, and can easily be applied to any type of balanced and unbalanced data based on several score functions like accuracy, sensitivity, specificity, etc.ResultsIn addition to our previously introduced optimisation algorithm (WCC), a total of 10 efficient, well-known and recently developed algorithms have been implemented in FeatureSelect. We applied our software to a range of different datasets and evaluated the performance of its algorithms. Acquired results show that the performances of algorithms are varying on different datasets, but WCC, LCA, FOA, and LA are suitable than others in the overall state. The results also show that wrapper methods are better than filter methods.ConclusionsFeatureSelect is a feature or gene selection software application which is based on wrapper methods. Furthermore, it includes some popular filter methods and generates various comparison diagrams and statistical measurements. It is available from GitHub (https://github.com/LBBSoft/FeatureSelect) and is free open source software under an MIT license.

Abstract. Dissolved oxygen (DO) holds an important role in water quality monitoring due to its impact on the aquatic organisms inhabiting the water body. Sampaloc lake, located in San Pablo City in Laguna Province, Philippines, one of the major fishing lakes, has been affected by the decrease of DO concentration leading to continuous fish kill incidents. This study used supervised machine learning-based model utilizing the filter and wrapper method in feature selection to predict the DO levels in Sampaloc Lake. The goal of the study is to provide a baseline for building an early warning system for fish kills events in the lake. Water quality, physical and hydroclimatic parameters are the range of features selected as predictors to create a reliable DO predictive model. Based on the feature selection process, water temperature, atmospheric temperature, biochemical oxygen demand, surface pressure and pH are the consistent important feature in predicting DO for both filter and wrapper method. Overall, the result of the eXtreme Gradient Boosting Regression (XGBR) Model of the filter method with R² = 0.95, RMSE = 0.14, and MAE = 0.10 showed a reliable performance among the predicted models (Multiple Linear Regression, Random Forest, Kernel Ridge Regression, and Bootstrap Aggregating) tested in the study.

Prediction of the stage of cancer plays an important role in planning the course of treatment and has been largely reliant on imaging tools which do not capture molecular events that cause cancer progression. Gene-expression data-based analyses are able to identify these events, allowing RNA-sequence and microarray cancer data to be used for cancer analyses. Breast cancer is the most common cancer worldwide, and is classified into four stages - stages 1, 2, 3, and 4 [2]. While machine learning models have previously been explored to perform stage classification with limited success, multi-class stage classification has not had significant progress. There is a need for improved multi-class classification models, such as by investigating deep learning models. Gene-expression-based cancer data is characterised by the small size of available datasets, class imbalance, and high dimensionality. Class balancing methods must be applied to the dataset. Since all the genes are not necessary for stage prediction, retaining only the necessary genes can improve classification accuracy. The breast cancer samples are to be classified into 4 classes of stages 1 to 4. Invasive ductal carcinoma breast cancer samples are obtained from The Cancer Genome Atlas (TCGA) and Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) datasets and combined. Two class balancing techniques are explored, synthetic minority oversampling technique (SMOTE) and SMOTE followed by random undersampling. A hybrid feature selection pipeline is proposed, with three pipelines explored involving combinations of filter and embedded feature selection methods: Pipeline 1 - minimum-redundancy maximum-relevancy (mRMR) and correlation feature selection (CFS), Pipeline 2 - mRMR, mutual information (MI) and CFS, and Pipeline 3 - mRMR and support vector machine-recursive feature elimination (SVM-RFE). The classification is done using deep learning models, namely deep neural network, convolutional neural network, recurrent neural network, a modified deep neural network, and an AutoKeras generated model. Classification performance post class-balancing and various feature selection techniques show marked improvement over classification prior to feature selection. The best multiclass classification was found to be by a deep neural network post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with a Cohen-Kappa score of 0.303 and a classification accuracy of 53.1%. For binary classification into early and late-stage cancer, the best performance is obtained by a modified deep neural network (DNN) post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with an accuracy of 81.0% and a Cohen-Kappa score (CKS) of 0.280. This pipeline also showed improved multiclass classification performance on neuroblastoma cancer data, with a best area under the receiver operating characteristic (auROC) curve score of 0.872, as compared to 0.71 obtained in previous work, an improvement of 22.81%. The results and analysis reveal that feature selection techniques play a vital role in gene-expression data-based classification, and the proposed hybrid feature selection pipeline improves classification performance. Multi-class classification is possible using deep learning models, though further improvement particularly in late-stage classification is necessary and should be explored further.

The aim of the study is to develop models of intrahospital mortality (IHM) prediction on an unbalanced sample of patients with coronary artery disease (CAD) post coronary artery bypass graft (CABG) surgery. Methods. Models for IHM prediction were built following the analysis of 866 electronic case histories based on the analysis of CAD patients, revascularized with the CABG operation. The patient cohort consisted of two groups. The first included 35 (4%) patients who died within the first 30 days after CABG, the second - 831 (96%) patients with a favorable operation outcome. We analyzed 99 factors, including the results of clinical, laboratory and instrumental studies obtained before CABG. For feature compilation, classical filtering and model selection methods were used (wrapper method). The primary drawback to applying a classical approach was the unbalanced sample as one cohort only consisted of 4% of subjects. In that case, it was not possible to apply the cross-validation procedure with three types of samples, standard quality metrics and multi-category factors. Results. Features searching approach using the multi-stage selection procedure, which combined the validation of predefined predictors, filtering methods and multifactor model development based on logistic regression, random forest (RF) and artificial neural networks (ANNs) was proposed. The models' accuracy was evaluated by a combined quality metric. RF and ANNs based models allowed not only to build more accurate forecasting tools, but also assisted in verifying five additional IHM predictors.

The high-dimensional feature vectors of hyper spectral data often impose a high computational cost as well as the risk of “over fitting ” when classification is performed. Therefore it is necessary to reduce the dimensionality through ways like feature selection. Currently, there are two kinds of feature selection methods: filter methods and wrapper methods. The form kind requires no feedback from classifiers and estimates the classification performance indirectly. The latter kind evaluates the “goodness ” of selected feature subset directly based on the classification accuracy. Many experimental results have proved that the wrapper methods can yield better performance, although they have the disadvantage of high computational cost. In this paper, we present a Genetic Algorithm (GA) based wrapper method for classification of hyper spectral data using Support Vector Machine (SVM), a state-of-art classifier that has found success in a variety of areas. The genetic algorithm (GA), which seek to solve optimization problems using the methods of evolution, specifically survival of the fittest, was used to optimize both the feature subset, i.e. band subset, of hyper spectral data and SVM kernel parameters simultaneously. A special strategy was adopted to reduce computation cost caused by the high-dimensional feature vectors of hyper spectral data when the feature subset part of chromosome was designed. The GA-SVM method was realized using the ENVI/IDL language, and was then tested by applying to a HYPERION hyper spectral image. Comparison of the optimized results and the un-optimized results showed that the GA-SVM method could significantly reduce the computation cost while improving the classification accuracy. The number of bands used for classification was reduced from 198 to