Feature Selection via Dynamic Feature Graph

Accurate classification of fetal biometric planes in ultrasound imaging is more important for effective prenatal screening and early diagnosis of fetal abnormalities. To enhance the diagnostic efficiency, the research proposed a novel method called “Dynamic Graph-Based Quantum Feature Selection” (DG-QFS) framework to improve the classification performance by integrating the quantum computing principles. Features are extracted from ultrasound images using a pre-trained deep learning model and processed through a quantum-driven feature selection pipeline that models the inter-feature relationships using dynamically entangled multi-qubit graphs. In the DG-QFS method, qubits represent extracted deep feature nodes, while a quantum entanglement score-based dynamic graph captures the complex dependencies. Entanglement score and dynamic graph centrality are used to select the most informative features. The refined feature set is classified using a lightweight multi-layer perceptron (MLP), stochastic gradient descent with adaptive learning rate. Examined the proposed model on a fetal plane ultrasound scan dataset, including 12,400 images of six categorical planes, such as brain, thorax, abdomen, femur, maternal cervix, and other views. Experimental results demonstrate that the proposed model achieves a classification accuracy of 96.73%, significantly outperforming baseline deep learning and conventional feature selection techniques regarding accuracy, generalization, and interpretability.

TodyNet: Temporal dynamic graph neural network for multivariate time series classification

With the continuous development of Internet technology, data gradually present a complicated and high-dimensional trend. These high-dimensional data have a large number of redundant features and irrelevant features, which bring great challenges to the existing machine learning algorithms. Feature selection is one of the important research topics in the fields of machine learning, pattern recognition and data mining, and it is also an important means in the data preprocessing stage. Feature selection is to look for the optimal feature subset from the original feature set, which would improve the classification accuracy and reduce the machine learning time. The traditional feature selection algorithm tends to ignore the kind of feature which has a weak distinguishing capacity as a monomer, whereas the feature group’s distinguishing capacity is strong. Therefore, a new dynamic interaction feature selection (DIFS) algorithm is proposed in this paper. Initially, under the theoretical framework of interactive information, it redefines the relevance, irrelevance and redundancy of the features. Secondly, it offers the computational formulas for calculating interactive information. Finally, under the eleven data sets of UCI and three different classifiers, namely, KNN, SVM and C4.5, the DIFS algorithm increases the classification accuracy of the FullSet by 3.2848% and averagely decreases the number of features selected by 15.137. Hence, the DIFS algorithm can not only identify the relevance feature effectively, but also identify the irrelevant and redundant features. Moreover, it can effectively improve the classification accuracy of the data sets and reduce the feature dimensions of the data sets.

Estimating the structure of directed acyclic graphs (DAGs) of features (variables) plays a vital role in revealing the latent data generation process and providing causal insights in various applications. Although there have been many studies on structure learning with various types of data, the structure learning on the dynamic graph has not been explored yet, and thus we study the learning problem of node feature generation mechanism on such ubiquitous dynamic graph data. In a dynamic graph, we propose to simultaneously estimate contemporaneous relationships and time-lagged interaction relationships between the node features. These two kinds of relationships form a DAG, which could effectively characterize the feature generation process in a concise way. To learn such a DAG, we cast the learning problem as a continuous score-based optimization problem, which consists of a differentiable score function to measure the validity of the learned DAGs and a smooth acyclicity constraint to ensure the acyclicity of the learned DAGs. These two components are translated into an unconstraint augmented Lagrangian objective which could be minimized by mature continuous optimization techniques. The resulting algorithm, named GraphNOTEARS, outperforms baselines on simulated data across a wide range of settings that may encounter in real-world applications. We also apply the proposed approach on two dynamic graphs constructed from the real-world Yelp dataset, demonstrating our method could learn the connections between node features, which conforms with the domain knowledge.

Android OS based applications offer services in various aspects of our daily lives such as banking, personal, professional, social, etc. Increased usage of Android applications makes them extremely vulnerable to various malware threats. A resilient and attack resistant machine learning based Android malware detector is desired to achieve a safe working environment. This work employs feature selection on static and dynamic features and proposes a hybrid feature selection method that can identify most informative features while eliminating the irrelevant ones. Information gain from filter and recursive feature elimination from wrapper feature selection methods outperform other evaluated feature selection techniques. Thereafter, different classification algorithms are trained on the features selected through hybrid feature selection technique and experimental results showed that XGBoost obtained maximum accuracy i.e., 98% and 89% for binary and multiclass classification respectively using only 50 features.

Dynamic multi-scale spatial-temporal graph convolutional network for traffic flow prediction

Chronic kidney disease (CKD) involves numerous variables, but only a few significantly impact the classification task. The statistically equivalent signature (SES) method, inspired by constraint-based learning of Bayesian networks, is employed to identify essential features in CKD. Unlike conventional feature selection methods, which typically focus on a single set of features with the highest predictive potential, the SES method can identify multiple predictive feature subsets with similar performance. However, most feature selection (FS) classifiers perform suboptimally with strongly correlated data. The FS approach faces challenges in identifying crucial features and selecting the most effective classifier, particularly in high-dimensional data. This study proposes using the Least Absolute Shrinkage and Selection Operator (LASSO) in conjunction with the SES method for feature selection in CKD identification. Following this, an ensemble deep-learning model combining long short-term memory (LSTM) and gated recurrent unit (GRU) networks is proposed for CKD classification. The features selected by the hybrid feature selection method are fed into the ensemble deep-learning model. The model’s performance is evaluated using accuracy, precision, recall, and F1 score metrics. The experimental results are compared with individual classifiers, including decision tree (DT), Random Forest (RF), logistic regression (LR), and support vector machine (SVM). The findings indicate a 2% improvement in classification accuracy when using the proposed hybrid feature selection method combined with the LSTM and GRU ensemble deep-learning model. Further analysis reveals that certain features, such as HEMO, POT, bacteria, and coronary artery disease, contribute minimally to the classification task. Future research could explore additional feature selection methods, including dynamic feature selection that adapts to evolving datasets and incorporates clinical knowledge to enhance CKD classification accuracy further.

The vast volume of redundant and irrelevant network traffic data poses significant hurdles for intrusion detection. Effective feature selection is crucial for eliminating irrelevant information. Presently, most filtering and embedded methods rely on fixed thresholds or ratios, necessitating prior knowledge. Conversely, wrapper methods are computationally intensive, and individual feature selection methods may introduce biases in evaluation. To address these challenges, this study introduces Adaptive Neighborhood based Feature Selection (AN-SFS), a dynamic feature selection approach that adapts to local statistical properties of the data. Unlike traditional methods, AN-SFS adjusts its threshold based on the characteristics of the current feature subset and incorporates statistical measures of neighboring features, capturing subtle relationships and dependencies. This adaptability enables AN-SFS to achieve robust and effective feature selection outcomes. Using NSL-KDD and UNSW-NB15 datasets, our model demonstrates superiority over conventional ML classifiers in detection rate, precision, and recall, achieving outstanding accuracy rates of 99.3% on NSL-KDD and 97.5% on UNSW-NB15, significantly outperforming contemporary methods. To further demonstrate the effectiveness of our feature selection approach, we conducted a series of comparisons with other statistical methods, including ANOVA, Pearson correlation, and the chi-square test. In each comparison, our approach consistently outperformed these alternatives, underscoring its superior efficacy.

Spectral feature selection (SFS) is superior to conventional feature selection methods in many aspects, by extra importing a graph matrix to preserve the subspace structure of data. However, the graph matrix of classical SFS that is generally constructed by original data easily outputs a suboptimal performance of feature selection because of the redundancy. To address this, this paper proposes a novel feature selection method via coupling the graph matrix learning and feature data learning into a unified framework, where both steps can be iteratively update until achieving the stable solution. We also apply a low-rank constraint to obtain the intrinsic structure of data to improve the robustness of learning model. Besides, an optimization algorithm is proposed to solve the proposed problem and to have fast convergence. Compared to classical and state-of-the-art feature selection methods, the proposed method achieved the competitive results on twelve real data sets.

Feature selection (FS) is a very important pre-processing technique in machine learning and data mining. It aims to select a small subset of relevant and informative features from the original feature space that may contain many irrelevant, redundant and noisy features. Feature selection usually leads to better performance, interpretability, and lower computational cost. In the literature, FS methods are categorized into three main approaches: Filters, Wrappers, and Embedded. In this paper we introduce a new feature selection method called graph feature selection (GFS). The main steps of GFS are the following: first, we create a weighted graph where each node corresponds to each feature and the weight between two nodes is computed using a matrix of individual and pairwise score of a Decision tree classifier. Second, at each iteration, we split the graph into two random partitions having the same number of nodes, then we keep moving the worst node from one partition to another until the global modularity is converged. Third, from the final best partition, we select the best ranked features according to a new proposed variable importance criterion. The results of GFS are compared to three well-known feature selection algorithms using nine benchmarking datasets. The proposed method shows its ability and effectiveness at identifying the most informative feature subset.

Small sample is an acute problem in many application domains, which may be partially addressed by feature selection or dimensionality reduction. For the purpose of distance learning, we describe a method for feature selection using equivalence constraints between pairs of datapoints. The method is based on L1 regularization and optimization. Feature selection is then incorporated into an existing non-parametric method for distance learning, which is based on the boosting of constrained generative models. Thus the final algorithm employs dynamical feature selection, where features are selected anew in each boosting iteration based on the weighted training data. We tested our algorithm on the classification of facial images, using two public domain databases. We show the results of extensive experiments where our method performed much better than a number of competing methods, including the original boosting-based distance learning method and two commonly used Mahalanobis metrics.

BackgroundGait dynamics are widely used to diagnose neurodegenerative illnesses like ALS, Parkinson’s, and Huntington’s. Cognitive bias might hinder clinical assessments and early detection. Machine learning may help detect aberrant gait patterns early.MethodsA novel ensemble classifier, the Dynamic Weighted Prediction Framework (DWPF), and an innovative feature selection methodology, Stability-Enhanced Dynamic Thresholding (SEDT), have been proposed for neurodegenerative disease detection. DWPF combines predictions from various heterogeneous base classifiers using dynamic weighted averaging with weights based on performance. By adapting to classifier efficacies, this method improves ensemble accuracy. The DWPF’s dynamic weight allocation based on real-time classifier performance makes the model robust across datasets and classification problems. This strategy balances each base classifier's reliability to improve accuracy and model resilience to overfitting and underfitting. SEDT uses stability enhancement and dataset-specific feature relevance thresholds to identify significant neurodegenerative disease features before classification. Unlike fixed threshold approaches, SEDT may adjust to different data subsets' feature relevance, preserving only the most important features for model training. The DWPF ensemble and SEDT feature selection approach are merged to detect neurodegenerative diseases in gait data. Extensive binary classification testing has been conducted to discriminate controls from ALS, Parkinson's, and Huntington’s patients.ResultsThe proposed SEDT + DWPF model was validated using Holdout Validation, K-Fold Cross-Validation, and Monte Carlo (Shuffle-Split) Cross-Validation. Monte Carlo Cross-Validation had the best ALS detection accuracy (93.50%), sensitivity (94.63%), and specificity (92.80%). K-Fold Cross-Validation detected Huntington's disease with the best accuracy (81.09%), sensitivity (74.19%), and specificity (89.28%). Monte Carlo Cross-Validation performed best for Parkinson's disease identification, with 84.58% accuracy, 78.95% sensitivity, and 89.71% specificity.ConclusionDWPF-SEDT integration improves neurodegenerative disease diagnosis using adaptive feature selection and robust ensemble classification. This method has great potential for early and accurate neurodegenerative disease diagnosis.

BackgroundLarge number of features are extracted from protein crystallization trial images to improve the accuracy of classifiers for predicting the presence of crystals or phases of the crystallization process. The excessive number of features and computationally intensive image processing methods to extract these features make utilization of automated classification tools on stand-alone computing systems inconvenient due to the required time to complete the classification tasks. Combinations of image feature sets, feature reduction and classification techniques for crystallization images benefiting from trace fluorescence labeling are investigated.ResultsFeatures are categorized into intensity, graph, histogram, texture, shape adaptive, and region features (using binarized images generated by Otsu’s, green percentile, and morphological thresholding). The effects of normalization, feature reduction with principle components analysis (PCA), and feature selection using random forest classifier are also analyzed. The time required to extract feature categories is computed and an estimated time of extraction is provided for feature category combinations. We have conducted around 8624 experiments (different combinations of feature categories, binarization methods, feature reduction/selection, normalization, and crystal categories). The best experimental results are obtained using combinations of intensity features, region features using Otsu’s thresholding, region features using green percentile G90 thresholding, region features using green percentile G99 thresholding, graph features, and histogram features. Using this feature set combination, 96% accuracy (without misclassifying crystals as non-crystals) was achieved for the first level of classification to determine presence of crystals. Since missing a crystal is not desired, our algorithm is adjusted to achieve a high sensitivity rate. In the second level classification, 74.2% accuracy for (5-class) crystal sub-category classification. Best classification rates were achieved using random forest classifier.ContributionsThe feature extraction and classification could be completed in about 2 s per image on a stand-alone computing system, which is suitable for real time analysis. These results enable research groups to select features according to their hardware setups for real-time analysis.

Traditional speech recognition methods based on static features of a word to justify whether the word is correctly recognized or not, which neglected the information carried by its contexts and the surrounding environment.In this paper,a 14.1% word error rate (WER) speech recognizer (SR) is used as the baseline system,and 10-dimension static features achived 24.9% decline of Classification Error Rate (CER). Context features and dynamic features are extracted in relation to the static features.The total 42-dimension features get an better CER of 7.4% than static features.But not all these features have a positive impact on the classification.Too many features not only take redundant information,but also make the classification process time-consuming.To solve this problem,feature extraction which can extract prime information from original features and feature selection method which can select effective features from the original feature set are proposed in this paper.The experimental results show that context features and dynamic features are effective features for classification,and the features can be considerably compressed through feature extraction and feature selection.

Feature selection is an important pre-processing step in many fields, such as data mining, machine learning and pattern recognition. This paper focuses on dynamically updating a subset of features with new samples arriving and provides a hypergraph model to deal with dynamic feature selection problem. Firstly, we discuss the relationship between feature selection of information system and minimum vertex cover of hypergraph, and feature selection is converted to a minimum vertex cover problem based on this relationship. Then, an algorithm for generating induced hypergraph from information system is presented, the induced hypergraph can be divided into two part: the original induced hypergraph and the added hypergraph with new samples arriving. Finally, a novel dynamic feature selection algorithm based on minimum vertex cover of hypergraph is proposed, and this algorithm only needs a small amount of computation. Experiments show that the proposed method is feasible and highly effective.