A Dynamic Surrogate-Assisted Hybrid Breeding Algorithm for High-Dimensional Imbalanced Feature Selection

High-dimensional datasets (HDDs) pose significant challenges in feature selection due to their complex nature. While metaheuristic algorithms like the Whale Optimisation Algorithm (WOA) have shown promise in addressing these challenges, stability issues are least addressed. Through an extensive literature review, it was identified that stability is manifested in the equilibrium of exploration and exploitation. Hence, this study introduced the Hierarchical Whale Optimisation Algorithm (HiWOA), an approach designed to enhance the WOA’s stability and performance in HDD feature selection tasks. The HiWOA incorporates a two-phase strategy comprising a nonlinear control parameter based on the arcsine function and a hierarchical position-update mechanism adapted from the Grey Wolf Optimiser. The proposed HiWOA was evaluated through 23 benchmark optimisation functions and feature selection experiments on 11 medical HDDs. The results indicate that the HiWOA outperformed the WOA and a modified variant (mWOA) in terms of a better fitness value, a more balanced exploration-exploitation ratio, and improved classification accuracy with fewer selected features. These findings demonstrate the HiWOA’s effectiveness in enhancing stability, making it a robust solution for high-dimensional optimisation and feature selection.

A velocity-based butterfly optimization algorithm for high-dimensional optimization and feature selection

Medical diagnosis is a crucial step for patient treatment. However, diagnosis is prone to bias due to imbalanced datasets. To overcome the imbalanced dataset problem, simple minority oversampling technique (SMOTE) was proposed that can generate new synthetic samples at data level to create the balance between minority and majority classes. However, the synthetic samples are generated on a random basis which causes class mixture problem; thus, resulting in deteriorating the classification performance and biased diagnosis. In order to overcome the SMOTE shortcomings, some modified methods were proposed that try to generate synthetic samples along the line segment of selected minority samples. Most of these methods adopt one of the two policies for selecting minority samples to generate synthetic samples: borderline region sampling or safe region sampling. However, they both suffer from over-generalisation problem. We propose a modified SMOTE-based resampling method called RSMOTE to alleviate the medical imbalanced dataset problem. We provide an in-depth analysis and verify the performance of RSMOTE over imbalanced medical datasets. In this paper, the proposed RSMOTE divides the minority sample domain into four regions (normal, semi-normal, semi-critical, and critical) based on the minority sample density analysis. RSMOTE discovers the minority sample region globally and applies the resampling near a specific group of samples. Our analysis and experiments verify that if synthetic samples are generated in the regions with high minority sample density, classification performance will be improved due to low risk of class mixture. Unlike some safe region methods, RSMOTE decides the region of minority samples on a global basis, thus removing the over-generalisation problem. Classic and additional evaluation metrics are considered to measure the effectiveness of the modified method: Recall, FP Rate, Precision, F-Measure, ROC area, and Average Aggregated Metric. We carried out experiments over various imbalanced medical datasets. Based on the minority sample density analysis, we propose RSMOTE method that divides the minority sample domain into four regions. The proposed RSMOTE includes four re-sampling methods that each of them carries out resampling on a specific region. According to the experimental results, resampling on the regions with high minority sample density obtained better results while those with lower minority sample density got the inferior results. Thus, we conclude that the RSMOTE is a more flexible resampling method for the imbalanced medical datasets that is capable of generating samples with various minority sample densities.

Power system congestion event prognosis (CEP) based on multivariate time series (MTS) learning is an effective way to improve the warning abilities against risky situations. However, it is still challenging to decide which measurement sequences of system variables should be selected to train MTS learning-based CEPs models, especially when facing high-dimensional candidate features from power systems. Focusing on this issue, this paper proposes a novel high-dimensional feature selection (FS) method to identify and select variables that contain beneficial data patterns for the prognosis of network congestion events. A wrapper framework embedded with multiple MTS learning models is first built to treat FS as a combinatorial optimization task and receive MTS as inputs directly without sequence compressions. Then, to improve the combinatorial optimization efficiency and FS results in the high-dimensional case, a hybrid structure called enhanced evolutionary computation (EEC) is developed by combining with information theory-based feature priority scores, which play the role of semi-guidance in the iterative search. Test results on both the real-world and synthetic datasets validate the effectiveness of the developed EEC structure and show that the selected features by the proposed FS method are more beneficial in identifying the early patterns of network congestion events, thus contributing to efficient and accurate CEP for power systems.

The challenges related to handling of the gigantic imbalanced data volumes are incredible and has set a new trail for its efficient processing. The inventive prospects contained by these huge imbalanced data sets have posed a priority of concern in recent research avenues. The several applications handling imbalanced Big Data sets have noted significance for precise classification while determining unidentified values from these data sets. Traditional classifiers are not able to discourse the imbalance of class distribution among the data samples. A class having fewer samples indicates difficulty in learning, whereas it points to a notable drop in the performance. Recent studies demonstrate that the classifier independent set of oversampling techniques are more capable to efficiently handle the issues raised in imbalanced data sets. An enhanced oversampling technique, viz., Minority–Majority Mix mean Oversampling Technique (MMMmOT), improving classification performance is discussed in detail in this paper. An appropriate consideration of majority as well as minority samples is planned to generate the synthetic samples. The proposed technique is investigated encircling data sets mainly from the UCI repository over Apache Hadoop. Furthermore, the stimulus of maintaining the imbalance ratio with better oversampling instances from the generated pool is analyzed. The results of classification performance are recognized using standard parameters like F-Measure and area under the curve. The achieved experimental outcomes clearly exhibit the preeminence of the presented technique over the traditional techniques.

For many high-dimensional genomic and epigenomic datasets, the outcome of interest is ordinal. While these ordinal outcomes are often thought of as the observed cutpoints of some latent continuous variable, some ordinal outcomes are truly discrete and are comprised of the subjective combination of several factors. The nonlinear stereotype logistic model, which does not assume proportional odds, was developed for these 'assessed' ordinal variables. It has previously been extended to the frequentist high-dimensional feature selection setting, but the Bayesian framework provides some distinct advantages in terms of simultaneous uncertainty quantification and variable selection. Here, we review the stereotype model and Bayesian variable selection methods and demonstrate how to combine them to select genomic features associated with discrete ordinal outcomes. We compared the Bayesian and frequentist methods in terms of variable selection performance. We additionally applied the Bayesian stereotype method to an acute myeloid leukemia RNA-sequencing dataset to further demonstrate its variable selection abilities by identifying features associated with the European LeukemiaNet prognostic risk score.

High dimension and complexity of network high-dimensional data lead to poor feature selection effect network high-dimensional data. To effectively solve this problem, feature selection algorithms for high-dimensional network data based on supervised discriminant projection (SDP) have been designed. The sparse representation problem of high-dimensional network data is transformed into an Lp norm optimization problem, and the sparse subspace clustering method is used to cluster high-dimensional network data. Dimensionless processing is carried out for the clustering processing results. Based on the linear projection matrix and the best transformation matrix, the dimensionless processing results are reduced by combining the SDP. The sparse constraint method is used to achieve feature selection of high-dimensional data in the network, and the relevant feature selection results are obtained. The experimental findings demonstrate that the suggested algorithm can effectively cluster seven different types of data and converges when the number of iterations approaches 24. The F1 value, recall, and precision are all kept at high levels. High-dimensional network data feature selection accuracy on average is 96.9%, and feature selection time on average is 65.1 milliseconds. The selection effect for network high-dimensional data features is good.

One of the more prevalent problems when working with bioinformatics datasets is class imbalance, when there are more instances in one class compared to the other class (es). This problem is made worse because frequently, the class of interest is also the minority class. A possible solution is data sampling, a powerful tool for combating class imbalance by adding or removing instances to make the dataset more balanced. In addition to the choice of including data sampling, one of the most important decisions when applying data is what the final class ratio should be. Commonly, the final class ratio when data is applied is 50:50, however it is an open question whether other ratios are more appropriate for certain imbalanced datasets (all datasets in this paper have 25.16% minority instances or less) where a 50:50 ratio requires extreme modification to the dataset. In this work we compare six different data approaches (feature selection with the pair wise combinations of three data techniques and two final class ratios) with feature selection without data with the goal of determining if the inclusion of data is beneficial and if so, what should be the final class ratio. In order to test the six data approaches and feature selection alone thoroughly, we utilize seven imbalanced and high-dimensional datasets, three feature selection techniques, and six classifiers. Our results show that for a majority of scenarios, random under along with either 35:65 or 50:50 is the best data approach. Statistical analysis shows that there is significant difference between the data approaches. However, despite this, we still recommend using random under along with 35:65 as the final class ratio. This is because of the frequency of both random under and 35:65 being the most frequent top performing data technique and class ratio respectively. Additionally, 35:65 will have fewer negative impacts than 50:50 (less data loss or over fitting, which makes it a better choice if all other factors are equal) and random under is more computationally efficient than any other form of sampling, including no sampling (both by not requiring any internal calculations and by producing a reduced, easier-to-work-with dataset). To our knowledge, this is the most comprehensive work which focuses on the choice of the inclusion and implementation of data with different final class ratios on bioinformatics datasets which exhibit such large levels of class imbalance.

Feature selection is an important pre-processing step in many fields, such as data mining, machine learning and pattern recognition. This paper focuses on dynamically updating a subset of features with new samples arriving and provides a hypergraph model to deal with dynamic feature selection problem. Firstly, we discuss the relationship between feature selection of information system and minimum vertex cover of hypergraph, and feature selection is converted to a minimum vertex cover problem based on this relationship. Then, an algorithm for generating induced hypergraph from information system is presented, the induced hypergraph can be divided into two part: the original induced hypergraph and the added hypergraph with new samples arriving. Finally, a novel dynamic feature selection algorithm based on minimum vertex cover of hypergraph is proposed, and this algorithm only needs a small amount of computation. Experiments show that the proposed method is feasible and highly effective.

Due to the widespread use of social media, people are exposed to fake news and misinformation. Spreading fake news has adverse effects on both the general public and governments. This issue motivated researchers to utilize advanced natural language processing concepts to detect such misinformation in social media. Despite the recent research studies that only focused on semantic features extracted by deep contextualized text representation models, we aim to show that content-based feature engineering can enhance the semantic models in a complex task like fake news detection. These features can provide valuable information from different aspects of input texts and assist our neural classifier in detecting fake and real news more accurately than using semantic features. To substantiate the effectiveness of feature engineering besides semantic features, we proposed a deep neural architecture in which three parallel convolutional neural network (CNN) layers extract semantic features from contextual representation vectors. Then, semantic and content-based features are fed to a fully connected layer. We evaluated our model on an English dataset about the COVID-19 pandemic and a domain-independent Persian fake news dataset (TAJ). Our experiments on the English COVID-19 dataset show 4.16% and 4.02% improvement in accuracy and f1-score, respectively, compared to the baseline model, which does not benefit from the content-based features. We also achieved 2.01% and 0.69% improvement in accuracy and f1-score, respectively, compared to the state-of-the-art results reported by Shifath et al. (A transformer based approach for fighting covid-19 fake news, arXiv preprint arXiv:2101.12027, 2021). Our model outperformed the baseline on the TAJ dataset by improving accuracy and f1-score metrics by 1.89% and 1.74%, respectively. The model also shows 2.13% and 1.6% improvement in accuracy and f1-score, respectively, compared to the state-of-the-art model proposed by Samadi et al. (ACM Trans Asian Low-Resour Lang Inf Process, https://doi.org/10.1145/3472620, 2021).

An efficient hybrid sine-cosine Harris hawks optimization for low and high-dimensional feature selection

Feature selection is an important processing step of data mining and aims at selecting a subset with high quality. The feature subset is obtained by reducing the irrelevant and redundant features based on maintaining satisfactory accuracy. Many algorithms might be trapped in local optima on solving high-dimensional feature selection. Hybrid rice optimization (HRO) is a newly presented algorithm, its population is divided into three types such as the maintainer, restorer, and sterile line to simulate the breeding process of Chinese hybrid rice. Three main stages are designed to update the restorer and sterile line. However, the maintainer line is not updated in HRO, which degrades the performance of the algorithm on high-dimensional feature selection. In this paper, a hybrid model named HROAS is proposed for feature selection by combining HRO with ant system (AS). AS is embedded as an operator in HRO to update the maintainer line due to its unique updating mechanism. The performance of HROAS is evaluated on nine standard benchmark datasets and compared with several classic and novel algorithms. The experimental results demonstrate the outperformance of HROAS.

Concept mapping is a knowledge representation method used to represent and understand concepts, entities, and the relationships between them, which are referred to as hyponymy–hypernymy semantic relations. These relations are primarily used to describe the hierarchical and categorical relationships between different concepts or entities. The detection of hyponymy–hypernymy semantic relations is an important task in the field of natural language processing, crucial to many downstream tasks such as information extraction, automatic reasoning, and personalized recommendations. These tasks often require understanding the semantic relations between concepts or entities in text for more accurate analysis and reasoning. Currently, algorithms for identifying and detecting hyponymy–hypernymy semantic relations face two main challenges: first, candidate hyponymy–hypernymy relation tuples do not exist in the same contextual sentence, failing to meet the co-occurrence requirement; second, distributed algorithms have issues with lexical memory. To address these issues, this paper proposes an improved algorithm for detecting hyponymy–hypernymy relations based on heterogeneous graph neural networks, aiming to detect hyponymy–hypernymy relations in various candidate word sets and then construct a hierarchical system. To meet the co-occurrence requirement of candidate word pairs, an open-source large model is utilized to generate contextual sentences for the candidate pairs. Sub-word features are adopted to capture the intrinsic semantic connections between nested phrases, thus alleviating the issue of reversed predictions for nested phrases. The representation of relation nodes is modified by encoding relation definitions through a pre-trained model, enabling the model to understand the semantic relations between concept nodes. To address the problem of overfitting in traditional graph attention networks, the calculation order of adjacency node aggregation is changed in the heterogeneous graph to capture dynamic attention features. In addition, a pipeline for hierarchical system construction is designed and implemented, combining the divide-and-conquer approach with loop detection algorithms. Compared to baseline metrics, the proposed method achieves a 4.14% improvement in accuracy and a 0.62% increase in F1 score on the EVALution dataset; a 4.89% increase in accuracy and a 0.71% improvement in F1 score on the Bansal dataset; and a 1.05% increase in accuracy, a 2.21% increase in recall, and a 1.79% improvement in F1 score on a self-annotated Chinese dictionary dataset.

An improved hybrid chameleon swarm algorithm for feature selection in medical diagnosis

With the explosive growth of data across various fields, effective data preprocessing has become increasingly critical. Evolutionary and swarm intelligence algorithms have shown considerable potential in feature selection. However, their performance often deteriorates in large-scale problems, due to premature convergence and limited exploration ability. To address these limitations, this paper proposes an algorithm named IHBOFS, a biomimetics-inspired optimization framework that integrates multiple adaptive strategies to enhance performance and stability. The introduction of the Good Point Set and Elite Opposition-Based Learning mechanisms provides the population with a well-distributed and diverse initialization. Furthermore, adaptive exploitation–exploration balancing strategies are designed for each subpopulation, effectively mitigating premature convergence. Extensive ablation studies on the CEC2022 benchmark functions verify the effectiveness of these strategies. Considering the discrete nature of feature selection, IHBOFS is further extended with continuous-to-discrete mapping functions and applied to six real-world datasets. Comparative experiments against nine metaheuristic-based methods, including Harris Hawk Optimization (HHO) and Ant Colony Optimization (ACO), demonstrate that IHBOFS achieves an average classification accuracy of 92.57%, confirming its superiority and robustness in high-dimensional feature selection tasks.