A hybrid binary dwarf mongoose optimization algorithm with simulated annealing for feature selection on high dimensional multi-class datasets

Distributed feature selection (DFS) strategy for microarray gene expression data to improve the classification performance

Occam's razor in dimension reduction: Using reduced row Echelon form for finding linear independent features in high dimensional microarray datasets

High-dimensional datasets with low sample sizes (HDLSS) are pivotal in the fields of biology and bioinformatics. One of core objective of HDLSS is to select most informative features and discarding redundant or irrelevant features. This is particularly crucial in bioinformatics, where accurate feature (gene) selection can lead to breakthroughs in drug development and provide insights into disease diagnostics. Despite its importance, identifying optimal features is still a significant challenge in HDLSS. To address this challenge, we propose an effective feature selection method that combines gradual permutation filtering with a heuristic tribrid search strategy, specifically tailored for HDLSS contexts. The proposed method considers inter-feature interactions and leverages feature rankings during the search process. In addition, a new performance metric for the HDLSS that evaluates both the number and quality of selected features is suggested. Through the comparison of the benchmark dataset with existing methods, the proposed method reduced the average number of selected features from 37.8 to 5.5 and improved the performance of the prediction model, based on the selected features, from 0.855 to 0.927. The proposed method effectively selects a small number of important features and achieves high prediction performance.

Network intrusion detection system are significant features that contribute to the enterprises and organization network success. In the past decade, Intrusion Detection system (IDS) using several methods have been proposed and implemented to safeguard the networks within an organization to be reliable, secure, and readily available. Although, some of these methods are built utilizing machine learning concepts. IDS utilizing machine learning techniques are accurate, effective and efficient in spotting attacks. However, the machine learning model performance reduces with high dimensional intrusion dataset. Therefore, it is sacrosanct to implement an efficient feature selection technique that can generate a great impact on the classification stage. Also, the feature selection phase is the most serious phase in IDS based on machine learning. This phase is expensive both in time and efforts. Furthermore, most of the proposed machine learning intrusion detection system suffered from low detection accuracy and high false positive rate when the models are experimented on high dimensional dataset. The aim of this paper is to propose a binary firefly algorithm (BFFA) based feature selection for IDS. We first performed normalization in the first stage of the model. Subsequently, the BFFA algorithm was used for feature selection stage. We adopted random forest algorithm for the classification phase. The experiment was performed on high dimensional University of New South Wales-NB 2015 (UNSW-NB15) dataset with seventy five percent of the data used for training the model and twenty percent for testing. The findings showed an accuracy of 99.72%, detection rate of 99.84%, precision of 99.27%, recall of 99.84% and F-score of 99.56%. The results were gauge with the state-of-the-art results and our results were found outstanding.

DBFS: An effective Density Based Feature Selection scheme for small sample size and high dimensional imbalanced data sets

SUMMARY Many interesting inverse problems in geophysics are non-linear and multimodal. Parametrization of these problems leads to an objective function, or measure of agreement between data and model predictions, that has a complex topography with many local minima. Optimization algorithms that rely on local gradients in the objective function or that search the model space locally may become trapped in these local minima. By combining simulated annealing with the downhill simplex method, a hybrid global search algorithm is presented in this paper for non-linear, multimodal, inverse problems. The hybrid algorithm shares the advantages of both local search methods that perform well if the local model is suitable, and global methods that are able to explore efficiently the full model space. The hybrid algorithm also utilizes a larger and more complex memory to store information on the objective function than simulated annealing algorithms. The effectiveness of this new scheme is evaluated in three problems: minimization of the multidimensional Rosenbrock function, non-linear, 1 -D, acoustic waveform inversion, and residual statics. The performance of the hybrid algorithm is compared with simulated annealing and genetic algorithms and is shown to converge more rapidly and to have a higher success rate of locating the global minimum for the cases investigated.

Arctic puffin optimization: A bio-inspired metaheuristic algorithm for solving engineering design optimization

Metaheuristic algorithms have been hybridized with the standard K-means to address the latter’s challenges in finding a solution to automatic clustering problems. However, the distance calculations required in the standard K-means phase of the hybrid clustering algorithms increase as the number of clusters increases, and the associated computational cost rises in proportion to the dataset dimensionality. The use of the standard K-means algorithm in the metaheuristic-based K-means hybrid algorithm for the automatic clustering of high-dimensional real-world datasets poses a great challenge to the clustering performance of the resultant hybrid algorithms in terms of computational cost. Reducing the computation time required in the K-means phase of the hybrid algorithm for the automatic clustering of high-dimensional datasets will inevitably reduce the algorithm’s complexity. In this paper, a preprocessing phase is introduced into the K-means phase of an improved firefly-based K-means hybrid algorithm using the concept of the central limit theorem to partition the high-dimensional dataset into subgroups of randomly formed subsets on which the K-means algorithm is applied to obtain representative cluster centers for the final clustering procedure. The enhanced firefly algorithm (FA) is hybridized with the CLT-based K-means algorithm to automatically determine the optimum number of cluster centroids and generate corresponding optimum initial cluster centroids for the K-means algorithm to achieve optimal global convergence. Twenty high-dimensional datasets from the UCI machine learning repository are used to investigate the performance of the proposed algorithm. The empirical results indicate that the hybrid FA-K-means clustering method demonstrates statistically significant superiority in the employed performance measures and reducing computation time cost for clustering high-dimensional dataset problems, compared to other advanced hybrid search variants.

Feature selection (FS) using meta-heuristic algorithms on high-dimensional datasets (HDD) is becoming more prevalent due to the continuous advancement in data mining. However, the difficulty in identifying the threshold of features in a dataset to be categorised as HDD remains an issue due to the different schools of thought on this matter. Therefore, this survey intended to determine the threshold for a number of features to be HDD, and subsequently identify the trend or potential FS method for HDD and the most preferred meta-heuristic algorithms and classifiers for both wrapper-based and filter-based FS methods to analyse HDD. This study performed an extensive systematic literature review by implementing the PRISMA guidelines on 62 research articles that were published between 2016 to 2021. This survey proposed a novel grouping technique called literal grouping and data grouping (LGDG) to accurately group the chosen articles based on HDD. The LGDG method serves as a guide for other researchers who intend to perform FS research related to HDD. Literal grouping refers to searching for selected papers using specific keywords, like HDD in this case. While data grouping compares the number of features in datasets towards the threshold, which is set at 2,000 features by the majority. Based on the analyses of all the LGDG groupings, the filter-based FS method gained more attention in recent years with competent results no less than wrapper-based, especially on HDD. Besides that, Moth Flame Optimisation works well in filter-based methods, whereas Cuckoo Optimisation Algorithm works well in wrapper-based, while Whale Optimisation Algorithm works well in both FS methods. As for the classifier’s preferences, SVM, DT, and NB are preferred by the filter-based, while KNN is preferred by the wrapper-based method. It can be recommended that reviewing other aspects such as multi-objective FS on HDD and including more FS methods could be included in future studies as an extension to this survey.

Rayleigh wave inversion using heat-bath simulated annealing algorithm

The removal of irrelevant and insignificant features from the high-dimensional dataset is a necessary prerequisite for the exploration of information. Meta-heuristic optimization techniques have been widely used in the field of knowledge discovery over the last few years. The whale optimization algorithm (WOA) is a swarm-based metaheuristic technique that is often used in the field of dimensionality reduction. Among the various WOA-based feature selection techniques in the literature, not a single technology illuminates the stability issue of WOA. Stability is often identified as a sensitivity to the disruption of input data during the process of selecting significant features. In this study, a new feature selection model based on improved WOA (iWOA) is proposed to select significant features from a high-dimensional microarray dataset. The stability of the results obtained is evaluated with the existing stability index that satisfies all the required characteristics of the stability measure. In addition, the results of the proposed model are compared with other contemporary meta-heuristics techniques. The proposed iWOA proposes its identification as a well-stable feature selection technique according to the strength of the stability index agreement.

Feature selection is performed to eliminate irrelevant features to reduce computational overheads. Metaheuristic algorithms have become popular for the task of feature selection due to their effectiveness and flexibility. Hybridization of two or more such metaheuristics has become popular in solving optimization problems. In this paper, we propose a hybrid wrapper feature selection technique based on binary butterfly optimization algorithm (bBOA) and Simulated Annealing (SA). The SA is combined with the bBOA in a pipeline fashion such that the best solution obtained by the bBOA is passed on to the SA for further improvement. The SA solution improves the best solution obtained so far by searching in its neighborhood. Thus the SA tries to enhance the exploitation property of the bBOA. The proposed method is tested on twenty datasets from the UCI repository and the results are compared with five popular algorithms for feature selection. The results confirm the effectiveness of the hybrid approach in improving the classification accuracy and selecting the optimal feature subset.

In the modern world, wireless body area networks (WBANs) play an essential role in psychological and biomedical applications. The use of WBANs in medical applications is limited due to various issues related to the sensors, viz., irregularity in data production, replacement and recharging of their batteries and the energy consumed by the networks. This manuscript addresses how these problems can be solved along with optimization of the energy consumption through efficient design of the system by applying routing protocols and heuristic-based optimization algorithms. In this paper, the particle swarm optimization (PSO) algorithm is a heuristic search algorithm that relies on an upgrade mechanism of the velocity and position of swarms. Although PSO has excellent exploration capability in global search, it becomes quickly stuck in local minima. To enhance the local search function of the current PSO algorithm, a simulated annealing (SA) algorithm has been incorporated in the exploitation phase. The newly developed hybrid PSO-SA (hPSO-SA) algorithm is validated with other state-of-the-art nature-inspired algorithms on eighteen benchmarks and five real engineering design problems. The statistical results of the proposed hPSO-SA algorithm are promising and indicate very good efficiency. The paper also aims at the application of the proposed algorithm to the WBAN design problem for minimization of the energy consumption through better selection of the relay node. The proposed hPSO-SA algorithm outperforms twelve other metaheuristic algorithms, taking hybrid variants for comparison.

Feature selection is the preliminary step in machine learning and data mining. It identifies the most important and relevant features within a dataset by eliminating the redundant or irrelevant features. The substantial benefits may include an improved performance in terms of high prediction accuracy, reduced computational complexity, and simply interpretable underlying models. In this paper, we present a novel framework to investigate and understand the importance of Monte Carlo tree search (MCTS) in feature selection for very high-dimensional datasets. We construct a binary feature selection tree where each node represents one of the two feature states: a feature is selected or not. The search starts with an empty root node reflecting that no feature is selected. Then, the search tree is expanded by adding nodes in an incremental fashion through MCTS-based simulations. Following tree and default policy, every iteration generates an initial feature subset, where a filter is used to select the top k features forming the candidate feature subset. The classification accuracy is used as the goodness or reward of the candidate feature subset and propagated backward up to the root node following the active path. Finally, the candidate subset with highest reward is selected as the best feature subset. Experiments are performed on 30 real-world datasets, including 14 very high-dimensional microarray datasets, and results are also compared with state-of-the-art methods in the literature, which proves the efficacy, validity, and significance of the proposed method.

A new hybrid filter/wrapper algorithm for feature selection in classification