New feature selection methods based on opposition-based learning and self-adaptive cohort intelligence for predicting patient no-shows

Classification problem in biological Omics data has gained in popularity in recent years. In consideration of the high dimension of the Omics dataset, the importance of feature selection technology becomes apparent. Feature selection can not only improve the classification accuracy but also avoid overfitting which is a common issue in machine learning. There are three main categories of feature selection methods: filter methods, wrapper methods, and hybrid methods. Yet it is difficult for researchers to choose among them. In this study, we conducted a comprehensive comparison for filter methods, wrapper methods, and hybrid methods. Specifically, we selected information gain as the filter method, genetic algorithm, and binary particle swarms optimization as the wrapper methods, IG-GA and IG-BPSO as the hybrid methods for comparison. The experimental results show that the IG-BPSO, a hybrid method, has the highest classification accuracy among these feature selection methods on three Omics datasets.

A new hybrid filter/wrapper algorithm for feature selection in classification

Particle swarm optimization (PSO) is a population-based algorithm designed to tackle various optimization problems. However, its performance deteriorates significantly when optimization problems are subjected to noise. PSO is strongly influenced by its previous best particles and global best one, which may lead to premature convergence and fall into local optima. This also holds true for various PSO variants dealing with optimization problems in noisy environments. Opposition-based learning (OBL) is well-known for its ability to increase population diversity. In this paper, we propose hybrid PSO algorithms that introduce OBL into PSO variants for improving the latter’s performance. The proposed hybrid algorithms employ probabilistic OBL for a swarm. In contrast to other integrations of PSO and OBL, we select the top fittest particles from the current swarm and its opposite swarm to improve the entire swarm’s fitness. Experiments on 20 benchmark functions subject to different levels of noise show that the proposed hybrid PSO algorithms outperform their counterpart PSO variants as well as composite differential evolution in most cases.

Feature selection plays a critical role in improving the efficiency, accuracy, and interpretability of machine learning models, particularly when dealing with high-dimensional datasets. Among various approaches, wrapper-based feature selection methods are known for their ability to capture feature interactions by directly optimizing model performance. This study presents a comprehensive comparative analysis of six wrapper feature selection techniques—Recursive Feature Elimination (RFE), Sequential Forward Selection (SFS), Sequential Backward Selection (SBS), Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Differential Evolution (DE)—in conjunction with five widely used classification algorithms: Decision Tree, K-Nearest Neighbour, Random Forest, Logistic Regression, and Support Vector Machine. Experiments are conducted on an educational dataset comprising 395 student records with 30 attributes obtained from the UCI repository, using different feature subset sizes (all features, top 20, top 15, and top 10). Model performance is evaluated using accuracy, precision, recall, F1-score, and AUC. The results demonstrate that wrapper methods significantly enhance classification performance while reducing dimensionality, with GA and RFE consistently emerging as the most effective techniques across multiple classifiers. DE also shows strong performance, particularly with Logistic Regression and Random Forest, whereas PSO generally underperforms in terms of AUC. Furthermore, reducing the feature set does not adversely affect predictive accuracy and, in several cases, leads to improved generalization. The findings confirm the effectiveness of wrapper methods for educational data mining and provide practical insights for selecting optimal feature–classifier combinations.

A simple and efficient filter feature selection method via document-term matrix unitization

There is a consistent rise in chronic diseases worldwide. These diseases decrease immunity and the quality of daily life. The treatment of these disorders is a challenging task for medical professionals. Dimensionality reduction techniques make it possible to handle big data samples, providing decision support in relation to chronic diseases. These datasets contain a series of symptoms that are used in disease prediction. The presence of redundant and irrelevant symptoms in the datasets should be identified and removed using feature selection techniques to improve classification accuracy. Therefore, the main contribution of this paper is a comparative analysis of the impact of wrapper and filter selection methods on classification performance. The filter methods that have been considered include the Correlation Feature Selection (CFS) method, the Information Gain (IG) method and the Chi-Square (CS) method. The wrapper methods that have been considered include the Best First Search (BFS) method, the Linear Forward Selection (LFS) method and the Greedy Step Wise Search (GSS) method. A Decision Tree algorithm has been used as a classifier for this analysis and is implemented through the WEKA tool. An attribute significance analysis has been performed on the diabetes, breast cancer and heart disease datasets used in the study. It was observed that the CFS method outperformed other filter methods concerning the accuracy rate and execution time. The accuracy rate using the CFS method on the datasets for heart disease, diabetes, breast cancer was 93.8%, 89.5% and 96.8% respectively. Moreover, latency delays of 1.08 s, 1.02 s and 1.01 s were noted using the same method for the respective datasets. Among wrapper methods, BFS’ performance was impressive in comparison to other methods. Maximum accuracy of 94.7%, 95.8% and 96.8% were achieved on the datasets for heart disease, diabetes and breast cancer respectively. Latency delays of 1.42 s, 1.44 s and 132 s were recorded using the same method for the respective datasets. On the basis of the obtained result, a new hybrid Attribute Evaluator method has been proposed which effectively integrates enhanced K-Means clustering with the CFS filter method and the BFS wrapper method. Furthermore, the hybrid method was evaluated with an improved decision tree classifier. The improved decision tree classifier combined clustering with classification. It was validated on 14 different chronic disease datasets and its performance was recorded. A very optimal and consistent classification performance was observed. The mean values for accuracy, specificity, sensitivity and f-score metrics were 96.7%, 96.5%, 95.6% and 96.2% respectively.

Various opposition-based learning (OBL) strategies were proposed to further improve solution quality since 2005. However, in all existing OBLs, only one opposite point is generated for a given candidate solution in population. There is the old saying that there is strength in numbers. The purpose of this study is to examine the feasibility and effectiveness of double-points opposition-based learning (DOBL). The basic concept of DOBL is first proposed and then some viable forms are introduced in this paper. In addition, previous evaluation function and calculation method are highly complex, which block its application in DOBL. Based on a new evaluation function, another approach is presented to calculate its mathematical expectation more easily. The results by theoretical analysis and simulation experiment over sampling problems indicate that DOBL is better than the conventional OBL. In engineering application, double-points opposition-based learning differential evolution (DODE) is first developed to accelerate its convergence speed. Experiment results over 58 optimization problems show that DODE has an eclectic convergence speed in a fair competitive environment. Furthermore, the contribution of opposite points and the effect of jumping rate are also discussed in detail. When both considering algorithm convergence and reliability, a small jumping rate is generally recommended for an unknown optimization problem.

Optimising the job-shop scheduling problem using a multi-objective Jaya algorithm

Classification has been widely studied due to its practical applications. Feature selection aims to improve the classification accuracy by selecting a small feature subset from the original full feature set. However, identification of relevant features is not trivial due to the large search space. Particle swarm optimisation (PSO) is an efficient meta-heuristic algorithm which has shown to be promising in feature selection. However, traditional PSO uses its personal best experience and its historical best experience to determine its search direction, but this learning strategy may limit its performance for feature selection due to the premature convergence. Therefore, the potential of PSO needs to be further explored. In this paper, a new evolutionary learning algorithm termed hybridising PSO with differential evolution (HPSO-DE) is proposed to develop new feature selection methods. In HPSO-DE, differential evolution is applied to breed promising and efficient exemplars for PSO to guide its search, which is expected to not only preserve the diversity of the population but also guide particles to fly to promising areas. HPSO-DE is compared with three classic PSO variants and five traditional feature selection methods on 15 classification problems. The results show that the proposed algorithm can effectively achieve a higher classification accuracy with a smaller feature subset than the compared methods.

Target spectrum based feature selection (TSFS): A new method based on chain coding for target detection problems

Particle swarm optimization (PSO) and differential evolution (DE) are meta-heuristics which have been found to be successful in a wide variety of optimization tasks. The high speed of convergence and the relative simplicity of PSO make it a highly viable candidate to be used in multi-objective optimization problems (MOPs). Therefore, several PSO approaches capable to handle MOPs (MOPSOs) have appeared in the past. There are some problems, however, where PSO-based algorithms have shown a premature convergence. On the other hand, multi- objective DEs (MODE) have shown lower speed of convergence than MOPSOs but they have been successfully used in problems where MOPSO have mistakenly converged. In this work, we have developed experiments to observe the convergence behavior, the online convergence, and the diversity of solutions of both meta-heuristics in order to have a better understanding about how particles and solutions move in the search space. To this end, MOPSO and MODE algorithms under (to our best effort) similar conditions were used. Moreover, the ZDT test suite was used on all experiments since it allows to observe Pareto fronts in two-dimensional scatter plots (more details on this are presented on the experiments section). Based on the observations found, modifications to two PSO-based algorithms from the state of the art were proposed resulting in a rise on their performance. It is concluded that MOPSO presents a poor distributed scheme that leads to a more aggressive search. This aggressiveness showed to be detrimental for the selected problems. On the other hand, MODE seemed to generate better distributed points on both decision and objective space allowing it to produce better results.

Particle swarm optimization (PSO) is a global algorithm which is inspired by birds flocking and fish schooling. PSO has shown good search ability in many complex optimization problems, but premature convergence is still a main problem. A novel hybrid PSO(NHPSO) was proposed, which employed hybrid strategies, including dynamic step length (DSL) and opposition-based learning (OBL). DSL is helpful to enhance local search ability of PSO, and OBL is beneficial for improving the quality of candidate solutions. In order to verify the performance of NHPSO, we test it on several benchmark functions. The simulation results demonstrate the effectiveness and efficiency of our approach.

Particle Swarm Optimization or Differential Evolution—A comparison

Fuel cells are complex multiphysical systems characterized by a stratified geometry consisting of thin layers of different materials. Their full characterization involves a number of physical parameters, which are important for quantifying and comparing the performance of the materials. The physical parameters are also used in fuel cell models, which allow the fast exploration of different operating scenarios and can be implemented in optimization procedures aimed at achieving the best structural and operating conditions. The system of equations involved in these models, e.g. the Nernst equation, the Butler-Volmer equation and the diffusion equation, are strongly non-linear; thus, the model itself is extremely sensitive to parameter variations. Careful ex-situ measurements of such parameters can be performed by means of a number of diagnostic techniques, whose results are however not completely consistent with fuel cell operation due to several effects, e.g. variable interactions and boundary effects in the thin layers of the fuel cell. Conversely, “in situ” measurements can provide meaningful operational values, but a very few techniques are available to determine a limited number of parameters.A solution can consist in multiple parameter identification from multiple fundamental measurements, such as polarization curve and electrochemical impedance spectroscopy, performed in different conditions, e.g. at different temperature, pressure, concentration, and humidification.If there is only one unknown parameter, the solution is easy, requiring just a statistical interpolating technique. In the case of multiple unknown parameters, which is the most frequent condition, the problem becomes increasingly difficult as the number of parameters increases as duplicity problem emerges, i.e., several groups of parameters may lead to the same performance (e.g. polarization curve).Recently, a number of numerical tools are under development to face this kind of problems, some based on stochastic mathematical models, each presenting specific features in terms of accuracy, convergence, stability, robustness, and calculation speed [1]. Among the most promising methods, some of which have already been applied to multiple fuel cell parameter identification, are the Ishibashi–Kimura method [2], the Lambert W-function method [3], the Levenberg-Marquardt algorithm (LMA), the Gauss-Newton algorithm (GNA), the hybrid artificial bee colony algorithm [4], the trust region method, the current switching method [5], the simulated annealing method [6], and the particle swarm optimization (PSO) [7], and the differential evolution method [8].The presentation will focus on the application of a particular class of stochastic optimization methods to the problem of multiple parameters identification. These methods have already been widely tested and applied successfully to the simultaneous search of sets of optimal parameters with different physical dimensions, making them ideal tools to cope with the present problem. Their recent relevant literature is vast. They are especially suited for multimodal and noisy functions over large search spaces, which makes them particularly appealing for the problem at hand. The main drawback of such methods is the high number of objective function evaluations needed for convergence; however, thanks to a particular formulation of the PEMFC model, such drawback can be of minor impact, i.e. the overall computational times are on the order of a few tens of seconds.We are going to present the application of two of the most well-known metaheuristic methods, namely Particle Swarm Optimization (PSO) and Differential Evolution (DE), developed in previous work and suitably adapted. It should be noted that PSO has already been applied to parameter identification for FCs, while the application of DE is novel in this context. This is somewhat surprising since, as results will show, DE provides better results and offers higher robustness. We will present the implementation details regarding PSO and DE. The optimization algorithms have not been tuned to the specific problem and algorithm control parameters have been chosen from reasonable values suggested in literature.

Particle swarm optimization (PSO) is a stochastic, population-based optimization method, which has been applied successfully to a wide range of problems. However, PSO is computationally expensive and suffers from premature convergence. In this paper, opposition-based learning is used to improve the performance of PSO. The performance of the proposed approaches is investigated and compared with PSO when applied to eight benchmark functions. The experiments conducted show that opposition-based learning improves the performance of PSO.