Adversarial Robustness Through Dynamic Ensemble Learning

Background: Ensemble selection is one of the most researched topics for ensemble learning. Researchers have been attracted to selecting a subset of base classifiers that may perform more helpful than the whole ensemble system classifiers. Dynamic Ensemble Selection (DES) is one of the most effective techniques in classification problems. DES systems obtain to select the most appropriate classifiers from the candidate classifier pool. Ensemble models that balance diversity and accuracy in the training process improve performance than the whole classifiers. Objective: In this paper, the novel techniques are proposed by combining Noise Filter (NF) and Dynamic Ensemble System (DES) to have better predictive accuracy. In other words, a noise filter and DES make the data cleaner and DES improve the performance of classification. Methods: The proposed NF-DES model had been demonstrated on twelve datasets, especially has three credit scoring datasets and a performance measure accuracy. Results: The results show that our proposed model has better than other models. Conclusion: The novel noise filer and dynamic ensemble learning with aim to improve the classification ability are presented. To improve performance of classification, noise filter with dynamic ensemble learning makes the noise data toward the correct class. Then, novel dynamic ensemble learning choose the appropriate subset classifiers in the pool of base classifiers.

This paper presents a novel dynamic ensemble learning (DEL) algorithm for designing ensemble of neural networks (NNs). DEL algorithm determines the size of ensemble, the number of individual NNs employing a constructive strategy, the number of hidden nodes of individual NNs employing a constructive–pruning strategy, and different training samples for individual NN’s learning. For diversity, negative correlation learning has been introduced and also variation of training samples has been made for individual NNs that provide better learning from the whole training samples. The major benefits of the proposed DEL compared to existing ensemble algorithms are (1) automatic design of ensemble; (2) maintaining accuracy and diversity of NNs at the same time; and (3) minimum number of parameters to be defined by user. DEL algorithm is applied to a set of real-world classification problems such as the cancer, diabetes, heart disease, thyroid, credit card, glass, gene, horse, letter recognition, mushroom, and soybean datasets. It has been confirmed by experimental results that DEL produces dynamic NN ensembles of appropriate architecture and diversity that demonstrate good generalization ability.

This article presents a review of selected articles about structural engineering applications of machine learning (ML) in the past few years. It is divided into the following areas: structural system identification, structural health monitoring, structural vibration control, structural design, and prediction applications. Deepneural networkalgorithms have beenthe subject of a large number of articles in civil and structural engineering.There are, however, otherML algorithms with great potential in civil and structural engineering that are worth exploring. Four novel supervised ML algorithms developed recently by the senior author and his associates with potential applications in civil/structural engineering are reviewed in this paper. They are the Enhanced Probabilistic Neural Network (EPNN), the Neural Dynamic Classification (NDC) algorithm, the Finite Element Machine (FEMa), and the Dynamic Ensemble Learning (DEL) algorithm

Improvement in Farm Yield Projection is a step toward addressing the increasing demand for food production globally. This depends on several factors, including environmental conditions (temperature, rainfall, humidity), management practices (irrigation, fertilizer use), genetic variations (crop variety), and regional aspects (soil type, climatic zones). Capturing the complex relationships between these features is necessary to create an accurate predictive model. This study suggests a dynamic predictive model that can account for non-linear interactions that affect yield. Attention neural networks are used to capture the nonlinear relationships between features. A dynamic ensemble classifier is developed utilizing these interactions and features using dynamic weights to choose the base learner. The model is further optimized for edge device deployment through lightweight architecture pruning. Additionally, a comprehensive ablation study validates the significance of attention mechanisms in improving predictive performance. Experimental results on a real-world dataset from Maharashtra, India show a 12.5 % improvement in accuracy over baseline models. The proposed framework is adaptable for multi-region, multi-crop applications, supporting precision agriculture and sustainable farming practices.

Many real-world datasets suffer from missing or incomplete data. In the healthcare setting, for example, certain patient measurement parameters, such as vitals and/or lab values, may be missing due to insufficient monitoring. When present, however, these features could be highly discriminative in predicting aspects of patient state. Therefore, it is desirable to incorporate these sparsely measured features into a predictive model. Training predictive algorithms on such datasets is complicated by the missing data. Overcoming this problem is usually achieved by first estimating values for the missing data, which is referred to as data imputation. Without strong prior knowledge about the relationship between features though, it is common to fill in missing values with their respective population mean or median. The accuracy of this approach is limited, however, and may simply inject noise into the data. We propose a two-stage machine learning algorithm that learns a dynamic classifier ensemble from an incomplete dataset without data imputation. The algorithm is very simple to implement and applicable across a wide range of problems. Our method first employs a variant of AdaBoost to learn a set of low-dimensional classifiers, each of which abstains from predicting if its dependent feature(s) are missing. Our novel contribution is the secondary dynamic ensemble learning stage in which the low-dimensional classifiers are combined using a dynamic weighting that depends on the pattern of measured features in the present input data. This allows the model to be resilient to missing data by adjusting the strength of certain classifiers to account for missing features. We apply our algorithm to early detection of hemodynamic instability in ICU patients. Providing an effective risk score of hemodynamic instability has the potential to give the clinician sufficient time to intervene, thereby reducing the chance of organ damage due to insufficient blood perfusion. We compare the results of our algorithm to other common missing data approaches, including mean imputation and multiple imputation methods, and discuss the advantages of the approach given the constraints of the application domain (e.g., high specificity to combat hospital alarm fatigue).

The rapid evolution of next-generation networks (NGNs), driven by 5G, IoT, edge computing, and software-defined networking, has introduced new opportunities alongside complex security challenges. Traditional intrusion detection and prevention systems (IDS), built on signature-based and anomaly-based methods, struggle to cope with the scale, heterogeneity, and dynamic threat landscape of NGNs. In response, artificial intelligence (AI) has emerged as a powerful enabler of modern IDPS. This review surveys AI-powered approaches, beginning with classical machine learning methods such as decision trees, support vector machines, and random forests, and then examining deep learning architectures including convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory networks (LSTMs), and autoencoders. It further analyses hybrid frameworks that integrate ensemble learning, federated learning, and meta-learning, as well as specialised methods tailored for SDN, IoT, edge, and cloud/5G environments. Benchmark datasets, including NSL-KDD, CICIDS2017, UNSW-NB15, Bot-IoT, IoT-23, and TON_IoT, are reviewed, highlighting their contributions and limitations. The paper identifies key challenges, including dataset scarcity, generalisation gaps, computational overhead, adversarial robustness, explainability, and privacy. Future directions emphasise the need for realistic NGN datasets, lightweight yet accurate architectures, privacy-preserving and federated frameworks, and integrated detection and prevention mechanisms. Overall, AI-powered IDPS demonstrate significant potential to secure NGNs, but realising this vision will require advances that balance accuracy, efficiency, interpretability, and resilience.

Vision Transformers (ViTs) have achieved remarkable performance in computer vision tasks but are vulnerable to adversarial attacks. Recent studies have demonstrated the feasibility of crafting transferable adversarial examples based on ViT models. However, the adversarial examples generated by ViTs exhibit poor generalization, primarily due to structural differences between models and the tendency to overfit, which significantly hinders cross-architecture transferability. In this paper, we propose a novel framework to improve the generalization and transferability of adversarial attacks across diverse models, focusing on two key strategies: Token Gradient Divergence (TGD) and Multi-level Frequency-aware Attack (MFA). TGD, as a gradient regularization method, addresses the structural gradient issue of surrogate models, which is one of the causes of overfitting. By increasing the gradient divergence between tokens and eliminating the influence of the class token gradient, TGD enhances the transferability of adversarial examples across models. Meanwhile, MFA employs an implicit ensemble approach to enhance attack generalization. Through multiple spectral augmentations, it increases input diversity and simulates ensemble learning. By targeting critical frequency regions across models, MFA enhances adversarial example adaptability to different architectures, significantly boosting cross-architecture transferability. Extensive experiments on both ViTs and CNNs demonstrate that TGD-MFA significantly outperforms state-of-the-art transfer-based attacks, achieving substantial improvements in adversarial transferability and robustness.

Active learning aims to reduce annotation cost by predicting which samples are useful for a human teacher to label. However it has become clear there is no best active learning algorithm. Inspired by various philosophies about what constitutes a good criteria, different algorithms perform well on different datasets. This has motivated research into ensembles of active learners that learn what constitutes a good criteria in a given scenario, typically via multi-armed bandit algorithms. Though algorithm ensembles can lead to better results, they overlook the fact that not only does algorithm efficacy vary across datasets, but also during a single active learning session. That is, the best criteria is non-stationary. This breaks existing algorithms' guarantees and hampers their performance in practice. In this paper, we propose dynamic ensemble active learning as a more general and promising research direction. We develop a dynamic ensemble active learner based on a non-stationary multi-armed bandit with expert advice algorithm. Our dynamic ensemble selects the right criteria at each step of active learning. It has theoretical guarantees, and shows encouraging results on 13 popular datasets.

The co-occurrence of multiple code smells in Android applications poses a more serious threat to software maintainability and stability than individual smells. However, most existing studies still concentrate on detecting single types of smells. This study aims to detect prevalent co-occurring Android-specific code smells, Message Chains (MC) and Member Ignoring Method (MIM). We propose an approach that integrates static program analysis with a dynamic stacking ensemble learning method. First, we design a heuristic-based static analysis algorithm for MC smell detection and extend it to identify the co-occurrence of MC and MIM. To support training, an automated sample generation technique, implemented in a tool named Android-specific Smell Detection (ASSD), produces labeled positive and negative samples. Finally, our approach employs a Dynamic Stacking Ensemble with Backward Elimination (DSE-BE), which combines five conventional machine learning models with three deep learning models. The experimental results demonstrate substantial improvements compared with manual detection methods, with the F1 score increasing from 0.774 to 0.938 and the MCC from 0.623 to 0.874. Moreover, the DSE-BE strategy not only outperforms individual models but also reduces ensemble complexity, improving computational efficiency. This method provides a robust solution for detecting co-occurring code smells in Android applications and holds strong potential for practical application.

Fine-grained bird species classification is a challenging task, mainly because the differences between species are often very subtle, and there is typically limited labeled data available for each species. This paper introduces a novel approach that combines multiple deep-learning models with a smart labeling strategy to address these issues. The method uses a dynamic ensemble of three advanced models: ResNet50, which excels in recognizing visual patterns; EfficientNet-B3, known for its efficiency and performance balance; and Swin Transformer, a newer model that captures both local and global features in images. Instead of relying on each model’s prediction separately, an adaptive mechanism (a multi-layer perceptron or MLP) weighs their predictions per sample, allowing the system to make smarter and more accurate decisions. Overall, the approach stands out due to its combination of a dynamic ensemble of models and an active learning strategy. This not only boosts performance but also makes the best use of limited labeled data, offering a more efficient way to tackle complex classification tasks like bird species identification. The results demonstrate that with fewer labels, the method can still achieve high accuracy and generalize well, providing a powerful tool for fine-grained classification in scenarios where data is scarce.

Occupational burnout among civil aviation security officers presents significant multidimensional risks to aviation safety. Traditional intervention strategies often fail to meet real-time demands. Grounded in dynamic resource-demand balancing theory, this study develops an integrated system linking data collection, modeling, prediction, and intelligent intervention. By fusing physiological signals, behavioral data, psychological assessments, and organizational environment data, we constructed an innovative early warning index system. Using a Stacking ensemble learning model that integrates XGBoost, Long Short-Term Memory (LSTM), and random forest algorithms, we achieved precise predictions. Furthermore, we designed a lightweight data collection terminal, a hierarchical early warning response mechanism, and a dynamic organizational resource allocation plan, forming a closed-loop management model of "monitoring, early warning, and intervention." This study offers a practical framework for burnout intervention in high-stress professions, enhancing the human resource resilience of security systems.

Accurately predicting treatment outcome is crucial for creating personalized treatment plans and follow-up schedules. Electronic health records (EHRs) contain valuable patient-specific information that can be leveraged to improve outcome prediction. We propose a reliable multi-objective ensemble deep learning (MoEDL) method that uses features extracted from EHRs to predict high risk of treatment failure after radiotherapy in patients with lung cancer. The dataset used in this study contains EHRs of 814 patients who had not achieved disease-free status and 193 patients who were disease-free with at least one year follow-up time after lung cancer radiation therapy. The proposed MoEDL consists of three phases: (1) training with dynamic ensemble deep learning; (2) model selection with adaptive multi-objective optimization; and (3) testing with evidential reasoning (ER) fusion. Specifically, in the training phase, we employ deep perceptron networks as base learners to handle various issues with EHR data. The architecture and key hyper-parameters of each base learner are dynamically adjusted to increase the diversity of learners while reducing the time spent tuning hyper-parameters. Furthermore, we integrate the snapshot ensembles (SE) restarting strategy, multi-objective optimization, and ER fusion to improve the prediction robustness and accuracy of individual networks. The SE restarting strategy can yield multiple candidate models at no additional training cost in the training stage. The multi-objective model simultaneously considers sensitivity, specificity, and AUC as objective functions, overcoming the limitations of single-objective–based model selection. For the testing stage, we utilized an analytic ER rule to fuse the output scores from each optimal model to obtain reliable and robust predictive results. Our experimental results demonstrate that MoEDL can perform better than other conventional methods.

Associative classifiers have shown competitive performance with state-of-the-art methods for predicting class labels. In addition to accuracy performance, associative classifiers produce human readable rules for classification which provides an easier way to understand the decision process of the model. Early models of associative classifiers suffered from the limitation of selecting proper threshold values which are dataset specific. Recent work on associative classifiers eliminates that restriction by searching for statistically significant rules. However, a high dimensional feature vector in the training data impacts the performance of the model. Ensemble models like Random Forest are also very powerful tools for classification but the decision process of Random Forest is not easily understandable like the associative classifiers. In this study we propose Dynamic Ensemble Associative Learning (DEAL) where we use associative classifiers as base learners on feature sub-spaces. In our approach we select a subset of the feature vector to train each of the base learners. Instead of a random selection, we propose a dynamic feature sampling procedure which automatically defines the number of base learners and ensures diversity and completeness among the subset of feature vectors. We use 10 datasets from the UCI repository and evaluate the performance of the model in terms of accuracy and memory requirement. Our ensemble approach using the proposed sampling method largely decreases the memory requirement in the case of datasets having a large number of features and this without jeopardising accuracy. In fact, accuracy is also improved in most cases. Moreover, the decision process of our DEAL approach remains human interpretable by collecting and ranking the rules generated by the base learners predicting the final class label.

Vehicle emissions have become a major source of air pollution in urban areas, especially for near-road environments, where the pollution characteristics are difficult to be captured by a single-scale air quality model due to the complex composition of the underlying surface. Here we developed a hybrid model CMAQ-RLINE_URBAN to quantitatively analyse the effects of vehicle emissions on urban roadside NO2 concentrations at a high spatial resolution of 50 m × 50 m. To estimate the influence of various street canyons on the dispersion of air pollutants, a Machine Learning-based Street Canyon Flow (MLSCF) scheme was constructed based on Computational Fluid Dynamic and ensemble learning methods. The results indicated that compared with the CMAQ model, the hybrid model improved the underestimation of NO2 concentration at near-road sites with MB changing from -10 μg/m3 to 6.3 μg/m3. The MLSCF scheme obviously increased concentrations at upwind receptors within deep street canyons due to changes in the wind environment caused by the vortex. In summer, the relative contribution of vehicles to NO2 concentrations in Beijing urban areas was 39 % on average, similar to results from CMAQ-ISAM model, but increased significantly with the decreased distance to the road centerline, especially reaching 75 % on urban freeways.

The increasing complexity of power systems as well as the rapid expansion of renewable energy resources and deregulation of electricity industry lead to continual demand for operating power systems at higher efficiencies. The modern power systems are pushed to operate close to their secure limits. When subjected to a severe contingency, the stressed power systems may lose its dynamic security, resulting in catastrophic consequences such as cascading failure and even wide-spread blackout. Under this new reality, online and real-time power system stability assessments have great impacts on avoiding blackout events. The traditional stability methods are mainly based on time-domain simulation, which are not fast enough to satisfy the online and real-time application requirements. In recent years, with the development of advanced real-time system monitoring and communication techniques, the data-driven methods have been identified as powerful tools to help power systems counteract large disturbance and avoid instability. To solve above threats, this Ph.D. research aims to develop data-analytics methodologies from power grid measurement data to recognize the system stability, covering both pre-fault dynamic security assessment (DSA) and post-fault real-time short-term voltage stability (STVS) assessment areas under some extreme scenarios and practical issues. Specific objectives of this Ph.D. research are to develop an intelligent system for the future smart grid. Based on several kinds of promising advanced data-analytics technologies, including randomized learning, incremental broad learning (BL), transfer learning (TL), adversarial machine learning (ML), and generative adversarial network (GAN), etc, an intelligent system will be developed to interpret the power grid stability characteristics, and to support stability-constrained operation control actions. Hence, it can make power systems achieve seven goals, including extensibility, efficiency, robustness, security, personalization, reliability, and effectiveness. The outcomes of this Ph.D. research will enable a more secure, intelligent, and knowledge-transparent future of the next-generation smart grid. The main contributions through this Ph.D. research are as follows: Firstly, the temporal-adaptive data-driven methods are proposed based on optimized hybrid ensemble learning mechanism for online STVS assessment. Compared to the traditional data-driven stability methods, the proposed methods can balance the tradeoff between the stability accuracy and speed and select the most suitable Pareto set. Secondly, in the area of DSA, an online updating DSA method is developed by using incremental BL technique. The incremental BL method only needs to calculate the pseudoinverse of new state matrix without the computations of the whole output weights matrix. In this way, it does not need re-training the whole model to continuously maintain and improve the online DSA accuracy. Besides, the proposed method can achieve under three different increment scenarios, including increment of enhancement hidden nodes, features, and new training instances. Thirdly, in the area of missing phasor measurement unit (PMU) data, two different fully robust data-driven stability methods are developed, respectively for pre-fault DSA and post-fault STVS assessment. The former one is based on GAN without observability-constrained feature subsets to ensure its availability under any PMU missing scenario. The latter one is based on super-resolution perception and incremental BL, which can achieve both satisfactory stability accuracy and speed. Fourthly, in order to accurately quantify the robustness of the ML-based model under adversarial perturbations on safety-critical power system stability problem, an adversarial robustness verification method for data-driven stability models is proposed. Besides, a model-free and attack-independent robust index are defined to evaluate the ability of the ML-based stability models against adversarial examples, which can be used to select the candidate model and provide formal robustness guarantee for stability application in practice. Rigorous mathematic proof is provided for the robust index under both linear and non-linear binary scenarios. Moreover, the proposed robust index can be used to mitigate the adversarial examples via adversarial training. Finally, three different TL-based DSA methods are proposed to use one known fault to evaluate other related but different unknown faults. The first two methods aim to reduce the distribution differences between the trained data and unknown data with the complete unlearned faults. The third one considers the different faults with incomplete data scenario. Through applying the adversarial training process, the DSA model trained by one fault can work for a different but related fault by confusing domain discriminator. With the property of feature extractor network, the proposed integrated TL-based method can solve the missing data issue for the unknown faults. All the methodologies have been tested on benchmark power systems to verify their effectiveness, and comparative and comparative studies with existing methods in the literature are conducted where applicable.