Adaptive ensemble learning strategy for semi-supervised soft sensing

Remote sensing estimation of rice chlorophyll content based on UAV image feature selection and PSO-optimized ensemble learning

Predictive models for early identification of heart disease must be precise and efficient because it is a major worldwide health concern. To improve classification performance while protecting data privacy, this study investigated a combined method that uses ensemble learning, feature selection, and federated learning (FL). The ensemble-based approaches proved the most predictive after testing several different machine learning (ML) models, including random forests, the light gradient boosting machine, support vector machines, k-nearest neighbors, convolutional neural networks, and long short-term memory. We used particle swarm optimization (PSO) for feature selection, which optimized the most relevant features in conjunction with voting and stacking approaches to further increase the model's performance. In addition, federated learning was implemented to allow decentralized training while preserving sensitive medical data. The results highlight the effectiveness of combining these techniques in the detection of heart disease, providing a scalable and privacy-preserving solution for real-world healthcare applications. Two benchmark datasets were used to validate the proposed approach, ensuring the reliability and generalizability of the findings. Furthermore, we used four performance metrics, namely accuracy, precision, recall, and F1score, to evaluate the selected models. Finally, federated learning was included to handle privacy issues and guarantee safe access to private medical data. This distributed method allows model training without centralizing patient data, so it is compatible with strict data privacy rules. With up to 95% precision, our method shows a notable increase in prediction accuracy according to the testing results. This work offers a strong, scalable, and safe solution for the early identification of cardiovascular diseases by combining ensemble learning, feature selection, and federated learning, opening the way for more general uses in medical diagnostics.

Diabetes is a complicated disease and a public health problem in the world. It has been shown that 50% of patients with diabetes in the world are not properly diagnosed. In this paper, a novel improved SVM based ensemble learning model is proposed for diagnosis of diabetes. Firstly, the given constraints reduction strategy are used to reduce the number of constraints in the classic SVM method. Secondly, the penalty method is used to transform the original problem into an unconstrained optimization problem. Then, the improved SVM results are obtained by gradient descent algorithm. Finally, an ensemble learning framework, which combines the improved SVM, logistic regression and CART is developed for diagnosis of diabetes. The results of experiment, which is carried out using the real-work datasets, show the proposed model outperforms other benchmark methods.

Numerous deep learning architectures have been designed for a variety of tasks in the past few years. However, it is almost impossible for one model to work well for all kinds of scenarios and datasets. Therefore, we present an ensemble deep learning framework in this paper, which not only decreases the information loss and over-fitting problems caused by single models, but also overcomes the imbalanced data issue in multimedia big data. First, a suite of deep learning algorithms are utilized for deep feature selection. Thereafter, an enhanced ensemble algorithm is developed based on the performance of each single Support Vector Machine classifier on each deep feature set. We evaluate our proposed ensemble deep learning framework on a large and highly imbalanced video dataset containing natural disaster events. Experimental results demonstrate the effectiveness of the proposed framework for semantic event detection, and show how it outperforms several state-of-the-art deep learning architectures, as well as handcrafted features integrated with ensemble and non-ensemble algorithms.

Herbs have historically been central to medicinal practices, representing one of the earliest forms of therapeutic intervention. While synthetic drugs are often highly effective in treating acute conditions, their use is frequently accompanied by adverse side effects. In addition, the growing dependence on synthetic pharmaceuticals has raised concerns regarding affordability, thereby fostering a renewed interest in herbal medicine as a cost-effective and holistic alternative. In response to this need, the current study introduces a computer vision framework for accurate herb identification. A novel dataset, Herbify, was compiled from two different herb datasets and refined through rigorous cleaning, preprocessing, and quality control procedures. The resulting dataset underwent standardization via the Preprocessing Algorithm for Herb Detection (PAHD), producing a refined dataset of 6104 images, representing 91 distinct herb species, with an average of about 67 images per species. Utilizing transfer learning, the research harnessed pre-trained Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), then integrated these models into an ensemble framework that leverages the unique strengths of each architecture. Experimental results indicate that EfficientNet v2-Large achieved a noteworthy F₁-score of 99.13%, while the ensemble of EfficientNet v2-Large and ViT-Large/16, termed EfficientL-ViTL, attained an even higher F₁-score of 99.56%. Additionally, the research also introduces ‘Herbify’ application, an AI-driven framework designed to identify herbs using the developed model. By directly tackling the principal obstacles in herb identification, the proposed system achieves a highly accurate and operationally viable classification mechanism. The experimental outcomes showcase top-tier performance in herb identification and emphasize the transformative potential of AI-based solutions in supporting botanical applications.Supplementary InformationThe online version contains supplementary material available at 10.1186/s13007-025-01421-5.

This research explores the performance criteria used to evaluate training effectiveness of simulation-based training and adaptation strategies for aviation pilots. Due to the pilot shortage, and with the emergence of new technologies, the future of pilot training must change. This research analyzes metrics from a multi-year research effort using pilot task performance and cognitive workload measures, through the electrocardiogram (ECG) signal, in simulation and live flight. It draws conclusions about performance criteria requirements for simulation-based training devices and adaptive learning systems. This dissertation contributes to the challenges in designing and configuring simulation-based training environments and the implementation of adaptive learning strategies to reduce cost and time to train. Research results show the effects of simulator fidelity across levels of pilot proficiency and the degree of simulator immersion. Evaluation of these objective performance criteria help to identify appropriate levels of simulator fidelity. Results also discuss adaptive learning strategies through adaptation schemes. More research is needed to determine if the assessment of task performance and cognitive workload measures as performance criteria provide an optimal environment to assess pedagogical strategies.

Background: Immune checkpoint inhibitors (ICIs) targeting PD-1 or PD-L1 have dramatically improved survival of patients with gastric cancer (GC). However, recent clinical trials, including KEYNOTE-059, KEYNOTE-061, and ATTRACTION-2, show that the objective response rate (ORR) for ICIs is still less than 20% in patients with advanced GC. Therefore, predicting who will respond to ICIs in advance is essential to make better treatment plans for GC patients. There are potential biomarkers to predict ICI response, but most of them can only be applied to a limited population of patients. For example, Kim ST., et al., (PMID: 30013197) show that patients with MSI-high or EBV-positive responded well to ICIs, but these subtypes account for less than 30 percent of GC patients (PMID:25079317). In this study, we propose a prediction model of ICI response using the 32 GC-specific genes that we identified from mutation data (PMID: 26635139) and showed their close association with survival and ICI response in GC (PMID: 35140202; PMID: 36508166). In addition, since the number of available GC-ICI samples is small, we design our prediction model in Bayesian and ensemble learning frameworks which are robust to small sample size problems. Methods: We collected mRNA sequencing data from GC-ICI treated patients from our institutions and publications (PMID: 30013197; PMID: 35254400; PMID: 33846173). We used 108 samples previously included in our previous work (PMID: 36508166) as the training data and 27 samples (13 responders and 14 non-responders) from (PMID: 3384617) as the test data. We built and trained a classifier to separate responders from non-responders using the expressions of the 32 genes. We proposed to use a Deep Gaussian process (GP) model, a multi-layered extension of a GP that is itself a Bayesian method. We optimized the parameters of each GP model in variational inference. To implement ensemble learning, we trained multiple DGP models with different initializations of the model parameters and data splitting (training/validation sets). For each training run, the best model was selected based on the prediction performance on the validation set, measured by AUC. The final prediction for a test sample was made by combining the prediction results of all the DGP models by taking the average. Results: We implemented the DGP-based prediction model using PyTorch in Python, and the codes will be publicly available on the GitHub repository (https://github.com/hwanglab/GC_ICI_response_prediction). We achieved a prediction performance of 0.813 AUC on the test data (sensitivity = 0.769 and specificity = 0.714). Conclusion: We propose the prediction model of ICI response, which can handle small sample size problems in the framework of Bayesian and ensemble learning. The experimental results using the GC-ICI data demonstrate that the 32 genes can be used as predictive markers for ICI response in GC. Citation Format: Sunho Park, Minji Kim, Changjin Hong, Tae Hyun Hwang. A deep Gaussian process (DGP)-based prediction model for immunecheckpoint inhibitor (ICI) response in gastric cancer (GC) [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 2316.

The nonlinearities and time‐varying characteristics are two major causes of low performance of soft sensors in process systems. Motivated of solving the two problems, this paper proposes an adaptive soft sensing method under the ensemble learning framework. An improved process state partition scheme is proposed to construct independent local models, which not only inherits the merits of the original process state partition method but also possesses the function of detecting and deleting redundant models. These prepared local models are weighted by a supervised weighting mechanism and then combined via the Bayesian inference to predict the y‐value of the query sample. Because the weighting mechanism can fully exploit the historical data set and quantify each local model's generalization ability for the query sample, it is potential to compute the combination weights more accurately. Simulations are conducted on two benchmark data sets from two real‐life chemical processes. Extensive performance evaluations of the proposed soft sensor are conducted, and results show its effectiveness. © 2015 Curtin University of Technology and John Wiley & Sons, Ltd.

Electronically controlled pneumatic (ECP) brake is widely used in heavy-haul train. Although the latest data-driven fault diagnosis can exploit the collection data from the braking system, it still has challenges for effective fault diagnosis model because of industrial data noise and insufficient fault samples. This article proposes a fault diagnosis model based on multidimensional feature fusion and ensemble learning for braking system of heavy-haul train (MFF-GBFD). First, the multidimensional features are extracted. By principal component analysis and feature fusion, the redundant features are eliminated. Then, the model is trained under ensemble learning framework with boosting strategy. Experiments are carried out on the data from the ECP braking system of DK-2 locomotive. The efforts show that the proposed MFF-GBFD model presents better performances as a result from the early-stage feature extraction, feature selection, and feature fusion. It also has higher accuracy and $F_1$ values compared with the traditional classification algorithms.

Surface roughness is primarily used to evaluate the surface finish of a workpiece, which significantly affects the quality performance of the finished components. Effective surface roughness prediction can reduce the time of trial-and-test, thus increasing productivity and reducing costs. Although earlier studies have proposed many different prediction approaches for different machining methods, few studies have been done to develop a generic approach for surface roughness prediction. Therefore, this paper presents a generic evolutionary ensemble learning (GEEL) framework for surface roughness prediction of different kinds of manufacturing process. The GEEL framework for surface roughness prediction consists of three modules, including pre-processing module, multi-algorithm regression module, and GA-based ensemble learning module. Validation experiments were conducted on fluid jet polishing (FJP) of 3D-printed Cobalt Chrome (CoCr) alloy and other seven cases of different machining methods. The results indicate that the mean square error (MSE) and mean absolute error (MAE), obtained by the proposed method were reduced to a certain extent than typical methods in predicting the surface roughness, exhibiting excellent prediction performance and robustness. At the same time, the method has good extensibility and is expected to become a generic framework for surface roughness prediction in manufacturing.

The Adaptive Boosting (AdaBoost) classifier is a widely used ensemble learning framework, and it can get good classification results on general datasets. However, it is challenging to apply the AdaBoost classifier directly to pulmonary nodule detection of labeled and unlabeled lung CT images since there are still some drawbacks to ensemble learning method. Therefore, to solve the labeled and unlabeled data classification problem, the semi-supervised AdaBoost classifier using an improved sparrow search algorithm (AdaBoost-ISSA-S4VM) was established. Firstly, AdaBoost classifier is used to construct a strong semi-supervised classifier using several weak classifiers S4VM (AdaBoost-S4VM). Next, in order to solve the accuracy problem of AdaBoost-S4VM, sparrow search algorithm (SSA) is introduced in the AdaBoost classifier and S4VM. Then, sine cosine algorithm and new labor cooperation structure are adopted to increase the global optimal solution and convergence performance of sparrow search algorithm, respectively. Furthermore, based on the improved sparrow search algorithm and adaptive boosting classifier, the AdaBoost-S4VM classifier is improved. Finally, the effective improved AdaBoost-ISSA-S4VM classification model was developed for actual pulmonary nodule detection based on the publicly available LIDC-IDRI database. The experimental results have proved that the established AdaBoost-ISSA-S4VM classification model has good performance on labeled and unlabeled lung CT images.

Recently, deep clustering methods have achieved remarkable results compared to traditional clustering approaches. However, its performance remains constrained by the absence of annotations. A thought-provoking observation is that there is still a significant gap between deep clustering and semi-supervised classification methods. Even with only a few labeled samples, the accuracy of semi-supervised learning is much higher than that of clustering. Given that we can annotate a small number of samples in a certain unsupervised way, the clustering task can be naturally transformed into a semi-supervised setting, thereby achieving comparable performance. Based on this intuition, we propose ClusMatch, a unified positive and negative pseudo-label learning based semi-supervised learning framework, which is pluggable and can be applied to existing deep clustering methods. Specifically, we first leverage the pre-trained deep clustering network to compute predictions for all samples, and then design specialized selection strategies to pick out a few high-quality samples as labeled samples for supervised learning. For the unselected samples, the novel unified positive and negative pseudo-label learning is introduced to provide additional supervised signals for semi-supervised fine-tuning. We also propose an adaptive positive-negative threshold learning strategy to further enhance the confidence of generated pseudo-labels. Extensive experiments on six widely-used datasets and one large-scale dataset demonstrate the superiority of our proposed ClusMatch. For example, ClusMatch achieves a significant accuracy improvement of 5.4% over the state-of-the-art method ProPos on an average of these six datasets.

Hybrid grey wolf and improved particle swarm optimization with adaptive intertial weight-based multi-dimensional learning strategy for load balancing in cloud environments

Early prediction of sepsis can help to identify potential risks in time and help take necessary measures to prevent more dangerous situations from occurring. In PhysioNet/Computing in Cardiology Challenge 2019, we integrate Long Short Term Memory (LSTM) recurrent neural network and ensemble learning to achieve early sepsis prediction. Specifically, we tackle the problem of class imbalance and data missing firstly, and then we manually extract features according to the prior knowledge from the medical field. In addition, we regard the prediction of sepsis as a time series prediction problem and pre-train LSTM-based models as feature extractors to obtain the deep features on time series that might be related to the onset of sepsis. Manual features and deep features are then used to train prediction models under the framework of ensemble learning, including Extreme Gradient Boosting (XGBoost) and Gradient Boosting Decision Tree (GBDT) regressor. The final normalized utility score our team (UCAS_DataMiner) have obtained was 0.313 on full hidden test set.

Semi-supervised matrixized least squares support vector machine