Soft sensor development for nonlinear and time‐varying processes based on supervised ensemble learning with improved process state partition

Traditional single model based soft sensors may have poor performance on quality prediction for batch processes because of the strong nonlinearity, multiple-phase, and time-varying characteristics. Therefore, a phase partition based ensemble learning framework upon least squares support vector regression (LSSVR) is proposed for soft sensor modeling. Firstly, multiway principal component analysis (MPCA) is employed to handle high-dimensional datasets and extract essential correlation information. Then, different operation phases of the process can be identified by the phase partition strategy based on Gaussian mixture model (GMM) method. Meanwhile, the optimal Gaussian component number is determined by Bayesian information criterion (BIC) technique. Further, multiple localized LSSVR models are constructed to characterize the various dynamic relationships between quality and process variables for local regions, while the grid search (GS) and ten-fold cross-validation methods are introduced to parameter optimization for each local model. Finally, the posterior probability for each test sample with respect to different phases can be estimated by Bayesian inference strategy, and local outputs are integrated to produce the final quality prediction results. Feasibility and superiority of the proposed soft sensor are validated through a case study for penicillin fermentation process. It can achieve satisfactory prediction accuracy and effectively tackle nonlinear and multi-phase modeling problems in chemical and biological processes.

Predictive models for early identification of heart disease must be precise and efficient because it is a major worldwide health concern. To improve classification performance while protecting data privacy, this study investigated a combined method that uses ensemble learning, feature selection, and federated learning (FL). The ensemble-based approaches proved the most predictive after testing several different machine learning (ML) models, including random forests, the light gradient boosting machine, support vector machines, k-nearest neighbors, convolutional neural networks, and long short-term memory. We used particle swarm optimization (PSO) for feature selection, which optimized the most relevant features in conjunction with voting and stacking approaches to further increase the model's performance. In addition, federated learning was implemented to allow decentralized training while preserving sensitive medical data. The results highlight the effectiveness of combining these techniques in the detection of heart disease, providing a scalable and privacy-preserving solution for real-world healthcare applications. Two benchmark datasets were used to validate the proposed approach, ensuring the reliability and generalizability of the findings. Furthermore, we used four performance metrics, namely accuracy, precision, recall, and F1score, to evaluate the selected models. Finally, federated learning was included to handle privacy issues and guarantee safe access to private medical data. This distributed method allows model training without centralizing patient data, so it is compatible with strict data privacy rules. With up to 95% precision, our method shows a notable increase in prediction accuracy according to the testing results. This work offers a strong, scalable, and safe solution for the early identification of cardiovascular diseases by combining ensemble learning, feature selection, and federated learning, opening the way for more general uses in medical diagnostics.

Remote sensing estimation of rice chlorophyll content based on UAV image feature selection and PSO-optimized ensemble learning

Adaptive ensemble learning strategy for semi-supervised soft sensing

Just-in-time learning (JITL) has recently been used for online soft sensor modeling. Unlike traditional global approaches, the JITL-based method employs a local model built from historical samples similar to a query sample so that both nonlinearities and changes in process characteristics can be handled well. A key issue in JITL is to establish a suitable similarity criterion for selecting relevant samples. Conventional JITL methods, which use distance-based similarity measures for local modeling, can be inappropriate for many industrial processes exhibiting time-varying and non-Gaussian behaviors. In this article, a GMM-based similarity measure is proposed to improve the prediction accuracy of a JITL soft sensor. By taking the non-Gaussianity of the process data and the characteristics of the query sample into account, a more suitable similarity criterion is defined for sample selection of a JITL soft sensor, and better modeling performance can be achieved. Case studies involving a numerical example and an industrial process are discussed to demonstrate the feasibility and effectiveness of the proposed method.

Numerous deep learning architectures have been designed for a variety of tasks in the past few years. However, it is almost impossible for one model to work well for all kinds of scenarios and datasets. Therefore, we present an ensemble deep learning framework in this paper, which not only decreases the information loss and over-fitting problems caused by single models, but also overcomes the imbalanced data issue in multimedia big data. First, a suite of deep learning algorithms are utilized for deep feature selection. Thereafter, an enhanced ensemble algorithm is developed based on the performance of each single Support Vector Machine classifier on each deep feature set. We evaluate our proposed ensemble deep learning framework on a large and highly imbalanced video dataset containing natural disaster events. Experimental results demonstrate the effectiveness of the proposed framework for semantic event detection, and show how it outperforms several state-of-the-art deep learning architectures, as well as handcrafted features integrated with ensemble and non-ensemble algorithms.

Diabetes is a complicated disease and a public health problem in the world. It has been shown that 50% of patients with diabetes in the world are not properly diagnosed. In this paper, a novel improved SVM based ensemble learning model is proposed for diagnosis of diabetes. Firstly, the given constraints reduction strategy are used to reduce the number of constraints in the classic SVM method. Secondly, the penalty method is used to transform the original problem into an unconstrained optimization problem. Then, the improved SVM results are obtained by gradient descent algorithm. Finally, an ensemble learning framework, which combines the improved SVM, logistic regression and CART is developed for diagnosis of diabetes. The results of experiment, which is carried out using the real-work datasets, show the proposed model outperforms other benchmark methods.

Herbs have historically been central to medicinal practices, representing one of the earliest forms of therapeutic intervention. While synthetic drugs are often highly effective in treating acute conditions, their use is frequently accompanied by adverse side effects. In addition, the growing dependence on synthetic pharmaceuticals has raised concerns regarding affordability, thereby fostering a renewed interest in herbal medicine as a cost-effective and holistic alternative. In response to this need, the current study introduces a computer vision framework for accurate herb identification. A novel dataset, Herbify, was compiled from two different herb datasets and refined through rigorous cleaning, preprocessing, and quality control procedures. The resulting dataset underwent standardization via the Preprocessing Algorithm for Herb Detection (PAHD), producing a refined dataset of 6104 images, representing 91 distinct herb species, with an average of about 67 images per species. Utilizing transfer learning, the research harnessed pre-trained Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), then integrated these models into an ensemble framework that leverages the unique strengths of each architecture. Experimental results indicate that EfficientNet v2-Large achieved a noteworthy F₁-score of 99.13%, while the ensemble of EfficientNet v2-Large and ViT-Large/16, termed EfficientL-ViTL, attained an even higher F₁-score of 99.56%. Additionally, the research also introduces ‘Herbify’ application, an AI-driven framework designed to identify herbs using the developed model. By directly tackling the principal obstacles in herb identification, the proposed system achieves a highly accurate and operationally viable classification mechanism. The experimental outcomes showcase top-tier performance in herb identification and emphasize the transformative potential of AI-based solutions in supporting botanical applications.Supplementary InformationThe online version contains supplementary material available at 10.1186/s13007-025-01421-5.

Integrated soft sensor using just-in-time support vector regression and probabilistic analysis for quality prediction of multi-grade processes

Data heterogeneity across clients in federated learning (FL) settings is a widely acknowledged challenge. In response, personalized federated learning (PFL) emerged as a framework to curate local models for clients' tasks. In PFL, a common strategy is to develop local and global models jointly - the global model (for generalization) informs the local models, and the local models (for personalization) are aggregated to update the global model. A key observation is that if we can improve the generalization ability of local models, then we can improve the generalization of global models, which in turn builds better personalized models. In this work, we consider class imbalance, an overlooked type of data heterogeneity, in the classification setting. We propose FedNH, a novel method that improves the local models' performance for both personalization and generalization by combining the uniformity and semantics of class prototypes. FedNH initially distributes class prototypes uniformly in the latent space and smoothly infuses the class semantics into class prototypes. We show that imposing uniformity helps to combat prototype collapse while infusing class semantics improves local models. Extensive experiments were conducted on popular classification datasets under the cross-device setting. Our results demonstrate the effectiveness and stability of our method over recent works.

Electronically controlled pneumatic (ECP) brake is widely used in heavy-haul train. Although the latest data-driven fault diagnosis can exploit the collection data from the braking system, it still has challenges for effective fault diagnosis model because of industrial data noise and insufficient fault samples. This article proposes a fault diagnosis model based on multidimensional feature fusion and ensemble learning for braking system of heavy-haul train (MFF-GBFD). First, the multidimensional features are extracted. By principal component analysis and feature fusion, the redundant features are eliminated. Then, the model is trained under ensemble learning framework with boosting strategy. Experiments are carried out on the data from the ECP braking system of DK-2 locomotive. The efforts show that the proposed MFF-GBFD model presents better performances as a result from the early-stage feature extraction, feature selection, and feature fusion. It also has higher accuracy and $F_1$ values compared with the traditional classification algorithms.

Surface roughness is primarily used to evaluate the surface finish of a workpiece, which significantly affects the quality performance of the finished components. Effective surface roughness prediction can reduce the time of trial-and-test, thus increasing productivity and reducing costs. Although earlier studies have proposed many different prediction approaches for different machining methods, few studies have been done to develop a generic approach for surface roughness prediction. Therefore, this paper presents a generic evolutionary ensemble learning (GEEL) framework for surface roughness prediction of different kinds of manufacturing process. The GEEL framework for surface roughness prediction consists of three modules, including pre-processing module, multi-algorithm regression module, and GA-based ensemble learning module. Validation experiments were conducted on fluid jet polishing (FJP) of 3D-printed Cobalt Chrome (CoCr) alloy and other seven cases of different machining methods. The results indicate that the mean square error (MSE) and mean absolute error (MAE), obtained by the proposed method were reduced to a certain extent than typical methods in predicting the surface roughness, exhibiting excellent prediction performance and robustness. At the same time, the method has good extensibility and is expected to become a generic framework for surface roughness prediction in manufacturing.

The Adaptive Boosting (AdaBoost) classifier is a widely used ensemble learning framework, and it can get good classification results on general datasets. However, it is challenging to apply the AdaBoost classifier directly to pulmonary nodule detection of labeled and unlabeled lung CT images since there are still some drawbacks to ensemble learning method. Therefore, to solve the labeled and unlabeled data classification problem, the semi-supervised AdaBoost classifier using an improved sparrow search algorithm (AdaBoost-ISSA-S4VM) was established. Firstly, AdaBoost classifier is used to construct a strong semi-supervised classifier using several weak classifiers S4VM (AdaBoost-S4VM). Next, in order to solve the accuracy problem of AdaBoost-S4VM, sparrow search algorithm (SSA) is introduced in the AdaBoost classifier and S4VM. Then, sine cosine algorithm and new labor cooperation structure are adopted to increase the global optimal solution and convergence performance of sparrow search algorithm, respectively. Furthermore, based on the improved sparrow search algorithm and adaptive boosting classifier, the AdaBoost-S4VM classifier is improved. Finally, the effective improved AdaBoost-ISSA-S4VM classification model was developed for actual pulmonary nodule detection based on the publicly available LIDC-IDRI database. The experimental results have proved that the established AdaBoost-ISSA-S4VM classification model has good performance on labeled and unlabeled lung CT images.

Background: Immune checkpoint inhibitors (ICIs) targeting PD-1 or PD-L1 have dramatically improved survival of patients with gastric cancer (GC). However, recent clinical trials, including KEYNOTE-059, KEYNOTE-061, and ATTRACTION-2, show that the objective response rate (ORR) for ICIs is still less than 20% in patients with advanced GC. Therefore, predicting who will respond to ICIs in advance is essential to make better treatment plans for GC patients. There are potential biomarkers to predict ICI response, but most of them can only be applied to a limited population of patients. For example, Kim ST., et al., (PMID: 30013197) show that patients with MSI-high or EBV-positive responded well to ICIs, but these subtypes account for less than 30 percent of GC patients (PMID:25079317). In this study, we propose a prediction model of ICI response using the 32 GC-specific genes that we identified from mutation data (PMID: 26635139) and showed their close association with survival and ICI response in GC (PMID: 35140202; PMID: 36508166). In addition, since the number of available GC-ICI samples is small, we design our prediction model in Bayesian and ensemble learning frameworks which are robust to small sample size problems. Methods: We collected mRNA sequencing data from GC-ICI treated patients from our institutions and publications (PMID: 30013197; PMID: 35254400; PMID: 33846173). We used 108 samples previously included in our previous work (PMID: 36508166) as the training data and 27 samples (13 responders and 14 non-responders) from (PMID: 3384617) as the test data. We built and trained a classifier to separate responders from non-responders using the expressions of the 32 genes. We proposed to use a Deep Gaussian process (GP) model, a multi-layered extension of a GP that is itself a Bayesian method. We optimized the parameters of each GP model in variational inference. To implement ensemble learning, we trained multiple DGP models with different initializations of the model parameters and data splitting (training/validation sets). For each training run, the best model was selected based on the prediction performance on the validation set, measured by AUC. The final prediction for a test sample was made by combining the prediction results of all the DGP models by taking the average. Results: We implemented the DGP-based prediction model using PyTorch in Python, and the codes will be publicly available on the GitHub repository (https://github.com/hwanglab/GC_ICI_response_prediction). We achieved a prediction performance of 0.813 AUC on the test data (sensitivity = 0.769 and specificity = 0.714). Conclusion: We propose the prediction model of ICI response, which can handle small sample size problems in the framework of Bayesian and ensemble learning. The experimental results using the GC-ICI data demonstrate that the 32 genes can be used as predictive markers for ICI response in GC. Citation Format: Sunho Park, Minji Kim, Changjin Hong, Tae Hyun Hwang. A deep Gaussian process (DGP)-based prediction model for immunecheckpoint inhibitor (ICI) response in gastric cancer (GC) [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 2316.

Soft sensors have gained wide popularity in the industrial processes for online quality prediction in the recent years. In the case of online deployment, it is important to incorporate fewer input variables to improve the performance of the soft sensor. Therefore, the goal of this paper is to present an approach for the development of more efficient and less complex soft sensors in order to maximize the accuracy as well as to minimize the number of input soft sensing variables. The approach is based on multi-state-dependent parameter (MSDP) models, in which model parameters are estimated in a multivariable state space employing the Kalman filter and fixed interval smoothing algorithms. The proposed MSDP-based soft sensor is applied to an industrial sulfur recovery unit (SRU) in order to predict of H2S and SO2 concentrations. The model is consequently compared with the other soft sensing techniques, which are based on the same benchmark data set of the case study. The prediction results show that the designed MSDP-based soft sensors are more robust and exhibit higher predictive performance than other presented soft sensing methods based on the root mean square errors and Pearson correlation coefficient criterions while using fewer input variables.