Integrated soft sensor using just-in-time support vector regression and probabilistic analysis for quality prediction of multi-grade processes

Soft sensors are becoming increasingly important in our world today as tools for inferring difficult-to-measure process variables to achieve good operational performance and economic benefits. Recent advancement in machine learning provides an opportunity to integrate machine learning models for soft sensing applications, such as Least Square Support Vector Regression (LSSVR) which copes well with nonlinear process data. However, the LSSVR model usually uses the radial basis function (RBF) kernel function for prediction, which has demonstrated its usefulness in numerous applications. Thus, this study extends the use of non-conventional kernel functions in the LSSVR model with a comparative study against widely used partial least square (PLS) and principal component regression (PCR) models, measured with root mean square error (RMSE), mean absolute error (MAE) and error of approximation (Ea) as the performance benchmark. Based on the empirical result from the case study of the penicillin fermentation process, the Ea of the multiquadric kernel (MQ) is lowered by 63.44% as compared to the RBF kernel for the prediction of penicillin concentration. Hence, the MQ kernel LSSVR has outperformed the RBF kernel LSSVR. The study serves as empirical evidence of LSSVR performance as a machine learning model in soft sensing applications and as reference material for further development of non-conventional kernels in LSSVR-based models because many other functions can be used as well in the hope to increase the prediction accuracy.

Combined Terrestrial Evapotranspiration Index prediction using a hybrid artificial intelligence paradigm integrated with relief algorithm-based feature selection

Traditional single model based soft sensors may have poor performance on quality prediction for batch processes because of the strong nonlinearity, multiple-phase, and time-varying characteristics. Therefore, a phase partition based ensemble learning framework upon least squares support vector regression (LSSVR) is proposed for soft sensor modeling. Firstly, multiway principal component analysis (MPCA) is employed to handle high-dimensional datasets and extract essential correlation information. Then, different operation phases of the process can be identified by the phase partition strategy based on Gaussian mixture model (GMM) method. Meanwhile, the optimal Gaussian component number is determined by Bayesian information criterion (BIC) technique. Further, multiple localized LSSVR models are constructed to characterize the various dynamic relationships between quality and process variables for local regions, while the grid search (GS) and ten-fold cross-validation methods are introduced to parameter optimization for each local model. Finally, the posterior probability for each test sample with respect to different phases can be estimated by Bayesian inference strategy, and local outputs are integrated to produce the final quality prediction results. Feasibility and superiority of the proposed soft sensor are validated through a case study for penicillin fermentation process. It can achieve satisfactory prediction accuracy and effectively tackle nonlinear and multi-phase modeling problems in chemical and biological processes.

To improve the prediction accuracy of crude oil price even in current complicated international situation, this paper proposed a novel model linking firefly algorithm (FA) with least squares support vector regression (LSSVR), namely FA-LSSVR. In this hybrid intelligent model, FA is used to find the optimal values of LSSVR parameters (i.e., penalty coefficient and kernel function parameters), in order to achieve fast and accurate prediction results. To evaluate the forecasting ability of FA-LSSVR, its performance is compared with other models, including hybrid intelligent methods (LSSVR models with other popular optimization methods), and single models with given predetermined parameters (i.e., support sector regression (SVR), LSSVR, back-propagation neural network (BPNN), autoregressive integrated moving average (ARIMA)). The empirical results reveal that FA-LSSVR outperforms other benchmarks in terms of prediction accuracy, time saving and robustness, suggesting that the proposed approach is a promising alternative to forecast the crude oil price.

Prediction of dissolved oxygen content in river crab culture based on least squares support vector regression optimized by improved particle swarm optimization

Data‐driven soft sensor models have been extensively utilized in industrial processes. Batch processes are usually employed to manufacture low‐volume and high value‐added products in chemical, materials, and pharmaceutical industries. The most distinctive features of batch process lie in nonlinear, repetition, and slow time varying characteristics. In this paper, a data‐driven soft sensor modelling method based on linear slow feature analysis (LSFA) and least squares support vector regression (LSSVR) is proposed. In this method, LSFA was used to effectively capture the driving force behind the data features that change as slowly as possible. Then, a LSSVR model was constructed between the extracted slow feature variables and quality variables. Finally, a numerical example, industrial penicillin fermentation processes, and cobalt oxalate synthesis process were utilized to confirm the prediction accuracy and model reliability of the proposed approach.

Least Squares Support Vector Regression (LS-SVR) is a powerful kernel-based learning tool for regression problems. However, since it is based on the ordinary least squares (OLS) approach for parameter estimation, the standard LS-SVR model is very sensitive to outliers. Robust variants of the LS-SVR model, such as the WLS-SVR and IRLS-SVR models, have been developed aiming at adding robustness to the parameter estimation process, but they still rely on OLS solutions. In this paper we propose a totally different approach to robustify the LS-SVR. Unlike previous models, we maintain the original LS-SVR loss function, while the solution of the resulting linear system for parameter estimation is obtained by means of the Recursive Least M-estimate (RLM) algorithm. We evaluate the proposed approach in nonlinear system identification tasks, using artificial and real-world datasets contaminated with outliers. The obtained results for infinite-steps-ahead prediction shows that proposed model consistently outperforms the WLS-SVR and IRLS-SVR models for all studied scenarios.

Hybrid application of unsupervised and supervised learning in forecasting absolute open flow potential for shale gas reservoirs

Comparative assessment on smart pre-processing methods for extracting information in FT-NIR measured data

A hybrid slantlet denoising least squares support vector regression model for exchange rate prediction

Spectral analysis is a practical technology used for rapid analysis of fruit maturity. Least-squares support vector regression (LSSVR) with radial basis function (RBF) kernel is an effective nonlinear method for the quantitative calibration in the visible-near-infrared (Vis-NIR) spectral region. However, the nonlinear effect and high-dimensional spectrum data influence the prediction accuracy and complexity of modeling procedures. This article presents a fuzzy optimization strategy to improve the performance of the RBF LSSVR model in Vis-NIR quantitative determination of pomelo maturity. In the proposed strategy, a linguistic iterative mode is introduced for optimizing RBF kernel parameters. The input variables of the model are the informative features extracted through fuzzy transform principal component analysis algorithm, and the output was a polynomial equation of the inputs. In this article, pomelo maturity is recorded using the quantitative indices of L*, a*, and b*. The Vis-NIR spectral data of Pomelo samples are first converted to a set of fuzzy-transformed principal components and inputted into a fuzzy-optimized LSSVR model for calibration, validation, and test. Parameter uncertainty is evaluated to verify the effectiveness of the proposed strategy. Experimental results show that the proposed fuzzy optimization strategy is feasible for reducing the computational complexity of the BRF LSSVR model, and the predictions on L*, a*, and b* are much appreciable. The proposed fuzzy optimization strategy based on the linguistic iteration mode is considered as the effective implementation for calibration models in the spectroscopy technology for rapid prediction of fruit maturity.

Generalization-Memorization learning endeavors to minimize empirical risk while simultaneously reducing expected risk. Although we often do not pay attention to whether the training samples are accurately memorized during regression, improving generalization performance with better memory is always a goal pursued by regression. To tackle this issue, we introduce two new regression models, Least Squares Generalization-Memorization Regression (LSGMR) and Soft Least Squares Generalization-Memorization Regression (SLSGMR), by introducing the memory kernel learning on Least Squares Support Vector Regression (LSSVR). We conduct tests on these models using synthetic dataset and showcase that the LSSVR model can be viewed as a special case of our proposed model. Our experiments highlight that, for numerous problems, the models incorporating the employed memory mechanisms, LSGMR and SLSGMR, prove highly effective in yielding superior results compared to LSSVR on noise regression.

A robust localized soft sensor for particulate matter modeling in Seoul metro systems

Online quality prediction for cobalt oxalate synthesis process using least squares support vector regression approach with dual updating

Robust Lp-norm least squares support vector regression with feature selection