Dynamic nonlinear soft sensor modelling method using linear slow feature analysis and least squares support vector regression for batch processes

Integrated soft sensor using just-in-time support vector regression and probabilistic analysis for quality prediction of multi-grade processes

Online quality prediction for cobalt oxalate synthesis process using least squares support vector regression approach with dual updating

Soft sensors are becoming increasingly important in our world today as tools for inferring difficult-to-measure process variables to achieve good operational performance and economic benefits. Recent advancement in machine learning provides an opportunity to integrate machine learning models for soft sensing applications, such as Least Square Support Vector Regression (LSSVR) which copes well with nonlinear process data. However, the LSSVR model usually uses the radial basis function (RBF) kernel function for prediction, which has demonstrated its usefulness in numerous applications. Thus, this study extends the use of non-conventional kernel functions in the LSSVR model with a comparative study against widely used partial least square (PLS) and principal component regression (PCR) models, measured with root mean square error (RMSE), mean absolute error (MAE) and error of approximation (Ea) as the performance benchmark. Based on the empirical result from the case study of the penicillin fermentation process, the Ea of the multiquadric kernel (MQ) is lowered by 63.44% as compared to the RBF kernel for the prediction of penicillin concentration. Hence, the MQ kernel LSSVR has outperformed the RBF kernel LSSVR. The study serves as empirical evidence of LSSVR performance as a machine learning model in soft sensing applications and as reference material for further development of non-conventional kernels in LSSVR-based models because many other functions can be used as well in the hope to increase the prediction accuracy.

Combined Terrestrial Evapotranspiration Index prediction using a hybrid artificial intelligence paradigm integrated with relief algorithm-based feature selection

Traditional single model based soft sensors may have poor performance on quality prediction for batch processes because of the strong nonlinearity, multiple-phase, and time-varying characteristics. Therefore, a phase partition based ensemble learning framework upon least squares support vector regression (LSSVR) is proposed for soft sensor modeling. Firstly, multiway principal component analysis (MPCA) is employed to handle high-dimensional datasets and extract essential correlation information. Then, different operation phases of the process can be identified by the phase partition strategy based on Gaussian mixture model (GMM) method. Meanwhile, the optimal Gaussian component number is determined by Bayesian information criterion (BIC) technique. Further, multiple localized LSSVR models are constructed to characterize the various dynamic relationships between quality and process variables for local regions, while the grid search (GS) and ten-fold cross-validation methods are introduced to parameter optimization for each local model. Finally, the posterior probability for each test sample with respect to different phases can be estimated by Bayesian inference strategy, and local outputs are integrated to produce the final quality prediction results. Feasibility and superiority of the proposed soft sensor are validated through a case study for penicillin fermentation process. It can achieve satisfactory prediction accuracy and effectively tackle nonlinear and multi-phase modeling problems in chemical and biological processes.

To improve the prediction accuracy of crude oil price even in current complicated international situation, this paper proposed a novel model linking firefly algorithm (FA) with least squares support vector regression (LSSVR), namely FA-LSSVR. In this hybrid intelligent model, FA is used to find the optimal values of LSSVR parameters (i.e., penalty coefficient and kernel function parameters), in order to achieve fast and accurate prediction results. To evaluate the forecasting ability of FA-LSSVR, its performance is compared with other models, including hybrid intelligent methods (LSSVR models with other popular optimization methods), and single models with given predetermined parameters (i.e., support sector regression (SVR), LSSVR, back-propagation neural network (BPNN), autoregressive integrated moving average (ARIMA)). The empirical results reveal that FA-LSSVR outperforms other benchmarks in terms of prediction accuracy, time saving and robustness, suggesting that the proposed approach is a promising alternative to forecast the crude oil price.

Prediction of dissolved oxygen content in river crab culture based on least squares support vector regression optimized by improved particle swarm optimization

Least Squares Support Vector Regression (LS-SVR) is a powerful kernel-based learning tool for regression problems. However, since it is based on the ordinary least squares (OLS) approach for parameter estimation, the standard LS-SVR model is very sensitive to outliers. Robust variants of the LS-SVR model, such as the WLS-SVR and IRLS-SVR models, have been developed aiming at adding robustness to the parameter estimation process, but they still rely on OLS solutions. In this paper we propose a totally different approach to robustify the LS-SVR. Unlike previous models, we maintain the original LS-SVR loss function, while the solution of the resulting linear system for parameter estimation is obtained by means of the Recursive Least M-estimate (RLM) algorithm. We evaluate the proposed approach in nonlinear system identification tasks, using artificial and real-world datasets contaminated with outliers. The obtained results for infinite-steps-ahead prediction shows that proposed model consistently outperforms the WLS-SVR and IRLS-SVR models for all studied scenarios.

A hybrid slantlet denoising least squares support vector regression model for exchange rate prediction

Robust Lp-norm least squares support vector regression with feature selection

The least square support vector regression (LSSVR) with the Gaussian kernel has two hyperparameters, sigma, $\sigma$, a bandwidth showing the spread of the kernel, and C, a regularization gauging the level of errors. In this paper, we want to systematically study how the hyperparameters are related to the LSSVR learning, thanks to the observation that the ergodic and stationary mean field game (MFG) implied hyperparameters are estimated based on the closed form solution given arbitrarily fixed bounded hyperparameters. Inversely, this closed form solution enables us to estimate MFG implied hyperparameters. The LSSVR with particle swarm optimization (LSSVR + PSO) allows a new way to estimate hyperparameters that doesn't involve cross-validations. The exact solvable MFG equivalent to our model has a stationary and ergodic solution of nonlinear Schr\"{o}dinger equation (NLS). The equilibrium solution of MFG is a pair of optimal action and the interaction density and this pair in NLS is the ergodic and stationary chemical potential and the interacting residual particle density, or $|\psi|^2$ in NLS. The ergodic solution of NLS ergodicises and stationarizes the LSSVR residual particles via the local interaction measure and we call this \emph{'the ergodic learning.'} As long as the non-zero volatility sigma and the non-zero mass ($\mu=1/C$) are bounded and the input and output data are required to have a non-degenerate support vector kernel, the input and output processes themselves don't need to be stationary, while all local interaction measures of the residual particles are ergodic. Depending on the (ergodic) learning classifications into condensed learning, intermediate learning and classical learning, based on theoretical temperatures, many candidate models for prediction are ergodically generated and the region of ergodicized and stationalized residuals via local interaction measure can be found for each hyperparameter pairs. As the C increases (or as we decrease the mass) or as the sigma increases, the volatility of learning increases. The learning tends to be more classical and less condensed. For each ergodic path generated as the ergodic solution to the MFG system, we profile the system response with respect to hyperparameters. Also, even in ergodic learning setting, phase transition behavior has been observed when PSO correctly keep the stationary energy conserved ergodically and stationarily. Lastly, it is shown that the zero crossing number of LSSVR $+$ PSO $=$ MFG is given by $E(N_0(T))=\frac{T}{2 \pi},$ given the data $\{(x_i, y_i)\}_{i=1}^T,$ x input, y output with $T>\tau^*=\sqrt{4 \pi} \Sigma^* \mu \sigma^2.$

Although electronic system has entered the digital age, analog circuit is still an essential part. Therefore the performance monitoring or evaluation of analog circuit is extremely important. However some problems about analog circuit performance monitoring is being, such as data acquisition online of the industry field with uncertainty, performance monitoring timeliness. Here an online performance monitoring method for analog circuit (OPM) subject to the data uncertainty is proposed. The main idea of OPM is to employ a learning machine same as least square support vector regression (LSSVR) to train and learn the data set. Considering the data from industrial field usually hold nonlinear feature, time varying feature and contain faults value, a novel robust LSSVR (RLSSVR) is presented to detection the performance of Electronic System. Moreover, the multi-kernel function is employed which has more flexibility than the single-kernel function, and can get more support vector numbers accuracy. At the same time, the novel robust learning algorithm is employed to process the data set which is with fault values. For the robust idea, the fault value is solved via the LSSVR model iteratively weights, and then the trained RLSSVR model is updated depend on the interaction between incremental learning and decrement learning. During the interaction, the interests of the history data and the control storage data are all be considered. Numerical experiment supported by the college analog electronic experiments, adopted eight indexes of one order low power amplifier to evaluate performance. The training data set were gotten via precision instrument evaluation in two years. Simulation results reveal that proposed RLSSVR can handle the regressive deviation caused by the nonlinear feature, time varying feature and contain faults value exist in the industry processing, and has speeder than the traditional LSSVR, and WLSSVR.

Generalization-Memorization learning endeavors to minimize empirical risk while simultaneously reducing expected risk. Although we often do not pay attention to whether the training samples are accurately memorized during regression, improving generalization performance with better memory is always a goal pursued by regression. To tackle this issue, we introduce two new regression models, Least Squares Generalization-Memorization Regression (LSGMR) and Soft Least Squares Generalization-Memorization Regression (SLSGMR), by introducing the memory kernel learning on Least Squares Support Vector Regression (LSSVR). We conduct tests on these models using synthetic dataset and showcase that the LSSVR model can be viewed as a special case of our proposed model. Our experiments highlight that, for numerous problems, the models incorporating the employed memory mechanisms, LSGMR and SLSGMR, prove highly effective in yielding superior results compared to LSSVR on noise regression.

Hybrid application of unsupervised and supervised learning in forecasting absolute open flow potential for shale gas reservoirs

Comparative assessment on smart pre-processing methods for extracting information in FT-NIR measured data