Comment on dwes-2021-7

Robust Lp-norm least squares support vector regression with feature selection

Examining the condition of groundwater resources and the impact of droughts is valuable for effective water resources management. Today, machine learning (ML) models are recognized as one of the useful tools in time series predictions. In this study, the groundwater condition of one of the most important aquifers in northwest Iran was investigated using MODFLOW, followed by estimating the groundwater resource index (GRI) utilizing the multivariate adaptive regression spline (MARS) and least squares support vector regression (LSSVR) for a period between 2001 and 2019. Meteorological and hydrological drought indicators along with precipitation and flow rate were used as input variables for prediction. The simulation results revealed a groundwater level decrease since the aquifer withdrawal amount is more than the recharge amount. Besides, results showed that there is a limited interaction between surface water and groundwater resources, mainly caused by the decrease in the river flow and aquifer groundwater level drop. Both ML models performed well in GRI estimation, using groundwater flow, streamflow drought index, standardized precipitation index, and runoff as input variables. The performance of the MARS model with RMSE, MAE, and NSE error evaluation criteria of 0.37, − 0.19, and 0.83, respectively, exerted slightly better results than LSSVR with RMSE, MAE, and NSE of 0.48, − 0.06, and 0.80, respectively. The findings reveal the appropriate performance of both models in forecasting drought indicators, highlighting the necessity of using ML models in hydrology and drought prediction problems.

In this paper, division algebras are proposed as an elegant basis upon which to extend support vector regression (SVR) to multidimensional targets. Using this framework, a multitarget SVR called epsilon(Z)-SVR is proposed based on an epsilon-insensitive loss function that is independent of the coordinate system or basis used. This is developed to dual form in a manner that is analogous to the standard epsilon-SVR. The epsilon(H)-SVR is compared and contrasted with the least-square SVR (LS-SVR), the Clifford SVR (C-SVR), and the multidimensional SVR (M-SVR). Three practical applications are considered: namely, 1) approximation of a complex-valued function; 2) chaotic time-series prediction in 3-D; and 3) communication channel equalization. Results show that the epsilon(H)-SVR performs significantly better than the C-SVR, the LS-SVR, and the M-SVR in terms of mean-squared error, outlier sensitivity, and support vector sparsity.

The least square support vector regression (LSSVR) with the Gaussian kernel has two hyperparameters, sigma, $\sigma$, a bandwidth showing the spread of the kernel, and C, a regularization gauging the level of errors. In this paper, we want to systematically study how the hyperparameters are related to the LSSVR learning, thanks to the observation that the ergodic and stationary mean field game (MFG) implied hyperparameters are estimated based on the closed form solution given arbitrarily fixed bounded hyperparameters. Inversely, this closed form solution enables us to estimate MFG implied hyperparameters. The LSSVR with particle swarm optimization (LSSVR + PSO) allows a new way to estimate hyperparameters that doesn't involve cross-validations. The exact solvable MFG equivalent to our model has a stationary and ergodic solution of nonlinear Schr\"{o}dinger equation (NLS). The equilibrium solution of MFG is a pair of optimal action and the interaction density and this pair in NLS is the ergodic and stationary chemical potential and the interacting residual particle density, or $|\psi|^2$ in NLS. The ergodic solution of NLS ergodicises and stationarizes the LSSVR residual particles via the local interaction measure and we call this \emph{'the ergodic learning.'} As long as the non-zero volatility sigma and the non-zero mass ($\mu=1/C$) are bounded and the input and output data are required to have a non-degenerate support vector kernel, the input and output processes themselves don't need to be stationary, while all local interaction measures of the residual particles are ergodic. Depending on the (ergodic) learning classifications into condensed learning, intermediate learning and classical learning, based on theoretical temperatures, many candidate models for prediction are ergodically generated and the region of ergodicized and stationalized residuals via local interaction measure can be found for each hyperparameter pairs. As the C increases (or as we decrease the mass) or as the sigma increases, the volatility of learning increases. The learning tends to be more classical and less condensed. For each ergodic path generated as the ergodic solution to the MFG system, we profile the system response with respect to hyperparameters. Also, even in ergodic learning setting, phase transition behavior has been observed when PSO correctly keep the stationary energy conserved ergodically and stationarily. Lastly, it is shown that the zero crossing number of LSSVR $+$ PSO $=$ MFG is given by $E(N_0(T))=\frac{T}{2 \pi},$ given the data $\{(x_i, y_i)\}_{i=1}^T,$ x input, y output with $T>\tau^*=\sqrt{4 \pi} \Sigma^* \mu \sigma^2.$

In this study, an innovative approach that combines least square support vector regression (LSSVR) with uncertainty theory to enhance its performance in dealing with low-quality or imprecise data from real-world be proposed. The resulting model, called uncertain least square support vector regression (ULSSVR), incorporates chance constraints and simplified parameter selection, which are critical to handle imprecise observations. A numerical algorithm called the conjugate residual method (CR) is introduced to reduce the computational complexity of the model solution. The experimental results using both small and medium-sized datasets demonstrate the superior performance of ULSSVR in terms of prediction accuracy and generalization ability compared to other models such as uncertain support vector regression (USVR), uncertain linear lodel, uncertain polynomial model, and uncertain growth models. ULSSVR not only improves prediction accuracy by at least 28.49% but also demonstrates faster computational speed. Overall, ULSSVR presents a promising solution for data science and internet applications where dealing with imprecise and low-quality data is a common challenge.

The prediction of asphalt performance can be very important in terms of increasing service life and performance while saving energy and money. In this study, a new hybrid artificial intelligence (AI) system, SOS–LSSVR, has been proposed to predict the permanent deformation potential of asphalt pavement mixtures. SOS–LSSVR utilizes the symbiotic organisms search (SOS) and the least squares support vector regression (LSSVR), which are seen as a complementary system. The prediction model can be established from all input and output data pairs for LSSVR, while SOS optimizes the system’s tuning parameters. To avoid sampling bias and to partition the dataset into testing and training, a cross-validation technique was chosen. The results can be compared to those of previous studies and other predictive methods. Through the use of four error indicators, SOS–LSSVR accuracy was verified in predicting the permanent deformation behavior of an asphalt mixture. The present study demonstrates that the proposed AI system is a valuable decision-making tool for road designers. Additionally, the success of SOS–LSSVR in building an accurate prediction model suggests that the proposed self-optimized prediction framework has found an underlying pattern in the current database and thus can potentially be implemented in various disciplines.

To improve the accuracy of short-term load forecasting, a differential evolution algorithm (DE) based least squares support vector regression (LSSVR) method is proposed in this paper. Through optimizing the regularization parameter and kernel parameter of the LSSVR by DE, a short-term load forecasting model which can take load affected factors such as meteorology, weather, and date types into account is built. The proposed LSSVR method is proved by implementing short-term load forecasting on the real historical data of Yangquan power system in China. The average forecasting error is less than 1.6%, which shows better accuracy and stability than the traditional LSSVR and Support vector regression. The result of implementation of short-term load forecasting demonstrates that the hybrid model can be used in the short-term forecasting of the power system more efficiently.

To reduce the influence of received signal strength indication (RSSI) on ranging error, as well as the influence of least squares support vector regression (LSSVR) on localization algorithm, a node three-dimensional localization algorithm based on RSSI and LSSVR parameter optimization is proposed. First, the RSSI average values at 1–25m in four different directions are collected by experiments and the weighted recursive mean optimization method is used to optimize the values of RF factor and propagation factor. Then, the parameters of RBF kernel function and grid width of LSSVR are optimized. Finally, the RSSI range values are used as the input of LSSVR localization model, and the LSSVR regression model is used to solve, in this way, the location estimation of unknown WSN nodes is realized. The simulation results show that the average localization error of the algorithm without parameter optimization is 21.82%, and the localization error of the algorithm after parameter optimization is 11.70%, which has higher localization accuracy. At the same time, a node three dimensional localization experiment platform was built to verify the proposed algorithm in the actual environment, and the test results verified the effectiveness and superiority of the proposed algorithm.

Soft sensors are becoming increasingly important in our world today as tools for inferring difficult-to-measure process variables to achieve good operational performance and economic benefits. Recent advancement in machine learning provides an opportunity to integrate machine learning models for soft sensing applications, such as Least Square Support Vector Regression (LSSVR) which copes well with nonlinear process data. However, the LSSVR model usually uses the radial basis function (RBF) kernel function for prediction, which has demonstrated its usefulness in numerous applications. Thus, this study extends the use of non-conventional kernel functions in the LSSVR model with a comparative study against widely used partial least square (PLS) and principal component regression (PCR) models, measured with root mean square error (RMSE), mean absolute error (MAE) and error of approximation (Ea) as the performance benchmark. Based on the empirical result from the case study of the penicillin fermentation process, the Ea of the multiquadric kernel (MQ) is lowered by 63.44% as compared to the RBF kernel for the prediction of penicillin concentration. Hence, the MQ kernel LSSVR has outperformed the RBF kernel LSSVR. The study serves as empirical evidence of LSSVR performance as a machine learning model in soft sensing applications and as reference material for further development of non-conventional kernels in LSSVR-based models because many other functions can be used as well in the hope to increase the prediction accuracy.

Although electronic system has entered the digital age, analog circuit is still an essential part. Therefore the performance monitoring or evaluation of analog circuit is extremely important. However some problems about analog circuit performance monitoring is being, such as data acquisition online of the industry field with uncertainty, performance monitoring timeliness. Here an online performance monitoring method for analog circuit (OPM) subject to the data uncertainty is proposed. The main idea of OPM is to employ a learning machine same as least square support vector regression (LSSVR) to train and learn the data set. Considering the data from industrial field usually hold nonlinear feature, time varying feature and contain faults value, a novel robust LSSVR (RLSSVR) is presented to detection the performance of Electronic System. Moreover, the multi-kernel function is employed which has more flexibility than the single-kernel function, and can get more support vector numbers accuracy. At the same time, the novel robust learning algorithm is employed to process the data set which is with fault values. For the robust idea, the fault value is solved via the LSSVR model iteratively weights, and then the trained RLSSVR model is updated depend on the interaction between incremental learning and decrement learning. During the interaction, the interests of the history data and the control storage data are all be considered. Numerical experiment supported by the college analog electronic experiments, adopted eight indexes of one order low power amplifier to evaluate performance. The training data set were gotten via precision instrument evaluation in two years. Simulation results reveal that proposed RLSSVR can handle the regressive deviation caused by the nonlinear feature, time varying feature and contain faults value exist in the industry processing, and has speeder than the traditional LSSVR, and WLSSVR.

To improve the prediction accuracy of crude oil price even in current complicated international situation, this paper proposed a novel model linking firefly algorithm (FA) with least squares support vector regression (LSSVR), namely FA-LSSVR. In this hybrid intelligent model, FA is used to find the optimal values of LSSVR parameters (i.e., penalty coefficient and kernel function parameters), in order to achieve fast and accurate prediction results. To evaluate the forecasting ability of FA-LSSVR, its performance is compared with other models, including hybrid intelligent methods (LSSVR models with other popular optimization methods), and single models with given predetermined parameters (i.e., support sector regression (SVR), LSSVR, back-propagation neural network (BPNN), autoregressive integrated moving average (ARIMA)). The empirical results reveal that FA-LSSVR outperforms other benchmarks in terms of prediction accuracy, time saving and robustness, suggesting that the proposed approach is a promising alternative to forecast the crude oil price.

Health monitoring and fault diagnosis of liquid rocket engine (LRE) are the most important concerning issue for the safety of rocket’s flying, especially for the man-carried aerospace engineering. Based on the sensor measurement signals of a certain type of hydrogen-oxygen rocket engine, this paper proposed a real-time fault detection approach using a genetic algorithm-based least squares support vector regression (GA-LSSVR) algorithm for the real-time fault detection of the rocket engine. In order to obtain effective training samples, the data is normalized in this paper. Then, the GA-LSSVR algorithm is derived through comprehensive considerations of the advantages of the Support Vector Regression (SVR) algorithm and Least Square Support Vector Regression (LSSVR). What is more, this paper provided the genetic algorithm to search for the optimal LSSVR parameters. In the end, the computational results of the suggested approach using the rocket practical experimental data are given out. Through the analysis of the results, the effectiveness and the detection accuracy of this presented real-time fault detection method using LSSVR GA-optimized is verified. The experiment results show that this method can effectively diagnose this hydrogen-oxygen rocket engine in real-time, and the method has engineering application value.

Prediction of dissolved oxygen content in river crab culture based on least squares support vector regression optimized by improved particle swarm optimization

An evaluation method based on support vector regression (SVR) is put forward for the purpose of predicting subjective perceptions of automobile seat comfort. The inputs included fourteen seat interface pressure measures, three anthropometric. The output was an overall comfort index derived from occupant responses to a survey. In process of experimental data analysis, the algorithm of the least squares support vector regression (LSSVR) was used. The experimental results show that support vector regression model in a number of superior performance on the widely-used artificial neural network prediction model, results of this study will help automotive manufacturers improve car seat in the comfort of the process to reduce costs and shorten the manufacturing time for the car seat provides the industrial design aspects of the man-machine engineering evaluation method.

Chapter 3 - Hydrological Time Series Forecasting Using Three Different Heuristic Regression Techniques