An interpretable regression approach based on bi-sparse optimization

Two-phase multi-kernel LP-SVR for feature sparsification and forecasting

Robust Lp-norm least squares support vector regression with feature selection

Although electronic system has entered the digital age, analog circuit is still an essential part. Therefore the performance monitoring or evaluation of analog circuit is extremely important. However some problems about analog circuit performance monitoring is being, such as data acquisition online of the industry field with uncertainty, performance monitoring timeliness. Here an online performance monitoring method for analog circuit (OPM) subject to the data uncertainty is proposed. The main idea of OPM is to employ a learning machine same as least square support vector regression (LSSVR) to train and learn the data set. Considering the data from industrial field usually hold nonlinear feature, time varying feature and contain faults value, a novel robust LSSVR (RLSSVR) is presented to detection the performance of Electronic System. Moreover, the multi-kernel function is employed which has more flexibility than the single-kernel function, and can get more support vector numbers accuracy. At the same time, the novel robust learning algorithm is employed to process the data set which is with fault values. For the robust idea, the fault value is solved via the LSSVR model iteratively weights, and then the trained RLSSVR model is updated depend on the interaction between incremental learning and decrement learning. During the interaction, the interests of the history data and the control storage data are all be considered. Numerical experiment supported by the college analog electronic experiments, adopted eight indexes of one order low power amplifier to evaluate performance. The training data set were gotten via precision instrument evaluation in two years. Simulation results reveal that proposed RLSSVR can handle the regressive deviation caused by the nonlinear feature, time varying feature and contain faults value exist in the industry processing, and has speeder than the traditional LSSVR, and WLSSVR.

In this study, an innovative approach that combines least square support vector regression (LSSVR) with uncertainty theory to enhance its performance in dealing with low-quality or imprecise data from real-world be proposed. The resulting model, called uncertain least square support vector regression (ULSSVR), incorporates chance constraints and simplified parameter selection, which are critical to handle imprecise observations. A numerical algorithm called the conjugate residual method (CR) is introduced to reduce the computational complexity of the model solution. The experimental results using both small and medium-sized datasets demonstrate the superior performance of ULSSVR in terms of prediction accuracy and generalization ability compared to other models such as uncertain support vector regression (USVR), uncertain linear lodel, uncertain polynomial model, and uncertain growth models. ULSSVR not only improves prediction accuracy by at least 28.49% but also demonstrates faster computational speed. Overall, ULSSVR presents a promising solution for data science and internet applications where dealing with imprecise and low-quality data is a common challenge.

Soft sensors are becoming increasingly important in our world today as tools for inferring difficult-to-measure process variables to achieve good operational performance and economic benefits. Recent advancement in machine learning provides an opportunity to integrate machine learning models for soft sensing applications, such as Least Square Support Vector Regression (LSSVR) which copes well with nonlinear process data. However, the LSSVR model usually uses the radial basis function (RBF) kernel function for prediction, which has demonstrated its usefulness in numerous applications. Thus, this study extends the use of non-conventional kernel functions in the LSSVR model with a comparative study against widely used partial least square (PLS) and principal component regression (PCR) models, measured with root mean square error (RMSE), mean absolute error (MAE) and error of approximation (Ea) as the performance benchmark. Based on the empirical result from the case study of the penicillin fermentation process, the Ea of the multiquadric kernel (MQ) is lowered by 63.44% as compared to the RBF kernel for the prediction of penicillin concentration. Hence, the MQ kernel LSSVR has outperformed the RBF kernel LSSVR. The study serves as empirical evidence of LSSVR performance as a machine learning model in soft sensing applications and as reference material for further development of non-conventional kernels in LSSVR-based models because many other functions can be used as well in the hope to increase the prediction accuracy.

In this paper, division algebras are proposed as an elegant basis upon which to extend support vector regression (SVR) to multidimensional targets. Using this framework, a multitarget SVR called epsilon(Z)-SVR is proposed based on an epsilon-insensitive loss function that is independent of the coordinate system or basis used. This is developed to dual form in a manner that is analogous to the standard epsilon-SVR. The epsilon(H)-SVR is compared and contrasted with the least-square SVR (LS-SVR), the Clifford SVR (C-SVR), and the multidimensional SVR (M-SVR). Three practical applications are considered: namely, 1) approximation of a complex-valued function; 2) chaotic time-series prediction in 3-D; and 3) communication channel equalization. Results show that the epsilon(H)-SVR performs significantly better than the C-SVR, the LS-SVR, and the M-SVR in terms of mean-squared error, outlier sensitivity, and support vector sparsity.

Prediction of dissolved oxygen content in river crab culture based on least squares support vector regression optimized by improved particle swarm optimization

To improve the prediction accuracy of crude oil price even in current complicated international situation, this paper proposed a novel model linking firefly algorithm (FA) with least squares support vector regression (LSSVR), namely FA-LSSVR. In this hybrid intelligent model, FA is used to find the optimal values of LSSVR parameters (i.e., penalty coefficient and kernel function parameters), in order to achieve fast and accurate prediction results. To evaluate the forecasting ability of FA-LSSVR, its performance is compared with other models, including hybrid intelligent methods (LSSVR models with other popular optimization methods), and single models with given predetermined parameters (i.e., support sector regression (SVR), LSSVR, back-propagation neural network (BPNN), autoregressive integrated moving average (ARIMA)). The empirical results reveal that FA-LSSVR outperforms other benchmarks in terms of prediction accuracy, time saving and robustness, suggesting that the proposed approach is a promising alternative to forecast the crude oil price.

Dissolved oxygen directly affects the growth status of fishes in intensive aquaculture, thus we set up a prediction model to determine the future changing trend of dissolved oxygen. The dissolved oxygen prediction model we proposed was based on the least squares support vector regression (LSSVR) model with fruit fly optimization algorithm (FOA) to find optimal parameters (γ and σ) of LSSVR. Because these two parameters can significantly affect the performance of the LSSVR, we studied the other two parameter optimization methods the particle swarm optimization (PSO) algorithm, the genetic algorithm (GA) and immune genetic algorithm(IGA) to compare them with the FOA algorithm. The calculated mean absolute percentage errors of the results of the four prediction models were 0.35%, 1.3%, 2.03% and 1.33%, respectively. The FOA-LSSVR model has a higher prediction accuracy and more reliable performance than the other models. When the predicted values of dissolved oxygen fall below the safety level, the farmer can start an oxygen increasing machine in advance to maintain the safety of fishes. The prediction model was used in Yangzhong, Jiangsu province, China, and it performed well and helped farmers make decisions and reduce aquaculture risks.

The least square support vector regression (LSSVR) with the Gaussian kernel has two hyperparameters, sigma, $\sigma$, a bandwidth showing the spread of the kernel, and C, a regularization gauging the level of errors. In this paper, we want to systematically study how the hyperparameters are related to the LSSVR learning, thanks to the observation that the ergodic and stationary mean field game (MFG) implied hyperparameters are estimated based on the closed form solution given arbitrarily fixed bounded hyperparameters. Inversely, this closed form solution enables us to estimate MFG implied hyperparameters. The LSSVR with particle swarm optimization (LSSVR + PSO) allows a new way to estimate hyperparameters that doesn't involve cross-validations. The exact solvable MFG equivalent to our model has a stationary and ergodic solution of nonlinear Schr\"{o}dinger equation (NLS). The equilibrium solution of MFG is a pair of optimal action and the interaction density and this pair in NLS is the ergodic and stationary chemical potential and the interacting residual particle density, or $|\psi|^2$ in NLS. The ergodic solution of NLS ergodicises and stationarizes the LSSVR residual particles via the local interaction measure and we call this \emph{'the ergodic learning.'} As long as the non-zero volatility sigma and the non-zero mass ($\mu=1/C$) are bounded and the input and output data are required to have a non-degenerate support vector kernel, the input and output processes themselves don't need to be stationary, while all local interaction measures of the residual particles are ergodic. Depending on the (ergodic) learning classifications into condensed learning, intermediate learning and classical learning, based on theoretical temperatures, many candidate models for prediction are ergodically generated and the region of ergodicized and stationalized residuals via local interaction measure can be found for each hyperparameter pairs. As the C increases (or as we decrease the mass) or as the sigma increases, the volatility of learning increases. The learning tends to be more classical and less condensed. For each ergodic path generated as the ergodic solution to the MFG system, we profile the system response with respect to hyperparameters. Also, even in ergodic learning setting, phase transition behavior has been observed when PSO correctly keep the stationary energy conserved ergodically and stationarily. Lastly, it is shown that the zero crossing number of LSSVR $+$ PSO $=$ MFG is given by $E(N_0(T))=\frac{T}{2 \pi},$ given the data $\{(x_i, y_i)\}_{i=1}^T,$ x input, y output with $T>\tau^*=\sqrt{4 \pi} \Sigma^* \mu \sigma^2.$

Integrated soft sensor using just-in-time support vector regression and probabilistic analysis for quality prediction of multi-grade processes

To reduce the influence of received signal strength indication (RSSI) on ranging error, as well as the influence of least squares support vector regression (LSSVR) on localization algorithm, a node three-dimensional localization algorithm based on RSSI and LSSVR parameter optimization is proposed. First, the RSSI average values at 1–25m in four different directions are collected by experiments and the weighted recursive mean optimization method is used to optimize the values of RF factor and propagation factor. Then, the parameters of RBF kernel function and grid width of LSSVR are optimized. Finally, the RSSI range values are used as the input of LSSVR localization model, and the LSSVR regression model is used to solve, in this way, the location estimation of unknown WSN nodes is realized. The simulation results show that the average localization error of the algorithm without parameter optimization is 21.82%, and the localization error of the algorithm after parameter optimization is 11.70%, which has higher localization accuracy. At the same time, a node three dimensional localization experiment platform was built to verify the proposed algorithm in the actual environment, and the test results verified the effectiveness and superiority of the proposed algorithm.

To improve the accuracy of short-term load forecasting, a differential evolution algorithm (DE) based least squares support vector regression (LSSVR) method is proposed in this paper. Through optimizing the regularization parameter and kernel parameter of the LSSVR by DE, a short-term load forecasting model which can take load affected factors such as meteorology, weather, and date types into account is built. The proposed LSSVR method is proved by implementing short-term load forecasting on the real historical data of Yangquan power system in China. The average forecasting error is less than 1.6%, which shows better accuracy and stability than the traditional LSSVR and Support vector regression. The result of implementation of short-term load forecasting demonstrates that the hybrid model can be used in the short-term forecasting of the power system more efficiently.

By accurate predicting of pipe bursts, it is possible to schedule pipe maintenance, rehabilitation and improve the level of services in water distribution networks (WDNs). In this study, we aimed to implement five artificial intelligence and machine learning regression models such as multivariate adaptive regression splines (MARS), M5' regression tree (M5'), Least square support vector regression (LS-SVR), fuzzy regression based on c-means clustering (FCMR) and regressive convolution neural network with support vector regression (RCNN-SVR) for predicting pipe burst rate and evaluating the performance of these models. The most effective parameters for regression models are pipes age, diameter, depth of installation, length, average and maximum hydraulic pressure. In the present study, collected data include 158 cases for polyethylene (PE) and 124 cases for asbestos cement (AC) pipes during 2012-2019. The results indicate that the RCNN-SVR model has a great performance of pipe burst rate (PBR) prediction.

An evaluation method based on support vector regression (SVR) is put forward for the purpose of predicting subjective perceptions of automobile seat comfort. The inputs included fourteen seat interface pressure measures, three anthropometric. The output was an overall comfort index derived from occupant responses to a survey. In process of experimental data analysis, the algorithm of the least squares support vector regression (LSSVR) was used. The experimental results show that support vector regression model in a number of superior performance on the widely-used artificial neural network prediction model, results of this study will help automotive manufacturers improve car seat in the comfort of the process to reduce costs and shorten the manufacturing time for the car seat provides the industrial design aspects of the man-machine engineering evaluation method.