Time Series Prediction Using LS-SVM with Particle Swarm Optimization

It is well known that the performance of Least Squares Support Vector Machine (LSSVM) is guaranteed by k-fold cross-validation (k-CS) or other optimized approaches to choose an appropriate setting of a number of parameters including such as the regularization parameter (7) and the kernel parameter (σ) and so on. However, it is mentioned in a large number of research on k-CS that it need more computational time and large computational burden in the process of optimizing parameters for LSSVM. Moreover, the obtained parameters by CS method are fixed and it lead easily to a poor generalization capabilities in various application. In order to avoid k-CS and to implement parameters update online, this paper proposes a novel method which applied Unscented Kalman Filter (UKF) to dynamically implement parameter updating problem for LSSVM. To estimate LSSVM's parameters online, the state and measurement equations of UKF are first constructed by considering LSSVM's parameter choice as state variable and treating LSSVM model as the measurement equation, respectively. Then, the UKF approach is used to update the LSSVM parameters timely according to the last obtained instance. Applying the proposed method, LSSVM parameters are not any more fixed as tuned parameters on the training dataset, but are adjusted dynamically as new measurements arrived. Finally, the viability and superiority of the proposed method are verified by simulation.

Accurate electricity price prediction is key to the orderly operation of the electricity market. However, the uncertain, stochastic and fluctuant characteristics of electricity pricees make prediction difficult. With the aim of solving this issue, this investigation proposed a multi-stage intelligent model integrating the Beveridge–Nelson decomposition (B-N-D) model, the least square support vector machine (LSSVM), and a nature-inspired optimization model named the whale optimization algorithm (WOA). Firstly, the B-N-D model was utilized to decompose the hourly electricity price time series into determinacy component, periodic trend, and stochastic item. Secondly, the WOA–LSSVM model was proposed to forecast the future hourly data of three components respectively, of which the optimal parameters of LSSVM were determined by using WOA. Finally, the future hourly electricity price data were computed by multiplying the forecasted data of those terms. To verify the validity of the proposed electricity price prediction model in this paper, five comparison approaches based on the B-N-D approach were selected, which are auto-regressive integrated moving average (ARIMA), single LSSVM, LSSVM optimized by the fruit-fly optimization algorithm (FOA), LSSVM optimized by particle swarm optimization (PSO) models, and WOA–LSSVM without B-N-D. By comparatively analyzing the error criteria values of the above models through testing on the objective data of the Pennsylvania–New Jersey–Maryland (PJM) electricity market collected from 11 December 2017 to 18 December 2017, from 15 January 2018 to 22 January 2018, and from 1 February 2018 to 25 February 2018, we conclude that the constructed intelligent model in this paper can greatly enhance the prediction precision of electricity prices.

Time series forecasting has a fundamental importance in various practical domains. Many models have been proposed in literature to model and predict the Time Series Data (TSD) efficiently. As the modeling and prediction depends on the nature of TSD, one model may not be opt for all applications. This paper presents a hybrid model based on Particle Swarm Optimization (PSO) with Least Square Support Vector Machine (LSSVM). PSO parameter is used for optimizing LSSVM parameters and the proposed model is tested on titan stock market data which is a time series data, Surface roughness data, strength of radiation shielding concrete (RSC) data are non TSD and evaluated using the standard performance measures like Mean absolute error (MAE), Mean Relative Error (MRE) and Root Mean Squared Error (RMSE). It can be observed from the obtained results that the proposed hybrid model provides better prediction accuracy than the LS-SVM model.

The application of support vector machine to the aircraft power supply system breakdown diagnosis is one of the research focuses in the tomorrow. Support vector machine (SVM) belongs a new type of machine learning technique, which uses structural hazard minimization rule to substitute for the conventional empirical hazard minimization according to great specimen. The powerful performance of the least squares support vector machine (LSSVM) needs to be reflected in the optimal selection of appropriate parameters, and the quality of parameters greatly affects the accuracy and efficiency of breakdown diagnosis. The traditional LSSVM parameters selection method is inefficient, the calculation is huge, and it takes expensive time to select the satisfactory solution. Aiming at the problem of parameters selection of LSSVM, the way of optimizing the parameters of LSSVM by apply simulated annealing genetic algorithm (SAGA) is proposed. SAGA uses the parallel sampling process of genetic algorithm to improve the time performance of optimization, and makes use of the simulated annealing algorithm to dominate the constriction of majorization to prevent precocity. On the one hand, genetic algorithm has strong ability to grasp the overall search process, on the other hand, simulated annealing algorithm is used to control the convergence of the algorithm to prevent premature phenomenon. The automatic optimal selection of LSSVM parameters is achieved. The Iris system classification and recognition in the UCI machine learning database and the breakdown diagnosis of the autopilot flight control box are used as experimental platforms to obtain data samples for simulation research. The simulation results show that LSSVM optimized by SAGA improves both the accuracy and efficiency of classification recognition. It not only effectively overcomes the problem of low efficiency caused by searching optimal parameters by experience, but also effectively improves the accuracy of classification recognition. The false alarm rate and redundant breakdowns are effectively reduced.

Improving the accuracy and speed has become a main concern of time-series prediction. Aiming at these problems existing in time-series prediction, three kinds of researches and improvements are made as follows. This paper proposes a prediction method of combining Empirical Mode Decomposition (EMD) with least squares support vector machines (LS-SVM), the experimental results show that under the same conditions, the testing error of EMD combining with LS-SVM method is 0.1943 which significantly better than any single method of LS-SVM or SVM or BP neural network (BPNN), thus it is better for non-stationary time series. The immune clonal memetic algorithm (ICMA) is employed for resolving the parameter optimization problem in LS-SVM model, by combining global optimization with local optimization, the experiments show that the testing error of this method is 0.0865, which is faster than the optimization with the genetic algorithm (GA) or grid search algorithm. To raise the prediction speed, an improved LS-SVM online prediction method is proposed, which combine selective pruning algorithm with fast incremental learning, the results of experiment show that the speed of this method is improved nearly double compared with the direct inverse LS-SVM's, and a quarter is raised than the recursive inversion LS-SVM, with higher real-time performance while ensuring the reasonable prediction accuracy.

The prediction of meteorological time series plays very important role in several fields. In this paper, an application of least squares support vector machine (LS-SVM) for short-term prediction of meteorological time series (e.g. solar irradiation, air temperature, relative humidity, wind speed, wind direction and pressure) is presented. In order to check the generalization capability of the LS-SVM approach, a K-fold cross-validation and Kolmogorov–Smirnov test have been carried out. A comparison between LS-SVM and different artificial neural network (ANN) architectures (recurrent neural network, multi-layered perceptron, radial basis function and probabilistic neural network) is presented and discussed. The comparison showed that the LS-SVM produced significantly better results than ANN architectures. It also indicates that LS-SVM provides promising results for short-term prediction of meteorological data.

Stock return forecast has been an important issue and difficult task for both shareholders and financial professionals. To tackle this problem, we introduce Least Square Support Vector Machine (LS-SVM), an improved algorithm that regresses faster than standard SVM, and Dynamic Inertia Weight Particle Swarm Optimization (W-PSO), that outperform standard PSO in parameter selection. The work of this paper is as following: First, forecast daily stock Return of Shanghai Security Exchanges of China using Back Propagation Neural Network (BPNN) and LS-SVM. Secondly, forecast the stock return using LS-SVM optimized by W- PSO. Finally, make a comparative analysis of the three algorithms. We reached conclusion that, in terms of forecast accuracy, LS-SVM outperforms BPNN, and when LS-SVM is optimized by W-PSO, the best result is achieved.

In nuclear power plant's peripheral environmental radiation monitoring wireless sensor network, the γ dose rate data collected in sensor node may be missed due to various factors, which will influence the validity of environmental radiation monitoring. To solve the problem, a missing data imputation algorithm based on least squares support vector machine(LSSVM) is proposed and particle swarm optimization is adopted to search the optimal set of the LSSVM model parameters. This LSSVM model imputes missing data utilizing node's previous monitoring data and neighbor node's current monitoring data jointly. The experimental results using the real radiation monitoring data around a nuclear power plant demonstrate that the proposed algorithm can achieve higher imputation accuracy than neural network model as well as direct LSSVM model, which has the non-optimized parameters.

Regularization parameter and kernel parameters play a crucial role on the accuracy and rapidity of the least squares support vector machine (LS-SVM) algorithm. Existing methods of tuning the parameters for LS-SVM have some disadvantages, such as difficulty of getting global optimum and poor convergent speed. In this work, we develop the parameter design of LS-SVM using quantum particle swarm optimization (QPSO) and apply the designed LS-SVM into node localization. According to the initial values, the optimal parameter values are obtained using the leave-10-out cross validation method. Simulation results show that the proposed method is compared to the coupling simulated annealing (CSA) LS-SVM and the improved particle swarm optimization (IPSO) LS-SVM.

Aiming at the difficulty of accurate prediction due to the randomness and nonstationary nature of blood glucose concentration series, a blood glucose concentration prediction model based on complementary ensemble empirical mode decomposition (CEEMD) and least squares support vector machine (LSSVM) is proposed. Firstly, CEEMD is used to convert the blood glucose concentration sequence into a series of intrinsic mode functions (IMFs) to reduce the impact of randomness and nonstationary signals on prediction performance. Then, a LSSVM prediction model is established for each mode IMF. The comprehensive learning particle swarm optimization (CLPSO) algorithm is used to optimize the kernel parameters of LSSVM. Finally, the prediction results of all IMFs are superimposed to yield the final blood glucose concentration prediction value. The experimental results show that the proposed prediction model has higher prediction accuracy in short-term blood glucose concentration values.

A heuristic training-based least squares support vector machines for power system stabilization by SMES

The objective of this paper was to determine soluble solids content (SSC) of intact Gannan navel orange by a portable near-infrared (NIR) spectrometer with the optical fiber in the wavelength range of 551~950nm. The effective wavelength regions (EWs) were chosen from the spectra pro-processed by second derivative by interval partial least square (iPLS) and backward interval partial least square (Bipls). Then the partial least square (PLS) and least square support vector machine (LS-SVM) models were developed with EWs. 60 unknown samples were used to evaluate the performance of them. The LS-SVM model was better than others with EWs chosen by Bilps. The correlation coefficient (R) and root mean square error of prediction (RMSEP) for LS-SVM (Bipls) were 0.86 and 0.55°Brix. The results showed that the portable NIR combination with LS-SVM was a feasible method to determine SSC of intact Gannan navel orange nondestructively.

Improving forecasts, particularly the accuracy, efficiency, and precision of time-series forecasts, is becoming critical for authorities to predict, monitor, and prevent the spread of the Coronavirus disease. However, the results obtained from the predictive models are imprecise and inefficient because the dataset contains linear and non-linear patterns, respectively. Linear models such as autoregressive integrated moving average cannot be used effectively to predict complex time series, so nonlinear approaches are better suited for such a purpose. Therefore, to achieve a more accurate and efficient predictive value of COVID-19 that is closer to the true value of COVID-19, a hybrid approach was implemented. Therefore, the objectives of this study are twofold. The first objective is to propose intelligence-based prediction methods to achieve better prediction results called autoregressive integrated moving average-least-squares support vector machine. The second objective is to investigate the performance of these proposed models by comparing them with the autoregressive integrated moving average, support vector machine, least-squares support vector machine, and autoregressive integrated moving average-support vector machine. Our investigation is based on three COVID-19 real datasets, i.e., daily new cases data, daily new death cases data, and daily new recovered cases data. Then, statistical measures such as mean square error, root mean square error, mean absolute error, and mean absolute percentage error were performed to verify that the proposed models are better than the autoregressive integrated moving average, support vector machine model, least-squares support vector machine, and autoregressive integrated moving average-support vector machine. Empirical results using three recent datasets of known the Coronavirus Disease-19 cases in Malaysia show that the proposed model generates the smallest mean square error, root mean square error, mean absolute error, and mean absolute percentage error values for training and testing datasets compared to the autoregressive integrated moving average, support vector machine, least-squares support vector machine, and autoregressive integrated moving average-support vector machine models. This means that the predicted value of the proposed model is closer to the true value. These results demonstrate that the proposed model can generate estimates more accurately and efficiently. Compared to the autoregressive integrated moving average, support vector machine, least-squares support vector machine, and autoregressive integrated moving average-support vector machine models, our proposed models perform much better in terms of percent error reduction for both training and testing all datasets. Therefore, the proposed model is possibly the most efficient and effective way to improve prediction for future pandemic performance with a higher level of accuracy and efficiency.

In this paper, Artificial Bee Colony (ABC) algorithm which inspired from the behavior of honey bees swarm is presented. ABC is a stochastic population-based evolutionary algorithm for problem solving. ABC algorithm, which is considered one of the most recently swarm intelligent techniques, is proposed to optimize least square support vector machine (LSSVM) to predict the daily stock prices. The proposed model is based on the study of stocks historical data, technical indicators and optimizing LSSVM with ABC algorithm. ABC selects best free parameters combination for LSSVM to avoid over-fitting and local minima problems and improve prediction accuracy. LSSVM optimized by Particle swarm optimization (PSO) algorithm, LSSVM, and ANN techniques are used for comparison with proposed model. Proposed model tested with twenty datasets representing different sectors in S&P 500 stock market. Results presented in this paper show that the proposed model has fast convergence speed, and it also achieves better accuracy than compared techniques in most cases.

Aiming at the nonlinearity, chaos, and small-sample of aero engine performance parameters data, a new ensemble model, named the least squares support vector machine (LSSVM) ensemble model with phase space reconstruction (PSR) and particle swarm optimization (PSO), is presented. First, to guarantee the diversity of individual members, different single kernel LSSVMs are selected as base predictors, and they also output the primary prediction results independently. Then, all the primary prediction results are integrated to produce the most appropriate prediction results by another particular LSSVM—a multiple kernel LSSVM, which reduces the dependence of modeling accuracy on kernel function and parameters. Phase space reconstruction theory is applied to extract the chaotic characteristic of input data source and reconstruct the data sample, and particle swarm optimization algorithm is used to obtain the best LSSVM individual members. A case study is employed to verify the effectiveness of presented model with real operation data of aero engine. The results show that prediction accuracy of the proposed model improves obviously compared with other three models.