Analysis of Long Short-Term Memory and Support Vector Regression Methods in Forecasting Electric Energy Sales: Case Study

The purpose of the study was to train and compare machine learning (ML) and deep learning (DL) models for long-term forecasting of minimum daily temperatures (frosts) - a key agrometeorological risk affecting crop productivity. The study was based on data from weather stations in St. Petersburg and the Leningrad region. To predict the year ahead, which aligns with the needs of operational agricultural production planning (sowing dates, harvest timing, protective measures), 8 methods were used: ForecasterAutoreg, Random Forest, Support Vector Regression (SVR), XGBoost, Convolutional Neural Network (CNN), SimpleRNN, Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM). The analysis was carried out on two datasets: for the periods 1936-2024 and 1881-1995. The quality of the models was assessed using the metrics MAE, MSE, RMSE, R2 and adjusted R2. The LSTM model showed the most accurate results on the main dataset (1936-2024): MAE 2.9, MSE 14.661, R2 0.789. The SVR method (MAE 3.461, R2 0.775) demonstrated the best metrics based on archived data (1881-1995). It has been established that DL models (LSTM, GRU, CNN) generally outperform classical ML methods based on modern data. The LSTM method is recognized as the most effective for integration into precision farming systems and agricultural monitoring in the region for the advance planning of agrotechnological measures to protect crops from frost, optimization of crop rotation systems, and minimization of crop loss risks.

The electrical equipment of offshore oil support ships operates in harsh marine environments with high failure rates, posing challenges to the development of preventive maintenance strategies. To address this issue, this article proposes an intelligent platform based on Long Short Term Memory (LSTM) algorithm to optimize preventive maintenance strategies for electrical equipment. Firstly, operational data of ship electrical equipment is collected, and data preprocessing and key feature extraction are carried out; subsequently, a prediction model based on LSTM is constructed to make real-time predictions on the health status of the equipment; then, the predictive model is integrated into the intelligent platform to achieve real-time monitoring of device status and dynamic optimization of maintenance strategies. The experimental results show that the LSTM based prediction model outperforms support vector regression (SVR) and random forest methods in terms of prediction accuracy and robustness. The average monthly failure rate of the equipment is 0.67 times, and the maintenance cost for 12 months is only $4750. In the above data conclusions, the intelligent platform based on LSTM algorithm can significantly improve the effectiveness of preventive maintenance strategies for marine oil support ship electrical equipment, highlighting the advantages of the proposed method.

Time Series (TS) models are well-known techniques that help to predict the weather in a certain time period. The traditional TS prediction models take more prediction time, overfitting and under-fitting of training data. In addition, state-of-art method like regression automata technique’s computational complexity is high due to the learning process based on heuristic method. In this study, learning process of the Cellular Automata (CA) is performed by the Support Vector Regression (SVR) method. The better discretize property of the SVR increases the learning rate and reduces the complexity of the CA. The proposed model is named as a Range Regression Automata Model based on Support Vector (RRAMSV) and this model calculates the multiple temperature attributes like soil temperature, air temperature, relative humidity and soil humidity. The proposed RRAMSV method has higher performance in most of the time-series prediction compared to Range Linear Regression (RLR), Range Logistic Regression (RLOR) and Convolutional Neural Network (CNN) — Long Short Term Memory (LSTM). The proposed RRAMSV method has Mean Absolute Error (MAE) of 0.06 and existing CNN-LSTM method has 1.65 MAE.

Electric power consumption short-term forecasting for individual households is an important and challenging topic in the fields of AI-enhanced energy saving, smart grid planning, sustainable energy usage and electricity market bidding system design. Due to the variability of each household’s personalized activity, difficulties exist for traditional methods, such as auto-regressive moving average models, machine learning methods and non-deep neural networks, to provide accurate prediction for single household electric power consumption. Recent works show that the long short term memory (LSTM) neural network outperforms most of those traditional methods for power consumption forecasting problems. Nevertheless, two research gaps remain as unsolved problems in the literature. First, the prediction accuracy is still not reaching the practical level for real-world industrial applications. Second, most existing works only work on the one-step forecasting problem; the forecasting time is too short for practical usage. In this study, a hybrid deep learning neural network framework that combines convolutional neural network (CNN) with LSTM is proposed to further improve the prediction accuracy. The original short-term forecasting strategy is extended to a multi-step forecasting strategy to introduce more response time for electricity market bidding. Five real-world household power consumption datasets are studied, the proposed hybrid deep learning neural network outperforms most of the existing approaches, including auto-regressive integrated moving average (ARIMA) model, persistent model, support vector regression (SVR) and LSTM alone. In addition, we show a k-step power consumption forecasting strategy to promote the proposed framework for real-world application usage.

Early warning systems depend heavily on the accuracy of landslide displacement forecasts. This study focuses on the Bazimen landslide located in the Three Gorges Reservoir region and proposes a hybrid prediction approach combining support vector regression (SVR) and long short-term memory (LSTM) networks. These models are optimized via the K-Nearest Neighbor (KNN) algorithm. Initially, cumulative displacement data were separated into trend and cyclic elements using a smoothing approach. SVR and LSTM were then used to predict the components, and KNN was introduced to optimize input factors and classify the results, improving accuracy. The final KNN-optimized SVR-LSTM model effectively integrates static and dynamic features, addressing limitations of traditional models. The results show that LSTM performs better than SVR, with an RMSE and MAPE of 24.73 mm and 1.87% at monitoring point ZG111, compared to 30.71 mm and 2.15% for SVR. The sequential hybrid model based on KNN-optimized SVR and LSTM achieved the best performance, with an RMSE and MAPE of 23.11 mm and 1.68%, respectively. This integrated model, which combines multiple algorithms, offers improved prediction of landslide displacement and practical value for disaster forecasting in the Three Gorges area.

Abnormal identification of oil monitoring based on LSTM and SVDD

Time series data reveals dynamic behavior of systems. A few real life examples are traffic flow, amount of rainfall, usage of electricity, share values, Forex rate etc.. Depending on the complexity of the system dynamics, algorithms differ to model the time series data accurately, so that the created model can be used for interpolation and more commonly extrapolation or prediction. For example, AR model performs well in stationary time series, but for non-stationary, it cannot capture the non-linear dynamics. In this research, we use Forex rate data, and experimented with various algorithms to capture the dynamics of the data. The success of the model is evaluated by accuracy in prediction. In our experiments, we applied two state-of-the-art models -Support Vector Regression (SVR) and Recurrent Neural Network (RNN). The target of the experiment is the prediction of longer future by recursion (feeding back predicted value to input for the next step prediction). The result shows that RNN with proper Long Short-Term Memory (LSTM) has better performance in predicting longer future.

The main purpose of modeling time series data is for forecasting. Time series modeling can be categorized into the univariate and multivariate approach. In univariate time series modeling, there are several methods of forecasting; one of them is ARIMA. Time series modeling can also consider the exogenous variables to improve the accuracy of forecasting or to obtain a more meaningful model. Recently, machine learning methods are widely employed in various field, including time series forecasting, particularly for solving nonlinear relationship among the variables. One of the most popular machine learning methods is Support Vector Machine (SVM). SVM can solve classification and regression problem. The SVM method used in time series is called Support Vector Regression (SVR). In SVR method, one of the most important things to improve the accuracy of forecasting is input selection. One of the approaches to select input in SVR is by choosing the significant lag obtained from the classical time series analysis and expand the ARIMA model to get multiplicative lag. In addition, the data on daily sales contains seasonal and calendar variation. The models are evaluated using Root Mean Square Error (RMSE) and Mean Absolute Percentage Error (MAPE). The best method is the SVR method using multiplicative lag from the best ARIMA model without dummy variable.

Accurate real-time bus arrival information is essential for an efficient public transport network, as it significantly impacts passenger experience, system reliability, reduced waiting times, dwell times, and operational efficiency. In Sri Lanka's public transport system, current bus arrival time computations primarily rely on static data, neglecting real-time information and critical factors influencing travel times. This highlights the need to identify the unique variables affecting bus arrival times within the Sri Lankan context and to develop robust prediction models that account for these influences. While traditional methods such as Historical Average Models, Regression, Time Series Analysis, and Kalman Filtering have been used in previous research for short-term travel time predictions, Machine Learning (ML) approaches have proven to deliver superior accuracy. ML models are regarded as the most effective for heterogeneous, lane-less traffic conditions with varying traffic volumes, such as those found in Sri Lanka. ML techniques excel in processing large, high-quality datasets and provide accurate predictions by accounting for all relevant variables influencing travel times. Although research has been conducted on developing various basic ML models for travel time prediction, there is a noticeable gap in studies comparing these models to determine the most suitable one for the Sri Lankan context. A Long- Short Term Memory (LSTM) neural network is a deep learning model that is capable of handling long-term dependencies. In the context of bus travel time prediction, LSTMs can leverage historical traffic and travel data to capture temporal patterns and fluctuations that influence travel times. By evaluating LSTMs against basic machine learning models, this study seeks to explore the advantages of applying deep learning techniques to transportation forecasting, ultimately contributing to more accurate and efficient predictive systems in transit planning. The ML models selected in this study include two basic traditional models K- Nearest Neighbours (KNN) and Support Vector Regression (SVR) and four advanced models that utilize ensemble techniques and advanced optimization as Random Forest Regression (RFR), Ada Boost, XG Boost and Gradient Boosting Machine (GBM). The performance of these models was compared with the LSTM model to identify the gap in their accuracies.

Durability estimation and short-term voltage degradation forecasting of vehicle PEMFC system: Development and evaluation of machine learning models

COVID-19 in Bangladesh: A Deeper Outlook into The Forecast with Prediction of Upcoming Per Day Cases Using Time Series.

Long short-term memory (LSTM) is widely applied in both academic and industrial fields. However, there is no reliable criterion on selecting hyperparameters of LSTM. Currently, although some widely used classic methods such as random search and grid search have obtained success to some extent, the problems in local optimum and convergence still exist. In this research, we propose to use grey wolf optimizer (GWO) to search for the hyperparameters of LSTM. Through the method, the superiority of metaheuristic in global optimization and the strength of LSTM in predicting are combined. In this model, number of hidden layer nodes and learning rate of LSTM are set as preys, and grey wolf pack has a simple but efficient mechanism to search for the optimal hyperparameters. The benchmark tests on several basic functions were utilized, and the results were verified by a comparative study with random search, support vector regression and several other regression methods. Specifically, we applied this algorithm in predicting the degradation trend of the airborne fuel pump. As a result, the ergodicity and convergence of the algorithm are proved mathematically based on Markov processes theory. The benchmark tests show that the GWO-LSTM model holds for predicting data with low overall slope and high partial fluctuation. The application in airborne fuel pump shows that, trained by dataset with 5700 points, the proposed model could predict sequence of 300 points with root mean square error 0.617 after 30 iterations of optimizing, which is 2.512 previously. The result further demonstrates that the proposed algorithm is applicable to make prediction with high accuracy. Overall, the effectiveness of GWO-LSTM model is verified from theoretical proof to benchmark tests and then to actual product application.

Applying and dissecting LSTM neural networks and regularized learning for dynamic inferential modeling

This study compares Support Vector Regression (SVR) and Long Short-Term Memory (LSTM) for forecasting the meteorological data in Nusa Tenggara, on some variables i.e., temperature, rainfall, wind speed, and humidity. It used historical data from 1980 to 2023 from the Meteorology, Climatology, and Geophysics Agency in Indonesia. This research evaluates the performance of both methods based on accuracy and computational efficiency. The SVR demonstrated faster training times, making it suitable for applications requiring quick predictions. Meanwhile, the LSTM excels in capturing complex temporal patterns. SVR and LSTM achieved a maximum R<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> of 0.633 for forecasting average temperature, indicating similar performance. The findings provide insights into choosing the appropriate model based on specific use cases in meteorology.

In hydrological research, accurate rainfall data is the primary subject for the minimization of potential loss of life and property that is mainly caused by floods. However, there is a difficulty in getting precise rainfall data for poorly gauged locations, especially in mountainous areas. Weather radar instruments can be the remedy accompanied by some errors. And, these errors should be removed before the implementation of this product. This paper presents the results of the research on radar rainfall estimate errors with support vector regression (SVR) method using the observed rain gauge data. The paper depicts the methodological base of the algorithm that covers additive and multiplicative corrections and the results of practical implementations considering the locations of gauge measurements. The preliminary results show that the SVR has a location-oriented performance. The multiplicative and additive correction factors show decreasing and polynomial trends respectively, as the distance from the radar location increase. Another particular outcome is that the SVR shows better results for the stations located in the mid-range (mainly for 40-60 km) contrary to the nearest ones. Since the systematic error in the radar data is nonlinear, the SVR method would show a promising result with a combination of other optimization techniques.