From simple to complex: a sequential method for enhancing time series forecasting with deep learning

Time series prediction is widely applied in transportation management, energy scheduling, and weather forecasting, with deep learning emerging as a dominant approach due to its powerful temporal feature extraction capabilities. However, the predictive performance of deep learning models heavily depends on training strategies, while traditional optimization methods suffer from limited efficiency, ultimately affecting model performance. To address this issue, this study designs a Hybrid Optimization Expert System (HOES) based on six evolutionary algorithms to optimize deep learning models for time series prediction. HOES integrates multiple evolutionary algorithms and employs a transmission mechanism, memory system, and punishment system to achieve collaborative optimization. Specifically, the transmission mechanism enhances global search capability, the memory system preserves historically optimal solutions to prevent search degradation, and the punishment system eliminates ineffective optimization strategies; together, these mechanisms improve optimization efficiency. Experiments on six public datasets—Traffic, Weather, Household, Wind Power, Solar Power, and ETT_m1 demonstrate that HOES enhances predictive accuracy and convergence rate. SJ-LSTM is used for validation as a representative time-series forecasting model. Specially, on the Solar power dataset, the HOES-optimized SJ-LSTM model achieved a 24% reduction in RMSE, a 30% reduction in MAE, compared to suboptimal algorithm. These findings indicate that HOES significantly enhances the global optimization capability of deep learning models, mitigates the risk of local optima, which is also well-suited for complex time series prediction tasks.

As one of the conventional statistical methods, the autoregressive integrated moving average (ARIMA) model has been one of the most widely used linear models in time series forecasting. However, the ARIMA model cannot easily capture the nonlinear patterns. Artificial neural network (ANN) can be utilized to construct more accurate forecasting model than ARIMA for nonlinear time series, but it is difficult to explain the meaning of the hidden layers of ANN and it does not produce a mathematical equation. In this study, by combining ARIMA with genetic programming (GP), a hybrid forecasting model will be used for elevator traffic flow time series which can improve the accuracy both the GP and the ARIMA forecasting models separately. At last, simulations are adopted to demonstrate the advantages of the proposed ARIMA-GP forecasting model.

This dissertation explores the critical role of loss functions in enhancing the predictive performance of deep machine learning models. Loss functions are an integral element of all the ongoing advances we witness daily in this domain. I design custom loss functions and their impacts on various machine learning tasks, particularly in computer vision. In the first stage of my research, I aim to improve the prediction performance of deep learning models by providing them with more precise feedback associated with task requirements. This led me to create the concept of assistive loss functions. My first proposed loss function, inspired by the Active Shape Model (ASM), is ASM Loss, designed to improve the detection of facial landmarks and head pose accuracy in facial images. Additionally, influenced by Knowledge Distillation (KD), I introduce KD-Loss, a loss designed specifically for regression tasks, focusing on detecting facial landmarks. Proceeding to the second phase of my research, I focus on regression problems. Building upon my prior work, I introduce Adaptive Coordinate-based Regression (ACR)-Loss, a context-aware loss function with face alignment implication. I provide a mechanism to dynamically measure how hard the localization of each landmark point is, and accordingly, ACR-Loss provides unique feedback to the model. Furthermore, I introduce Intensity-aware Categorical (IC)-Loss for detecting sagittal cervical spine landmark points in medical images. It effectively deploys characteristics from classification problems to enhance regression optimization. Both ACR-Loss and IC-Loss comprise contextual information, leading to a significant enhancement in localization precision. In the third phase, I turned my attention toward classification problems within the domain of deep machine learning. I introduce Adaptive Correlation-based (Ad-corre) Loss, designed to enhance the discriminative capabilities of deep learning models, with a specific focus on improving classification accuracy in automatic facial expression recognition. Furthermore, I introduce Informative (Info) Loss, a specialized loss function developed for medical applications, specifically aimed at distinguishing between Mild Cognitive Impairment and Normal Cognition groups based on transcriptions of interviews conducted as part of the I-CONECT study. Overall, I have proposed custom loss functions to improve the predictive performance in both regression and classification problems. Furthermore, I have introduced assistive loss functions, designed to enhance the performance of the main loss functions. The collective outcomes of this extensive exploration highlight the critical role of loss functions, adapting and optimizing them for various domains in deep machine learning. This research suggests and proposes a variety of methodologies for designing adaptive and context-aware loss functions within deep machine learning.

Accurate well rate forecasting is critical for field development. In the oil industry, recurrent-based deep learning models have been used for production forecasting. Modern research is shifting towards using the novel transformer architecture in natural language and time series applications to better handle long-term temporal dependencies using attention layers. In this paper, we present a novel approach of applying a transformer-based deep learning model for probabilistic production forecasting. The Temporal Fusion Transformer (TFT) model, a modern transformer-based model for time series forecasting, was used to provide a better oil rate prediction in the Norwegian Volve field. The historical bottomhole pressure, wellhead pressure, wellhead temperature and choke size opening real time series data were used as past input features in the TFT model to forecast ahead the oil rate of two wells. The model was trained with a quantile loss function to produce a probabilistic prediction with both upper and lower bounds of uncertainties. After optimization the TFT model was blindly tested on the last 20% of the data to evaluate its prediction performance compared Block Recurrent Neural Network architectures (BlockRNN). The real-time production data used in this multivariate forecasting problem was found to be complex with no clear trend or seasonality. The BlockRNN model failed to produce a good prediction of oil rate compared to the TFT model. The TFT model was better at encoding the input features including, the choke size, wellhead pressure, and temperature, and understanding their long-range dependencies with the oil rate.The model was able to minimize the testing Mean Squared Error (MSE) for the two wells F-11H and F-12H reaching values of 0.08 and 0.03, respectively. In addition, the model was able to forecast a prediction bandwidth in between the 10th and 90th quantiles to account for uncertainty ranges which laid in-between the blind test intervals. Overall, the TFT model was proven successful in accurately forecasting complex trends of oil rates overcoming the limitations of conventional, BlockRNN uses a memory to understand and make predictions about new data models of information loss over long-term multivariate time series prediction. Our work presents a novel approach of using the TFT probabilistic deep learning model for multivariate oil rate forecasting in the oil and gas industry. The model showed very promising results outperforming conventional BlockRNN-based models in addition to providing a range of the forecasting uncertainty using quantile regression. Knowing the uncertainty range helps in making critical decision particularly in the well intervention and field development.

This study demonstrates the application of deep learning approach specifically the deep learning for cyberfraud incidence classification and time series prediction in the South African financial institutions. Secondary data from the South African Banking Risk Information Centre (SABRIC) was employed and the data was trained under the deep learning paradigm using the Long Short-Term Memory (LSTM) model and adaptive moment estimation (ADAM) algorithm for fraud incidence classification and time series prediction of fraud incidences. Overall, there were 94.1% correct classifications as opposed to 5.9% incorrect classifications. Moreover, the accuracy, precision, recall and F1-score of the LSTM classification model were 71.668%, 87.5%, 99.1% and 78.78% respectively. This indicates that the developed LSTM model is suitable for classification purposes. In addition, the model’s performance improves as new datasets are fed in. This is evident as the root mean square error (RMSE) reduced from 253.5116 obtained initially to 150.9 after new data was fed in. This study contributes conceptually, theoretically and empirical to knowledge on cyberfraud mitigation. The results show that the LSTM model can be deployed for fraud classification and time series analysis of fraud incidences. The outcome of this study may promote cyber resilience and sustain the fight against the perpetration of cyber-related fraud in South Africa’s financial institutions. The use of the LSTM model for cyberfraud classification and time series prediction of cyberfraud incidences in the South African financial institutions demonstrated in this study is unique.

The primary objective of this study is to evaluate the accuracy of different forecasting models for monthly wind farm electricity production. This study compares the effectiveness of three forecasting models: Autoregressive Integrated Moving Average (ARIMA), Seasonal ARIMA (SARIMA), and Support Vector Regression (SVR). This study utilizes data from two wind farms located in Poland—‘Gizałki’ and ‘Łęki Dukielskie’—to exclude the possibility of biased results due to specific characteristics of a single farm and to allow for a more comprehensive comparison of the effectiveness of both time series analysis methods. Model parameterization was optimized through a grid search based on the Mean Absolute Percentage Error (MAPE). The performance of the best models was evaluated using Mean Bias Error (MBE), MAPE, Mean Absolute Error (MAE), and R2Score. For the Gizałki farm, the ARIMA model outperformed SARIMA and SVR, while for the Łęki Dukielskie farm, SARIMA proved to be the most accurate, highlighting the importance of optimizing seasonal parameters. The SVR method demonstrated the lowest effectiveness for both datasets. The results indicate that the ARIMA and SARIMA models are effective for forecasting wind farm energy production. However, their performance is influenced by the specificity of the data and seasonal patterns. The study provides an in-depth analysis of the results and offers suggestions for future research, such as extending the data to include multidimensional time series. Our findings have practical implications for enhancing the accuracy of wind farm energy forecasts, which can significantly improve operational efficiency and planning.

The applications of deep learning and artificial intelligence have permeated daily life, with time series prediction emerging as a focal area of research due to its significance in data analysis. The evolution of deep learning methods for time series prediction has progressed from the Convolutional Neural Network (CNN) and the Recurrent Neural Network (RNN) to the recently popularized Transformer network. However, each of these methods has encountered specific issues. Recent studies have questioned the effectiveness of the self-attention mechanism in Transformers for time series prediction, prompting a reevaluation of approaches to LTSF (Long Time Series Forecasting) problems. To circumvent the limitations present in current models, this paper introduces a novel hybrid network, Temporal Convolutional Network-Linear (TCN-Linear), which leverages the temporal prediction capabilities of the Temporal Convolutional Network (TCN) to enhance the capacity of LSTF-Linear. Time series from three classical chaotic systems (Lorenz, Mackey-Glass, and Rossler) and real-world stock data serve as experimental datasets. Numerical simulation results indicate that, compared to classical networks and novel hybrid models, our model achieves the lowest RMSE, MAE, and MSE with the fewest training parameters, and its R2 value is the closest to 1.

Problem statement: The aim of this research was to study further some latest progress of wavelet transform for time series forecasting, particularly about Neural Networks Multiscale Autoregressive (NN-MAR). Approach: There were three main issues that be considered further in this research. The first was some properties of scale and wavelet coefficients from Maximal Overlap Discrete Wavelet Transform (MODWT) decomposition, particularly at seasonal time series data. The second focused on the development of model building procedures of NN-MAR based on the properties of scale and wavelet coefficients. Then, the third was empirical study about the implementation of the proposed procedure and comparison study about the forecast accuracy of NN-MAR to other forecasting models. Results: The results showed that MODWT at seasonal time series data also has seasonal pattern for scale coefficient, whereas the wavelet coefficients are stationer. The result of model building procedure development yielded a new proposed procedure of NN-MAR model for seasonal time series forecasting. In general, this procedure accommodated input lags of scale and wavelet coefficients and other additional seasonal lags. In addition, the result showed that the proposed procedure works well for determining the best NN-MAR model for seasonal time series forecasting. Conclusion: The comparison study of forecast accuracy showed that the NN-MAR model yields better forecast than MAR and ARIMA models.

Poly-linear regression with augmented long short term memory neural network: Predicting time series data

Emotion recognition (ER) is crucial for understanding human behaviours, social interactions, and psychological well-being. Electroencephalography (EEG) has emerged as a promising tool for capturing the neural correlates of emotions. This work is a systematic review of articles in ER using EEG signals. A total of 120 articles from 1041 articles were selected based on PRISMA guidelines using defined inclusion and exclusion criteria, published between 2018 and 2024. This article aims to provide an in-depth understanding of the current landscape of ER from EEG signals utilizing deep learning (DL). This review offers valuable guidance for researchers and practitioners seeking more refined and reliable emotion classification systems. To explore the effectiveness of DL models in EEG-based ER, several potential DL models, such as convolutional neural network, long short-term memory (LSTM), gated recurrent unit (GRU), hybrid bidirectional LSTM (BiLSTM), bidirectional GRU, and advanced DL models such as convolutional recurrent neural network and EEG-Conformer models are applied to two popular datasets, SEED and GAMEEMO, respectively, to depict the full process of ER. Additionally, the performance of DL models is also compared with the performance of basic machine learning (ML) models such as SVM, k-nearest neighbors, logistic regression, and boosting algorithms such as AdaBoost, XGBoost and LightGBM. Through extensive experiments and performance evaluations, the performance of different models when applied to the datasets mentioned above is compared. The accuracy, precision, recall, and F1-scores are analysed to determine the most effective model for EEG-based ER. The findings of this study demonstrate that the performance of hybrid DL models is more efficacious than that of ML models. The best-performing model (BiLSTM) classified the emotions, with an accuracy of 90.54% when applied to the GAMEEMO dataset. This research contributes to the growing body of literature on ER and provides insights into the feasibility of using EEG signals to understand emotional states, and presents a structured roadmap for future exploration. The findings can aid in the development of more accurate and reliable ER systems, which can have wide-ranging applications in psychology, social sciences, and human-computer interactions.

Disease prediction based on modeling the correlations between compounded indicator factors is a widely used technique in high incidence chronic disease prevention diagnosis. Predictive models based on personal health information have been developed historically by using simple regression fitting over relatively few factors. Regression approaches have been favored in previous prediction modeling approaches because they are simplest and do not assume any non-linearity in the model for contributions of the chosen factors. In practice, many factors are correlated and have underlying non-linear relationships to the predicted outcome. Deep learning offers a means to construct a more complex modeling approach, along with automation and adaptation. The aim of this paper is to assess the ability of a deep learning model to predict the heart disease incidence using a common benchmark dataset (University of California, Irvine (UCI) dataset). The performance of deep learning model has been compared with four popular machine learning models (two linear and two nonlinear) in predicting the incidence of heart disease using data from 567 participants from two cohorts taken from UCI database. The deep learning model was able to achieve the best accuracy of 94% and an AUC score of 0.964 when compared to other models. The performance of deep learning and nonlinear machine learning models was significantly better compared to the linear machine learning models with increase in the dataset size.

Time series forecasting plays a critical role in decision-making across diverse domains such as finance, healthcare, and environmental monitoring. While classical statistical models like ARIMA remain interpretable and efficient, they often struggle with nonlinear patterns and dynamic dependencies. This review systematically examines how artificial intelligence (AI) and optimization techniques enhance forecasting accuracy and robustness. We evaluate modern deep learning architectures (e.g., LSTM, GRU, Transformers), hybrid frameworks (e.g., VMD-LSTM, CNN-GRU), and optimization-augmented models. A meta-analysis of over 150 studies reveals that deep learning-based approaches, particularly those enhanced with Adam and RMSProp optimizers, improve forecasting accuracy by up to 14% compared to traditional methods. Hybrid models demonstrate superior performance in multi-step predictions and handling volatility. The analysis includes financial datasets (S&P 500, NASDAQ) and environmental data (Beijing $$PM_{2.5}$$ ). Despite their power, AI-driven models face challenges including interpretability, computational cost, and data dependency. Future directions highlight explainable AI, transfer learning, and lightweight architectures to address these limitations. This work serves as a reference for researchers exploring the evolving landscape of time series forecasting through AI and optimization integration.

A hybrid ARIMA and support vector machines model in stock price forecasting

A new look at forecasting annual corporate earnings in the U.S.A.

In this study, composite earnings per share models are estimated for 35 chemical, food, and utility firms during the 1979-1980 period. It is generally held that financial analysts produce superior earnings forecast when compared to time series model forecasts, however, the results of this study indicate that the average mean square forecasting error of analyst forecasts may be reduced by combining analyst and univariate time series model forecasts. Moreover, despite the high degree o! correlation existing among analyst and time series forecasts, the ordinary least squares estimation of the composite earnings model is a better forecasting model than the composite earnings models estimated with ridge regression and latent root regression techniques. Standardization of regression variables also is addressed.