Payload-Based Traffic Classification Using Multi-Layer LSTM in Software Defined Networks

Many countries around the globe, including Tanzania, are investing in renewable energy, partly because of the need to increase electric power generation and meet the ever-increasing electricity demands, and partly because of the need to combat global climate change caused by carbon emissions from fossil fuel-based electricity production. Kishapu Solar Project, located in Kishapu District, Shinyanga Region, Tanzania, is one of the renewable energy projects with a planned generation of 150 MW of electric power once completed, using solar photovoltaic technology. Accurate forecasting of solar radiation is important in managing and maintaining solar energy plants. Inaccurate forecasting of solar radiation can lead to poor harvest of solar energy and poor management of solar energy plants, which directly impacts electricity contribution to the national electricity grid. Long Short-Term Memory (LSTM) and its variant, PeepHole LSTM, are examples of Deep Learning models that can be used to predict solar radiation. The purpose of PeepHole LSTM is to introduce “peephole” connections in the LSTM model, which allow LSTM gates to directly inspect memory cell state and hence have richer information to process. However, there is still a research gap on the impact of “peephole” connections on the prediction performance of the LSTM model, especially in the context of forecasting solar radiation in Kishapu District, where the Kishapu Solar Project is located. The purpose of this study is to fill this research gap by developing both LSTM and PeepHole LSTM models and comparing their performances in forecasting solar radiation in Kishapu District. The 14-year Kishapu District daily solar radiation (Downward Shortwave Radiation) dataset was first pre-processed by scaling (normalizing) it using Standard Scaler and then split into training set (the first 70%), validation set (the next 15%) and test set (the last 15%) before being utilized to train and test the developed LSTM and PeepHole LSTM models. Both LSTM and PeepHole LSTM models used the previous 5 days' solar radiation values as inputs to predict the output (next day (6th day) solar radiation value). The trained and better-performing model between the two was finally saved in .h5 format and integrated with a Gradio based Web Application to provide a user interface for officials in Kishapu District and at Kishapu Solar Project to predict daily solar radiation. The findings from performance evaluation revealed that the LSTM model outperformed the PeepHole LSTM model in predicting solar radiation by achieving test Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) scores of 26.22 W/m2 and 36.65 W/m2, respectively, compared to MAE and RMSE scores of 30.84 W/m2 and 41.88 W/m2, respectively, achieved by the PeepHole LSTM model, concluding that “peephole” connections reduce the performance of the LSTM model rather than increasing it when used to forecast solar radiation in Kishapu District. This study’s recommendation to Artificial Intelligence (AI) software developers and researchers is to use the LSTM rather than the PeepHole LSTM Deep Learning model when forecasting solar radiation in Kishapu District and in locations with similar climatic conditions

Wearable electroencephalography (EEG) devices for seizure detection accuracy and reliability are deep learning (DL) applications in the field of epilepsy diagnosis. In this study, we sought to increase the accuracy of seizure detection using advanced DL algorithms on the Children’s Hospital Boston - Massachusetts Institute of Technology (CHB-MIT) EEG database. First, a fully convolutional network (FCN) was trained and assessed using accuracy and recall/precision metrics, and the early stopping technique was used to avoid overfitting. To assess the performance, the FCN was evaluated in terms of various metrics, including accuracy, precision, recall, F1-score, and receiver operating characteristic (ROC)-area under the curve (AUC). In addition, two-dimensional (2D) convolutional neural networks (CNNs) and long short-term memory (LSTM) models were used to model the database, and their performance was thoroughly measured using different metrics, graphs, and confusion matrices. Using LSTM variants, such as standard LSTM, bidirectional LSTM, stacked LSTM, and LSTM attention mechanisms, hybrid convolutional LSTM (ConvLSTM) models were trained and compared. The comparison was conducted based on the training and validation accuracy and loss, as well as the graphs resulting from the precision–recall curves. Apart from DL approaches, EEG signal analysis using time–frequency techniques, such as wavelet transform and short-time Fourier transform, has also been investigated. These methods assisted in the analysis of the time–frequency features of EEG signals in combination with DL models. This study demonstrates that the performance of wearable EEG devices can be augmented using a combination of DL and seizure signal processing techniques. The FCN achieved an accuracy of 92%, with a recall for seizures of 33%, an F1-score of 0.03, and strong ROC-AUC results. The 2D CNN achieved 96% accuracy, a seizure recall of 70%, an F1-score of 0.12, and an ROC-AUC score of 78%. The baseline LSTM struggled with effectiveness at 53% accuracy with a seizure recall of 18%. In contrast, the LSTM model, which incorporated synthetic minority oversampling technique (SMOTE) balancing, was able to reach improvements of up to 89% accuracy, with a precision of 91%, a recall of 86%, an F1-score of 0.89, and a strong ROC curve performance. Among the models, the LSTM with SMOTE was the best performer, with 89% accuracy, 91% precision, 86% recall, and an F1-score of 0.89. These results provide evidence that applying techniques for data balancing in combination with certain DL network architectures significantly improves the detection of seizures using wearable EEG devices worn on the body. We believe that real-time monitoring and high-performance systems are feasible using optimized DL frameworks. The analysis of the performance of different models allows for the understanding of the possibilities of optimizing the architectures of DL algorithms for the modern diagnosis of epilepsy in real time. The source code used to carry out the experiments is publicly available at CHB-MIT EEG Dataset Python Scripts (https://www.kaggle.com/code/adnankust/adnaneeg1).

Tanzania heavily depends on United States Dollar (USD) foreign currency to import various goods and services into the country. Failure to correctly forecast exchange rates between USD and Tanzanian Shillings (TZS) may pose risks such as inability to import intended goods and services, possibility of losing money in stock exchange markets and other investment businesses in case of unexpected currency appreciation or depreciation as well as poor investment decisions in foreign exchange markets. To address this, this study has developed and comparatively evaluated performances of LSTM (Long Short-Term Memory) and BiLSTM (Bidirectional LSTM) deep learning models for forecasting daily USD to TZS exchange rates. The findings reveal that, BiLSTM model outperforms LSTM in forecasting daily USD to TZS exchange rates, achieving a MAPE (Mean Absolute Percentage Error) score of 0.363 on test set (unseen data) compared to a MAPE score of 1.471 achieved by LSTM model. This study recommends to the prospective Artificial Intelligence (AI) researchers and software developers to use BiLSTM instead of LSTM model to forecast (predict) USD to TZS exchange rates. Also, this study has developed USD to TZS exchange rates dataset which can be used by AI researchers, saving them time and costs involved with creating datasets from scratch. This study has also developed ready to use BiLSTM and LSTM models which can be used by Tanzanian business men and women involved in stock exchange markets, foreign exchange markets and other businesses, to predict daily USD to TZS exchange rates and make appropriate business and investment decisions.

Validating the robustness of an internet of things based atrial fibrillation detection system

BackgroundPredicting hypoglycemia while maintaining a low false alarm rate is a challenge for the wide adoption of continuous glucose monitoring (CGM) devices in diabetes management. One small study suggested that a deep learning model based on the long short-term memory (LSTM) network had better performance in hypoglycemia prediction than traditional machine learning algorithms in European patients with type 1 diabetes. However, given that many well-recognized deep learning models perform poorly outside the training setting, it remains unclear whether the LSTM model could be generalized to different populations or patients with other diabetes subtypes.ObjectiveThe aim of this study was to validate LSTM hypoglycemia prediction models in more diverse populations and across a wide spectrum of patients with different subtypes of diabetes.MethodsWe assembled two large data sets of patients with type 1 and type 2 diabetes. The primary data set including CGM data from 192 Chinese patients with diabetes was used to develop the LSTM, support vector machine (SVM), and random forest (RF) models for hypoglycemia prediction with a prediction horizon of 30 minutes. Hypoglycemia was categorized into mild (glucose=54-70 mg/dL) and severe (glucose<54 mg/dL) levels. The validation data set of 427 patients of European-American ancestry in the United States was used to validate the models and examine their generalizations. The predictive performance of the models was evaluated according to the sensitivity, specificity, and area under the receiver operating characteristic curve (AUC).ResultsFor the difficult-to-predict mild hypoglycemia events, the LSTM model consistently achieved AUC values greater than 97% in the primary data set, with a less than 3% AUC reduction in the validation data set, indicating that the model was robust and generalizable across populations. AUC values above 93% were also achieved when the LSTM model was applied to both type 1 and type 2 diabetes in the validation data set, further strengthening the generalizability of the model. Under different satisfactory levels of sensitivity for mild and severe hypoglycemia prediction, the LSTM model achieved higher specificity than the SVM and RF models, thereby reducing false alarms.ConclusionsOur results demonstrate that the LSTM model is robust for hypoglycemia prediction and is generalizable across populations or diabetes subtypes. Given its additional advantage of false-alarm reduction, the LSTM model is a strong candidate to be widely implemented in future CGM devices for hypoglycemia prediction.

The effective integration of photovoltaic (PV) systems into the grid which facilitates better planning and resource allocation depends on accurate day-ahead solar power forecasts. This paper proposes a Long Short-Term Memory (LSTM) model for forecasting solar power generation one day in advance with the algorithm inspired by Ant Colony Optimization (ACO). A year's worth of real-time data for a 4.5 kW PV system is used to train and test the model with input characteristics including temperature, PV power production and solar irradiation. Over the course of three seasons—summer, winter, and monsoon—the performance of the ACO-optimized LSTM model is contrasted with that of a simple LSTM model and an LSTM model enhanced by particle swarm optimization (PSO). This encompasses metrics like R-squared (R²) values, Mean Absolute Error (MAE) and Root Mean Square Error (RMSE). The PSO-optimized LSTM model (MAE of 0.31 kW, RMSE of 0.38 kW and R² of 0.88) and the baseline LSTM model (MAE of 0.35 kW, RMSE of 0.42 kW and R² of 0.85) couldn't match the ACO-optimized LSTM model's summertime MAE of 0.27 kW, RMSE of 0.33 kW and R² of 0.92. Similar trends also surfaced throughout the monsoon and winter seasons, with the ACO-optimized model consistently outperforming the others. In the winter, for instance, it obtained an R² of 0.93, an RMSE of 0.26 kW and an MAE of 0.22 kW. With regard to hyper parameter optimization and overall performance, these findings demonstrate the ACO-optimized LSTM model's superiority over the baseline and PSO-optimized LSTM models highlighting its improved prediction precision for all seasons. The effectiveness of ACO in bolstering solar power forecast models is confirmed by this study and provides a solid basis for deep learning model improvement in practical PV applications.

An accurate electrical Short-term Load Forecasting (STLF) is an eminent factor in the power generation, electrical load dispatching and energy planning for the power supply companies, specifically in developing countries. This paper proposes a novel temporal feature selection-based Long Short-term Memory (LSTM) model developed by the combination of standard Artificial Neural Network (ANN) layer and LSTM for electrical short term load forecasting. The LSTM model has excellent capability of predicting the stochastic nature of an hour ahead electrical loads. The standard ANN layer consisting 11 neurons is used as an input to LSTM cells. Such a combination of ANN layer with LSTM was never proposed before. The proposed model accommodates variations in weather as well as temporal inputs like humidity, holidays, and date-time features in the hourly load data of the power supply company situated in Johor, Malaysia. This paper gives the insights of hyper parameter tuning to capture the more generalized electrical load patterns in the dataset without compromising the time complexity of the proposed model. The proposed approach was compared with five existing approaches, namely: ANN, LSTM model 1, LSTM model 2, LSTM model 3 and Convolutional Neural Network-LSTM (CNN-LSTM) using hourly load dataset of Johor. The experimental results demonstrate that the proposed approach outperformed the existing approaches in terms of root mean square error, mean absolute percentage error and Diebold-Mariano statistical inference test within 95% confidence interval.

Abstract. The information available on the internet nowadays is diverse and moves very quickly. Information is becoming easier to obtain by the general public with the numerous online media outlets, including news portals that provide up-to-date information insights. Various news portals earn revenue from advertising using pay-per-click methods that encourage article writers to use clickbait techniques to attract visitors. However, the negative effects of clickbait include a decrease in journalism quality and the spread of hoaxes. This problem can be prevented by using text classification to classify clickbait in news titles. One method that can be used for text classification is a neural network. Artificial neural networks use algorithms that can independently adjust input coefficient weights. This makes this algorithm highly effective for modeling non-linear statistical data. The artificial neural network algorithm, especially the Long Short-Term Memory (LSTM), has been widely used in various natural language processing fields with satisfying results, including text classification. To improve the performance of the neural network model, adjustments can be made to the model's hyperparameters. Hyperparameters are parameters that cannot be obtained through data and must be defined before the training process. In this research, the Long Short-Term Memory (LSTM) model was used in clickbait classification in news titles. Sixteen neural network models were trained with different hyperparameter configurations for each model. Hyperparameter tuning was carried out using the random search algorithm. The dataset used was the CLICK-ID dataset published by William &amp; Sari, 2020[1], with a total of 15,000 annotated data. The research results show that the developed LSTM model has a validation accuracy of 0.8030, higher than William &amp; Sari's research, and a validation loss of 0.4876. Using this model, researchers were able to classify clickbait in news titles with fairly good accuracy. Purpose: The study was to develop and evaluate a LSTM model with hyperparameter tuning for clickbait classification on news headlines. The thesis also aims to compare the performance of simple LSTM and bidirectional LSTM for this task. Methods: This study uses CLICK-ID dataset and applies different text preprocessing techniques. The dataset later was used to build and train 16 LSTM models with different hyperparameters and evaluates them using validation accuracy and loss. This study uses random search for hyperparameter tuning. Result: The results of the study show that the best model for clickbait classification on news headlines is a bidirectional LSTM model with one layer, 64 units, 0.2 dropout rate, and 0.001 learning rate. This model achieves a validation accuracy of 0.8030 and a validation loss of 0.4876. The results also show that hyperparameter tuning using random search can improve the performance of the LSTM models by avoiding zero probabilities and finding the optimal values for the hyperparameters. Novelty: This study compares and analyzes the different preprocessing methods on text and the different configurations of the models to find the best model for clickbait classification on news headlines. The study also uses hyperparameter tuning to tune the model into the best model and finding the optimal values for the hyperparameters.

Wave runup prediction is necessary for offshore structure designs and early warnings. Data-driven methods based on machine learning have inspired reduced-order solutions for wave–structure interaction problems. This study provides the quantification of deep learning algorithms’ potential for wave runup prediction. Two prominent deep learning models were utilized to predict the wave runups along the fore column of semisubmersible under head seas. The long short-term memory (LSTM) and the temporal convolutional networks (TCNs) were comprehensively compared based on the datasets from a model test carried out in the deep ocean basin. The LSTM and TCN model structures were optimized to compare prediction accuracy and computational complexity reasonably. The results reveal that (1) both developed TCN and LSTM models had a satisfied prediction accuracy of over 90%. Their predictions were extended to 10 s into the future with accuracies over 80% and 45%, respectively. (2) With the noise-extended datasets, the LSTM model was robust to noises, while the TCN model showed better prediction performance on the extreme wave runup events. (3) The incident wave and dominant rotation provided the major information for wave runup prediction. TCN and LSTM models’ prediction accuracies were 91.5% and 89.3% based on the simplified input tensors composed of incident wave and pitch. The comparison showed the great potential of the TCN model to predict the nonlinear wave runup with less time and memory costs. The input tensors’ design and optimization based on physical understanding also play a significant role in the prediction performance.

The volatility of Australian companies' stock prices in 2020, caused by China's trade restrictions, poses a significant challenge for predicting financial gain or loss. This research contributes to future scholarship in predicting stock prices under specific circumstances or during special time periods. The study proposes a novel approach to stock price prediction, incorporating news sentiment analysis into a deep learning model. The research collected news items potentially affecting the stock price, incorporating them into an analysis model to generate a new feature for the Long Short-Term Memory (LSTM) model. The LSTM model used in this study was bidirectional, with two sets of gates per layer, and a three-layer model with different units. Each layer employed a dropout layer and a dense layer in the final stage. The study also utilized the feature engineering of lookback, selecting a window of time in the past to predict the next day's stock prices. Following multiple hyperparameter tunings and feature engineering adjustments, the results and graphs demonstrate a successful prediction for all three of the chosen companies, even during an unstable stock market. The overall trend lines achieve optimal predictions for the stock prices, illustrating both upward and downward trends.

The long short-term memory (LSTM) model is widely used in multiple areas, mainly for speech recognition, natural language processing and activity recognition. In the last few years, we started to see many variants of LSTM for recurrent neural networks since its inception in 1997. However, there weren’t many studies that have addressed the LSTM’s gating mechanism. In this paper, we propose a novel LSTM framework where we modify the architecture of the LSTM unit by adding a new layer that we call the “outlier gate”. The latter controls the flow of information that goes into the LSTM cell. This added signal allows us to avoid both the carry-over effect that the outliers have on the forecasted point and a bias in the estimates of our LSTM model – caused by unusual or non-repetitive events. The proposed architecture led us to an end-to-end trainable model that we applied in this paper to a financial time-series forecasting problem. Our results demonstrate that the new proposed LSTM architecture achieves better performance than the state-of-the-art original LSTM model.

In recent years, various types of Intelligent Transportation Systems(ITSs) have produced a large amount of structured traffic flow data. Using these structured data and deep learning models to predict urban short-term traffic flow is an important focus of current research. In order to solve the problem of low accuracy of Long-Short Term Memory(LSTM) model in urban traffic flow prediction, this paper proposes an LSTM model based on attention mechanism. The proposed model can learn the importance of each past value to the current value from the long sequence of traffic data at the past moment, which makes it possible to extract more valuable features. Constructed a dataset using the traffic data in the core section of Wuhan for experiments, and the performance of the improved model is compared with the original LSTM model. The results show that when the input data sequence increases from 16 to 64, the MAPE of our proposed model is reduced by 3.76%, while the MAPE of the LSTM model is reduced by 1.51%, which proves the effectiveness of our proposed method.

The traffic matrix (TM) is an important type of information in network management, which is needed in network load balancing and routing configuration. Due to technical and cost reasons, it is difficult to directly measure the matrix, but we can use prediction instead of direct measurement. The Long Short-Term Memory(LSTM) model in deep learning is very suitable for time series forecasting problems. However, due to the characteristics of deep learning, the data samples used to train the network will leave their own traces on the final model, which allows attackers to restore training samples through member inference attacks, causing privacy leakage. This paper focuses on combining the differential privacy mechanism with the LSTM model. In the gradient descent stage of training, a well-controlled noise is added to protect the final model. In the experiments, we verify the feasibility of the method proposed in this paper on the data set Abilene.KeywordsTraffic matrix predictionLong short-term memoryDifferential privacy

Water quality prediction (WQP) plays an essential role in water quality management for aquaculture to make aquaculture production profitable and sustainable. In this work, we propose hybrid deep learning (DL) models, convolutional neural network (CNN) with the long short-term memory (LSTM) and gated recurrent unit (GRU) for aquaculture WQP. CNN can effectively fetch the aquaculture water quality characteristics, whereas GRU and LSTM can learn long-term dependencies in the time series data. We conduct experiments using the two different water quality datasets and present an extensive study on the impact of hyperparameters on the performance of the proposed hybrid DL models. Furthermore, the performance of hybrid CNN-LSTM and CNN-GRU models are compared with different baseline LSTM, GRU and CNN DL models and also with attention-based LSTM and attention-based GRU DL models. The results show that the hybrid CNN-LSTM outperformed all other models in terms of prediction accuracy and computation time.

Abstract. Contamination of surface waters with microbiological pollutants is a major concern to public health. Although long-term and high-frequency Escherichia coli (E. coli) monitoring can help prevent diseases from fecal pathogenic microorganisms, such monitoring is time-consuming and expensive. Process-driven models are an alternative means for estimating concentrations of fecal pathogens. However, process-based modeling still has limitations in improving the model accuracy because of the complexity of relationships among hydrological and environmental variables. With the rise of data availability and computation power, the use of data-driven models is increasing. In this study, we simulated fate and transport of E. coli in a 0.6 km2 tropical headwater catchment located in the Lao People's Democratic Republic (Lao PDR) using a deep-learning model and a process-based model. The deep learning model was built using the long short-term memory (LSTM) methodology, whereas the process-based model was constructed using the Hydrological Simulation Program–FORTRAN (HSPF). First, we calibrated both models for surface as well as for subsurface flow. Then, we simulated the E. coli transport with 6 min time steps with both the HSPF and LSTM models. The LSTM provided accurate results for surface and subsurface flow with 0.51 and 0.64 of the Nash–Sutcliffe efficiency (NSE) values, respectively. In contrast, the NSE values yielded by the HSPF were −0.7 and 0.59 for surface and subsurface flow. The simulated E. coli concentrations from LSTM provided the NSE of 0.35, whereas the HSPF gave an unacceptable performance with an NSE value of −3.01 due to the limitations of HSPF in capturing the dynamics of E. coli with land-use change. The simulated E. coli concentration showed the rise and drop patterns corresponding to annual changes in land use. This study showcases the application of deep-learning-based models as an efficient alternative to process-based models for E. coli fate and transport simulation at the catchment scale.