Data-driven approaches for runoff prediction using distributed data

Compressive strength is an important mechanical property of high-strength concrete (HSC), but testing methods are usually uneconomical, time-consuming, and labor-intensive. To this end, in this paper, a long short-term memory (LSTM) model was proposed to predict the HSC compressive strength using 324 data sets with five input independent variables, namely water, cement, fine aggregate, coarse aggregate, and superplasticizer. The prediction results were compared with those of the conventional support vector regression (SVR) model using four metrics, root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and correlation coefficient (R2). The results showed that the prediction accuracy and reliability of LSTM were higher with R2 = 0.997, RMSE = 0.508, MAE = 0.08, and MAPE = 0.653 compared to the evaluation metrics R2 = 0.973, RMSE = 1.595, MAE = 0.312, MAPE = 2.469 of the SVR model. The LSTM model is recommended for the pre-estimation of HSC compressive strength under a given mix ratio before the laboratory compression test. Additionally, the Shapley additive explanations (SHAP)-based approach was performed to analyze the relative importance and contribution of the input variables to the output compressive strength.

Electricity is the most substantial energy form that significantly affects the development of modern life, work efficiency, quality of life, production, and competitiveness of the society in the ever-growing global world. In this respect, forecasting accurate electricity energy consumption (EEC) is fairly essential for any country?s energy consumption planning and management regarding its growth. In this study, four time-series methods; long short-term memory (LSTM) neural network, adaptive neuro-fuzzy inference system (ANFIS) with subtractive clustering (SC), ANFIS with fuzzy cmeans (FCM), and ANFIS with grid partition (GP) were implemented for the short-term one-day ahead EEC prediction. Root mean square error (RMSE), correlation coefficient (R), mean absolute error (MAE) and mean absolute percentage error (MAPE) were considered as statistical accuracy criteria. Those forecasted results by the LSTM, ANFIS-FCM, ANFIS-SC and ANFIS-GP models were evaluated by comparing with the actual data using statistical accuracy metrics. According to the testing process, the best MAPE values were obtained to be 4.47%, 3.21%, 2.34%, and 1.91% for the ANFIS-GP, ANFIS-SC, ANFIS-FCM, and LSTM, respectively. Furthermore, the best RMSE values were found as 25.94 GWh, 41.17 GWh, 29.50 GWh, and 80.14 GWh for the LSTM, ANFIS-SC, ANFIS-FCM, and ANFIS-GP models, respectively. As a consequence, the LSTM model generally outperformed all ANFIS models. The results revealed that forecasting of short-term daily EEC time series using the LSTM approach can provide high accuracy results.

Many countries around the globe, including Tanzania, are investing in renewable energy, partly because of the need to increase electric power generation and meet the ever-increasing electricity demands, and partly because of the need to combat global climate change caused by carbon emissions from fossil fuel-based electricity production. Kishapu Solar Project, located in Kishapu District, Shinyanga Region, Tanzania, is one of the renewable energy projects with a planned generation of 150 MW of electric power once completed, using solar photovoltaic technology. Accurate forecasting of solar radiation is important in managing and maintaining solar energy plants. Inaccurate forecasting of solar radiation can lead to poor harvest of solar energy and poor management of solar energy plants, which directly impacts electricity contribution to the national electricity grid. Long Short-Term Memory (LSTM) and its variant, PeepHole LSTM, are examples of Deep Learning models that can be used to predict solar radiation. The purpose of PeepHole LSTM is to introduce “peephole” connections in the LSTM model, which allow LSTM gates to directly inspect memory cell state and hence have richer information to process. However, there is still a research gap on the impact of “peephole” connections on the prediction performance of the LSTM model, especially in the context of forecasting solar radiation in Kishapu District, where the Kishapu Solar Project is located. The purpose of this study is to fill this research gap by developing both LSTM and PeepHole LSTM models and comparing their performances in forecasting solar radiation in Kishapu District. The 14-year Kishapu District daily solar radiation (Downward Shortwave Radiation) dataset was first pre-processed by scaling (normalizing) it using Standard Scaler and then split into training set (the first 70%), validation set (the next 15%) and test set (the last 15%) before being utilized to train and test the developed LSTM and PeepHole LSTM models. Both LSTM and PeepHole LSTM models used the previous 5 days' solar radiation values as inputs to predict the output (next day (6th day) solar radiation value). The trained and better-performing model between the two was finally saved in .h5 format and integrated with a Gradio based Web Application to provide a user interface for officials in Kishapu District and at Kishapu Solar Project to predict daily solar radiation. The findings from performance evaluation revealed that the LSTM model outperformed the PeepHole LSTM model in predicting solar radiation by achieving test Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) scores of 26.22 W/m2 and 36.65 W/m2, respectively, compared to MAE and RMSE scores of 30.84 W/m2 and 41.88 W/m2, respectively, achieved by the PeepHole LSTM model, concluding that “peephole” connections reduce the performance of the LSTM model rather than increasing it when used to forecast solar radiation in Kishapu District. This study’s recommendation to Artificial Intelligence (AI) software developers and researchers is to use the LSTM rather than the PeepHole LSTM Deep Learning model when forecasting solar radiation in Kishapu District and in locations with similar climatic conditions

Hand, foot, and mouth disease (HFMD) is a common febrile illness caused by enteroviruses in the Picornaviridae family. The major symptoms of HFMD are fever and a vesicular rash on the hand, foot, or oral mucosa. Acute meningitis and encephalitis are observed in rare cases. HFMD epidemics occur annually in Japan, usually in the summer season. Relatively large-scale outbreaks have occurred every two years since 2011. In this study, the epidemic patterns of HFMD in Japan are predicted four weeks in advance using a deep learning method. The time-series data were analyzed by a long short-term memory (LSTM) approach called a Recurrent Neural Network. The LSTM model was trained on the numbers of weekly HFMD cases in each prefecture. These data are reported in the Infectious Diseases Weekly Report, which compiles the national surveillance data from web sites at the National Institute of Infectious Diseases, Japan, under the Infectious Diseases Control Law. Consequently, our trained LSTM model distinguishes between relatively large-scale and small-scale epidemics. The trained model predicted the HFMD epidemics in 2018 and 2019, indicating that the LSTM approach can estimate the future epidemic patterns of HFMD in Japan.

Durability estimation and short-term voltage degradation forecasting of vehicle PEMFC system: Development and evaluation of machine learning models

Extracting cybersecurity entities and the relationships between them from online textual resources such as articles, bulletins, and blogs and converting these resources into more structured and formal representations has important applications in cybersecurity research and is valuable for professional practitioners. Previous works to accomplish this task were mainly based on utilizing feature-based models. Feature-based models are time-consuming and need labor-intensive feature engineering to describe the properties of entities, domain knowledge, entity context, and linguistic characteristics. Therefore, to alleviate the need for feature engineering, we propose the usage of neural network models, specifically the long short-term memory (LSTM) models to accomplish the tasks of Named Entity Recognition (NER) and Relation Extraction (RE). We evaluated the proposed models on two tasks. The first task is performing NER and evaluating the results against the state-of-the-art Conditional Random Fields (CRFs) method. The second task is performing RE using three LSTM models and comparing their results to assess which model is more suitable for the domain of cybersecurity. The proposed models achieved competitive performance with less feature-engineering work. We demonstrate that exploiting neural network models in cybersecurity text mining is effective and practical.

Reservoirs and dams are vital human-built infrastructures that play essential roles in flood control, hydroelectric power generation, water supply, navigation, and other functions. The realization of those functions requires efficient reservoir operation, and the effective controls on the outflow from a reservoir or dam. Over the last decade, artificial intelligence (AI) techniques have become increasingly popular in the field of streamflow forecasts, reservoir operation planning and scheduling approaches. In this study, three AI models, namely, the backpropagation (BP) neural network, support vector regression (SVR) technique, and long short-term memory (LSTM) model, are employed to simulate reservoir operation at monthly, daily, and hourly time scales, using approximately 30 years of historical reservoir operation records. This study aims to summarize the influence of the parameter settings on model performance and to explore the applicability of the LSTM model to reservoir operation simulation. The results show the following: (1) for the BP neural network and LSTM model, the effects of the number of maximum iterations on model performance should be prioritized; for the SVR model, the simulation performance is directly related to the selection of the kernel function, and sigmoid and RBF kernel functions should be prioritized; (2) the BP neural network and SVR are suitable for the model to learn the operation rules of a reservoir from a small amount of data; and (3) the LSTM model is able to effectively reduce the time consumption and memory storage required by other AI models, and demonstrate good capability in simulating low-flow conditions and the outflow curve for the peak operation period.

The long short-term memory (LSTM) model is widely used in multiple areas, mainly for speech recognition, natural language processing and activity recognition. In the last few years, we started to see many variants of LSTM for recurrent neural networks since its inception in 1997. However, there weren’t many studies that have addressed the LSTM’s gating mechanism. In this paper, we propose a novel LSTM framework where we modify the architecture of the LSTM unit by adding a new layer that we call the “outlier gate”. The latter controls the flow of information that goes into the LSTM cell. This added signal allows us to avoid both the carry-over effect that the outliers have on the forecasted point and a bias in the estimates of our LSTM model – caused by unusual or non-repetitive events. The proposed architecture led us to an end-to-end trainable model that we applied in this paper to a financial time-series forecasting problem. Our results demonstrate that the new proposed LSTM architecture achieves better performance than the state-of-the-art original LSTM model.

Wearable electroencephalography (EEG) devices for seizure detection accuracy and reliability are deep learning (DL) applications in the field of epilepsy diagnosis. In this study, we sought to increase the accuracy of seizure detection using advanced DL algorithms on the Children’s Hospital Boston - Massachusetts Institute of Technology (CHB-MIT) EEG database. First, a fully convolutional network (FCN) was trained and assessed using accuracy and recall/precision metrics, and the early stopping technique was used to avoid overfitting. To assess the performance, the FCN was evaluated in terms of various metrics, including accuracy, precision, recall, F1-score, and receiver operating characteristic (ROC)-area under the curve (AUC). In addition, two-dimensional (2D) convolutional neural networks (CNNs) and long short-term memory (LSTM) models were used to model the database, and their performance was thoroughly measured using different metrics, graphs, and confusion matrices. Using LSTM variants, such as standard LSTM, bidirectional LSTM, stacked LSTM, and LSTM attention mechanisms, hybrid convolutional LSTM (ConvLSTM) models were trained and compared. The comparison was conducted based on the training and validation accuracy and loss, as well as the graphs resulting from the precision–recall curves. Apart from DL approaches, EEG signal analysis using time–frequency techniques, such as wavelet transform and short-time Fourier transform, has also been investigated. These methods assisted in the analysis of the time–frequency features of EEG signals in combination with DL models. This study demonstrates that the performance of wearable EEG devices can be augmented using a combination of DL and seizure signal processing techniques. The FCN achieved an accuracy of 92%, with a recall for seizures of 33%, an F1-score of 0.03, and strong ROC-AUC results. The 2D CNN achieved 96% accuracy, a seizure recall of 70%, an F1-score of 0.12, and an ROC-AUC score of 78%. The baseline LSTM struggled with effectiveness at 53% accuracy with a seizure recall of 18%. In contrast, the LSTM model, which incorporated synthetic minority oversampling technique (SMOTE) balancing, was able to reach improvements of up to 89% accuracy, with a precision of 91%, a recall of 86%, an F1-score of 0.89, and a strong ROC curve performance. Among the models, the LSTM with SMOTE was the best performer, with 89% accuracy, 91% precision, 86% recall, and an F1-score of 0.89. These results provide evidence that applying techniques for data balancing in combination with certain DL network architectures significantly improves the detection of seizures using wearable EEG devices worn on the body. We believe that real-time monitoring and high-performance systems are feasible using optimized DL frameworks. The analysis of the performance of different models allows for the understanding of the possibilities of optimizing the architectures of DL algorithms for the modern diagnosis of epilepsy in real time. The source code used to carry out the experiments is publicly available at CHB-MIT EEG Dataset Python Scripts (https://www.kaggle.com/code/adnankust/adnaneeg1).

An accurate electrical Short-term Load Forecasting (STLF) is an eminent factor in the power generation, electrical load dispatching and energy planning for the power supply companies, specifically in developing countries. This paper proposes a novel temporal feature selection-based Long Short-term Memory (LSTM) model developed by the combination of standard Artificial Neural Network (ANN) layer and LSTM for electrical short term load forecasting. The LSTM model has excellent capability of predicting the stochastic nature of an hour ahead electrical loads. The standard ANN layer consisting 11 neurons is used as an input to LSTM cells. Such a combination of ANN layer with LSTM was never proposed before. The proposed model accommodates variations in weather as well as temporal inputs like humidity, holidays, and date-time features in the hourly load data of the power supply company situated in Johor, Malaysia. This paper gives the insights of hyper parameter tuning to capture the more generalized electrical load patterns in the dataset without compromising the time complexity of the proposed model. The proposed approach was compared with five existing approaches, namely: ANN, LSTM model 1, LSTM model 2, LSTM model 3 and Convolutional Neural Network-LSTM (CNN-LSTM) using hourly load dataset of Johor. The experimental results demonstrate that the proposed approach outperformed the existing approaches in terms of root mean square error, mean absolute percentage error and Diebold-Mariano statistical inference test within 95% confidence interval.

Abstract. The information available on the internet nowadays is diverse and moves very quickly. Information is becoming easier to obtain by the general public with the numerous online media outlets, including news portals that provide up-to-date information insights. Various news portals earn revenue from advertising using pay-per-click methods that encourage article writers to use clickbait techniques to attract visitors. However, the negative effects of clickbait include a decrease in journalism quality and the spread of hoaxes. This problem can be prevented by using text classification to classify clickbait in news titles. One method that can be used for text classification is a neural network. Artificial neural networks use algorithms that can independently adjust input coefficient weights. This makes this algorithm highly effective for modeling non-linear statistical data. The artificial neural network algorithm, especially the Long Short-Term Memory (LSTM), has been widely used in various natural language processing fields with satisfying results, including text classification. To improve the performance of the neural network model, adjustments can be made to the model's hyperparameters. Hyperparameters are parameters that cannot be obtained through data and must be defined before the training process. In this research, the Long Short-Term Memory (LSTM) model was used in clickbait classification in news titles. Sixteen neural network models were trained with different hyperparameter configurations for each model. Hyperparameter tuning was carried out using the random search algorithm. The dataset used was the CLICK-ID dataset published by William & Sari, 2020[1], with a total of 15,000 annotated data. The research results show that the developed LSTM model has a validation accuracy of 0.8030, higher than William & Sari's research, and a validation loss of 0.4876. Using this model, researchers were able to classify clickbait in news titles with fairly good accuracy. Purpose: The study was to develop and evaluate a LSTM model with hyperparameter tuning for clickbait classification on news headlines. The thesis also aims to compare the performance of simple LSTM and bidirectional LSTM for this task. Methods: This study uses CLICK-ID dataset and applies different text preprocessing techniques. The dataset later was used to build and train 16 LSTM models with different hyperparameters and evaluates them using validation accuracy and loss. This study uses random search for hyperparameter tuning. Result: The results of the study show that the best model for clickbait classification on news headlines is a bidirectional LSTM model with one layer, 64 units, 0.2 dropout rate, and 0.001 learning rate. This model achieves a validation accuracy of 0.8030 and a validation loss of 0.4876. The results also show that hyperparameter tuning using random search can improve the performance of the LSTM models by avoiding zero probabilities and finding the optimal values for the hyperparameters. Novelty: This study compares and analyzes the different preprocessing methods on text and the different configurations of the models to find the best model for clickbait classification on news headlines. The study also uses hyperparameter tuning to tune the model into the best model and finding the optimal values for the hyperparameters.

The absorption of capillary water is one of the most crucial factors in the flow of groundwater in rocks (CWA). Although meticulous experimental studies are needed to determine a rock’s CWA, predictive techniques might cut down on the expense and effort. There are various data mining methods for this purpose, but the considered algorithms in this study were not proposed so far for predicting the CWA. Different rock samples were taken for this purpose from various locations, yielding diverse rocks. For the prediction procedures, four support vector regression (SVR) models were created: a traditional SVR, two ensembled models, and a hybrid SVR model using the whale optimization technique (WOA - SVR). Results show that all models have acceptable performance in predicting the CWA with R2 larger than 0.797 and 0.806 for the training and testing data, respectively, representing the acceptable correlation between observed and predicted values. Regarding developed models, the conventional SVR model has the worst performance of all models. All statistical evaluation criteria were improved by assembling models, which present the ability of additive regression and bagging predictions in improving prediction processes. The hybrid WOA - SVR model has the best performance considering all indices. This hybrid model could also gain the lowest values of error indices between all SVR models, which leads to outperforming the WOA - SVR model compared to other methods.

Case studies of group discussions are considered an effective way to assess communication skills (CS). This method can help researchers evaluate participants’ engagement with each other in a specific realistic context. In this article, multimodal analysis was performed to estimate CS indices using a three-task-type group discussion dataset, the MATRICS corpus. The current research investigated the effectiveness of engaging both static and time-series modeling, especially in task-independent settings. This investigation aimed to understand three main points: first, the effectiveness of time-series modeling compared to nonsequential modeling; second, multimodal analysis in a task-independent setting; and third, important differences to consider when dealing with task-dependent and task-independent settings, specifically in terms of modalities and prediction models. Several modalities were extracted (e.g., acoustics, speaking turns, linguistic-related movement, dialog tags, head motions, and face feature sets) for inferring the CS indices as a regression task. Three predictive models, including support vector regression (SVR), long short-term memory (LSTM), and an enhanced time-series model (an LSTM model with a combination of static and time-series features), were taken into account in this study. Our evaluation was conducted by using the R 2 score in a cross-validation scheme. The experimental results suggested that time-series modeling can improve the performance of multimodal analysis significantly in the task-dependent setting (with the best R 2 = 0.797 for the total CS index), with word2vec being the most prominent feature. Unfortunately, highly context-related features did not fit well with the task-independent setting. Thus, we propose an enhanced LSTM model for dealing with task-independent settings, and we successfully obtained better performance with the enhanced model than with the conventional SVR and LSTM models (the best R 2 = 0.602 for the total CS index). In other words, our study shows that a particular time-series modeling can outperform traditional nonsequential modeling for automatically estimating the CS indices of a participant in a group discussion with regard to task dependency.

One-step vs horizon-step training strategies for multi-step traffic flow forecasting with direct particle swarm optimization grid search support vector regression and long short-term memory

In recent years, various types of Intelligent Transportation Systems(ITSs) have produced a large amount of structured traffic flow data. Using these structured data and deep learning models to predict urban short-term traffic flow is an important focus of current research. In order to solve the problem of low accuracy of Long-Short Term Memory(LSTM) model in urban traffic flow prediction, this paper proposes an LSTM model based on attention mechanism. The proposed model can learn the importance of each past value to the current value from the long sequence of traffic data at the past moment, which makes it possible to extract more valuable features. Constructed a dataset using the traffic data in the core section of Wuhan for experiments, and the performance of the improved model is compared with the original LSTM model. The results show that when the input data sequence increases from 16 to 64, the MAPE of our proposed model is reduced by 3.76%, while the MAPE of the LSTM model is reduced by 1.51%, which proves the effectiveness of our proposed method.