Deep Learning for Fraud Detection in Real-Time Transaction Networks

This article explores the suitability of a long short-term memory recurrent neural network (LSTM-RNN) and artificial intelligence (AI) method for low-flow time series forecasting. The long short-term memory works on the sequential framework which considers all of the predecessor data. This forecasting method used daily discharged data collected from the Basantapur gauging station located on the Mahanadi River basin, India. Different metrics [root-mean-square error (RMSE), Nash–Sutcliffe efficiency (ENS), correlation coefficient (R) and mean absolute error] were selected to assess the performance of the model. Additionally, recurrent neural network (RNN) model is also used to compare the adaptability of LSTM-RNN over RNN and naive method. The results conclude that the LSTM-RNN model (R = 0.943, ENS = 0.878, RMSE = 0.487) outperformed RNN model (R = 0.935, ENS = 0.843, RMSE = 0.516) and naive method (R = 0.866, ENS = 0.704, RMSE = 0.793). The finding of this research concludes that LSTM-RNN can be used as new reliable AI technique for low-flow forecasting.

Remaining useful life prediction of PEMFC based on long short-term memory recurrent neural networks

Non-intrusive load monitoring system (NILMS) have become a hot research topic. However, the majority of existing methods for NILM are unreliability, especially, the generalization of model and the accuracy of identification in the multi-load with the similar load energy consumption values. This paper proposes a reliable algorithm architecture fused deep learning and transfer learning to improve the reliability of load combination identification model. In the method, a load event outlier detection way based on adaptive sliding-window (ASW) is presented to make better use of experimental data; a feature extractor based on deep convolutional neural networks (DNN) is used to complete the high-quality (Hi-Q) representation of on-line multi-load features; the long short-term memory recurrent neural network (LSTM-RNN) with transfer learning is used to address the generalization of model and the inaccuracy of load combination identification problems. The three data sets from the different domain are employed for training, testing and verifying of the proposed model. The experimental results show that the proposed method can significantly improve the performance of model in NILMS.

With more and more frequency, users communicate with each other on social media. Many users start on Twitter or Facebook to find friends who have the same hobby. Our study proposes a method to estimate the users’ interests (hobby) based on tweets on Twitter. One tweet does not, in and of itself, contain a lot of information, and some tweets are not related to the user’s hobby. Therefore, we propose a reliable hobby estimation method by extracting features from multiple, sequential tweets. The proposed method uses Recurrent Neural Networks (RNN) which can accommodate time-series information. We also used a Convolutional Neural Networks (CNN) which can treat contextual information. We used an averaged vector of word distributed representation as a feature. Using the proposed method based on Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN), we obtained a 23.72% improvement as compared with a baseline method using a Random Forest (RF) regression as a machine learning algorithm.

In this paper, an empirical evaluation of different classification techniques is conducted on user movement data. The datasets used here for experiments are composed of accelerometer data collected from various devices, including smartphones and smart watches. The user movement data was processed and fed into five traditional machine learning algorithms. The classification performances were then compared with a deep learning technique, the Long Short Term Memory-Recurrent Neural Network (LSTM-RNN). LSTM-RNN achieved its highest accuracy at 89% as opposed to 97% from a traditional machine learning algorithm, specifically, K-Nearest Neighbors (k-NN), on wrist-worn accelerometer data.

Lithium battery packs are the main driving energy source for electric vehicles. A battery pack equalization charging solution using a constant current source for variable rate charging is presented in this paper. The charging system consists of a main constant current source and independent auxiliary constant current sources. Auxiliary constant current sources are controlled by the battery management system (BMS), which can change the current rate of the corresponding single battery, and achieve full charging of each single cell in the series battery pack. At the same time, the state of charge (SOC) is regarded as time series data to establish a long short-term memory recurrent neural network (LSTM-RNN) model, and it is possible to obtain the single battery with lower capacity, so that the charging efficiency and battery pack consistency can be improved. The experimental results show that the open circuit voltage difference between the single cells is less than 50 mV after the charging of 20 strings of lithium battery packs by using this method, which achieve the purpose of equalization charging.

Automatic recognition of frustration, by analysing facial and vocal expressions, can help user experience designers to identify interaction obstacles. To encourage the development of automated systems such as these, we present a novel audiovisual database: the Multimodal Game Frustration Database (MGFD), consisting of ca. 5 hours of audiovisual data, collected from 67 Chinese students speaking in English. For data collection, we developed ‘Crazy Trophy’, a Wizard-of-Oz voice activated web-game designed with a variety of usability problems and aimed to induce increasing amounts of frustration. We also present a baseline for binary multimodal frustration classification (frustration vs no-frustration). For this, we compare the performance of a conventional method, Support Vector Machine classifier, and a state-of-the-art method utilising Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN), extracting both audio (Mel-frequency Cepstral Coefficients) and video (facial action units) features. Using LSTM-RNN and a feature-based multi-model fusion strategy, the best result acheived for the baseline was 60.3 % UAR. To enable further research in this area, the game (‘Crazy Trophy’), the database (MGFD), and the partitioning considered in the presented baseline, are made accessible to the research community.

With the evolution of network threat, identifying attack from both external and internal is getting more and more difficult. To detect both known and unknown malicious attacks, several machine learning algorithms are utilized. However, these algorithms have still some limitations such as high false positive and false alarm rate. To overcome above challenge, we propose a threat analysis method combined with kernel principal component analysis (PCA) and long short-term memory recurrent neural network (LSTM-RNN). To achieve high accurate detection rate, data preprocessing, feature extraction, attack detection is seamlessly integrated into an end-to-end detection method. To assess the method, the well-known NSL-KDD dataset has been used. Experimental results show that the proposed threat analysis method greatly outperforms several attack detection methods that use SVM, neural network or Bayesian methods.

In this paper, we propose a Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) and multiple kernel learning (MKL) based multi-modal affect recognition scheme (LSTM-MKL). It takes the LSTM-RNN advantage to model the long range dependencies between successive observations, and uses the MKL power to model the non-linear correlations between the inputs and outputs. For each of the affect dimensions (arousal, valence, expectancy, and power), two LSTM-RNN models are trained, one for each modality. In the recognition phase, the audio and visual features are input to the corresponding learned LSTM models, which in turn produce initial estimates of the affect dimensions. The LSTM outputs are further input into a multi-kernel support vector regression (MK-SVR) for the final recognition. Experimental results carried out on the AVEC2012 database, show that compared to the traditional SVR-LLR (Support Vector Machine - local linear regression) or MK-SVR fusion scheme, the proposed LSTM-MKL fusion scheme obtains higher recognition results, with an correlation coefficient (COR) of 0.354, compared to a COR of 0.124 for SVR-LLR, and 0.168 for MK-SVR, respectively.

Remaining useful life (RUL) prognosis is one of the most important techniques in concrete structure health management. This technique evaluates the concrete structure strength through determining the advent of failure, which is very helpful to reduce maintenance costs and extend structure life. Degradation information with the capability of reflecting structure health can be considered as a principal factor to achieve better prognosis performance. In traditional data-driven RUL prognosis, there are drawbacks in which features are manually extracted and threshold is defined to mark the specimen’s breakdown. To overcome these limitations, this paper presents an innovative SAE-DNN structure capable of automatic health indicator (HI) construction from raw signals. HI curves constructed by SAE-DNN have much better fitness metrics than HI curves constructed from statistical parameters such as RMS, Kurtosis, Sknewness, etc. In the next stage, HI curves constructed from training degradation data are then used to train a long short-term memory recurrent neural network (LSTM-RNN). The LSTM-RNN is utilized as a RUL predictor since its special gates allow it to learn long-term dependencies even when the training data is limited. Model construction, verification, and comparison are performed on experimental reinforced concrete (RC) beam data. Experimental results indicates that LSTM-RNN generally estimates more accurate RULs of concrete beams than GRU-RNN and simple RNN with the average prediction error cycles was less than half compared to those of the simple RNN.

Intrusion Detection System (IDS) is built to detect both known and unknown malicious attacks. Several machine learning algorithms are used widely in IDS such as neural network, SVM, KNN etc. However, these algorithms have still some limitations such as high false positive and false alarm rate. To overcome above challenge, we propose a novel effective attack detection method based on kernel principal component analysis (PCA) and long short-term memory recurrent neural network (LSTM-RNN). To achieve high accurate detection rate, data preprocessing, feature extraction, attack detection is seamlessly integrated into an end-to-end detection method. To assess the method, the well-known NSL-KDD dataset has been used. The results of experimental show that the proposed attack detection strategy greatly outperforms several attack detection strategy that use SVM, neural network or Bayesian methods.

Intrusion Detection Systems (IDS) play a vital role in monitoring and preventing unauthorized access within critical network infrastructures. However, current IDS approaches often suffer from limitations in detection accuracy, scalability, and adaptability to evolving threats. To address these challenges, this study proposes an optimized Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) model for network-based intrusion detection. The model is specifically designed to capture temporal dependencies in sequential network traffic data, enabling accurate classification of benign and malicious activity while maintaining computational efficiency. The proposed model underwent rigorous validation to assess both its detection accuracy and generalizability. This was achieved by utilizing two distinct datasets that encompass realistic network traffic and a wide array of attack scenarios, including Denial of Service (DoS), Probe, User-to-Root (U2R), and Remote-to-Local (R2L) intrusions. The performance of the model is compared with several other recent models. The proposed model achieved high classification results among all the compared models, with an accuracy of 99.94%, precision of 99.98%, recall of 99.93%, and an F1-score of 99.95%. These outstanding results underscore the model’s robust capability to accurately identify and flag malicious activities within complex network environments. The model’s generalization power was similarly affirmed on a second independent dataset. The model maintained strong performance with an accuracy of 99.42%, precision of 99.20%, recall of 99.69%, and an F1-score of 99.45%. The successful demonstration of both high detection accuracy and strong generalization capability highlights the practical utility and inherent reliability of our proposed model for real-world threat detection.

Deep underground laboratories offer advantages for conducting high-precision observations of weak geophysical signals, benefiting from a low background noise level. Enhancing strong, noisy ground electromagnetic (EM) field data using synchronously recorded underground EM signals, which typically exhibit a high signal-to-noise ratio, is both valuable and feasible. In this study, we propose an EM field reconstruction method employing a Long Short-Term Memory (LSTM) recurrent neural network with referenced deep underground EM observations. Initially, a deep learning model was developed to capture the time-varying features of underground multi-component EM fields using the LSTM recurrent neural network. Subsequently, this model was applied to process synchronously observed strong, noisy data from other conventional observation systems, such as those at the surface, to achieve noise suppression through signal reconstructions. Both the theoretical analysis and the practical observational data suggest that the proposed method effectively suppresses noise and reconstructs clean EM signals. This method is efficient and time-saving, representing an effective approach to fully utilizing the advantages of deep underground observation data. Furthermore, this method could be extended to the processing and analysis of other geophysical data.

Electricity load forecasting is a crucial undertaking within all the deregulated markets globally. Among the research challenges on a global scale, the investigation of deep transfer learning (DTL) in the field of electricity load forecasting represents a fundamental effort that can inform artificial intelligence applications in general. In this paper, a comprehensive study is reported regarding day-ahead electricity load forecasting. For this purpose, three sequence-to-sequence (Seq2seq) deep learning (DL) models are used, namely the multilayer perceptron (MLP), the convolutional neural network (CNN) and the ensemble learning model (ELM), which consists of the weighted combination of the outputs of MLP and CNN models. Also, the study focuses on the development of different forecasting strategies based on DTL, emphasizing the way the datasets are trained and fine-tuned for higher forecasting accuracy. In order to implement the forecasting strategies using deep learning models, load datasets from three Greek islands, Rhodes, Lesvos, and Chios, are used. The main purpose is to apply DTL for day-ahead predictions (1–24 h) for each month of the year for the Chios dataset after training and fine-tuning the models using the datasets of the three islands in various combinations. Four DTL strategies are illustrated. In the first strategy (DTL Case 1), each of the three DL models is trained using only the Lesvos dataset, while fine-tuning is performed on the dataset of Chios island, in order to create day-ahead predictions for the Chios load. In the second strategy (DTL Case 2), data from both Lesvos and Rhodes concurrently are used for the DL model training period, and fine-tuning is performed on the data from Chios. The third DTL strategy (DTL Case 3) involves the training of the DL models using the Lesvos dataset, and the testing period is performed directly on the Chios dataset without fine-tuning. The fourth strategy is a multi-task deep learning (MTDL) approach, which has been extensively studied in recent years. In MTDL, the three DL models are trained simultaneously on all three datasets and the final predictions are made on the unknown part of the dataset of Chios. The results obtained demonstrate that DTL can be applied with high efficiency for day-ahead load forecasting. Specifically, DTL Case 1 and 2 outperformed MTDL in terms of load prediction accuracy. Regarding the DL models, all three exhibit very high prediction accuracy, especially in the two cases with fine-tuning. The ELM excels compared to the single models. More specifically, for conducting day-ahead predictions, it is concluded that the MLP model presents the best monthly forecasts with MAPE values of 6.24% and 6.01% for the first two cases, the CNN model presents the best monthly forecasts with MAPE values of 5.57% and 5.60%, respectively, and the ELM model achieves the best monthly forecasts with MAPE values of 5.29% and 5.31%, respectively, indicating the very high accuracy it can achieve.

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.