Prediction Method for Lysine Acetylation Sites Based on LSTM Network

Feature mining for encrypted malicious traffic detection with deep learning and other machine learning algorithms

As one of the key components in mechanical systems, rotatory machine plays a significant role in safe and stable operation. Accurate prediction of the Remaining Useful Life (RUL) of rotatory machine contributes to realization of intelligent operation and maintenance for mechanical manufacturing. In order to overcome the limitations of traditional machine learning algorithms in dealing with complex nonlinear signals, a novel prediction framework for RUL of rotatory machine based on deep learning is proposed in this paper. One-dimensional convolutional neural network is utilized to extract local features from the original signal sequence. In addition, the proposed framework analyzes sensor signals and predicts RUL by combining Long Short-Term Memory (LSTM) network with attention mechanism. Multi-layer LSTM is set up to extract useful temporal features layer by layer and improve the robustness of the model, while attention mechanism is able to effectively solve the problem of information loss in the long-distance signal transmission of LSTM. Through the feature extraction of multi-layer LSTM and the strong supervision ability of attention mechanism, the RUL of rotatory machine can be accurately predicted. The experimental results show that the proposed method for RUL estimation is efficient and has higher prediction accuracy than the traditional machine learning algorithms.

Objective. Neural decoding is an important tool in neural engineering and neural data analysis. Of various machine learning algorithms adopted for neural decoding, the recently introduced deep learning is promising to excel. Therefore, we sought to apply deep learning to decode movement trajectories from the activity of motor cortical neurons. Approach. In this paper, we assessed the performance of deep learning methods in three different decoding schemes, concurrent, time-delay, and spatiotemporal. In the concurrent decoding scheme where the input to the network is the neural activity coincidental to the movement, deep learning networks including artificial neural network (ANN) and long-short term memory (LSTM) were applied to decode movement and compared with traditional machine learning algorithms. Both ANN and LSTM were further evaluated in the time-delay decoding scheme in which temporal delays are allowed between neural signals and movements. Lastly, in the spatiotemporal decoding scheme, we trained convolutional neural network (CNN) to extract movement information from images representing the spatial arrangement of neurons, their activity, and connectomes (i.e. the relative strengths of connectivity between neurons) and combined CNN and ANN to develop a hybrid spatiotemporal network. To reveal the input features of the CNN in the hybrid network that deep learning discovered for movement decoding, we performed a sensitivity analysis and identified specific regions in the spatial domain. Main results. Deep learning networks (ANN and LSTM) outperformed traditional machine learning algorithms in the concurrent decoding scheme. The results of ANN and LSTM in the time-delay decoding scheme showed that including neural data from time points preceding movement enabled decoders to perform more robustly when the temporal relationship between the neural activity and movement dynamically changes over time. In the spatiotemporal decoding scheme, the hybrid spatiotemporal network containing the concurrent ANN decoder outperformed single-network concurrent decoders. Significance. Taken together, our study demonstrates that deep learning could become a robust and effective method for the neural decoding of behavior.

In the digital banking landscape, the increasing volume of online transactions has heightened the risk of fraudulent activities, necessitating the development of more effective detection systems. This study investigates the efficacy of various machine learning and deep learning algorithms in identifying fraudulent transactions, emphasizing Long Short-Term Memory (LSTM) networks. We implemented and evaluated multiple algorithms, including Logistic Regression, Random Forest, Gradient Boosting Machines (GBM), and XGBoost, on a large-scale credit card transaction dataset. Our results demonstrate that the LSTM model outperforms traditional machine learning algorithms, achieving an accuracy of 98.5%, precision of 87.2%, recall of 85.0%, and an Area Under the Curve (AUC) score of 0.94. These findings highlight the superior capability of LSTM networks to capture complex patterns in sequential transaction data, making them an asset for real-time fraud detection in banking. This research underscores the need for financial institutions to adopt advanced deep learning techniques to enhance their fraud detection systems, thereby minimizing financial losses and improving customer trust.

Accurate rainfall forecasting is crucial for effective urban planning and disaster management in Dhaka, the capital of Bangladesh, a city highly vulnerable to urban flooding and extreme weather events. Traditional forecasting methods often struggle to capture the region's complex rainfall patterns, resulting in inaccurate rainfall forecasts. This study evaluates the performance of two traditional machine learning algorithms, Random Forest Regression and Multi-layer Perceptron (MLP), alongside one deep learning algorithm, the Long Short-Term Memory (LSTM) network. These models are trained and tested to forecast rainfall over 1 to 5-day lead times, emphasizing their ability to handle temporal dependencies in time series data. Atmospheric and hydrologic variables, including temperature, surface pressure, evaporation, solar surface radiation, total column rainwater, large-scale precipitation, and total cloud cover, from the ECMWF (European Centre for Medium-Range Weather Forecasts) Reanalysis v5 (ERA5) dataset, were used as model inputs. Model forecasts were validated against ERA5 rainfall data and compared with the forecasts from the Global Forecast System (GFS) model. Results indicate that the Random Forest model outperforms all others, achieving an RMSE of 6.11 mm and Pearson’s correlation coefficient (R) of 0.74 for a 1-day lead time. The LSTM model achieved an RMSE of 7.46 mm, while the MLP performed less effectively than both RF and LSTM, with an RMSE of 7.61 mm. In comparison, the GFS forecasts displayed an RMSE of 9.16 mm. The RF model outperformed the other models at all lead times; however, its accuracy decreased as the lead time increased. This study highlights the potential of machine learning to improve short to medium range rainfall forecasts, contributing to timely decision-making for urban resilience and resource management.

Metal–organic frameworks (MOFs), a class of porous nanomaterials, have been widely used in gas adsorption-based applications due to their high porosities and chemical tunability. To facilitate the discovery of high-performance MOFs for different applications, a variety of machine learning models have been developed to predict the gas adsorption capacities of MOFs. Most of the predictive models are developed using traditional machine learning algorithms. However, the continuously increasing sizes of MOF datasets and the complicated relationships between MOFs and their gas adsorption capacities make deep learning a suitable candidate to handle such big data with increased computational power and accuracy. In this study, we developed models for predicting gas adsorption capacities of MOFs using two deep learning algorithms, multilayer perceptron (MLP) and long short-term memory (LSTM) networks, with a hypothetical set of about 130,000 structures of MOFs with methane and carbon dioxide adsorption data at different pressures. The models were evaluated using 10 iterations of 10-fold cross validations and 100 holdout validations. The MLP and LSTM models performed similarly with high prediction accuracy. The models for predicting gas adsorption at a higher pressure outperformed the models for predicting gas adsorption at a lower pressure. The deep learning models are more accurate than the random forest models reported in the literature, especially for predicting gas adsorption capacities at low pressures. Our results demonstrated that deep learning algorithms have a great potential to generate models that can accurately predict the gas adsorption capacities of MOFs.

<p dir="ltr">Recurrent Neural Networks (RNNs) are a major class of Artificial Neural Networks (ANNs). Suitable network architecture is vital for RNNs to achieve high performance. However, designing the architecture of deep RNNs (DRNNs) is a complicated and time-consuming process. Each layer of a DRNN contains numerous hyper-parameters. It is challenging to optimize the hyper-parameters of all layers jointly to achieve the best possible learning performance. Trial-and-error methods for designing DRNNs have been proven to be laborious and highly costly in practice. Therefore, an efficient automatic architecture search technique is needed to design DRNNs.</p><p dir="ltr">Genetic Algorithm (GA) is a popular evolutionary computation based approach. Existing research works already explored the use of GAs in designing ANNs. But, most of the existing approaches focus on the design of fixed-depth ANNs. However, it is not easy to fix the depth of DRNNs in advance, since different problems need DRNNs of different depths. Hence, the potential of GA has to be further explored to evolve the architecture of DRNNs of varying depths.</p><p dir="ltr">The primary goal of this thesis is to develop advanced GA approaches for designing Long Short Term Memory (LSTM) based DRNN architectures of varying depths effectively and efficiently.</p><p dir="ltr">Firstly, this thesis proposes a GA-based algorithm called Two-Stage Surrogate-Assisted GA (TS-SA-GA) with a progressive incremental strategy to design the architecture of LSTM networks of varying depths. The progressive approach can effectively extend well-designed shallow networks to high performing deep networks. Moreover, the new GA-based algorithm adopts newly designed knowledge-driven crossover and mutation operators to identify and repair LSTM network designs affected by inappropriate use of activation functions, thereby significantly reducing the chances for GA to evolve hard-to-train LSTM network architectures. Furthermore, this thesis proposes a two-stage surrogate method to predict the trainability and fitness of LSTM networks. This improves the efficiency and effectiveness of the GA-based algorithm to evolve LSTM networks.</p><p dir="ltr">Secondly, this thesis proposes a new algorithm called Evolutionary Design of LSTM Ensembles (ED-LSTM-Ensemble) based on multi-objective optimization techniques to evolve LSTM networks and ensembles simultaneously to directly utilize the ensemble performance to drive the evolution of base LSTM networks. Additionally, this thesis proposes a connection weight inheritance strategy to evolve LSTM networks and ensembles efficiently and effectively.</p><p dir="ltr">Thirdly, this thesis proposes a new algorithm called Skip Connections through Evolutionary and Differential Architecture search (SCEDA) to design LSTM networks together with appropriate skip connections automatically. Designing LSTM networks together with skip connections is highly challenging. This thesis hybridize gradient-based DAS and evolutionary architecture search to optimize the architecture of LSTM networks with suitable skip connections in a single evolutionary process to improve the efficiency and effectiveness of the search process.</p><p dir="ltr">Finally, this thesis conducts an in-depth empirical analysis of the impact of activation functions on the trainability of LSTM networks based on the concept of Edge Of Chaos. The analysis reveals the strong interrelation between the activation functions across multiple layers of an LSTM network and the trainability of the network. The analysis also highlights the importance of controlling activation functions to avoid generating untrainable LSTM networks. On the basis of the analysis, this thesis proposes a machine learning model to guide the use of activation functions in LSTM networks.</p>

Deep leaning in food safety and authenticity detection: An integrative review and future prospects

Vehicle classification from low-frequency GPS data with recurrent neural networks

Understanding and exploiting feeding patterns in swine could allow a reduced feed waste and minimized sorting losses. The objectives of this study were to evaluate the ability to predict whether a pig reached a target weight at finishing by using several algorithms and to compare the prediction using varying amounts of data during the growing period. Data were collected on 655 pigs from 75 to 166 days of age. Pigs were housed with 8 to 15 pigs and a Feed Intake Recording Equipment in each pen. Feed consumption, occupation time, and body weight per visit were recorded when a pig visited the feeder. Lasso Regression (LS), a machine learning algorithm: Random Forest (RF), and a deep learning algorithm: Long-short Term Memory (LSTM) network, were used to forecast whether pigs can reach 129 kg at the finishing stage (159–166 d). Times of visits, a sum of feed consumption, a sum of occupation time in the feeder every day, and age were used as predictors. Data were split into 6 slices by 14 days and used to calibrate the models and their predictive ability was tested with data corresponding to the last 8 days of the study period. The greatest correlation coefficients were 0.799, 0.828, and 0.868 using slice 6 (145–158 d) to train the LS, RF, and LSTM, respectively. The LS and LSTM algorithms had a smaller root mean squared error, 0.863 and 0.895 compared to the RF with 1.375 in the prediction. Overall, LS and LSTM performed best. Predictions using data closest to the finishing stage proved better. This study connects the dynamics of feeding behavior and feed intake data to growth using prediction methods that will hopefully accelerate the mainstream application of electronic feeders in pig production systems.

This paper presents a high-spatial-resolution dataset with remote sensing images of the Brazilian Cerrado for land use and land cover classification. The Biome Cerrado Dataset (CerraData) is a large database created from 150 scenes of the CBERS-4A satellite. Images were created by merging the near-infrared, green, and blue bands. Moreover, pan-sharpening was performed between all the scenes and their respective panchromatic bands, resulting in a final spatial resolution of two meters. A total of 2.5 million tiles of 256x256 pixels were derived from these scenes. From this total, 50 thousand tiles were labeled. We also conducted a few-shot learning experiment considering a training set with only 100 samples, 11 deep neural networks (DNNs), and two traditional machine learning (ML) algorithms, i.e., support vector machine (SVM) and random forest (RF). Results show that the DNN DenseNet-161 was the best model but its performance can be improved if it is used only as a feature extractor, leaving the classification task for the traditional ML algorithms. However, by decreasing the size of the training set, smarter approaches are needed. The labeled subset of CerraData as well as the source code we developed to support this study are available on-line: https://github.com/ai41uc/CerraData-code-data.

An exhaustive measurement of re-sampling detection in lossy compressed images using deep learning approach

Over the years, communication through Short Message Service (SMS) has been a primary tool for mobile subscribers. SMS has varied applications in health, industry, finances, education and social networking among others. The growth of mobile devices and SMS usage has consequently increased the attack surface for cyber-criminals culminating to the proliferation of malicious activities introduced using SMS spam, phishing, spyware, malware etc. Ham messages are normal messages people trade with one another and are usually not unwanted by the recipient, while spam messages are unsolicited junk and redundant messages that may be sent to a large number of people at once and are usually unwanted. Various spam detection models have been developed using various traditional machine and deep learning techniques. However, most studies where comparison between deep and traditional machine learning algorithms is done, have unfortunately omitted K-Nearest Neighbors and Support Vector Machine (SVM) which are empirically deemed as the most popular traditional machine learning algorithms. In this study, therefore, we develop a deep learning model based on Recurrent Neural Network (RNN) for Spam and Ham SMS classification and compare its performance against SVM model for the same University of California (UCl) SMS dataset. The results show that RNN has a slightly higher training and validation accuracy of 0.98 compared to SVM at 0.94, however, the false positive rate of SVM is marginally lower. Exploring application of deep learning with better optimization algorithms such as RNN improves accuracy, reduces computational complexities i.e. memory consumption and speed, and thus minimizing false positive rates. For future work, we suggest the use of varied performance metrics to validate the model in a distributed dataset environment.

The Internet has become an indispensable part of modern people's lives. The sentiment analysis of text generated by the Internet has gradually become a research hot spot. Through the sentiment analysis of texts, information such as the public's emotional status, views on some social phenomena, and preferences for a product can be obtained. It contributes to commercial value and social stability. The common research methods are based on traditional machine learning algorithms. According to hand-labeled sentiment lexicons, we use machine learning algorithms such as naive Bayes, support vector machines, and maximum entropy methods are to perform sentiment analysis on textual information. To reduce the dependence on hand-built emotional dictionary and highlight the role of keywords in the review text, this paper proposes the weighted word2vec, adds the Attention mechanism to the Long-Short Term Memory (LSTM) model. Experiment result shows that the method is significantly better than traditional machine learning methods.

The proliferation of misinformation, as insidious and pervasive as water, presents an unprecedented challenge to public discourse and comprehension. Often propagated to further specific ideologies or political objectives, misinformation not only misleads the populace but also fuels online advertising revenue generation. As such, the urgent need to pinpoint and eliminate misinformation from digital platforms has never been more critical. In response to this dilemma, this paper proposes a solution built on the backbone of massive data generation in today’s digital landscape. By leveraging advanced technologies, such as AI-driven systems with deep learning models and natural language processing capabilities, we can monitor and analyze an extensive scope of social media data. This, in turn, facilitates the identification of misinformation across multiple platforms and alerts users to potential propaganda. Central to our study is the development of misinformation classifiers based on a deep bi-directional long short-term memory (Bi-LSTM) model. This model is further enhanced by employing a genetic algorithm (GA), which automates the search for an optimal neural architecture, thereby significantly impacting the training behavior of the deep learning algorithm and the performance of the model being trained. To validate our approach, we compared the efficacy of our proposed model with nine traditional machine learning algorithms and a deep learning model rooted in long short-term memory (LSTM). The results affirmed the superiority of our GA-tuned Bi-LSTM model, which outperformed all other models in detecting misinformation with remarkable accuracy. Our intention with this paper is not to present our model as a comprehensive solution to misinformation but rather as a technological tool that can aid in the process, supplementing and bolstering the existing methodologies in the field of misinformation detection.