Deep Semantics-Enhanced Neural Code Search

Forecasting daily PM2.5 concentrations in Wuhan with a spatial-autocorrelation-based long short-term memory model

Traffic safety has been thought of as a basic feature of transportation, recent developments in civil aviation have emphasized the need for risk identification and safety prediction. This study aims to increase en-route flight safety through the development of prediction models for flight conflicts. Firstly, flight conflicts time series and traffic parameters are extracted from historical ADS-B data. In the second step, a Long Short-Term Memory (LSTM) model is trained to make a one-step-ahead prediction on the flight conflict time series. The results show that the LSTM model has the greatest prediction effect (MAE 0.3901) with comparison to other models. Based on that, we add traffic parameters (volume, density, velocity) into the LSTM model as new input variables and issue a comprehensive analysis of the relative predictive power of traffic parameters. The accuracy of prediction model is validated with a mean error of less than 3%. Based on the improvements of model performance brought by traffic parameters, LSTM models with a single traffic parameter are proposed for further discussion. The results illustrate that volume is the most important factor in promoting prediction accuracy and density has an advantage of improvement in the aspect of model stability.

The accurate prediction of carbon dioxide (CO2) emissions in the building industry can provide data support and theoretical insights for sustainable development. This study proposes a hybrid model for predicting CO2 emissions that combines a multi-strategy improved particle swarm optimization (MSPSO) algorithm with a long short-term memory (LSTM) model. Firstly, the particle swarm optimization (PSO) algorithm is enhanced by combining tent chaotic mapping, mutation for the least-fit particles, and a random perturbation strategy. Subsequently, the performance of the MSPSO algorithm is evaluated using a set of 23 internationally recognized test functions. Finally, the predictive performance of the MSPSO-LSTM hybrid model is assessed using data from the building industry in the Yangtze River Delta region as a case study. The results indicate that the coefficient of determination (R2) of the model reaches 0.9677, which is more than 10% higher than that of BP, LSTM, and CNN non-hybrid models and demonstrates significant advantages over PSO-LSTM, GWO-LSTM, and WOA-LSTM hybrid models. Additionally, the mean square error (MSE) of the model is 2445.6866 Mt, and the mean absolute error (MAE) is 4.1010 Mt, both significantly lower than those of the BP, LSTM, and CNN non-hybrid models. Overall, the MSPSO-LSTM hybrid model demonstrates high predictive accuracy for CO2 emissions in the building industry, offering robust support for the sustainable development of the industry.

Long short-term memory - Fully connected (LSTM-FC) neural network for PM2.5 concentration prediction

Sensitivity analysis of isoprene and aerosol emission in a suburban plantation using long short-term memory model

High-resolution prediction of the spatial distribution of PM2.5 concentrations in China using a long short-term memory model

This study develops a ship propeller diagnostic system to address the issue of insufficient ship maintenance capacity and enhance operational efficiency. It uses the Remaining Useful Life (RUL) prediction technology to establish a sensing platform for ship propellers to capture vibration signals during ship operations. The Diagnosis and RUL Prediction Model is designed to assess bearing aging status and the RUL of the propeller. The synchronized signal reshaping technology is employed in the Diagnosis and RUL Prediction Model to process the original vibration signals as input to the model. The vibration signals obtained are used to analyze the temporal and spectral energy of propeller bearings. Exponential functions are used to generate the health index as model outputs. Model inputs and outputs are simultaneously input into a Long Short-Term Memory (LSTM) model for training, culminating as the Diagnosis and RUL Prediction Model. Compared to Recurrent Neural Network and Support Vector Regression models used in previous studies, the Diagnosis and RUL Prediction Model developed in this study achieves a Mean Squared Error (MSE) of 0.018 and a Mean Absolute Error (MAE) of 0.039, demonstrating outstanding performance in prediction results and computational efficiency. This study integrates the Diagnosis and RUL Prediction Model, bearing aging experimental data, and real-world vibration measurements to develop the diagnosis and RUL prediction system for ship propellers. Experiments with ship propellers show that when the bearing of the propeller enters the worn stage, this diagnostic system for ship propellers can accurately determine the current status of the bearing and its remaining useful life. This study offers a practical solution to insufficient ship maintenance capacity and contributes to improving the operational efficiency of ships.

IntroductionDrug safety has increasingly become a serious public health problem that threatens health and damages social economy. The common detection methods have the problem of high false positive rate. This study aims to introduce deep learning models into the adverse drug reaction (ADR) signal detection and compare different methods.MethodsThe data are based on adverse events collected by Center for ADR Monitoring of Guangdong. Traditional statistical methods were used for data preliminary screening. We transformed data into free text, extracted text information and made classification prediction by using the Long Short-Term Memory (LSTM) model. We compared it with the existing signal detection methods, including Logistic Regression, Random Forest, K-NearestNeighbor, and Multilayer Perceptron. The feature importance of the included variables was analyzed.ResultsA total of 2,376 ADR signals were identified between January 2018 and December 2019, comprising 448 positive signals and 1,928 negative signals. The sensitivity of the LSTM model based on free text reached 95.16%, and the F1-score was 0.9706. The sensitivity of Logistic Regression model based on feature variables was 86.83%, and the F1-score was 0.9063. The classification results of our model demonstrate superior sensitivity and F1-score compared to traditional methods. Several important variables “Reasons for taking medication”, “Serious ADR scenario 4”, “Adverse reaction analysis 5”, and “Dosage” had an important influence on the result.ConclusionThe application of deep learning models shows potential to improve the detection performance in ADR monitoring.

Effective monitoring of tool wear status can improve productivity and reduce losses. In previous studies, extensive feature selection was required when using the traditional machine learning method. The gating mechanism in the traditional long short-term memory (LSTM) model may incur information loss and a weaker representation of global sequential dependencies in handling long sequences. This paper aims to enhance the performance of the LSTM model in tool wear prediction by combining feature and temporal attention. Firstly, the original vibration signal is divided into sub-sequences and related features extracted. Secondly, the ability to capture global sequential dependencies using the LSTM model is improved by feature and temporal attention. Finally, a fully connected layer is used to predict tool wear values. Compared to traditional LSTM, the proposed method performs best in three evaluation metrics, RMSE, MAE, and the coefficient of determination.

SummaryFor the past few years, Bitcoin plays a vital role in both the economical and financial industries. In order to gain a huge return on investment, the investors are eager to forecast the future value of Bitcoin. However, Bitcoin price variation is quite nonlinear and chaotic in nature, so it creates more difficulty in forecasting future value. Researchers found that the multiplicative long short term memory (LSTM) model will be more efficient for predicting those complex variations. So, target mission is about to develop an optimized multiplicative LSTM with an Attention mechanism using Technical Indicators derived from historical data. A modified cuckoo search optimization model is proposed to tune the hyperparameter of the Deep Learning model. This novel optimization algorithm eliminates the local optimum and slower convergence problem of the cuckoo search optimization algorithm. Deibold Mariano test is performed to statistically evaluate the proposed model and it is inferred that the recommended methodology is statistically fit. Regression metrics such as root mean square error, mean square error and mean absolute error has been used for comparative evaluation with related benchmark techniques such as genetic algorithm optimized LSTM (GA–LSTM), particle swarm optimized LSTM (PSO–LSTM) and cuckoo search optimized LSTM (CSO–LSTM). The empirical result shows that the recommended methodology outperforms the taken benchmark models and provides better accuracy.

Water managers need accurate rainfall forecasts for a wide spectrum of applications, ranging from water resources evaluation and allocation, to flood and drought predictions. In the past years, several frameworks based on Artificial Intelligence have been developed to improve the traditional Numerical Weather Prediction (NWP) forecasts, thanks to their ability of learning from past data, unravelling hidden relationships among variables and handle large amounts of inputs. Among these approaches, Long Short-Term Memory (LSTM) models emerged for their ability to predict sequence data, and have been successfully used for rainfall and flow forecasting, mainly with short lead-times. In this study, we explore three different multi-variate LSTM-based models, i.e. vanilla LSTM, stacked LSTM and bidirectional LSTM, to forecast daily precipitation for the upcoming 30 days in the area of Rhine Delta, the Netherlands. We use both local atmospheric and global climate variables from the ERA-5 reanalysis dataset to predict rainfall, and we introduce a fuzzy index for the models to account for seasonality effects. The framework is developed within the H2020 project CLImate INTelligence (CLINT), and its outcomes have the potential to improve forecasting precipitation deficit in the study area.

Abstract. The information available on the internet nowadays is diverse and moves very quickly. Information is becoming easier to obtain by the general public with the numerous online media outlets, including news portals that provide up-to-date information insights. Various news portals earn revenue from advertising using pay-per-click methods that encourage article writers to use clickbait techniques to attract visitors. However, the negative effects of clickbait include a decrease in journalism quality and the spread of hoaxes. This problem can be prevented by using text classification to classify clickbait in news titles. One method that can be used for text classification is a neural network. Artificial neural networks use algorithms that can independently adjust input coefficient weights. This makes this algorithm highly effective for modeling non-linear statistical data. The artificial neural network algorithm, especially the Long Short-Term Memory (LSTM), has been widely used in various natural language processing fields with satisfying results, including text classification. To improve the performance of the neural network model, adjustments can be made to the model's hyperparameters. Hyperparameters are parameters that cannot be obtained through data and must be defined before the training process. In this research, the Long Short-Term Memory (LSTM) model was used in clickbait classification in news titles. Sixteen neural network models were trained with different hyperparameter configurations for each model. Hyperparameter tuning was carried out using the random search algorithm. The dataset used was the CLICK-ID dataset published by William & Sari, 2020[1], with a total of 15,000 annotated data. The research results show that the developed LSTM model has a validation accuracy of 0.8030, higher than William & Sari's research, and a validation loss of 0.4876. Using this model, researchers were able to classify clickbait in news titles with fairly good accuracy. Purpose: The study was to develop and evaluate a LSTM model with hyperparameter tuning for clickbait classification on news headlines. The thesis also aims to compare the performance of simple LSTM and bidirectional LSTM for this task. Methods: This study uses CLICK-ID dataset and applies different text preprocessing techniques. The dataset later was used to build and train 16 LSTM models with different hyperparameters and evaluates them using validation accuracy and loss. This study uses random search for hyperparameter tuning. Result: The results of the study show that the best model for clickbait classification on news headlines is a bidirectional LSTM model with one layer, 64 units, 0.2 dropout rate, and 0.001 learning rate. This model achieves a validation accuracy of 0.8030 and a validation loss of 0.4876. The results also show that hyperparameter tuning using random search can improve the performance of the LSTM models by avoiding zero probabilities and finding the optimal values for the hyperparameters. Novelty: This study compares and analyzes the different preprocessing methods on text and the different configurations of the models to find the best model for clickbait classification on news headlines. The study also uses hyperparameter tuning to tune the model into the best model and finding the optimal values for the hyperparameters.

Travel planning is a process that allows users to obtain maximum benefit from their time, cost and energy. When planning a route from one place to another, it is an important option to present alternative travel areas on the route. This study proposes a travel route planning (TRP) architecture using a Long Short-Term Memory (LSTM) and Extreme Gradient Boosting (XGBoost) model to improve both travel efficiency and environmental sustainability in route selection. This model incorporates carbon emissions directly into the route planning process by unifying user preferences, location recommendations, route optimization, and multimodal vehicle selection within a comprehensive framework. By merging environmental sustainability with user-focused travel planning, it generates personalized, practical, and low-carbon travel routes. The carbon emissions observed with TRP’s artificial intelligence (AI) recommendation route are presented comparatively with those of the user-determined route. XGBoost, Random Forest (RF), Categorical Boosting (CatBoost), Light Gradient Boosting Machine (LightGBM), (Extra Trees Regressor) ETR, and Multi-Layer Perception (MLP) models are applied to the TRP model. LSTM is compared with Recurrent Neural Networks (RNNs) and Gated Recurrent Unit (GRU) models. Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Squared Error (MSE), and Normalized Root Mean Square Error (NRMSE) error measurements of these models are carried out, and the best result is obtained using XGBoost and LSTM. TRP enhances environmental responsibility awareness within travel planning by integrating sustainability-oriented parameters into the decision-making process. Unlike conventional reservation systems, this model encourages individuals and organizations to prioritize eco-friendly options by considering not only financial factors but also environmental and socio-cultural impacts. By promoting responsible travel behaviors and supporting the adoption of sustainable tourism practices, the proposed approach contributes significantly to the broader dissemination of environmentally conscious travel choices.

This study aims to analyze user sentiment toward the NU Online application by applying a Long Short-Term Memory (LSTM) model. The research was based on 13,576 user reviews collected from the Google Play Store, which underwent preprocessing, sentiment labeling, and Exploratory Data Analysis (EDA). To ensure balanced classification, undersampling was used, resulting in 6,612 reviews equally divided into positive and negative classes. The text data was processed using tokenization and padding before being input into the LSTM model. Model training involved the use of binary crossentropy, Adam optimizer, EarlyStopping, and ReduceLROnPlateau techniques. The model achieved 93% precision, recall, and F1-score, with low error rates and strong generalization ability. EDA results showed that positive feedback mainly focused on worship features like salat schedules, while negative reviews addressed technical issues such as the azan sound. User review peaks occurred during religious periods and major updates. A Gradio-based web interface was also developed to display results and enable user-friendly access to visual sentiment insights. This implementation proves the practical potential of integrating LSTM with an interactive platform for effective sentiment analysis

Short-term traffic flow prediction plays an important role in intelligent transportation system. Numerous researchers have paid much attention to it in the past decades. However, the performance of traditional traffic flow prediction methods is not satisfactory, for those methods cannot describe the complicated nonlinearity and uncertainty of the traffic flow precisely. Neural networks were used to deal with the issues, but most of them failed to capture the deep features of traffic flow and be sensitive enough to the time-aware traffic flow data. In this paper, we propose a deep bi-directional long short-term memory (DBL) model by introducing long short-term memory (LSTM) recurrent neural network, residual connections, deeply hierarchical networks and bi-directional traffic flow. The proposed model is able to capture the deep features of traffic flow and take full advantage of time-aware traffic flow data. Additionally, we introduce the DBL model, regression layer and dropout training method into a traffic flow prediction architecture. We evaluate the prediction architecture on the dataset from Caltrans Performance Measurement System (PeMS). The experiment results demonstrate that the proposed model for short-term traffic flow prediction obtains high accuracy and generalizes well compared with other models.