A Novel Trajectory Prediction Method Based on CNN, BiLSTM, and Multi-Head Attention Mechanism

The 4D trajectory is a multi-dimensional time series with plentiful spatial-temporal features and has a high degree of complexity and uncertainty. Aiming at these features of aircraft flight trajectory and the problem that it is difficult for existing trajectory prediction methods to extract spatial-temporal features from the trajectory data at the same time, we propose a novel 4D trajectory prediction hybrid architecture based on deep learning, which combined Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM). An 1D convolution is used to extract the spatial dimension feature of the trajectory, and LSTM is used to mine the temporal dimension feature of the trajectory. Hence the high-precision prediction of the 4D trajectory is realized based on the sufficient fusion of the above features. We use real Automatic Dependent Surveillance -Broadcast (ADS-B) historical trajectory data for experiments and compare the proposed method with a single LSTM model and BP model on the same data set. The experimental results show that the trajectory prediction accuracy of the CNN-LSTM hybrid model is superior to a single model. The prediction error is reduced by an average of 21.62% compared to the LSTM model and by an average of 52.45% compared to the BP model. It provides a certain reference for the trajectory prediction research and Air Traffic Management decision-making.

To better understand and evaluate the GNSS positioning performance, it is convenient to adopt corresponding measures to reduce the impact of errors on positioning. A GNSS positioning error estimation scheme based on an improved multi-layer perceptron model is proposed. The multi-head attention mechanism and gating operation are integrated into the multi-layer perceptron model to adaptively select and filter features, enhancing the model’s ability to understand input features. First, the original positioning error of the satellite is obtained through the Kalman filter positioning method. The data are then preprocessed to extract available features. Finally, the features are input into the constructed model for training and testing to obtain the estimated positioning error value. Two types of comparative experiments were completed. The performance of the presented model is evaluated by the root mean square error. Experimental results show that the proposed method performs well in terms of performance indicators, and has obvious advantages over other state-of-the-art methods. In particular, the root mean square error of the presented method in the first dataset is 0.239 m, which is 39.2% and 17% lower than the current state-of-the-art long short-term memory network and convolutional neural network, respectively. The presented method can provide higher-precision estimated values for studying the GNSS positioning error estimation problem.

To overcome the limitations of existing deep learning-based image recognition approaches for diagnosing the operating conditions of pumping wells–namely, slow processing speed and high parameter complexity–this paper proposes a lightweight diagnostic method based on a one-dimensional convolutional neural network (1D CNN). Unlike traditional image-based techniques, the proposed method eliminates the need to convert raw data into dynamometer card images, thereby reducing computational overhead and significantly decreasing the number of model parameters. The architecture integrates 1D CNN with a bidirectional long short-term memory (BiLSTM) network to effectively capture both local patterns and global temporal dependencies inherent in different operating states of pumping wells. Additionally, a multi-head attention (MHA) mechanism is incorporated to enhance the representation of critical time points, thereby improving the model's sensitivity to key features and its overall classification performance. Experiments conducted on an augmented dataset–generated via a simulation-based algorithm–demonstrate that the proposed 1DCNN-BiLSTM-MHA model achieves a classification accuracy of 99.22%. Experimental results demonstrate that the proposed model possesses strong feature learning capabilities. Comparative analysis with several baseline models highlights its superior diagnostic performance and reliability, along with reduced parameter count and improved computational speed.

Estimating reference crop evapotranspiration using improved convolutional bidirectional long short-term memory network by multi-head attention mechanism in the four climatic zones of China

Data-driven forecasting framework for daily reservoir inflow time series considering the flood peaks based on multi-head attention mechanism

Generally, emotions in short videos are transmitted through characters behaviour and text content in the video. The mainstream recognition model is based on a convolutional neural network. However, its processing ability for video data is limited, its recognition accuracy is not high enough, and it is limited in processing complex text. To solve these problems, a new text recognition emotion classification model is designed by improving the text feature fusion module on the basis of the convolutional neural network model. Moreover, a new human behaviour recognition emotion classification model is designed by introducing multi-head attention mechanisms on the basis of a 3D convolutional neural network model. These results confirmed that the accuracy of the improved text recognition model was around 75%, while the original convolutional model's average accuracy was only about 67%. The 3D convolutional model under the multi-head attention mechanism had the highest recognition accuracy, with a recognition accuracy of 94.5% on the UCF101 database, which was 12 to 39 percentage points higher than the model under other attention mechanisms. The improved convolutional network model for short video text classification and behaviour recognition has more advantages than traditional models. These research results have certain value for classification models and can serve as technical references.

COVID-19 fake news detection: A hybrid CNN-BiLSTM-AM model

GAOformer: An adaptive spatiotemporal feature fusion transformer utilizing GAT and optimizable graph matrixes for offshore wind speed prediction

The traditional methods of English text classification have two disadvantages. One is that they cannot fully represent the semantic information of the text. The other is that they cannot fully extract and integrate the global and local information of the text. Therefore, we propose a multi-feature fusion model based on long and short term memory network and improved artificial bee colony algorithm for English text classification. In this method, the character-level vector and word-level vector representations of English text are calculated using a pre-training model to obtain a more comprehensive text feature vector representation. Then the multi-head attention mechanism is used to capture the dependencies in the text sequence to improve the semantic understanding of the text. Through feature fusion, the channel features are optimized and the spatial features and time series features are combined to improve the classification performance of the hybrid model. In the stage of network training, the weighted linear combination of maximum Shannon entropy and minimum cross entropy is used as the return degree evaluation function of the bee colony algorithm, and the scale factor is introduced to adjust the solution search strategy of leading bees and following bees, and the improved artificial bee colony algorithm is combined with the classification network to realize the automatic optimization and adjustment of network parameters. Experiments are carried out on public data set. Compared with traditional convolutional neural networks, the classification accuracy of the new model increases by 2% on average, and the accuracy of data set increases by 2.4% at the highest.

In order to accurately predict the lane‐changing trajectory of the vehicle and improve the driving safety of the vehicle, a lane‐changing trajectory prediction model based on the combination of convolutional neural network (CNN) and bidirectional long short‐term memory (BiLSTM) neural network is proposed by comprehensively considering the historical driving behavior, the spatial characteristics of surrounding vehicles and the bidirectional time sequence information of the vehicle trajectory. Firstly, the vehicle trajectory data are filtered and smoothed, and it is divided into three categories: left lane change, right lane change, and straight driving, and a lane change trajectory sample set is constructed. Secondly, CNN‐BiLSTM model is constructed to identify the sample set of lane‐changing trajectory. Considering the interaction between vehicles in the driving process, the information of predicted vehicle, and surrounding vehicles is taken as the input of the model. The extracted feature vector is input to the BiLSTM layer for prediction after the CNN layer feature extraction, and the horizontal and vertical coordinates of the target vehicle at the next time are output. Thirdly, the trajectory data of the US‐101 dataset in NGSIM is selected to verify the performance of the CNN‐BiLSTM model, and at the same time, it is compared with models such as CNN‐LSTM, long short‐term memory (LSTM), BiLSTM, and CNN‐GRU‐Att. Finally, the verification result shows that the overall fitting degree of the vehicle lane change trajectory prediction of the proposed model reaches 99.50%, and the mean square error and mean absolute error are 0.0003076 and 0.01417, which are improved compared with other models. In the meanwhile, the research on multistep trajectory prediction in different prediction time domains is carried out. It was found that the longer the prediction time domain is, the lower the prediction performance of the model decreases, but the prediction accuracy still reached more than 96%, and it was able to accurately predict the lane change trajectory.

Greenhouse temperature prediction plays an important role in smart agriculture and smart monitoring. However, the following problems still exist in predicting the temperature of a greenhouse. First of all, temperature changes are affected by many factors, such as light intensity, carbon dioxide concentration and so on. Secondly, the temperature change is locally time dependent within the time range. Therefore, it is very challenging to accurately predict the greenhouse, and the existing prediction methods cannot comprehensively consider the above problems. In order to improve the accuracy and precision of greenhouse temperature prediction, and promote the timely growth of crops. This paper proposes a greenhouse temperature prediction model (TGTP) based on Transformer neural network. First, in order to solve the multiple linearity problem caused by the strong correlation between data features and reduce the loss of information contained in the original data, the method based on Pearson correlation coefficient is used to select the multi-source influencing factors of temperature changes, and build training set, test set and validation set based on this. The test set and the verification set; then, the data is standardized and input into the Transformer neural network for training, and the long-term temperature change law is captured through the multi-head attention mechanism; finally, the verification set is input to the model for prediction, and the greenhouse temperature prediction result is output. The comparison results show that the prediction effect of TGTP is better than the existing prediction methods.

Traffic flow prediction is crucial for public safety and traffic management, and remains a big challenge because of many complicated factors, e.g., multiple spatio-temporal dependencies, holidays, and weather. Some work leveraged 2D convolutional neural networks (CNNs) and long short-term memory networks (LSTMs) to explore spatial relations and temporal relations, respectively, which outperformed the classical approaches. However, it is hard for these work to model spatio-temporal relations jointly. To tackle this, some studies utilized LSTMs to connect high-level layers of CNNs, but left the spatio-temporal correlations not fully exploited in low-level layers. In this work, we propose novel spatio-temporal CNNs to extract spatio-temporal features simultaneously from low-level to high-level layers, and propose a novel gated scheme to control the spatio-temporal features that should be propagated through the hierarchy of layers. Based on these, we propose an end-to-end framework, multiple gated spatio-temporal CNNs (MGSTC), for citywide traffic flow prediction. MGSTC can explore multiple spatio-temporal dependencies through multiple gated spatio-temporal CNN branches, and combine the spatio-temporal features with external factors dynamically. Extensive experiments on two real traffic datasets demonstrates that MGSTC outperforms other state-of-the-art baselines.

Prediction of PM2.5 concentration in urban agglomeration of China by hybrid network model

A short-term wind power prediction model based on BiLSTM–CNN–WGAN-GP (LCWGAN-GP) is proposed in this paper, aiming at the problems of instability and low prediction accuracy of short-term wind power prediction. Firstly, the original wind energy data are decomposed into subsequences of natural mode functions with different frequencies by using the variational mode decomposition (VMD) algorithm. The VMD algorithm relies on a decision support system for the decomposition of the data into natural mode functions. Once the decomposition is performed, the nonlinear and dynamic behavior are extracted from each natural mode function. Next, the BiLSTM network is chosen as the generation model of the generative adversarial network (WGAN-GP) to obtain the data distribution characteristics of wind power’s output. The convolutional neural network (CNN) is chosen as the discrimination model, and the semi-supervised regression layer is utilized to design the discrimination model to predict wind power. The minimum–maximum game is formed by the BiLSTM and CNN network models to improve the quality of sample generation and further improve the prediction accuracy. Finally, the actual data of a wind farm in Jiuquan City, Gansu Province, China is taken as an example to prove that the proposed method has superior performance compared with other prediction algorithms.

As one of the core tasks of natural language processing technology, text classification methods general-ly face the problems of insufficient global semantic capture and limited feature focusing ability when processing long texts or complex semantics. To address this issue, a deep learning model that integrates improved convolutional neural networks, unbalanced bidirectional long short-term memory networks, and multi-head attention mechanisms is proposed. Utilizing an improved bidirectional long short-term memory network to capture global semantic information, while dynamically focusing on key features through a multi head attention mechanism to enhance the model's adaptability to classification tasks. The performance of the model is validated through experiments on AG News (short text) and IMDb (long text) datasets. The results show that in short text classification, the proposed method has an accu-racy rate of 96% and a classification error rate of only 1.46%. In the task of long text classification, the method proposed in the study has a product under the curve of 0.98. In adversarial attack testing, the accuracy rates of adversarial samples generated by different methods are 92.85% and 90.63%, respec-tively, with the lowest robustness degradation rates of 3.72% and 5.49%, respectively. In cross domain generalization testing, it shows the least classification errors and superior cross domain adaptability. These results validate the high performance, robustness, and wide applicability of the method. The re-search indicates that this approach can validly improve the performance of text classification and pro-vide new solutions for natural language processing related tasks in long text and multi-category scenar-ios.