Offshore wind power output prediction based on convolutional attention mechanism

Accurate ultra-short-term wind speed forecasting is great significance to ensure large scale integration of wind power into the power grid, but the randomness, instability, and non-linear nature of wind speed make it very difficult to be predicted accurately. To solve this problem, shifted window stationary attention transformer (SWSA transformer) is proposed based on a global attention mechanism for ultra-short-term forecasting of wind speed. SWSA transformer can sufficiently extract these complicated features of wind speed to improve the prediction accuracy of wind speed. First, positional embedding and temporal embedding are added at the bottom of the proposed method structure to mark wind speed series, which enables complicated global features of wind speed to be more effectively extracted by attention. Second, a shifted window is utilized to enhance the ability of attention to capture features from the edge sequences. Third, a stationary attention mechanism is applied to not only extract features of wind speed but also optimize the encoder-decoder network for smoothing wind speed sequences. Finally, the predicted values of wind speed are obtained using the calculation in the decoder network. To verify the proposed method, tests are performed utilizing data from an real offshore wind farm. The results show that the proposed method outperforms many popular models evaluated by many indexes including gated recurrent unit, Gaussian process regression, long-short term memory, shared weight long short-term memory network, and shared weight long short-term memory network -Gaussian process regression, in terms of mean absolute error, mean square error (MSE), root mean square error, mean absolute percentage error, mean square percentage error, and coefficient of determination (R2).

Offshore wind turbines (OWTs) have complex operating conditions and are prone to accidents. Timely and accurate vibration prediction can reduce the probability of failure of OWTs. To address this issue, this study proposed a vibration prediction model based on long short-term memory (LSTM) network. The proposed prediction model was applied to an OWT in an offshore wind farm in China, and three evaluation indicators, i.e. mean absolute error (MAE), mean absolute percentage error (MAPE), and root mean square error (RMSE), were used to verify the vibration prediction performance of LSTM. The results demonstrated that the prediction accuracy of LSTM was better than that of back propagation (BP) neural network, and LSTM had optimum engineering practicability. The prediction accuracy of LSTM and BP neural networks decreased with an increase in prediction durations. However, the prediction accuracy of LSTM was higher. This study improves the online safety monitoring of OWTs.

Current temperature prediction methods often focus on time-series information while neglecting the contributions of different meteorological factors and the context of varying time steps. Accordingly, this study developed a Dual-Attention-BiLSTM (a bidirectional long short-term memory network with dual attention mechanisms) network model, which integrates a bidirectional long short-term memory (BiLSTM) network model with random forest-based feature selection and two self-designed attention mechanisms. A sensitivity analysis was conducted to evaluate the influence of the attention mechanisms. This study focuses on Shijiazhuang City, China, which has a temperate continental monsoon climate with significant seasonal and daily variations. The data were sourced from ERA5-Land, comprising hourly near-surface temperature and related meteorological variables for the year of 2022. The results indicate that integrating the two attention mechanisms significantly improves the model’s prediction performance compared to using BiLSTM alone. The mean absolute error between simulation results ranges from 0.80 °C to 1.08 °C, with a reduction of 0.17 °C to 0.39 °C, and the root mean square error ranges from 1.17 °C to 1.37 °C, with a reduction of 0.12 °C to 0.22 °C.

Hydrate plugging and pipeline leak can impair the normal operation of natural gas pipeline and may lead to serious accidents. Since natural gas pipeline safety monitoring based on active acoustic excitation can detect and locate not only the two abnormal events but also normal components such as valves and pipeline elbows, recognition and classification of these events are of great importance to provide maintenance guidance for the pipeline operators and avoid false alarm. In this article, long short-term memory (LSTM) network is introduced and applied to classify detection signals of hydrate plugging, pipeline leak, and elbow. Adaptive moment estimation (Adam) algorithm is introduced and utilized to accelerate the long short-term memory network convergence in training. Experimental results demonstrate that the network with three layers and 64 units per cell performs the best. The cross-entropy loss in training is 0.0005, and classification accuracies are all 100% in training, validation, and testing which verify the validity of the long short-term memory network. Therefore, the method based on the long short-term memory network and adaptive moment estimation algorithm can work efficiently on pipeline events classification and has great guiding significance for safety assurance of natural gas transmission.

Modern decision-making in fixed income asset management benefits from intelligent systems, which involve the use of state-of-the-art machine learning models and appropriate methodologies. We conduct the first study of bond yield forecasting using long short-term memory (LSTM) networks, validating its potential and identifying its memory advantage. Specifically, we model the 10-year bond yield using univariate LSTMs with three input sequences and five forecasting horizons. We compare those with multilayer perceptrons (MLP), univariate and with the most relevant features. To demystify the notion of black box associated with LSTMs, we conduct the first internal study of the model. To this end, we calculate the LSTM signals through time, at selected locations in the memory cell, using sequence-to-sequence architectures, uni and multivariate. We then proceed to explain the states’ signals using exogenous information, for what we develop the LSTM-LagLasso methodology. The results show that the univariate LSTM model with additional memory is capable of achieving similar results as the multivariate MLP using macroeconomic and market information. Furthermore, shorter forecasting horizons require smaller input sequences and vice-versa. The most remarkable property found consistently in the LSTM signals, is the activation/deactivation of units through time, and the specialisation of units by yield range or feature. Those signals are complex but can be explained by exogenous variables. Additionally, some of the relevant features identified via LSTM-LagLasso are not commonly used in forecasting models. In conclusion, our work validates the potential of LSTMs and methodologies for bonds, providing additional tools for financial practitioners.

In order for the detection ability of floating small targets in sea clutter to be improved, on the basis of the complete ensemble empirical mode decomposition (CEEMD) algorithm, the high-frequency parts and low-frequency parts are determined by the energy proportion of the intrinsic mode function (IMF); the high-frequency part is denoised by wavelet packet transform (WPT), whereas the denoised high-frequency IMFs and low-frequency IMFs reconstruct the pure sea clutter signal together. According to the chaotic characteristics of sea clutter, we proposed an adaptive training timesteps strategy. The training timesteps of network were determined by the width of embedded window, and the chaotic long short-term memory network detection was designed. The sea clutter signals after denoising were predicted by chaotic long short-term memory (LSTM) network, and small target signals were detected from the prediction errors. The experimental results showed that the CEEMD-WPT algorithm was consistent with the target distribution characteristics of sea clutter, and the denoising performance was improved by 33.6% on average. The proposed chaotic long- and short-term memory network, which determines the training step length according to the width of embedded window, is a new detection method that can accurately detect small targets submerged in the background of sea clutter.

Seismic response prediction of RC bridge piers through stacked long short-term memory network

In human-machine interaction systems, continuous movement estimation methods occupy an important position because they are more natural and intuitive than pattern-recognition methods. Essentially, arm position is decided by the shoulder and elbow joint angles. However, the various deformations of muscles around the shoulder and elbow often lead to difficulties in sensor fixation, which results in a loss of synchronization between the surface electromyography (sEMG) signals and joint angles. In order to accurately estimate movement angles using sEMG in situations where the sEMG is not synchronized with joint angles, we utilized a bi-directional long short-term memory (Bi-LSTM) network rather than other deep learning methods to estimate non-dominant arm movements, based on the sEMG signal from the dominant arm. This estimation protocol was designed to avoid a multiplicity of sensors and to simulate more complicated loss of synchronization problems). The performance of the Bi-LSTM was compared with multilayer perceptrons (MLPs), convolutional neural networks (CNNs), and a long short-term memory network (LSTM). The Pearson correlation coefficient (cc) between the estimated and target joint angle sequences was calculated to evaluate the performance of each neural network. The Wilcoxon signed-rank results showed that the Bi-LSTM model significantly outperformed the MLP, CNN, and LSTM models (tested with completely untrained newly recorded free movements).

This paper aims to predict the occurrence of pests and diseases for cotton based on long short term memory (LSTM) network. First, the problem of occurrence of pests and diseases was formulated as time series prediction. Then LSTM was adopted to solve the problem. LSTM is a special kind of recurrent neutral network (RNN), which introduces gate mechanism to prevent the vanished or exploding gradient problem. It has been shown good performance in solving time series problem and can handle the long-term dependency problem, as mentioned in many literatures. The experimental results showed that LSTM performed good on the prediction of occurrence of pests and diseases in cotton fields, and yielded an Area Under the Curve (AUC) of 0.97. The paper further verified that the weather factors indeed have strong impact on the occurrence of pests and diseases, and the LSTM network has great advantage on solving the long-term dependency problem.

Aim and Background: In recent periods, micro-array data analysis using soft computing and machine learning techniques gained more interest among researchers to detect prostate cancer. Due to the small sample size of micro-array data with a larger number of attributes, traditional machine learning techniques face difficulty detecting prostate cancer. Methodology: The selection of relevant genes exploits useful information about micro-array data, which enhances the accuracy of detection. In this research, the samples are acquired from the gene expression omnibus database, particularly related to the prostate cancer GEO IDs such as GSE 21034, GSE 15484 and GSE 3325/GSE 3998. In addition, ensemble feature optimization technique and Bidirectional Long Short Term Memory (Bi-LSTM) network are employed for detecting prostate cancer from the microarray data of gene expression. Results: The ensemble feature optimization technique includes 4 metaheuristic optimizers that select the top 2000 genes from each GEO IDs, which are relevant to prostate cancer. Next, the selected genes are given to the Bi-LSTM network for classifying the normal and prostate cancer subjects. Conclusion: The simulation analysis revealed that the ensemble based Bi-LSTM network obtained 99.13%, 98.97%, and 94.12% of accuracy on the GEO IDs like GSE 3325/GSE 3998, GSE 21034, and GSE 15484.

As one of the key components in mechanical systems, rotatory machine plays a significant role in safe and stable operation. Accurate prediction of the Remaining Useful Life (RUL) of rotatory machine contributes to realization of intelligent operation and maintenance for mechanical manufacturing. In order to overcome the limitations of traditional machine learning algorithms in dealing with complex nonlinear signals, a novel prediction framework for RUL of rotatory machine based on deep learning is proposed in this paper. One-dimensional convolutional neural network is utilized to extract local features from the original signal sequence. In addition, the proposed framework analyzes sensor signals and predicts RUL by combining Long Short-Term Memory (LSTM) network with attention mechanism. Multi-layer LSTM is set up to extract useful temporal features layer by layer and improve the robustness of the model, while attention mechanism is able to effectively solve the problem of information loss in the long-distance signal transmission of LSTM. Through the feature extraction of multi-layer LSTM and the strong supervision ability of attention mechanism, the RUL of rotatory machine can be accurately predicted. The experimental results show that the proposed method for RUL estimation is efficient and has higher prediction accuracy than the traditional machine learning algorithms.

Accurately predicting the remaining useful life (RUL) of lithium-ion batteries is crucial for ensuring the safe and reliable operation of battery systems. However, due to the highly nonlinear degradation behavior of lithium-ion batteries under complex working conditions, traditional prediction methods often fail to achieve satisfactory accuracy. This study proposes a hybrid prediction model that integrates an attention (ATT) mechanism, a convolutional neural network (CNN), and a bidirectional long short-term memory (BiLSTM) network, termed ATT-CNN-BiLSTM. First, a one-dimensional CNN is employed to extract local degradation patterns and inter-channel correlated features from multi-dimensional sensor time-series data. Then, the BiLSTM network captures long-term dependencies within the degradation process. Furthermore, an attention mechanism is introduced to dynamically adjust the contribution of key time-step features and suppress noise interference. Experimental results based on the NASA battery aging datasets demonstrate that the proposed ATT-CNN-BiLSTM model reduces RMSE and MAE by 49.08% and 54.67%, respectively, compared with the LSTM model, and by 10.89% and 17.85%, respectively, compared with the ACNN-LSTM-MMD model. Ablation experiments also confirm that further confirm that the ATT-CNN-BiLSTM model decreases RMSE and MAE by 26.45% and 33.33%, respectively, compared with CNN-BiLSTM, and by 21.85% and 23.36%, respectively, compared with ATT-BiLSTM.

Multi-fidelity surrogate modeling using long short-term memory networks

A combined missing data filling approach based on the spectral analysis and the Long Short-Term Memory (LSTM) network is put forward in this paper to solve the data missing problem in wind speed. Firstly, the periodicity of wind speed data is determined by the periodogram and spectral density estimation results. Then two periodicity-related prediction filling strategies named the forward periodic prediction filling and the inverse periodic prediction filling are designed and realized through LSTM networks along with a non-periodicity-related sequence prediction filling strategy called the sequence prediction filling. Finally, the results of the three prediction filling models are combined according to the best weight vector obtained by the parameter optimization algorithm. Error comparison results demonstrate that the proposed approach performs well in wind speed missing data filling.

Correcting mean areal precipitation forecasts to improve urban flooding predictions by using long short-term memory network