A novel NMF-DiCCA deep learning method and its application in wind turbine blade icing failure identification

Recently, Brain-Computer Interface (BCI) technology has been applied more and more in the field of clinical rehabilitation, which provides an effective way of communication for patients with brain disability and stroke. Because electroencephalogram (EEG) signals are extremely complex and contain many redundant signals, the recognition effect of BCI system based on EEG is not very well. The purpose of this paper is to solve the problem of brain intention classification in brain-computer interface system, which combines Deep Separation Convolutional Neural Network (DSCNN) and Gate Recurrent Unit (GRU) network to classify the motor imagination task of EEG. Firstly, one-dimensional timing EEG signals are transformed into two-dimensional array, and the temporal and spatial features of EEG signals are extracted by separate convolution. Then, these EEG signals containing spatio-temporal features are convolved once to extract spatial features, and the temporal features is extracted by GRU. Finally, the experiments shows that the final intention recognition accuracy reach 97.76% via the open physiological motor imagery data set EEGMMIDB, which is superior to some advanced research methods for motor imagery task recognition at present and helpful to restore the rehabilitation ability of patients with brain injury.

BackgroundThe prediction of coronavirus disease in 2019 (COVID-19) in broader regions has been widely researched, but for specific areas such as urban areas the predictive models were rarely studied. It may be inaccurate to apply predictive models from a broad region directly to a small area. This paper builds a prediction approach for small size COVID-19 time series in a city.MethodsNumbers of COVID-19 daily confirmed cases were collected from November 1, 2022 to November 16, 2023 in Xuzhou city of China. Classical deep learning models including recurrent neural network (RNN), long and short-term memory (LSTM), gated recurrent unit (GRU) and temporal convolutional network (TCN) are initially trained, then RNN, LSTM and GRU are integrated with a new attention mechanism and transfer learning to improve the performance. Ten times ablation experiments are conducted to show the robustness of the performance in prediction. The performances among the models are compared by the mean absolute error, root mean square error and coefficient of determination.ResultsLSTM outperforms than others, and TCN has the worst generalization ability. Thus, LSTM is integrated with the new attention mechanism to construct an LSTMATT model, which improves the performance. LSTMATT is trained on the smoothed time series curve through frequency domain convolution augmentation, then transfer learning is adopted to transfer the learned features back to the original time series resulting in a TLLA model that further improves the performance. RNN and GRU are also integrated with the attention mechanism and transfer learning and their performances are also improved, but TLLA still performs best.ConclusionsThe TLLA model has the best prediction performance for the time series of COVID-19 daily confirmed cases, and the new attention mechanism and transfer learning contribute to improve the prediction performance in the flatten part and the jagged part, respectively.

This study examines the predictive performance of Feature Attention (FA), Temporal Attention (TA), and Feature and Temporal Attention (FATA) within Gated Recurrent Unit (GRU), Long Short‐Term Memory (LSTM), and Transformer architectures using price data from four Chinese carbon markets (CEA, BEA, GDEA, and HBEA). Drawing on multiple forecasting accuracy measures and significance testing, the results show that attention mechanisms can enhance forecasting accuracy in certain market‐model combinations, but their effectiveness critically depends on the alignment among market conditions, model architectures, and attention mechanisms. In markets with high average prices and volatility, FA achieves the best performance with GRU and LSTM; in lower price, moderately volatile markets, TA combined with Transformer is more effective; and in the high‐price, high‐volatility CEA market, FATA shows promise when paired with Transformer, but lacks robustness across markets. These findings highlight a pronounced compatibility pattern among market conditions, model architectures, and attention mechanisms, suggesting that the deployment of attention mechanisms in carbon price forecasting should be tailored to specific market conditions and model structures rather than applied universally.

To address the limitations of traditional models in capturing complex features for concrete strength prediction, this study proposes a hybrid deep learning approach that integrates multiple attention mechanisms with gated recurrent units (GRU). The methodology employs a multi-scale validation framework, conducting three-dimensional validation across three datasets: the Kaggle standard dataset, the lightweight foam concrete dataset, and the self-compacting concrete dataset. Six attention mechanisms (SE attention, dot-product attention, self-attention, etc.) are comprehensively compared to optimise the GRU network structure. A Newton–Raphson-based optimiser (NRBO) enables hyperparameter adaptive tuning. Experimental results show significant improvements over the baseline GRU model: mean R2 increased by 6.99%, while RMSE and MAE decreased by 38.5% and 37.5%, respectively. SHAP interpretability analysis confirms that attention mechanisms effectively capture key parameters like SP and VMA in the self-compacting concrete dataset. Based on the findings, this study recommends using self-attention for datasets smaller than 200 samples and selecting the higher-accuracy model between self-attention and stacked attention mechanisms for larger datasets.

Deep learning has been applied to the recognition of motor imagery electroencephalograms (MI-EEG) in brain-computer interface, and the performance results depend on data representation as well as neural network structure. Especially, MI-EEG is so complex with the characteristics of non-stationarity, specific rhythms, and uneven distribution; however, its multidimensional feature information is difficult to be fused and enhanced simultaneously in the existing recognition methods. In this paper, a novel channel importance (NCI) based on time-frequency analysis is proposed to develop an image sequence generation method (NCI-ISG) for enhancing the integrity of data representation and highlighting the contribution inequalities of different channels as well. Each electrode of MI-EEG is converted to a time-frequency spectrum by utilizing short-time Fourier transform; the corresponding part to 8-30Hz is combined with random forest algorithm for computing NCI; and it is further divided into three sub-images covered by α (8-13Hz), β1 (13-21Hz), and β2 (21-30Hz) bands; their spectral powers are further weighted by NCI and interpolated to 2-dimensional electrode coordinates, producing three main sub-band image sequences. Then, a parallel multi-branch convolutional neural network and gate recurrent unit (PMBCG) is designed to successively extract and identify the spatial-spectral and temporal features from the image sequences. Two public four-class MI-EEG datasets are adopted; the proposed classification method respectively achieves the average accuracies of 98.26% and 80.62% by 10-fold cross-validation experiment; and its statistical performance is also evaluated by multi-indexes, such as Kappa value, confusion matrix, and ROC curve. Extensive experiment results show that NCI-ISG + PMBCG can yield great performance on MI-EEG classification compared to state-of-the-art methods. The proposed NCI-ISG can enhance the feature representation of time-frequency-space domains and match well with PMBCG, which improves the recognition accuracies of MI tasks and demonstrates the preferable reliability and distinguishable ability. This paper proposes a novel channel importance (NCI) based ontime-frequencyanalysis to develop an image sequences generation method (NCI-ISG) for enhancing the integrity of data representation and highlighting the contribution inequalities of different channels as well. Then, a parallel multi-branch convolutional neural network and gate recurrent unit (PMBCG) is designed to successively extract and identify the spatial-spectral and temporal features from the image sequences.

Recurrent neural networks are widely used in sequential data modeling. For example, long-short term memory network (LSTM) and gated recurrent unit(GRU) are two typical traffic prediction methods. It is noted that the gate structure is the key component of LSTM and GRU while increasing the number of training parameters against traditional RNN. In this paper, inspired by the function of attention mechanism in regulating information flow, we propose a simple yet effective method for traffic prediction which embeds the attention mechanism within the recurrent module attempting to focus on the important information of inside features. The proposed model structure is named as RAU, which is short for the recurrent attention unit. We evaluated the proposed methods on five real-world datasets. Extensive experiments show that the proposed method can achieve comparable prediction performances against LSTM and GRU, while decrease more than 30% and 50% parameters, respectively.

At present, the amount of network equipment, servers, and network traffic is increasing exponentially, and the way in which operators allocate and efficiently utilize network resources has attracted considerable attention from traffic forecasting researchers. However, with the advent of the 5G era, network traffic has also shown explosive growth, and network complexity has increased dramatically. Accurately predicting network traffic has become a pressing issue that must be addressed. In this paper, a multilayer perceptron ensemble learning method based on convolutional neural networks (CNN) and gated recurrent units (GRU) spatiotemporal feature extraction (MECG) is proposed for network traffic prediction. First, we extract spatial and temporal features of the data by convolutional neural networks (CNN) and recurrent neural networks (RNN). Then, the extracted temporal features and spatial features are fused into new spatiotemporal features through integrated learning of a multilayer perceptron, and a spatiotemporal prediction model is built in the sequence-to-sequence framework. At the same time, the teacher forcing mechanism and attention mechanism are added to improve the accuracy and convergence speed of the model. Finally, the proposed method is compared with other deep learning models for experiments. The experimental results show that the proposed method not only has apparent advantages in accuracy but also shows some superiority in time training cost.

Predicting remaining useful life of a machine based on embedded attention parallel networks

With the aim of enhancing the accuracy of current models for forecasting vegetable prices and improving market structures, this study focuses on the prices of bell peppers at the Nanhuanqiao Market in Suzhou. In this paper, we propose a hybrid Convolutional Neural Network (CNN) and Gated Recurrent Unit (GRU) model for vegetable price forecasting based on Principal Component Analysis (PCA) and Attention Mechanism (ATT). Initially, we utilized the Pearson correlation coefficient to filter out the factors impacting prices. Then, we applied PCA to reduce dimensionality, extracting key price features. Next, we captured local sequence patterns with CNN, while handling time-series features with GRU. Finally, these outputs were integrated via ATT to generate the final prediction. Our results indicate that the hybrid CNN-GRU model, enhanced by PCA and ATT, achieved a Root Mean Square Error (RMSE) as low as 0.1642. This performance is 11.11%, 11.11%, and 15.79% better than that of the PCA-CNN, PCA-GRU, and CNN-GRU-ATT models, respectively. Furthermore, in order to prove the effectiveness of our proposed model, the proposed model is compared with the state-of-the-art models and classical machine learning algorithms under the same dataset, the results indicate that our proposed hybrid deep learning model based on PCA and ATT shows the best performance. Consequently, our model offers a valuable reference for vegetable price prediction.

Aiming to address the suboptimal diagnostic efficacy of conventional approaches in rolling bearing fault diagnosis under complex operational scenarios—particularly when subjected to intense noise contamination—this study introduces a novel fault diagnosis framework leveraging the synergistic attention mechanisms of MSCNN and GRU.The workflow begins with conditioning vibration signals sourced from the CRUW bearing dataset, followed by robust extraction of multi-scale fault features through a multi-scale convolutional neural network (MSCNN). To enhance temporal modeling, a gated recurrent unit (GRU) is integrated to extract temporal dependencies from the signals. Further improvements are achieved by enhancing the base model with Transformer layers and a convolutional attention module (CBAM), respectively. Experimental validation is conducted to compare the diagnostic performance of three configurations MSCNN+GRU, MSCNN+GRU+Transformer, and MSCNN+GRU+CBAM—under varying noise conditions. Outcomes demonstrate that the MSCNN+GRU+CBAM model exhibits superior fault detection accuracy and optimal generalization capacity across different load and rotational speed settings, confirming its notable advantage in rolling bearing fault diagnosis.

Evaluating cognitive load using electroencephalogram (EEG) signals is a crucial research area in the field of Brain-Computer Interfaces (BCI). However, achieving high accuracy and generalization in feature extraction and classification for cognitive load assessment remains a challenge, primarily due to the low signal-to-noise ratio of EEG signals and the inter-individual variability in EEG data. In this study, we propose a novel deep learning architecture that integrates temporal information features and functional connectivity features to enhance EEG signal analysis. Functional connectivity features capture inter-channel information, while temporal features are extracted from continuous signal segments using a Long Short-Term Memory (LSTM) network enhanced with an attention mechanism. The fusion strategy combines these two information streams to leverage their complementary strengths, resulting in improved classification performance. We evaluated our architecture on two publicly available datasets, and the results demonstrate its robustness in cognitive load recognition. Achieving performance comparable to the best existing methods on two public datasets. Ablation studies further substantiate the contributions of each module, demonstrating the importance of combining temporal and functional connectivity features for optimal results. These findings underscore the robustness and versatility of the proposed approach, paving the way for more effective EEG-based BCI applications.

Continuous annealing transition state evaluation of ultra-high strength steel via deep clustering network with time-series data

Previous research on reviews ranking has two branches: supervised method and unsupervised method. In this paper, we propose a supervised method to predict the review's importance, and the latent structure features are used to represent the review's writing styles. Our method mainly consists of three stages: Extracting lexical features to represent the review's lexical structure; Using lexical features as the input of a variant LDA topic model to learn latent structure features to represent review's writing styles; Obtaining the importance of a single review by a classifier. The experimental results show that our method performs better than the unsupervised baseline in accuracy, and our method is also more robust when compared with ordinary supervised baseline.

With the urbanization, urban transportation has become a key factor restricting the development of a city. In a big city, it is important to improve the efficiency of urban transportation. The key to realize short-term traffic flow prediction is to learn its complex spatial correlation, temporal correlation and randomness of traffic flow. In this paper, the convolution neural network (CNN) is proposed to deal with spatial correlation among different regions, considering that the large urban areas leads to a relatively deep Network layer. First three gated recurrent unit (GRU) were used to deal with recent time dependence, daily period dependence and weekly period dependence. Considering that each historical period data to forecast the influence degree of the time period is different, three attention mechanism was taken into GRU. Second, a two-layer full connection network was applied to deal with the randomness of short-term flow combined with additional information such as weather data. Besides, the prediction model was established by combining these three modules. Furthermore, in order to verify the influence of spatial correlation on prediction model, an urban functional area identification model was introduced to identify different functional regions. Finally, the proposed model was validated based on the history of New York City taxi order data and reptiles for weather data. The experimental results show that the prediction precision of our model is obviously superior to the mainstream of the existing prediction methods.

Given the substantial volatility and non-stationarity of cryptocurrency prices, forecasting them has become a complex task within the realm of financial time series analysis. This study introduces an innovative hybrid prediction model, VMD-AGRU-RESVMD-LSTM, which amalgamates the disintegration–integration framework with deep learning techniques for accurate cryptocurrency price prediction. The process begins by decomposing the cryptocurrency price series into a finite number of subseries, each characterized by relatively simple volatility patterns, using the variational mode decomposition (VMD) method. Next, the gated recurrent unit (GRU) neural network, in combination with an attention mechanism, predicts each modal component’s sequence separately. Additionally, the residual sequence, obtained after decomposition, undergoes further decomposition. The resultant residual sequence components serve as input to an attentive GRU (AGRU) network, which predicts the residual sequence’s future values. Ultimately, the long short-term memory (LSTM) neural network integrates the predictions of modal components and residuals to yield the final forecasted price. Empirical results obtained for daily Bitcoin and Ethereum data exhibit promising performance metrics. The root mean square error (RMSE) is reported as 50.651 and 2.873, the mean absolute error (MAE) stands at 42.298 and 2.410, and the mean absolute percentage error (MAPE) is recorded at 0.394% and 0.757%, respectively. Notably, the predictive outcomes of the VMD-AGRU-RESVMD-LSTM model surpass those of standalone LSTM and GRU models, as well as other hybrid models, confirming its superior performance in cryptocurrency price forecasting.