Adaptive temporal convolutional network with multi-head EMA-gated attention for continuous radar-based human activity recognition

Recently, human activity recognition (HAR) has gained significant attention as a research field, leading to the development of diverse technologies driven by its broad range of application scenarios. Radar technology has attracted much attention because of its unique advantages such as not being limited by environmental conditions such as light, shadow, and occlusion. In this article, a continuous HAR system based on multidomain radar data fusion (CMDN) is proposed. Firstly, in order to capture more detailed motion features of the human body, we apply the short-time fractional Fourier transform (STFrFT) to map radar data into the fractional domain, yielding a novel representation of human motion. Secondly, we develop an activity detector based on variable window length short-time average/long-time average (VW-STA/LTA) to accurately identify the start/end points of continuous human actions, addressing the challenge of difficult sequence segmentation in continuous activity recognition tasks. Finally, based on the multi-input multitask (MIMT) recognition network, the features of each domain are processed in parallel, and multiple input representations are fused to obtain the continuous activity classification results with high precision.

Motor imagery electroencephalogram (MI-EEG) decoding algorithms face multiple challenges. These include incomplete feature extraction, susceptibility of attention mechanisms to distraction under low signal-to-noise ratios, and limited capture of long-range temporal dependencies. To address these issues, this paper proposes a multi-branch differential attention temporal network (MDAT-Net). First, the method constructed a multi-branch feature fusion module to extract and fuse diverse spatio-temporal features from different scales. Next, to suppress noise and stabilize attention, a novel multi-head differential attention mechanism was introduced to enhance key signal dynamics by calculating the difference between attention maps. Finally, an adaptive residual separable temporal convolutional network was designed to efficiently capture long-range dependencies within the feature sequence for precise classification. Experimental results showed that the proposed method achieved average classification accuracies of 85.73%, 90.04%, and 96.30% on the public datasets BCI-IV-2a, BCI-IV-2b, and HGD, respectively, significantly outperforming several baseline models. This research provides an effective new solution for developing high-precision motor imagery brain-computer interface systems.

Masked multi-head self-attention for causal speech enhancement

Applications like smart homes, security surveillance, healthcare, and workplace safety now depend heavily on Human Activity Recognition (HAR) and monitoring. Automated systems can respond quickly, stop dangerous situations, and provide individualized services when they can precisely identify and categorize human activity. The intricate temporal dependencies present in human motion are frequently difficult to capture by conventional HAR techniques that rely on hand-crafted features and shallow learning models [1]. This paper investigates the effectiveness of a pure Recurrent Neural Network (RNN) architecture for activity recognition using smartphone sensor data, despite the fact that deep learning models like CNNs and LSTMs have dominated recent HAR research. Without the need for manually created features, we suggest a simplified RNN-based framework that can identify temporal dependencies in unprocessed accelerometer and gyroscope data. Pure RNNs can attain competitive accuracy, as shown by experiments on the UCI HAR and WISDM datasets, which yielded results of 92.4% and 90.7%, respectively. The results imply that, when properly optimized, RNNs despite their simplicity remain feasible for HAR tasks.

Effectively recognizing human activities requires at least 32 joint related degrees of freedom to be estimated so as to reliably track the human body in 3D. The particle filter is robust to distracting clutter by maintaining multiple hypotheses for each of these joint angles. Real-time tracking is difficult however with the computational overhead of such a large search space. This paper optimizes this search space utilizing feedback from a continuous human activity recognition (CHAR) system and improves the robustness and efficiency of each particle calculation using a novel body model. The joint angles are estimated for the next frame using a particle filter with forward smoothing. A new paradigm enables the temporal segmentation of continuous motion into dynemes. Using HMM, the CHAR system attempts to infer the human movement skill that could have produced the observed sequence of dynemes. Hundreds of movement skills, from gait to saltos, are successfully tracked and recognized.

Human activity recognition using mobile sensors is becoming increasingly important. Scaling up from individuals to groups, that is, Group Activity Recognition (GAR), has attracted significant attention recently. This paper investigates energy consumption for GAR and proposes a novel distributed middleware called GroupSense for mobile GAR. We implemented and tested GroupSense, which incorporates a protocol for the exchange of information required for GAR. We also investigated the battery drain of continuous activity recognition in a range of simple GAR scenarios. We then conclude with lessons learnt for GAR.

Research on smartphone-based human activity recognition (HAR) is prevalent in the field of healthcare, especially for elderly activity monitoring. Researchers usually propose to use of accelerometers, gyroscopes or magnetometers that are equipped in smartphones as an individual sensing modality for human activity recognition. However, any of these alone is limited in capturing comprehensive movement information for accurate human activity analysis. Thus, we propose a smartphone-based HAR approach by leveraging the inertial signals captured by these three sensors to classify human activities. These heterogeneous sensors deliver information on various aspects of nature, motion and orientation, offering a richer set of features for more accurate representations of the activities. Hence, a deep learning approach that amalgamates long short-term memory (LSTM) in temporal convolutional network (TCN) is proposed. We use independent temporal convolutional networks, coined as temporal convolutional streams, to independently analyse the temporal data of each sensing modality. We name this architecture multi-stream TC-LSTM. The performance of multi-stream TC-LSTM is assessed on the self-collected elderly activity database. Empirical results exhibit that multi-stream TC-LSTM outperforms the existing machine learning and deep learning models, with an F1 score of 98.3 %

Human action recognition is used in areas such as surveillance, entertainment, and healthcare.This paper proposes a system to recognize both single and continuous human actions from monocular video sequences, based on 3D human modeling and cyclic hidden Markov models (CHMMs).First, for each frame in a monocular video sequence, the 3D coordinates of joints belonging to a human object, through actions of multiple cycles, are extracted using 3D human modeling techniques.The 3D coordinates are then converted into a set of geometrical relational features (GRFs) for dimensionality reduction and discrimination increase.For further dimensionality reduction, k-means clustering is applied to the GRFs to generate clustered feature vectors.These vectors are used to train CHMMs separately for different types of actions, based on the Baum-Welch re-estimation algorithm.For recognition of continuous actions that are concatenated from several distinct types of actions, a designed graphical model is used to systematically concatenate different separately trained CHMMs.The experimental results show the effective performance of our proposed system in both single and continuous action recognition problems.

Bearings play a crucial role in rotating machinery, and their failure significantly impacts performance and safety. Early fault detection is challenging due to weak signals, especially under variable operating conditions. Deep learning techniques, such as Convolutional Neural Networks (CNN) and attention mechanisms, have shown great potential in improving fault detection accuracy in recent years. However, handling complex fault patterns and extracting early fault features remain challenging. To address these issues, this paper proposes a new method that combines Temporal Convolutional Networks (TCN), CNN, and Multi-Head Attention. TCN captures short-term dependencies, CNN extracts long-term features, and Multi-Head Attention enhances focus on key fault-related features. The model was tested on the Case Western Reserve University (CWRU) bearing fault dataset and compared with baseline models, showing a significant improvement in accuracy, achieving 99.3%. This study provides valuable insights for handling complex and multi-fault scenarios, offering essential implications for industrial applications.

Sequential attention mechanism for weakly supervised video anomaly detection

Traffic prediction has attracted a lot of attention in recent years. However, it is challenging due to the dynamic and heterogeneous spatial–temporal correlations. Most of the existing methods adopt the attention mechanism to capture the dynamic spatial features, but it is difficult for the attention mechanism to learn the real-time change of spatial dependencies, which restrict accurate spatial dependencies learning. Furthermore, most methods are out at elbows when solving heterogeneous spatial–temporal data. To overcome these problems, we propose a novel traffic prediction model called Dynamic spatial–temporal Heterogeneous Graph Convolution Network. Different from the existing methods, we have designed a dynamic localized graph and a corresponding adaptive localized graph convolution network, which are capable of simultaneously capturing dynamic and heterogeneous spatial correlations. We propose a gated adaptive temporal convolution network to capture the temporal heterogeneity of traffic data and enjoy global receptive fields. Finally, a global correlations fusion network is provided to incorporate the global spatial–temporal correlations. Compared with 11 baselines, our proposed model achieves state-of-the-art performance in the accuracy of prediction.

Temporal graph convolutional network for multi-agent reinforcement learning of action detection

The existing literature on forecasting time series data is primarily based on univariate analysis and techniques such as Univariate Autoregressive (UAR), Univariate Moving Average (UMA), Simple Exponential Smoothing (SES), deep learning models, and, most notably, univariate Long Short-Term Memory (LSTM) built based on univariate variable where the next lag of time series is leveraged for forecasting the next cycle of data. This paper takes this line of research to the next level by focusing on forecasting time series data based on “multivariate” modeling and analysis. To have a better insight of the performance of various deep learning-based models when multivariate analysis is performed, the paper builds and reports the forecasting accuracy for techniques such as the Transformer-based Multi-head Attention network, Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (BI-LSTM), Temporal Convolution Network (TCN), and conventional Vector Autoregressive (VAR) models. The findings revealed that the TCN model achieved the average lowest RMSE values of 0.0589 for stock data and 0.1554 for cryptocurrency data. Notably, the Multi-Head Attention model achieved average <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$R^{2}$ </tex-math></inline-formula> values of 0.92 for stock data and −1.98 for cryptocurrency data with respect to five variables (i.e., open, high, low, close and volume). According to the empirical studies conducted and reported in this paper, the transformer-based Multi-head Attention network outperformed other models such as LSTM, BI-LSTM, and more importantly conventional Vector Auto-Regression Models (VAR) in stocks and cryptocurrencies time series data where several variables were leveraged in building these multivariate-based models.

The major issues with business cycle value prediction are that the data sources are heterogeneous, complex spatiotemporal dependencies, and a lack of long-term prediction accuracy. The model of dynamic prediction, which is developed in this paper, is founded on the fusion of multimodal data and deep learning and incorporates four kinds of heterogeneous data, financial indicators, market transactions, macroeconomic data, and textual sentiment. The former applies a Temporal Convolutional Network (TCN) to learn the temporal features of each modality and a Multi-Head Attention (MHA) mechanism to learn cross-modal interaction relationships. Second, gated fusion unit is intended to weigh and fuse heterogeneous features dynamically and Bidirectional Long Short-Term Memory (BiLSTM) network is employed to model cycle evolution law. Lastly, residual connections with Dropout regularization are also added to enhance the generalization capacity of the model. The model on an empirical dataset of 3500 companies in the 2010-2024 period had a mean absolute percentage error (MAPE) of 6.2% and root mean square error (RMSE) of 0.043 in the expansion phase and 8.7% in the contraction phase, giving it a useful tool and theoretical foundation on dynamic monitoring of business cycle and corporate decision-making.

Traffic speed prediction is difficult because of the complicated dynamic spatiotemporal correlations. Recent studies in spatiotemporal models have achieved impressive outcomes for traffic speed prediction. But many studies use graphs in graph convolutional networks to learn spatial features that are often static. Additionally, effectively modeling long-range temporal features is crucial for prediction accuracy. In order to overcome these challenges, a Spatial–Temporal Adaptive Graph Convolutional Network with Transformer (ST_AGCNT) is designed in this paper. Specifically, an adaptive graph convolution network (AGCN) is designed to extract spatial dependency. An adaptive graph that fuses predefined matrices and learnable matrix is proposed to learn the correlations between nodes. The predefined matrices provide the model with richer prior information, while the learnable matrix can extract the dynamic nature of the nodes. And a temporal transformer (TT) is proposed to extract the long-range temporal dependency. In addition, to learn more information to achieve better results, different historical segments are modeled. Experiments conducted on a real-world traffic dataset confirm the effectiveness of the proposed model when compared to other baseline models. This model demonstrated excellent performance in prediction tasks across different time steps, effectively accomplishing traffic speed forecasting. It provides data support for improving traffic efficiency and reducing resource waste, contributing to the sustainable development of traffic management.