Small-Data-Driven Temporal Convolutional Capsule Network for Locomotion Mode Recognition of Robotic Prostheses.

Recurrent neural network (RNN) has been widely used for sequential learning which has achieved a great success in different tasks. The temporal convolutional network (TCN), a variant of one-dimensional convolutional neural network (CNN), was also developed for sequential learning in presence of sequence data. RNN and TCN typically captures long-term and short-term features in temporal or spatial domain, respectively. This paper presents a new sequential learning, called the convolutional recurrent network (CRN), which fulfills TCN as an encoder and RNN as a decoder so that the global semantics as well as the local dependencies are simultaneously characterized from sequence data. To facilitate the interpretation and robustness in neural models, we further develop the stochastic modeling for CRN based on variational inference. The merits of CNN and RNN are then incorporated in inference of latent space which sufficiently produces a generative model for sequential prediction. Experiments on language model shows the effectiveness of stochastic CRN when compared with the other sequential machines.

Accurate load forecasting is essential for ensuring safe, stable, and economical operation of energy internet. Temporal convolutional networks (TCNs) have demonstrated superior performance, when compared to recurrent neural network models, since their introduction in electrical load forecasting. However, the current TCN‐based models are unable to obtain a large receptive field and strong long‐time feature extraction capability owing to the specific kernel size of the 1D convolution structure. This paper proposes a temporal inception convolutional network based on multi‐head attention (TICN‐Att) for ultra‐short‐term load prediction. By introducing an inception structure into the TCN, the proposed model can extract multi‐dimensional information from the input features, through the multiple hidden convolutional kernels of different scales, without stacking layers depth‐wise. Simultaneously, by introducing a multi‐head attention mechanism, the TICN‐Att model has a long time‐dependent extraction capability, similar to that of the long short‐term memory network models. The generalization and validity of the model are tested using the global energy forecasting competition (GEFCOM2014) dataset, electrical load data of a city in Jiangsu (China), and PJM power system dataset. The experimental results demonstrated that the proposed model has the best prediction effect, compared to the other state‐of‐the‐art models.

Masked multi-head self-attention for causal speech enhancement

Network intrusion detection plays an important role in the network security, but the increasingly complex network environment brings a serious challenge to intrusion detection. Although the existing efficient Convolutional Neural Network (CNN)-based network traffic intrusion detection models do not require manual design of the traffic features, but they do not make full use of the structured information of network traffic. In this paper, we propose a novel network intrusion detection model based on Temporal Convolutional Networks (TCN), it extracts the key features in the dataset through exploiting the characteristics of byte sequence in the network traffic packets. Compared with traditional recurrent neural networks (RNN), TCN shows a better performance in sequence modeling tasks and it can process the sequences in parallel for faster training. To solve the problem of poor detection accuracy caused by the "death" of some neurons on ReLU during the training stage, we use the ELU activation function in the TCN instead of ReLU. Finally, we compare our proposed TCN-based intrusion detection model with the state-of-the-art methods on the CTU public dataset, and the experimental results show that the use of TCN can obtain higher performance within less time consumption, in terms of higher average accuracy, higher average recall and higher average F1-measure.

Research on smartphone-based human activity recognition (HAR) is prevalent in the field of healthcare, especially for elderly activity monitoring. Researchers usually propose to use of accelerometers, gyroscopes or magnetometers that are equipped in smartphones as an individual sensing modality for human activity recognition. However, any of these alone is limited in capturing comprehensive movement information for accurate human activity analysis. Thus, we propose a smartphone-based HAR approach by leveraging the inertial signals captured by these three sensors to classify human activities. These heterogeneous sensors deliver information on various aspects of nature, motion and orientation, offering a richer set of features for more accurate representations of the activities. Hence, a deep learning approach that amalgamates long short-term memory (LSTM) in temporal convolutional network (TCN) is proposed. We use independent temporal convolutional networks, coined as temporal convolutional streams, to independently analyse the temporal data of each sensing modality. We name this architecture multi-stream TC-LSTM. The performance of multi-stream TC-LSTM is assessed on the self-collected elderly activity database. Empirical results exhibit that multi-stream TC-LSTM outperforms the existing machine learning and deep learning models, with an F1 score of 98.3 %

Multivariate time-series forecasting is one of the crucial and persistent challenges in time-series forecasting tasks. As a kind of data with multivariate correlation and volatility, multivariate time series impose highly nonlinear time characteristics on the forecasting model. In this paper, a new multivariate time-series forecasting model, multivariate temporal convolutional attention network (MTCAN), based on a self-attentive mechanism is proposed. MTCAN is based on the Convolution Neural Network (CNN) model, using 1D dilated convolution as the basic unit to construct asymmetric blocks, and then, the feature extraction is performed by the self-attention mechanism to finally obtain the prediction results. The input and output lengths of this network can be determined flexibly. The validation of the method is carried out with three different multivariate time-series datasets. The reliability and accuracy of the prediction results are compared with Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Convolutional Long Short-Term Memory (ConvLSTM), and Temporal Convolutional Network (TCN). The prediction results show that the model proposed in this paper has significantly improved prediction accuracy and generalization.

Data in time series format, such as biological signals from medical sensors or machine signals from sensors in industrial environments are rich sources of information that can give crucial insights on the present and future condition of a person or machine. The task of predicting future values of time series has been initially approached with simple machine learning methods, and lately with deep learning. Two models that have shown good performance in this task are the temporal convolutional network and the attention module. However, despite the promising results of deep learning methods, their black-box nature makes them unsuitable for real-world applications where the predictions need to be explainable in order to be trusted. In this paper we propose an architecture comprised of a temporal convolutional network with an attention mechanism that makes predictions while presenting the timesteps of the input that were most influential for future outputs. We apply it on two datasets and we show that we gain interpretability without degrading the accuracy compared to the original temporal convolutional models. We then go one step further and we combine our configuration with various machine learning methods on top, creating a pipeline that achieves interpretability both across timesteps and input features. We use it to forecast a different variable from one of the above datasets and we study how the accuracy is affected compared to the original black-box approach.

Due to the objectivity of emotional expression in the central nervous system, EEG-based emotion recognition can effectively reflect humans' internal emotional states. In recent years, convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have made significant strides in extracting local features and temporal dependencies from EEG signals. However, CNNs ignore spatial distribution information from EEG electrodes; moreover, RNNs may encounter issues such as exploding/vanishing gradients and high time consumption. To address these limitations, we propose an attention-based temporal graph representation network (ATGRNet) for EEG-based emotion recognition. Firstly, a hierarchical attention mechanism is introduced to integrate feature representations from both frequency bands and channels ordered by priority in EEG signals. Second, a graph convolutional neural network with top-k operation is utilized to capture internal relationships between EEG electrodes under different emotion patterns. Next, a residual-based graph readout mechanism is applied to accumulate the EEG feature node-level representations into graph-level representations. Finally, the obtained graph-level representations are fed into a temporal convolutional network (TCN) to extract the temporal dependencies between EEG frames. We evaluated our proposed ATGRNet on the SEED, DEAP and FACED datasets. The experimental findings show that the proposed ATGRNet surpasses the state-of-the-art graph-based mehtods for EEG-based emotion recognition.

In smart mariculture, traditional methods are not only difficult to adapt to the complex, dynamic and changeable environment in open waters, but also have many problems, such as poor accuracy, high time complexity and poor long-term prediction. To solve these deficiencies, a new water quality prediction method based on TCN (temporal convolutional network) is proposed to predict dissolved oxygen, water temperature, and pH. The TCN prediction network can extract time series features and in-depth data features by introducing dilated causal convolution, and has a good effect of long-term prediction. At the same time, it is predicted that the network can process time series data in parallel, which greatly improves the time throughput of the model. Firstly, we arrange the 23,000 sets of water quality data collected in the cages according to time. Secondly, we use the Pearson correlation coefficient method to analyze the correlation information between water quality parameters. Finally, a long-term prediction model of water quality parameters based on a time domain convolutional network is constructed by using prior information and pre-processed water quality data. Experimental results show that long-term prediction method based on TCN has higher accuracy and less time complexity, compared with RNN (recurrent neural network), SRU (simple recurrent unit), BI-SRU (bi-directional simple recurrent unit), GRU (gated recurrent unit) and LSTM (long short-term memory). The prediction accuracy can reach up to 91.91%. The time costs of training model and prediction are reduced by an average of 64.92% and 7.24%, respectively.

The accuracy of short-term electricity price forecasting has profound influence on the business decisions made by the participants in the electricity market. Thus, we propose a short-term electricity price combination prediction model based on Complete Ensemble Empirical Mode Decomposition(CEEMD) as well as Temporal Convolutional Attention-based Network(TCAN). First, we defined the correlations among the features according to the Pearson Correlation Analysis and selected the desired features for our model. We then defined the single prediction period to be 24 hours based on the Unit Root test. In addition, we decomposed and denoised the data based on CEEMD and built the attention mechanism into the temporal convolution network. Using the expansion cascade network to extract the relationship between the inputs, combined with the self-attention mechanism to extract the internal information and learn the position dependence, effectively improve the model training speed and prediction accuracy. In order to verify the validity of the model, the actual electricity price data of DK1 in the Nordic electricity market from April to May 2020 were forecasted. The results show that the proposed method can effectively improve the accuracy and reliability of electricity price forecasting, and can provide reliable decision-making basis.

Time series data are increasingly prevalent across domains such as finance, healthcare, manufacturing, and IoT, making accurate forecasting and anomaly detection critical for decision-making and system reliability. Traditional statistical methods (e.g., ARIMA, Holt-Winters) often fail to capture complex temporal dependencies and high-dimensional interactions inherent in modern time series. Recent advances in artificial intelligence particularly deep learning architectures such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), temporal convolutional networks (TCNs), graph neural networks (GNNs) and Transformers have demonstrated marked improvements in modeling both univariate and multivariate series, as well as in detecting anomalies that deviate from learned norms (Darban, Webb, Pan, Aggarwal, & Salehi, 2022; Chiranjeevi, Ramya, Balaji, Shashank, & Reddy, 2024) [1,2]. Moreover, ensemble techniques and hybrid signal-processing + deep-learning pipelines show enhanced sensitivity and adaptability in real-world anomaly detection scenarios (Iqbal, Amin, Alsubaei, & Alzahrani, 2024) [3]. In this work, we provide a unified survey and comparative analysis of AI-driven time series forecasting and anomaly detection methods, highlight key industrial application domains, evaluate performance trade-offs (e.g., accuracy vs. latency, supervised vs. unsupervised learning), and discuss emerging challenges including interpretability, data drift, real-time deployment on edge devices, and integration of causal reasoning. Our findings suggest that while AI approaches significantly outperform classical techniques in many settings, careful consideration of data characteristics, evaluation metrics and deployment environment remains essential for effective adoption.

Hand gestures play a vital role in sign language, serving as a primary means of non-verbal communication for individuals who are deaf or mute, enabling them to express themselves and interact with others. Although numerous sign language recognition systems have been developed, most are either costly or lack adaptability for real-world use. Recent advances in deep learning, particularly Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), have significantly improved gesture recognition by enabling systems to learn complex patterns from large datasets of images or video sequences. This paper introduces a novel algorithm called Sign Language Accessibility Enhancement Using Ensemble of Deep Learning Techniques for Hand Gesture Recognition in Deaf and Dumb Communication (SLAEDL-HGRDDC). The model combines multiple components into a robust framework to enhance recognition accuracy and system scalability. Initially, Gaussian Filtering (GF) is used to reduce noise and improve image quality. Then, Inception-ResNet-V2 is adopted to extract intricate spatial features from gesture images. The recognition process utilizes an ensemble of three deep learning classifiers – Long Short-Term Memory (LSTM), Gated Recurrent Units (GRUs), and Temporal Convolutional Networks (TCN). Finally, the Modified Al-Biruni Earth Radius Optimization (MBER) algorithm is employed for optimal hyperparameter tuning. Extensive simulations demonstrate that SLAEDL-HGRDDC outperforms traditional systems in various performance metrics.

The General Movements Assessment (GMA) is a non-invasive procedure to diagnose developmental defects that may lead to Cerebral Palsy (CP) and other physical disabilities in infants under the age of three months. This project aims to categorize movements of patients using deep neural networks (DNN) which in the long run can be used in automating GMA. The primary objective here is to determine the optimal neural network (NN) model for classifying videos. For this purpose, three network models were developed to assess their ability to manage the videos, namely, Long short-term memory (LSTM), Convolutional LSTM (ConvLSTM), and Temporal Convolutional Network (TCN). The implementation results suggested that using a TCN structure may be well-suited for use with videos. Convolutional neural networks (CNNs) are the most often used image classification architecture, and these systems often serve as the basis for all three versions. The results of the three models in video classifying indicate that classifying videos with TCN can be an effective and robust process, and the ConvLSTM model can also be very effective with further development.

The applications of deep learning and artificial intelligence have permeated daily life, with time series prediction emerging as a focal area of research due to its significance in data analysis. The evolution of deep learning methods for time series prediction has progressed from the Convolutional Neural Network (CNN) and the Recurrent Neural Network (RNN) to the recently popularized Transformer network. However, each of these methods has encountered specific issues. Recent studies have questioned the effectiveness of the self-attention mechanism in Transformers for time series prediction, prompting a reevaluation of approaches to LTSF (Long Time Series Forecasting) problems. To circumvent the limitations present in current models, this paper introduces a novel hybrid network, Temporal Convolutional Network-Linear (TCN-Linear), which leverages the temporal prediction capabilities of the Temporal Convolutional Network (TCN) to enhance the capacity of LSTF-Linear. Time series from three classical chaotic systems (Lorenz, Mackey-Glass, and Rossler) and real-world stock data serve as experimental datasets. Numerical simulation results indicate that, compared to classical networks and novel hybrid models, our model achieves the lowest RMSE, MAE, and MSE with the fewest training parameters, and its R2 value is the closest to 1.

Monaural Speech Enhancement Using a Multi-Branch Temporal Convolutional Network