STCN: A Lightweight Sleep Staging Model with Multiple Channels

A deep learning framework for constitutive modeling based on temporal convolutional network

Veg-W2TCN: A parallel hybrid forecasting framework for non-stationary time series using wavelet and temporal convolution network model

Monaural Speech Enhancement Using a Multi-Branch Temporal Convolutional Network

Hybrid solar radiation forecasting model with temporal convolutional network using data decomposition and improved artificial ecosystem-based optimization algorithm

Accurate short-term GHI forecasting using a novel temporal convolutional network model

Accurate and reliable estimation of reference crop evapotranspiration (ET0) in the North Henan Plain is crucial for agricultural water resource management, production, and food supply in China. This study aims to evaluate the performance of deep learning (DL) methods in ET0 estimation and assess the applicability of the developed DL model beyond the training domain. This study utilized historical meteorological data from Zhengzhou City, northern Henan, spanning 2010–2024. Meteorological variables were selected through correlation analysis and maximum information coefficient (MIC). A novel DL model—the TCN-Attention model (TA)—was constructed by incorporating a self-attention mechanism into the temporal convolutional network (TCN) model. This model was compared with two classical DL models—Long Short-Term Memory (LSTM) and TCN. Results indicate: (1) Sunshine duration (n), relative humidity (RH), and maximum temperature (Tmax) are the three most significant features influencing summer maize evapotranspiration; (2) prediction accuracy under the same input scenarios: TA model > TCN model > LSTM model; (3) in scenarios where only temperature data is input, the TA model has the highest prediction accuracy, surpassing the H-S empirical method; and (4) for limited meteorological data, the combination of temperature and humidity was found to be most effective, showing good adaptability and accuracy at different time steps (hourly: R2 = 0.982; daily: R2 = 0.975; weekly: R2 = 0.928). This study highlights the potential of the TA model for estimating reference crop evapotranspiration in the northern Henan Plain, which may provide theoretical guidance for crop irrigation management under future climate change.

With the development and popularization of online shopping, a large number of commodities on the online retail platform should be managed through the multilevel category system, in which there are thousands of categories. Automatic commodity classification has become an issue. Classifying commodities according to their text titles can improve the efficiency of online retail platforms. The text titles of commodities contain the information of commodities, but the text titles seldom follow the grammar rules and the text length varies greatly. We propose a classification model, Temporal Multiple-Convolutional Network (TMN), which combines Temporal Convolutional Network(TCN) model and Multiple-Convolutional Neural Network(MCNN) model. The TCN model is firstly used to model sequence data based on both word embedding and character embedding respectively, and generate two output vectors. Then, the MCNN models are used to extract features from the abstract vectors. Finally, output layer employs softmax function to classify commodities. We conduct experiment on the online retail platform dataset provided by Inspur Group Co., Ltd. The results show that the TMN has a commodity classification accuracy of 84.2%, which is superior to that of state-of-the-art models.

Masked multi-head self-attention for causal speech enhancement

SummaryThe usage of multi‐rotor unmanned aerial vehicles (UAVs) has rapidly increased in a variety of industries recently, which has caused a quick rise in the quantity of research works on the topic. Remaining battery life prediction, anomaly detection and instantaneous power consumption prediction are among the topics that attract the most attention of researchers. This article presents the development and utilization of a modified temporal convolutional network (TCN) model, a commonly employed approach for anomaly detection and instantaneous power consumption prediction in unmanned aerial vehicles (UAVs). The first modification to the TCN model was to use the Leaky ReLu activation function in place of the rectified linear unit (ReLu) activation function from the original TCN model. In the next step, instantaneous power consumption prediction for UAVs was performed by using the data obtained from both sensors and simple moving average (SMA) algorithm. As a result of the tests performed with the created simulation setup, it has been clearly shown that the proposed method gives better results compared to other deep learning models used for comparison with the lowest RMSE of 0.0496.

A recurrent neural network (RNN) model is developed for simulating and predicting shear behavior of both a fine-grained residual soil and a dune sand. The RNN model with one hidden layer of 20 nodes appears very effective in modeling complex soil behavior, due to its feedback connections from a hidden layer to an input layer. A dynamic gradient descent learning algorithm is used to train the network. By training part of the experimental data, which include strain-controlled undrained tests and stress-controlled drained tests performed on a residual Hawaiian volcanic soil, the network is able to capture significant variability of shear behavior existing in the residual soil. The unusual characteristics that the denser soil samples dilate under a higher stress level and the looser soil samples contract under a lower stress level are well represented by the RNN model. The RNN model also shows encouraging results in simulation and prediction of behavior of a dune sand which experienced loading-unloading-reloading conditions. Excellent agreements between the measured data and the modeling results are observed in both stress-strain behavior and volumetric-change characteristics. As compared with a traditional model, the RNN model shows more effectiveness and less effort.Key words: neural network, modeling, soil behavior, shear tests, simulation, prediction.

Accurate production forecasting is crucial for effective reservoir management and field development planning. Traditional methods; such as numerical simulation, material balance equations coupled with well models, and advanced decline curve analysis (DCA) require extensive data and involve tradeoffs between accuracy and complexity. Recently, deep learning techniques, specifically artificial neural network (ANN) and recurrent neural network (RNN) models have demonstrated promising performance in this domain. This study applies ANN and RNN for production forecasting and compares their effectiveness with traditional DCA models.An actual production dataset for a gas field containing 13 wells is used to compare ANN and RNN models to traditional decline curve analysis models. After thorough data cleaning and preparation, the dataset was divided into training (historical) and test data sets. The training data was used to build ANN and RNN models in addition to estimating the model parameters for traditional DCA models. Traditional DCA models that were investigated include Arps (exponential, hyperbolic, and harmonic), Duong, Power Law Exponential Decline (PLE), and Stretched Exponential Decline (SEPD). Finally, the prediction of different models was evaluated and compared using the testing dataset. The comparison also included long, medium, and short-term prediction.Based on the comparison of daily gas production rate forecast, the ANN model demonstrated notable improvements over Arps models for the majority of wells, with a mean absolute percent error (MAPE) of 3.9% compared to 9.1%. However, across all wells, the average MAPE was slightly lower at 6.3% versus 8.1%. Conversely, the RNN model exhibited varied performance compared to Arps models. It showed poorer long-term performance (13% versus 4.5%), similar performance in the medium term (4.4% versus 4.2%), and superior performance in the short term (2.7% versus 4.2%). Furthermore, among DCA models specifically developed for unconventional wells, SEPD exhibited nearly identical performance to Arps (8.3% compared to 8.1%). In contrast, Duong model demonstrated the poorest performance with MAPE of 21.9%.This study provides insights into model analysis and introduces guidelines in using artificial and recurrent neural networks for predicting production rates. The ANN and RNN models were developed through careful testing with different designs and techniques. All the models in this research were created using Python scripts and powerful open-source libraries like SciPy and TensorFlow.

Accurate load forecasting is essential for ensuring safe, stable, and economical operation of energy internet. Temporal convolutional networks (TCNs) have demonstrated superior performance, when compared to recurrent neural network models, since their introduction in electrical load forecasting. However, the current TCN‐based models are unable to obtain a large receptive field and strong long‐time feature extraction capability owing to the specific kernel size of the 1D convolution structure. This paper proposes a temporal inception convolutional network based on multi‐head attention (TICN‐Att) for ultra‐short‐term load prediction. By introducing an inception structure into the TCN, the proposed model can extract multi‐dimensional information from the input features, through the multiple hidden convolutional kernels of different scales, without stacking layers depth‐wise. Simultaneously, by introducing a multi‐head attention mechanism, the TICN‐Att model has a long time‐dependent extraction capability, similar to that of the long short‐term memory network models. The generalization and validity of the model are tested using the global energy forecasting competition (GEFCOM2014) dataset, electrical load data of a city in Jiangsu (China), and PJM power system dataset. The experimental results demonstrated that the proposed model has the best prediction effect, compared to the other state‐of‐the‐art models.

Deep learning models, including recurrent neural network (RNN) and gated recurrent unit (GRU), were used to construct the clinical prognostic prediction models for peritoneal dialysis (PD) patients based on routine clinical data. The performance of the RNN and GRU were compared with logistic regression (LR), which is commonly used in medical researches. The possible underlining clinical implications based on the result from the GRU model were also investigated. We used the clinical data from the PD center of Peking University Third Hospital as the data source. Both the baseline data at the beginning of dialysis, and the follow-up and prognostic data of the patients were used by the RNN and GRU prediction models. The hyper-parameters were tuned based on the 10-fold cross-validation. The risk prediction performance of each model was evaluated via area under the receiver operation characteristic curve (AUROC), recall rate and F1-score on the testset. A total of 656 patients with the 261 occurrences of death were included in the experiment. The total number of all diagnostic records were 13 091. The results on the testset showed that the AUROC of the LR model, RNN model, and GRU model was 0.701 4, 0.786 0, and 0.814 7, respectively. The predictive performances of the GRU and RNN models were significantly better than that of the LR model. The performances of the GRU and RNN models assessed by recall rate and F1-score were also significantly better than that of the LR model, in which the GRU model reached the best performance. In addition, the recall rates were different among different causes of death or by different prediction time windows. The recurrent neural network model, especially the GRU model, is more effective in predicting PD patients' prognosis as compared with the LR model. This new model may be helpful for clinicians to provide timely intervention, thus improving the quality of care of PD.

Locomotion mode recognition has been shown to substantially contribute to the precise control of robotic lower-limb prostheses under different walking conditions. In this study, we proposed a temporal convolutional capsule network (TCCN) which integrates the spatial-temporal-based, dilation-convolution-based, dyna- mic routing and vector-based features for recognizing locomotion mode recognition with small data rather than big-data-based neural networks for robotic prostheses. TCCN proposed in this study has four characteristics, which extracts the (1) spatial-temporal information in the data and then makes (2) dilated convolution to deal with small data, and uses (3) dynamic routing, which produces some similarities to the human brain to process the data as a (4) vector, which is different from other scalar-based networks, such as convolutional neural network (CNN). By comparison with a traditional machine learning, e.g., support vector machine(SVM) and big-data-driven neural networks, e.g., CNN, recurrent neural network(RNN), temporal convolutional network(TCN) and capsule network(CN). The accuracy of TCCN is 4.1% higher than CNN under 5-fold cross-validation of three-locomotion-mode and 5.2% higher under the 5-fold cross-validation of five-locomotion modes. The main confusion we found appears in the transition state. The results indicate that TCCN may handle small data balancing global and local information which is closer to the way how the human brain works, and the capsule layer allows for better processing vector information and retains not only magnitude information, but also direction information.

Recurrent neural network (RNN) has been widely used for sequential learning which has achieved a great success in different tasks. The temporal convolutional network (TCN), a variant of one-dimensional convolutional neural network (CNN), was also developed for sequential learning in presence of sequence data. RNN and TCN typically captures long-term and short-term features in temporal or spatial domain, respectively. This paper presents a new sequential learning, called the convolutional recurrent network (CRN), which fulfills TCN as an encoder and RNN as a decoder so that the global semantics as well as the local dependencies are simultaneously characterized from sequence data. To facilitate the interpretation and robustness in neural models, we further develop the stochastic modeling for CRN based on variational inference. The merits of CNN and RNN are then incorporated in inference of latent space which sufficiently produces a generative model for sequential prediction. Experiments on language model shows the effectiveness of stochastic CRN when compared with the other sequential machines.