Application of Offshore Visibility Forecast Based on Temporal Convolutional Network and Transfer Learning.

Unmanned aerial vehicles (UAVs) are forecast to be widely used in the military and civilian domains. The remaining flying time is a critical parameter to monitor during a flight to ensure the safety of electric UAVs (e-UAVs). However, accurate remaining flying time prediction under different load conditions requires a large amount of data and is computationally expensive for online applications. To address these issues, a deep learning approach based on temporal convolutional networks and transfer learning is developed for lithium-ion battery systems for e-UAVs. A temporal convolutional network is used to extract features from monitoring data and predict the remaining flying time of flights under one load condition. A layer transfer strategy is then used to transfer the knowledge learned from one load condition to another load condition. Battery health monitoring data collected from a fixed-wing e-UAV are used to demonstrate the effectiveness of the proposed method. Experimental results show that the proposed temporal convolutional network with the transfer learning strategy can predict the remaining flying time of the e-UAV under two load conditions more efficiently and accurately than a temporal convolutional network without transfer learning.

Multi-source adversarial transfer learning for ultrasound image segmentation with limited similarity

Domain- and task-specific transfer learning for medical segmentation tasks

BackgroundThe prediction of coronavirus disease in 2019 (COVID-19) in broader regions has been widely researched, but for specific areas such as urban areas the predictive models were rarely studied. It may be inaccurate to apply predictive models from a broad region directly to a small area. This paper builds a prediction approach for small size COVID-19 time series in a city.MethodsNumbers of COVID-19 daily confirmed cases were collected from November 1, 2022 to November 16, 2023 in Xuzhou city of China. Classical deep learning models including recurrent neural network (RNN), long and short-term memory (LSTM), gated recurrent unit (GRU) and temporal convolutional network (TCN) are initially trained, then RNN, LSTM and GRU are integrated with a new attention mechanism and transfer learning to improve the performance. Ten times ablation experiments are conducted to show the robustness of the performance in prediction. The performances among the models are compared by the mean absolute error, root mean square error and coefficient of determination.ResultsLSTM outperforms than others, and TCN has the worst generalization ability. Thus, LSTM is integrated with the new attention mechanism to construct an LSTMATT model, which improves the performance. LSTMATT is trained on the smoothed time series curve through frequency domain convolution augmentation, then transfer learning is adopted to transfer the learned features back to the original time series resulting in a TLLA model that further improves the performance. RNN and GRU are also integrated with the attention mechanism and transfer learning and their performances are also improved, but TLLA still performs best.ConclusionsThe TLLA model has the best prediction performance for the time series of COVID-19 daily confirmed cases, and the new attention mechanism and transfer learning contribute to improve the prediction performance in the flatten part and the jagged part, respectively.

Temporal deep learning for satellite-based sugarcane monitoring and cycle age estimation

Accurate load forecasting is essential for ensuring safe, stable, and economical operation of energy internet. Temporal convolutional networks (TCNs) have demonstrated superior performance, when compared to recurrent neural network models, since their introduction in electrical load forecasting. However, the current TCN‐based models are unable to obtain a large receptive field and strong long‐time feature extraction capability owing to the specific kernel size of the 1D convolution structure. This paper proposes a temporal inception convolutional network based on multi‐head attention (TICN‐Att) for ultra‐short‐term load prediction. By introducing an inception structure into the TCN, the proposed model can extract multi‐dimensional information from the input features, through the multiple hidden convolutional kernels of different scales, without stacking layers depth‐wise. Simultaneously, by introducing a multi‐head attention mechanism, the TICN‐Att model has a long time‐dependent extraction capability, similar to that of the long short‐term memory network models. The generalization and validity of the model are tested using the global energy forecasting competition (GEFCOM2014) dataset, electrical load data of a city in Jiangsu (China), and PJM power system dataset. The experimental results demonstrated that the proposed model has the best prediction effect, compared to the other state‐of‐the‐art models.

Natural gas is often said to be the most environmentally friendly fossil fuel. Its usage has increased significantly in recent years. Meanwhile, accurate forecasting of natural gas spot prices has become critical to energy management, economic growth, and environmental protection. This work offers a novel model based on the temporal convolutional network (TCN) and dynamic learning rate for predicting natural gas spot prices over the following two weekdays. The residual block structure of TCN provides good prediction accuracy, and the dilated causal convolutions minimize the amount of computation. The dynamic learning rate setting was adopted to enhance the model’s prediction accuracy and robustness. Compared with three existing models, i.e., the one-dimensional convolutional neural network (1D-CNN), gate recurrent unit (GRU), and long short-term memory (LSTM), the proposed model can achieve better performance over other models with mean absolute percentage error (MAPE), mean absolute error (MAE), and root mean squared error (RMSE) scores of 4.965%, 0.216, and 0.687, respectively. These attractive advantages make the proposed model a promising candidate for long-term stability in natural gas spot price forecasting.

Abstract. The present work proposes a prediction model of significant wave height (SWH) and average wave period (APD) based on variational mode decomposition (VMD), temporal convolutional networks (TCNs), and long short-term memory (LSTM) networks. The wave sequence features were obtained using VMD technology based on the wave data from the National Data Buoy Center. Then the SWH and APD prediction models were established using TCNs, LSTM, and Bayesian hyperparameter optimization. The VMD–TCN–LSTM model was compared with the VMD–LSTM (without TCN cells) and LSTM (without VMD and TCN cells) models. The VMD–TCN–LSTM model has significant superiority and shows robustness and generality in different buoy prediction experiments. In the 3 h wave forecasts, VMD primarily improved the model performance, while the TCN had less of an influence. In the 12, 24, and 48 h wave forecasts, both VMD and TCNs improved the model performance. The contribution of the TCN to the improvement of the prediction result determination coefficient gradually increased as the forecasting length increased. In the 48 h SWH forecasts, the VMD and TCN improved the determination coefficient by 132.5 % and 36.8 %, respectively. In the 48 h APD forecasts, the VMD and TCN improved the determination coefficient by 119.7 % and 40.9 %, respectively.

Data in time series format, such as biological signals from medical sensors or machine signals from sensors in industrial environments are rich sources of information that can give crucial insights on the present and future condition of a person or machine. The task of predicting future values of time series has been initially approached with simple machine learning methods, and lately with deep learning. Two models that have shown good performance in this task are the temporal convolutional network and the attention module. However, despite the promising results of deep learning methods, their black-box nature makes them unsuitable for real-world applications where the predictions need to be explainable in order to be trusted. In this paper we propose an architecture comprised of a temporal convolutional network with an attention mechanism that makes predictions while presenting the timesteps of the input that were most influential for future outputs. We apply it on two datasets and we show that we gain interpretability without degrading the accuracy compared to the original temporal convolutional models. We then go one step further and we combine our configuration with various machine learning methods on top, creating a pipeline that achieves interpretability both across timesteps and input features. We use it to forecast a different variable from one of the above datasets and we study how the accuracy is affected compared to the original black-box approach.

Rainfall–runoff modeling is a complex nonlinear time-series problem in the field of hydrology. Various methods, such as physical-driven and data-driven models, have been developed to study the highly random rainfall–runoff process. In the past 2 years, with the advancement of computing hardware resources and algorithms, deep-learning methods, such as temporal convolutional network (TCN), have been shown to be good prospects in time-series prediction tasks. The aim of this study is to develop a prediction model based on TCN structure to simulate the hourly rainfall–runoff relationship. We use two datasets in the Jingle and Kuye watersheds to test the model under different network structures and compare with the other four models. The results show that the TCN model outperforms the Excess Infiltration and Excess Storage Model (EIESM), artificial neural network, and long short-term memory and improves the flood forecasting accuracy at different foreseeable periods. It is shown that the TCN has a faster convergence rate and is an effective method for hydrological forecasting.

In pattern classification, we may have a few labeled data points in the target domain, but a number of labeled samples are available in another related domain (called the source domain). Transfer learning can solve such classification problems via the knowledge transfer from source to target domains. The source and target domains can be represented by heterogeneous features. There may exist uncertainty in domain transformation, and such uncertainty is not good for classification. The effective management of uncertainty is important for improving classification accuracy. So, a new belief-based bidirectional transfer classification (BDTC) method is proposed. In BDTC, the intraclass transformation matrix is estimated at first for mapping the patterns from source to target domains, and this matrix can be learned using the labeled patterns of the same class represented by heterogeneous domains (features). The labeled patterns in the source domain are transferred to the target domain by the corresponding transformation matrix. Then, we learn a classifier using all the labeled patterns in the target domain to classify the objects. In order to take full advantage of the complementary knowledge of different domains, we transfer the query patterns from target to source domains using the K-NN technique and do the classification task in the source domain. Thus, two pieces of classification results can be obtained for each query pattern in the source and target domains, but the classification results may have different reliabilities/weights. A weighted combination rule is developed to combine the two classification results based on the belief functions theory, which is an expert at dealing with uncertain information. We can efficiently reduce the uncertainty of transfer classification via the combination strategy. Experiments on some domain adaptation benchmarks show that our method can effectively improve classification accuracy compared with other related methods.

Accurate lower-limb joint kinematics estimation is critical for patient monitoring, rehabilitation, and exoskeleton control. While previous studies have employed wearable sensor-based deep learning (DL) models for estimating joint kinematics, these methods often require extensive new datasets to adapt to unseen gait patterns. Meanwhile, researchers in computer vision have advanced human pose estimation models, which are easy to deploy and capable of real-time inference. However, such models are infeasible in scenarios where cameras cannot be used. To address these limitations, we propose a computer vision-based DL adaptation framework for realtime joint kinematics estimation. This framework requires only a small dataset (i.e., 1-2 gait cycles) and does not rely on professional motion capture setups. Using transfer learning, we adapted our temporal convolutional network (TCN) to stiff knee gait data, allowing the model to reduce root mean square error by 9.7 % and 19.9 % compared to a TCN trained on only able-bodied and stiff knee dataset, respectively. Our framework demonstrated a potential for smartphone camera-trained DL model to estimate real-time joint kinematics across novel users in clinical populations with applications in wearable robots.

The concept of Industry 4.0 uses cyber-physical systems and the Internet of Things to create "smart factories" that enable automated and connected production. However, the complex and diverse nature of cyber-physical systems makes them vulnerable to physical threats, where any fault in their physical components can lead to serious consequences, from financial losses to human safety. To reduce the risk of such failures, it is essential to have techniques in place to monitor the condition of these physical components closely. In recent years, data-driven predictive maintenance methods have become common due to the growth in data and advancements in machine learning. This thesis focuses on fault diagnosis, a key part of predictive maintenance, which aims to detect and identify faults in physical components. Incorporating advanced techniques, including machine learning algorithms and sensor data analysis, enhanced fault diagnosis systems' accuracy and efficiency. However, several real-world barriers challenge the effectiveness of these techniques. This research is structured around three primary objectives. The first objective aims to establish an efficient fault diagnosis system that is highly sensitive to fault occurrences while being resilient to uncertainties such as variability in fault patterns and data imbalance. To address variability in fault patterns, we developed a multi-step approach integrating clustering techniques and an adaptive neuro-fuzzy inference system, which significantly improved diagnosis robustness even under noisy conditions. For imbalanced datasets, a two-step method was introduced, combining permutation entropy for fault detection and envelope analysis for fault isolation. This approach was enhanced by an automatic frequency band selection method leveraging wavelet transform and statistical criteria, resulting in high fault diagnosis accuracy and reliability, even with limited labeled faulty data. The second objective focuses on developing fault diagnosis systems that can maintain high performance despite domain shifts and limited training data in both source and target domains. To address the challenge of domain shifts, a transfer learning technique was introduced, and a systematic literature review was conducted. This study bridges a technical gap by providing a comprehensive taxonomy, state-of-the-art methods, and insights into the challenges of applying transfer learning to predictive maintenance. Additionally, a conceptual framework was proposed to explore scenarios for transferring diagnostic knowledge from different source domains to target domains with insufficient training datasets. In a subsequent study, a new diagnostic framework was developed using digital twins and partial transfer learning to address the lack of real training datasets in both source and target domains as well as label space inconsistencies between them. The framework includes a general approach for developing digital twins of real machines to generate synthetic diagnostic data as source domain data. Furthermore, by incorporating a deep learning network and a double-layer attention mechanism, a partial transfer learning diagnostic model was developed to transfer diagnostic knowledge from the digital twin to the real target domain despite label space inconsistencies. The proposed framework was validated through an industrial case study involving rolling bearings in rotating equipment. The third objective focuses on developing an efficient fault diagnosis system capable of maintaining reliability under unseen and evolving working conditions. To achieve this, a novel self-adaptive fault diagnosis system was proposed, incorporating the MAPE-K loop, a domain generalization network model, and digital twin technologies. This self-adaptive fault diagnosis system leverages the MAPE-K loop to adapt in real time to evolving working conditions. Digital twins were employed to continuously generate data aligned with the real system’s operational context, enabling accurate fault diagnosis in dynamic environments. The domain generalization network model enhances generalization to unseen target working conditions by addressing overfitting and feature loss through multi-domain data augmentation, adversarial learning, and domain discrepancy modules. Validation using multiple rotating machinery datasets demonstrated that the developed fault diagnosis system outperformed state-of-the-art methods across various scenarios. In summary, this dissertation proposes a comprehensive and practical approach to addressing variability in fault patterns, imbalance datasets, data scarcity, domain shifts, and unseen and evolving working conditions challenges in existing fault diagnosis systems.

Multivariate time-series forecasting is one of the crucial and persistent challenges in time-series forecasting tasks. As a kind of data with multivariate correlation and volatility, multivariate time series impose highly nonlinear time characteristics on the forecasting model. In this paper, a new multivariate time-series forecasting model, multivariate temporal convolutional attention network (MTCAN), based on a self-attentive mechanism is proposed. MTCAN is based on the Convolution Neural Network (CNN) model, using 1D dilated convolution as the basic unit to construct asymmetric blocks, and then, the feature extraction is performed by the self-attention mechanism to finally obtain the prediction results. The input and output lengths of this network can be determined flexibly. The validation of the method is carried out with three different multivariate time-series datasets. The reliability and accuracy of the prediction results are compared with Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Convolutional Long Short-Term Memory (ConvLSTM), and Temporal Convolutional Network (TCN). The prediction results show that the model proposed in this paper has significantly improved prediction accuracy and generalization.

Research on smartphone-based human activity recognition (HAR) is prevalent in the field of healthcare, especially for elderly activity monitoring. Researchers usually propose to use of accelerometers, gyroscopes or magnetometers that are equipped in smartphones as an individual sensing modality for human activity recognition. However, any of these alone is limited in capturing comprehensive movement information for accurate human activity analysis. Thus, we propose a smartphone-based HAR approach by leveraging the inertial signals captured by these three sensors to classify human activities. These heterogeneous sensors deliver information on various aspects of nature, motion and orientation, offering a richer set of features for more accurate representations of the activities. Hence, a deep learning approach that amalgamates long short-term memory (LSTM) in temporal convolutional network (TCN) is proposed. We use independent temporal convolutional networks, coined as temporal convolutional streams, to independently analyse the temporal data of each sensing modality. We name this architecture multi-stream TC-LSTM. The performance of multi-stream TC-LSTM is assessed on the self-collected elderly activity database. Empirical results exhibit that multi-stream TC-LSTM outperforms the existing machine learning and deep learning models, with an F1 score of 98.3 %