Transportation Mode Detection Using Temporal Convolutional Networks Based on Sensors Integrated into Smartphones

As mobile internet technology rapidly advances, the importance of smartphones outfitted with diverse sensors has grown, particularly for transportation mode detection (TMD). Information regarding transportation modes is essential for improving urban development, traffic control, and route planning. Despite extensive research in TMD utilizing conventional machine learning techniques, the results often fall below expectations, largely due to their dependence on handcrafted features. Moreover, while long short‐term memory (LSTM) methods are employed for TMD, they generally fail to capture specific local nuances in complex time‐series data and encounter difficulties optimizing feature selection. To address these issues, we propose a refined TMD model incorporating a sophisticated spatial attention‐based transductive LSTM (TLSTM) network, further improved by reinforcement learning (RL) with an innovative scope loss function for efficient feature selection. The TLSTM framework is adept at identifying minor temporal variations in data, advancing beyond traditional LSTM models by applying transductive learning (TL) to concentrate on data points proximate to the test instance. The scope loss function balances the utilization of known data with the exploration of new data vistas. Additionally, our model integrates an upgraded differential evolution (DE) algorithm for intricate hyperparameter optimization, which includes a novel mutation strategy coupled with k‐means clustering to pinpoint crucial clusters. Extensive evaluations on the SHL, HTC, and US‐TMD datasets demonstrate that our advanced TMD algorithm outperforms existing competitive models. The F‐measure recorded by our system on these datasets stands at 90.140%, 88.315%, and 89.110%, respectively. This state‐of‐the‐art approach highlights the efficacy of TLSTM amalgamated with spatial attention and RL techniques in boosting the precision and dependability of TMD.

Accurate load forecasting is essential for ensuring safe, stable, and economical operation of energy internet. Temporal convolutional networks (TCNs) have demonstrated superior performance, when compared to recurrent neural network models, since their introduction in electrical load forecasting. However, the current TCN‐based models are unable to obtain a large receptive field and strong long‐time feature extraction capability owing to the specific kernel size of the 1D convolution structure. This paper proposes a temporal inception convolutional network based on multi‐head attention (TICN‐Att) for ultra‐short‐term load prediction. By introducing an inception structure into the TCN, the proposed model can extract multi‐dimensional information from the input features, through the multiple hidden convolutional kernels of different scales, without stacking layers depth‐wise. Simultaneously, by introducing a multi‐head attention mechanism, the TICN‐Att model has a long time‐dependent extraction capability, similar to that of the long short‐term memory network models. The generalization and validity of the model are tested using the global energy forecasting competition (GEFCOM2014) dataset, electrical load data of a city in Jiangsu (China), and PJM power system dataset. The experimental results demonstrated that the proposed model has the best prediction effect, compared to the other state‐of‐the‐art models.

Context-aware computing has become a certainty due to the widespread use of smartphone devices equipped with sensors. A wide range of services, such as vehicular traffic monitoring and smart parking, can be accomplished with the help of awareness of user mobility. Transportation mode detection (TMD) using machine learning algorithms and the data captured from smartphone embedded sensors have attracted research community attention. In this research, ensemble learning is utilized to differentiate between transportation modes, including walking, standing, riding a train, driving a car, and riding a bus. The ensemble learning consists of three classifiers; each classifier votes independently on the instances, and the majority vote is applied for robust generalization. The proposed method was validated using three datasets; the samples included in these datasets were gathered by smartphone sensors (belonging to heterogeneous users), such as rotation vector sensors, accelerometers, uncalibrated gyroscopes, linear acceleration, orientation, speed, game rotation vector, sound, and gyroscopes. The proposed ensemble learning method achieves an accuracy of 89%, 93%, and 95% on the first, second, and third datasets, respectively, when 10% and 90% of the data are used for testing and training, respectively. On the other set of experiments, in which 30% and 70% of the data are used for testing and training, respectively, the proposed method yields accuracies of 86.8%, 92.1%, and 94.9% on the first, second, and third datasets, respectively. The proposed method shows promising results compared to existing human activity recognition (HAR) methods.

Abstract. The present work proposes a prediction model of significant wave height (SWH) and average wave period (APD) based on variational mode decomposition (VMD), temporal convolutional networks (TCNs), and long short-term memory (LSTM) networks. The wave sequence features were obtained using VMD technology based on the wave data from the National Data Buoy Center. Then the SWH and APD prediction models were established using TCNs, LSTM, and Bayesian hyperparameter optimization. The VMD–TCN–LSTM model was compared with the VMD–LSTM (without TCN cells) and LSTM (without VMD and TCN cells) models. The VMD–TCN–LSTM model has significant superiority and shows robustness and generality in different buoy prediction experiments. In the 3 h wave forecasts, VMD primarily improved the model performance, while the TCN had less of an influence. In the 12, 24, and 48 h wave forecasts, both VMD and TCNs improved the model performance. The contribution of the TCN to the improvement of the prediction result determination coefficient gradually increased as the forecasting length increased. In the 48 h SWH forecasts, the VMD and TCN improved the determination coefficient by 132.5 % and 36.8 %, respectively. In the 48 h APD forecasts, the VMD and TCN improved the determination coefficient by 119.7 % and 40.9 %, respectively.

The transportation system is the backbone of nation's economy and plays a vital role in the environmental changes in the region. In order to make the transportation system more efficient, the mobility patterns using the data collected from smartphone users can be exploited. For such context-aware systems, transportation mode detection can be of particular interest in different intelligence applications to increase the quality of transportation, traffic safety, and other services. In this paper, various machine learning (ML) techniques are used to classify five different modes of transportation. Specifically, we consider support vector machine (SVM), K-nearest neighbors (KNN), decision tree (DT), bagging, and random forest (RF) algorithms for classification. The performance of all the considered algorithms is compared on the basis of F1 score, accuracy, precision, and recall and tested on freely available TMD dataset.

Multivariate time-series forecasting is one of the crucial and persistent challenges in time-series forecasting tasks. As a kind of data with multivariate correlation and volatility, multivariate time series impose highly nonlinear time characteristics on the forecasting model. In this paper, a new multivariate time-series forecasting model, multivariate temporal convolutional attention network (MTCAN), based on a self-attentive mechanism is proposed. MTCAN is based on the Convolution Neural Network (CNN) model, using 1D dilated convolution as the basic unit to construct asymmetric blocks, and then, the feature extraction is performed by the self-attention mechanism to finally obtain the prediction results. The input and output lengths of this network can be determined flexibly. The validation of the method is carried out with three different multivariate time-series datasets. The reliability and accuracy of the prediction results are compared with Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Convolutional Long Short-Term Memory (ConvLSTM), and Temporal Convolutional Network (TCN). The prediction results show that the model proposed in this paper has significantly improved prediction accuracy and generalization.

Research on smartphone-based human activity recognition (HAR) is prevalent in the field of healthcare, especially for elderly activity monitoring. Researchers usually propose to use of accelerometers, gyroscopes or magnetometers that are equipped in smartphones as an individual sensing modality for human activity recognition. However, any of these alone is limited in capturing comprehensive movement information for accurate human activity analysis. Thus, we propose a smartphone-based HAR approach by leveraging the inertial signals captured by these three sensors to classify human activities. These heterogeneous sensors deliver information on various aspects of nature, motion and orientation, offering a richer set of features for more accurate representations of the activities. Hence, a deep learning approach that amalgamates long short-term memory (LSTM) in temporal convolutional network (TCN) is proposed. We use independent temporal convolutional networks, coined as temporal convolutional streams, to independently analyse the temporal data of each sensing modality. We name this architecture multi-stream TC-LSTM. The performance of multi-stream TC-LSTM is assessed on the self-collected elderly activity database. Empirical results exhibit that multi-stream TC-LSTM outperforms the existing machine learning and deep learning models, with an F1 score of 98.3 %

The accuracy of short-term electricity price forecasting has profound influence on the business decisions made by the participants in the electricity market. Thus, we propose a short-term electricity price combination prediction model based on Complete Ensemble Empirical Mode Decomposition(CEEMD) as well as Temporal Convolutional Attention-based Network(TCAN). First, we defined the correlations among the features according to the Pearson Correlation Analysis and selected the desired features for our model. We then defined the single prediction period to be 24 hours based on the Unit Root test. In addition, we decomposed and denoised the data based on CEEMD and built the attention mechanism into the temporal convolution network. Using the expansion cascade network to extract the relationship between the inputs, combined with the self-attention mechanism to extract the internal information and learn the position dependence, effectively improve the model training speed and prediction accuracy. In order to verify the validity of the model, the actual electricity price data of DK1 in the Nordic electricity market from April to May 2020 were forecasted. The results show that the proposed method can effectively improve the accuracy and reliability of electricity price forecasting, and can provide reliable decision-making basis.

Data in time series format, such as biological signals from medical sensors or machine signals from sensors in industrial environments are rich sources of information that can give crucial insights on the present and future condition of a person or machine. The task of predicting future values of time series has been initially approached with simple machine learning methods, and lately with deep learning. Two models that have shown good performance in this task are the temporal convolutional network and the attention module. However, despite the promising results of deep learning methods, their black-box nature makes them unsuitable for real-world applications where the predictions need to be explainable in order to be trusted. In this paper we propose an architecture comprised of a temporal convolutional network with an attention mechanism that makes predictions while presenting the timesteps of the input that were most influential for future outputs. We apply it on two datasets and we show that we gain interpretability without degrading the accuracy compared to the original temporal convolutional models. We then go one step further and we combine our configuration with various machine learning methods on top, creating a pipeline that achieves interpretability both across timesteps and input features. We use it to forecast a different variable from one of the above datasets and we study how the accuracy is affected compared to the original black-box approach.

Locomotion mode recognition has been shown to substantially contribute to the precise control of robotic lower-limb prostheses under different walking conditions. In this study, we proposed a temporal convolutional capsule network (TCCN) which integrates the spatial-temporal-based, dilation-convolution-based, dyna- mic routing and vector-based features for recognizing locomotion mode recognition with small data rather than big-data-based neural networks for robotic prostheses. TCCN proposed in this study has four characteristics, which extracts the (1) spatial-temporal information in the data and then makes (2) dilated convolution to deal with small data, and uses (3) dynamic routing, which produces some similarities to the human brain to process the data as a (4) vector, which is different from other scalar-based networks, such as convolutional neural network (CNN). By comparison with a traditional machine learning, e.g., support vector machine(SVM) and big-data-driven neural networks, e.g., CNN, recurrent neural network(RNN), temporal convolutional network(TCN) and capsule network(CN). The accuracy of TCCN is 4.1% higher than CNN under 5-fold cross-validation of three-locomotion-mode and 5.2% higher under the 5-fold cross-validation of five-locomotion modes. The main confusion we found appears in the transition state. The results indicate that TCCN may handle small data balancing global and local information which is closer to the way how the human brain works, and the capsule layer allows for better processing vector information and retains not only magnitude information, but also direction information.

With the progress in sensor technology and the spread of mobile devices, transportation mode detection (TMD) is gaining importance for health and urban traffic improvements. As mobile devices become more lightweight, they require more efficient, low-power models to handle limited resources effectively. Despite extensive research on TMD, challenges remain in capturing non-stationary temporal dynamics and nonlinear fitting capabilities. Additionally, many existing models exhibit high space complexity, making lightweight deployment on devices with limited computing and memory resources difficult. To address these issues, we propose a novel deep TMD model based on discrete wavelet transform (DWT) and knowledge distillation (KD), called KDTMD. This model consists of two main modules, i.e., DWT and KD. For the DWT module, since non-stationary time variations and event distribution shifts complicate sensor time series analysis, we use the DWT modules to disentangle the sensor time series into two parts: a low-frequency part that indicates the trend and a high-frequency part that captures events. The separated trend data is less influenced by event distribution shifts, effectively mitigating the impact of non-stationary time variations. For the KD module, it includes the teacher model and student model. Specifically, for teacher model, to address the nonlinearities and interpretability, we incorporate T-KAN, which is composed of multiple layers of linear KAN that employ learnable B-spline functions to achieve a richer feature representation with fewer parameters. For student model, we develop the S-CNN, which is trained efficiently by T-KAN through KD. The KDTMD model achieves 97.27% accuracy and 97.29% F1-Score on the SHL dataset, and 96.56% accuracy and 96.72% F1-Score on the HTC dataset. Additionally, the parameters of the KDTMD model are only about 10% of the smallest baseline.

In this study, we introduce a novel approach to speech enhancement through the design of a complex temporal convolutional network (Complex-TCN). This model leverages the power of complex networks, enabling the simultaneous capture of both magnitude and phase information inherent in speech signals. By employing a temporal convolutional network, the Complex-TCN excels at extracting contextual information within the time domain of speech. Our findings underscore the substantial performance improvements achieved through the synergistic use of the temporal convolutional network and the incorporation of complex representations.

Visibility forecasting in offshore areas faces the problems of low observational data and complex weather. This paper proposes an intelligent prediction method of offshore visibility based on temporal convolutional network (TCN) and transfer learning to solve the problem. First, preprocess the visibility data sets of the source and target domains to improve the quality of the data. Then, build a model based on temporal convolutional network and transfer learning (TCN_TL) to learn the visibility data of the source domain. Finally, after transferring the knowledge learned from a large amount of data in the source domain, the model learns the small data set in the target domain. After completing the training, the model data of the European Mid-Range Weather Forecast Center (ECMWF) meteorological field were selected to test the model performance. The method proposed in this paper has achieved relatively good results in the visibility forecast of Qiongzhou Strait. Taking Haikou Station in the spring and winter of 2018 as an example, the forecast error is significantly lower than that before the transfer learning, and the forecast score is increased by 0.11 within the 0-1 km level and the 24 h forecast period. Compared with the CUACE forecast results, the forecast error of TCN_TL is smaller than that of the former, and the TS score is improved by 0.16. The results show that under the condition of small data sets, transfer learning improves the prediction performance of the model, and TCN_TL performs better than other deep learning methods and CUACE.

Transient muscle movements influence the temporal structure of myoelectric signal patterns, often leading to unstable prediction behavior from movement-pattern classification methods. We show that temporal convolutional network sequential models leverage the myoelectric signal's history to discover contextual temporal features that aid in correctly predicting movement intentions, especially during interclass transitions. We demonstrate myoelectric classification using temporal convolutional networks to effect 3 simultaneous hand and wrist degrees-of-freedom in an experiment involving nine human-subjects. Temporal convolutional networks yield significant (p<;0.001) performance improvements over other state-of-the-art methods in terms of both classification accuracy and stability.

Unmanned aerial vehicles (UAVs) are forecast to be widely used in the military and civilian domains. The remaining flying time is a critical parameter to monitor during a flight to ensure the safety of electric UAVs (e-UAVs). However, accurate remaining flying time prediction under different load conditions requires a large amount of data and is computationally expensive for online applications. To address these issues, a deep learning approach based on temporal convolutional networks and transfer learning is developed for lithium-ion battery systems for e-UAVs. A temporal convolutional network is used to extract features from monitoring data and predict the remaining flying time of flights under one load condition. A layer transfer strategy is then used to transfer the knowledge learned from one load condition to another load condition. Battery health monitoring data collected from a fixed-wing e-UAV are used to demonstrate the effectiveness of the proposed method. Experimental results show that the proposed temporal convolutional network with the transfer learning strategy can predict the remaining flying time of the e-UAV under two load conditions more efficiently and accurately than a temporal convolutional network without transfer learning.