• Home
  • Search
  • Towards robust and label-efficient time series representation learning
  • Cite Icon1
  • https://doi.org/10.32657/10356/170673Copy DOI Icon

Towards robust and label-efficient time series representation learning

  • Jan 1, 2023
  • Ahmed Ibrahim Ahmed Eldele Emadeldeen
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Time series data is sequential measurements collected over time from various sources in different applications, e.g., healthcare and manufacturing. With the increased generation of time series data from these applications, their analysis is becoming more important to get insights. Deep learning has shown a potential and proven capability in automatic learning from massive data, by identifying complex patterns and representations directly from data. However, the current deep learning-based models suffer significant limitations. First, they lack the ability to efficiently learn time series temporal relations while utilizing parallel processing. Second, these models require large amounts of labeled data for training, which can be difficult to obtain, especially with complex time series data. Third, the generalization capability of these models is limited, where they suffer performance deterioration when transferring knowledge from a labeled source domain to an out-of-distribution unlabeled target domain. In this thesis, we address these problems and provide solutions for the real-world deployment of deep learning models for time series data. We first propose a novel attention-based deep learning architecture called AttnSleep to classify EEG-based sleep stages, being one of the common time series healthcare data types. Specifically, we propose a powerful feature extractor that learns from different frequency bands in EEG signals. We also propose a temporal context encoder module to learn the temporal dependencies among extracted features using a causal multi-head attention mechanism. Last, we develop a class-aware loss function to address the class-imbalance problem in sleep data without incurring any additional computational costs. Next, we propose two frameworks to address the label scarcity problem in different settings. The first framework, TS-TCC, is a self-supervised learning approach that learns useful representations from unlabeled data. TS-TCC utilizes time series-specific augmentations to generate two views for each sample. We then learn the temporal representations via a novel cross-view temporal prediction task. Furthermore, we propose a contextual contrasting module that further learns discriminative representations. The second framework, CA-TCC, improves the learned representations from TS-TCC in semi-supervised settings, by training the model in four phases. First, we perform self-supervised training with TS-TCC. Then, we fine-tune the pretrained model with the available few labeled samples. Following that, we use the fine-tuned model to assign pseudo labels to the unlabeled set. Finally, we leverage these pseudo labels to realize a class-aware contrastive loss for semi-supervised training. These two frameworks showed significant performance improvement with having few labeled samples compared to traditional supervised training. Last, we tackle the domain shift problem and propose two novel frameworks to address this issue. In the first framework, we introduce an adversarial domain adaptation technique named ADAST, that addresses two challenges, namely the loss of domain-specific information during feature extraction and the ignorance of class information in the target domain during domain alignment. To overcome these challenges, we incorporate an unshared attention mechanism and an iterative self-training strategy with dual distinct classifiers. In the second framework, we attempt to overcome the complexity of adversarial training and present a novel approach called CoTMix to address the domain shift with a simple yet effective contrastive learning strategy. In specific, we propose a cross-domain temporal mixup strategy to create source-dominant and target-dominant domains. These domains serve as augmented views for the source and target domains in contrastive learning. Unlike prior works, CoTMix maps the source and target domains to an intermediate domain. These frameworks showed improved robustness of deep learning models on time series data.

Similar Papers
  • Dissertation

Label-efficient visual recognition with transfer learning

  • Jan 01, 2025
  • Jingyi Zhang
  • PDF
  • Research Article
  • Citations9

Generalized Zero-Shot Domain Adaptation for Unsupervised Cross-Domain PolSAR Image Classification

  • Jan 01, 2022
  • IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
  • Rong Gui +4
  • Research Article
  • Citations52

Deep Asymmetric Transfer Network for Unbalanced Domain Adaptation

  • Apr 25, 2018
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Daixin Wang +2
  • Research Article
  • Citations15

DualDA-Net: Dual-Head Rectification for Cross-Domain Object Detection of Remote Sensing

  • Jan 01, 2023
  • IEEE Transactions on Geoscience and Remote Sensing
  • Yangguang Zhu +4
  • Research Article
  • Citations27

Multi-Source Collaborative Contrastive Learning for Decentralized Domain Adaptation

  • May 01, 2023
  • IEEE Transactions on Circuits and Systems for Video Technology
  • Yikang Wei +3
  • Research Article
  • Citations4

Enhancing unsupervised fault diagnosis method for rolling bearings using domain adaptive transfer coding techniques

  • May 27, 2025
  • Measurement Science and Technology
  • Jiantao Lu +3
  • Research Article

Air Target ISAR Recognition Based on Data Augmentation and Transfer Learning

  • May 23, 2026
  • Sensors (Basel, Switzerland)
  • Moqian Wang +4
  • Research Article
  • Citations157

Heterogeneous Domain Adaptation: An Unsupervised Approach.

  • Nov 30, 2020
  • IEEE Transactions on Neural Networks and Learning Systems
  • Feng Liu +2
  • Research Article
  • Citations25

ACT: Semi-supervised Domain-adaptive Medical Image Segmentation with Asymmetric Co-Training.

  • Jan 01, 2022
  • Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention
  • Xiaofeng Liu +6
  • Research Article
  • Citations1

Distance Metric-Based Open-Set Domain Adaptation for Speaker Verification

  • Jan 01, 2024
  • IEEE/ACM Transactions on Audio Speech and Language Processing
  • Jianchen Li +5
  • Research Article

Unlocking Cross-Domain Synergies for Domain Adaptive Semantic Segmentation.

  • Jan 01, 2025
  • IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
  • Qin Xu +5
  • Conference Article
  • Citations4

Deep Joint Discriminative Feature Learning and Class-Aware Domain Alignment for Unsupervised Domain Adaptation

  • Feb 01, 2020
  • Han Zhao +1
  • PDF
  • Research Article
  • Citations13

Integrating Optical and SAR Time Series Images for Unsupervised Domain Adaptive Crop Mapping

  • Apr 20, 2024
  • Remote Sensing
  • Luwei Feng +4
  • Research Article
  • Citations11

Progressive Source-Aware Transformer for Generalized Source-Free Domain Adaptation

  • Jan 01, 2024
  • IEEE Transactions on Multimedia
  • Song Tang +5
  • Research Article
  • Citations7

Unsupervised Domain Expansion for Visual Categorization

  • Nov 12, 2021
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Jie Wang +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.