• Home
  • Search
  • Entropy-Regularized Joint CTC–Attention Learning for Low-Resource Continuous Sign Language Recognition
  • https://doi.org/10.21203/rs.3.rs-8387768/v1Copy DOI Icon

Entropy-Regularized Joint CTC–Attention Learning for Low-Resource Continuous Sign Language Recognition

Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

<title>Abstract</title> Continuous Sign Language Recognition (CSLR) seeks to transcribe unsegmented sign language videos into gloss sequences without frame-level supervision, presenting persistent challenges in temporal alignment, long-range dependency modeling, and reliable sequence-level generalization. While recent advances have achieved strong performance in high-resource languages, Kurdish Sign Language (KrdSL) remains largely unexplored due to the absence of sentence-level benchmarks. To address this gap, we introduce <bold>KrdSL-1400</bold> , the first continuous Kurdish Sign Language dataset, comprising 1,400 annotated video sequences covering 40 linguistically structured sentences performed by seven native signers, providing a standardized benchmark for low-resource CSLR. We propose a <bold>hybrid spatio-temporal CSLR framework</bold> that combines deep convolutional visual encoding with sequence-aware temporal modeling and a <bold>multi-task joint CTC–attention decoding strategy</bold> explicitly designed to address alignment uncertainty. The CTC objective enforces monotonic alignment, while an <bold>entropy-regularized multi-head attention mechanism</bold> dynamically emphasizes linguistically salient temporal segments, enabling robust sequence prediction without reliance on pose estimation or handcrafted features. Training dynamics exhibit stable and consistent convergence, with closely aligned training and validation WER curves indicating strong generalization. Quantitative evaluation shows that a CTC baseline achieves a WER of <bold>13.5%</bold> , which is reduced to <bold>10.5%</bold> using single-head attention, while the proposed model attains the best performance with a WER of <bold>9.5%</bold> , corresponding to an approximate <bold>30% relative improvement</bold> . <bold>Cross-dataset evaluation</bold> on the large-scale <bold>PHOENIX-2014-T</bold> benchmark further demonstrates generalization, achieving a WER of <bold>13.7%</bold> and outperforming recent attention-based and transformer-based CSLR approaches.

Similar Papers
  • Research Article

Continuous Chinese Sign Language Recognition Using Millimeter-Wave Radar with DeepSeek Semantic Enhancement

  • Jan 01, 2025
  • IEEE Sensors Journal
  • Lei Zhang +5
  • Conference Article

A Continuous Sign Language Recognition Method Based on Temporal Modeling and Semantic Alignment

  • Nov 14, 2025
  • J Li +5
  • Research Article
  • Citations19

Swin-MSTP: Swin transformer with multi-scale temporal perception for continuous sign language recognition

  • Feb 01, 2025
  • Neurocomputing
  • Sarah Alyami +1
  • Research Article
  • Citations16

Improving Continuous Sign Language Recognition with Consistency Constraints and Signer Removal

  • Mar 08, 2024
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Ronglai Zuo +1
  • Book Chapter
  • Citations1

Spatial-Temporal Graph Transformer for Skeleton-Based Sign Language Recognition

  • Jan 01, 2023
  • Zhengye Xiao +4
  • Research Article
  • Citations12

Gloss Prior Guided Visual Feature Learning for Continuous Sign Language Recognition.

  • Jan 01, 2024
  • IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
  • Leming Guo +5
  • Conference Article
  • Citations89

Transition movement models for large vocabulary continuous sign language recognition

  • Jan 01, 2004
  • Wen Gao +3
  • Research Article
  • Citations500

Sign Language Recognition: A Deep Survey

  • Aug 06, 2020
  • Expert Systems with Applications
  • Razieh Rastgoo +2
  • Research Article

A dual-stream deep learning framework for continuous sign language recognition to enhance communication accessibility in the Ha'il region.

  • Feb 03, 2026
  • Scientific reports
  • Haifa Harrouch +3
  • Research Article
  • Citations8

A multi-modal fusion framework for continuous sign language recognition based on multi-layer self-attention mechanism

  • Aug 10, 2022
  • Journal of Intelligent &amp; Fuzzy Systems
  • Cuihong Xue +5
  • Conference Article
  • Citations3

Continuous Sign Language Recognition Using MediaPipe

  • Oct 19, 2023
  • Khang B Tran +2
  • Conference Article
  • Citations24

Vision-based system for continuous Arabic Sign Language recognition in user dependent mode

  • May 01, 2008
  • Khaled Assaleh +4
  • Conference Article
  • Citations26

Continuous gesture recognition system for Korean sign language based on fuzzy logic and hidden Markov model

  • Aug 07, 2002
  • Jung-Bae Kim +3
  • Book Chapter

Conversational Algerian Sign Language Recognition Using the Deep Learning Approach

  • May 29, 2025
  • Lila Meddeber +1
  • PDF
  • Research Article

Sequential particle filter with covariance features classified with artificial neural nets for continuous Indian sign language recognition

  • Dec 21, 2017
  • International Journal of Engineering &amp; Technology
  • P Praveen Kumar +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.