• Home
  • Search
  • Towards Open-Vocabulary Video Semantic Segmentation
  • Cite Icon1
  • https://doi.org/10.1109/tmm.2025.3557719Copy DOI Icon

Towards Open-Vocabulary Video Semantic Segmentation

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Semantic segmentation in videos has been a focal point of recent research. However, existing models encounter challenges when faced with unfamiliar categories. To address this, we introduce the Open Vocabulary Video Semantic Segmentation (OV-VSS) task, designed to accurately segment every pixel across a wide range of open-vocabulary categories, including those that are novel or previously unexplored. To enhance OV-VSS performance, we propose a robust baseline, OV2VSS, which integrates a spatial-temporal fusion module, allowing the model to utilize temporal relationships across consecutive frames. Additionally, we incorporate a random frame enhancement module, broadening the model's understanding of semantic context throughout the entire video sequence. Our approach also includes video text encoding, which strengthens the model's capability to interpret textual information within the video context. Comprehensive evaluations on benchmark datasets such as VSPW and Cityscapes highlight OV-VSS's zero-shot generalization capabilities, especially in handling novel categories. The results validate OV2VSS's effectiveness, demonstrating improved performance in semantic segmentation tasks across diverse video datasets.

Similar Papers
  • Research Article
  • Citations16

A Semantic Change Detection Network Based on Boundary Detection and Task Interaction for High-Resolution Remote Sensing Images.

  • Sep 01, 2025
  • IEEE transactions on neural networks and learning systems
  • Yingjie Tang +5
  • Conference Article
  • Citations27

Continual Learning for Class- and Domain-Incremental Semantic Segmentation

  • Jul 11, 2021
  • Tobias Kalb +3
  • Book Chapter
  • Citations46

Chapter 12 - Semantic scene segmentation for robotics

  • Jan 01, 2022
  • Deep Learning for Robot Perception and Cognition
  • Juana Valeria Hurtado +1
  • Research Article
  • Citations8

Video Generalized Semantic Segmentation via Non-Salient Feature Reasoning and Consistency

  • Feb 29, 2024
  • Knowledge-Based Systems
  • Yuhang Zhang +6
  • Conference Article
  • Citations115

Convolutional gated recurrent networks for video segmentation

  • Sep 01, 2017
  • Mennatullah Siam +3
  • Research Article
  • Citations32

Joint Multiclass Object Detection and Semantic Segmentation for Autonomous Driving

  • Jan 01, 2023
  • IEEE Access
  • Shakhboz Abdigapporov +3
  • Research Article
  • Citations7

Large multimodal model for open vocabulary semantic segmentation of remote sensing images

  • Dec 30, 2024
  • European Journal of Remote Sensing
  • Bing Liu +5
  • Research Article
  • Citations1

Style Optimization Networks for real-time semantic segmentation of rainy and foggy weather

  • Sep 07, 2024
  • Signal Processing: Image Communication
  • Yifang Huang +5
  • Research Article
  • Citations44

Combining max-pooling and wavelet pooling strategies for semantic image segmentation

  • Jun 12, 2021
  • Expert Systems with Applications
  • André De Souza Brito +4
  • Book Chapter
  • Citations44

Can Ground Truth Label Propagation from Video Help Semantic Segmentation?

  • Jan 01, 2016
  • Siva Karthik Mustikovela +2
  • Research Article
  • Citations34

Modified Double U-Net Architecture for Medical Image Segmentation

  • Feb 01, 2023
  • IEEE Transactions on Radiation and Plasma Medical Sciences
  • Sagar Deep Deb +1
  • Research Article
  • Citations2

A Semantic Segmentation Method for Road Environment Images Based on Hybrid Convolutional Auto-Encoder

  • Aug 31, 2022
  • Traitement du Signal
  • Xiaona Song +6
  • PDF
  • Research Article

Semantic Segmentation of 3D Medical Images with 3D Convolutional Neural Networks

  • Apr 01, 2020
  • CLEI Electronic Journal
  • Alejandra Márquez Herrera +2
  • Conference Article
  • Citations4

Improving Semantic Segmentation of 3D Medical Images on 3D Convolutional Neural Networks

  • Sep 01, 2019
  • Alejandra Marquez Herrera +2
  • Research Article
  • Citations10

A Novel Semantic Segmentation Algorithm for RGB-D Images Based on Non-Symmetry and Anti-Packing Pattern Representation Model

  • Jan 01, 2023
  • IEEE Access
  • Yunping Zheng +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.