The removal of rain streaks and raindrops is crucial for enhancing the image visibility and mitigating the weather degradations. However, most existing approaches rely on the paired rainy and clean images, which are challenging to obtain in real-world scenarios. To this end, we propose a novel structure-preserving frequency-regularized text-guided optimal transport (SFTOT) framework, which formulates the unpaired rain streaks and raindrops removal as an optimal transport problem. Specifically, we introduce a structure-preserving transport cost, incorporating the structural similarity constraint to minimize the duality gap between the primal and dual formulations, while preserving the structural details of reconstructed images. Furthermore, by embedding the inherent frequency sparsity of rain streaks and raindrops into the transport cost, we derive a frequency-regularized optimal transport objective, ensuring consistency in frequency distributions between the generated and clean images. Additionally, we employ a pre-trained one-step stable diffusion model as the restoration network, which is fine-tuned using the low-rank adaptation (LoRA) adapters and zero convolutional layers, while integrating the domain-specific text prompts for both degraded and clean images to guide the generation process. Extensive experiments demonstrate that our method surpasses the existing well-performing unpaired learning approaches, achieving notable improvements in both the fidelity and photo-realism.

Despite significant advancements in large visual-language models (LVLMs), hallucinations remain a major bottleneck in their practical applications. One key factor contributing to hallucinations is the over-reliance on language priors during the autoregressive text generation process. Visual Contrastive Decoding (VCD), a popular technique for mitigating hallucinations, perturbs the visual input and compares the perturbed output with the original. However, it often overlooks the gradual attenuation of visual information within the decoder, limiting the model's ability to generate text based on actual visual content. We propose a novel, training-free method—Visual-Enhanced Contrastive Decoding (VECD)—which addresses this issue by amplifying visual information within the decoder, thereby reducing hallucinations caused by excessive reliance on language priors. VECD dynamically selects later layers for visual injection, while retaining only essential visual tokens in early layers. This approach enhances the generation process by adaptively balancing visual and language priors. By comparing outputs with and without visual amplification, we derive a refined probability distribution for the next token. Moreover, we improve the beam search algorithm by introducing a visually guided token selection strategy, enabling the generation of text that aligns more closely with the image content. Our extensive experiments show that VECD significantly reduces hallucinations and improves the quality of generated text, demonstrating its effectiveness as a practical solution.

Video Question Answering (VideoQA) aims to answer a question based on the content of a given video. Recent methods adapt image-text pre-trained models to the VideoQA task by designing learnable temporal modules within the image encoder. However, these methods struggle to fully comprehend the questions and effectively extract temporal information due to 1) over-reliance on candidate answers and 2) lack of explicit temporal modeling. Specifically, since the question is fixed in different question-answer pairs, existing models tend to focus on the varying candidate answers. Moreover, existing methods merely utilize the classification loss to constrain the confidence of candidate answers, failing to differentiate the effectiveness of temporal information and to explicitly guide temporal modeling. In this paper, we introduce the Question Understanding and Temporality Guiding (QU-TG) method to address the aforementioned limitations. To reduce over-reliance on candidate answers, we propose providing diverse questions through question selection and enhancing the model's comprehensive understanding of questions through question-video matching. To conduct explicit temporal modeling guiding, we propose negative video prevention and positive video guidance to conduct explicit temporal modeling guiding. Negative video prevention incorporates a prevention loss to discourage the model from making predictions based on erroneous temporal cues, whereas positive video guidance utilizes classification loss to encourage the model to derive correct answers from positive videos. Extensive experiments on the NExT-QA, IntentQA, STAR-QA, and Causal-VidQA datasets demonstrate the effectiveness and generalization of our method.

Unsupervised domain adaptation (UDA) is critical for cross-modal 3D semantic segmentation in real scenarios, as it significantly reduces the annotation costs in new domains. Existing methods primarily focus on various alignment across domains or modalities, however, domain shift and modal differences are inbuilt, and learning under the guidance of source semantic knowledge instead of forced alignment may be more favorable for UDA tasks. Motivated by this insight, we propose cross Domain Label-Guided learning (xDLG) for cross-modal UDA 3D semantic segmentation, which utilizes multimodal pseudo-labels generated online as a bridge for cross-domain label knowledge transfer. Specifically, we propose two types of label-guided learning: Label-Guided Semantic Prototype Learning (LG-SPL) and Label-Guided Pseudo-Label Learning (LG-PLL). First, in LG-SPL, we utilize massive class instances from the source domain to correct the semantic prototypes in the target domain, embedding genuine semantic similarity to tackle the issue of class confusion. Next in LG-PLL, we conceptualize the refinement of pseudo-labels as a reverse diffusion process, leveraging label information from the source domain as a guiding signal to adjust the semantic distribution of pseudo-labels on a global scale. Furthermore, we incorporate domain-specific cues from target 2D images as supplementary conditions throughout the denoising stages, ensuring semantic consistency at finer details. We evaluate our method on 3D semantic segmentation tasks. The results demonstrate that the proposed method achieves state-of-the-art performance across multiple cross-modal UDA scenarios.

Cross-modal hashing seeks to encode heterogeneous image-text data into compact binary codes for efficient retrieval. While significant efforts have been made to study sampling strategies, most of these approaches are tightly integrated with loss function engineering, lacking an independent focus on sampling methods. In this paper, we challenge the convention by revealing that batch-level sampling strategy is equally pivotal as loss design for learning discriminative hash codes. Specifically, by introducing a novel distribution-aware sampling strategy, a Distance Weighted Sampling Hashing (DDWSH) framework is proposed to dynamically select stable and informative training pairs. Unlike conventional random or semi-hard sampling, our method weights pairwise distances within each batch to approximate global data distribution, thereby mitigating training instability caused by biased sampling. To rigorously validate our claims, we conduct the first comprehensive crossover study between sampling strategies (random/semi-hard/ours) and loss functions (contrastive/triplet/ours) across three benchmark datasets. Experiments demonstrate that: Universality: Our sampling boosts all loss functions' performance (average +4.3% mAP vs. semi-hard mining), Superiority: DDWSH competes with complex loss function design and achieves state-of-the-art results, and Stability: It reduces performance variance by 60.2% compared to semi-hard sampling under varying batch compositions average. This systematic analysis establishes sampling as an independent research dimension in deep hashing, beyond a mere part of loss function engineering. The source code for DDWSH is freely available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/QinLab-WFU/DDWSH</uri>.

Implementing fine-grained emotion control is crucial for emotion generation tasks because it enhances the expressive capability of the generative model, allowing it to accurately and comprehensively capture and express various nuanced emotional states, thereby improving the emotional quality and personalization of generated content. Generating fine-grained facial animations that accurately portray emotional expressions using only a portrait and an audio recording presents a challenge. In order to address this challenge, we propose a visual attribute-guided audio decoupler. This enables the obtention of content vectors solely related to the audio content, enhancing the stability of subsequent lip movement coefficient predictions. To achieve more precise emotional expression, we introduce a fine-grained emotion coefficient prediction module. Additionally, we propose an emotion intensity control method using a fine-grained emotion matrix. Through these, effective control over emotional expression in the generated videos and finer classification of emotion intensity are accomplished. Subsequently, a series of 3DMM coefficient generation networks are designed to predict 3D coefficients, followed by the utilization of a rendering network to generate the final video. Our experimental results demonstrate that our proposed method, EmoSpeaker, outperforms existing emotional talking face generation methods in terms of expression variation and lip synchronization. Project page: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://peterfanfan.github.io/EmoSpeaker/</uri>

Although Multimodal Large Language Models (MLLMs) excel at various image-related tasks, they encounter challenges in precisely aligning coordinates with spatial information within images, particularly in position-aware tasks such as visual grounding. This limitation arises from two key factors. First, MLLMs lack explicit spatial references, making it difficult to associate textual descriptions with precise image locations. Second, their feature extraction processes prioritize global context over fine-grained spatial details, leading to weak localization capability. To address these issues, we introduce VPP-LLaVA, an MLLM enhanced with Visual Position Prompt (VPP) to improve its grounding capability. VPP-LLaVA integrates two complementary mechanisms: the global VPP overlays a learnable, axis-like tensor onto the input image to provide structured spatial cues, while the local VPP incorporates position-aware queries to support fine-grained localization. To effectively train our model with spatial guidance, we further introduce VPP-SFT, a curated dataset of 0.6 M high-quality visual grounding samples. Designed in a compact format, it enables efficient training and is significantly smaller than datasets used by other MLLMs (e.g., 21 M samples in MiniGPT-v2), yet still provides a strong performance boost. The resulting model, VPP-LLaVA, not only achieves state-of-the-art results on standard visual grounding benchmarks but also demonstrates strong zero-shot generalization to challenging unseen datasets.

In the past few years, 360° video has started to infiltrate various aspects of daily life. Although there have been significant developments in 360° video coding technology, understanding of the human visual gaze mechanism has been somewhat overlooked. In this paper, we propose a rate control scheme for 360° Versatile Video Coding (VVC) based on a human visual gaze mechanism, targeting at improving the coding performance and bitrate accuracy. More specifically, based on the Equi-rectangular Projection (ERP) format, latitude information is systematically analyzed and a stripe-level bit allocation scheme is established, to better mitigate the projection distortion. Subsequently, the Lagrange parameter λ is further optimized with distortion dependency and identification of the visual gaze guided key Coding Tree Units (CTUs). The proposed rate control scheme is implemented on the VVC Test Model for 360° video. Experimental results show that the proposed rate control scheme can achieve BD-rate savings in terms of Weighted to Spherically uniform-Peak Signal-to-Noise Ratio (WS-PSNR) and Sphere-Peak Signal-to-Noise Ratio (S-PSNR) under the various configurations, respectively. Meanwhile, a healthier buffer status and better visual quality can be observed, further demonstrating the advantages of the proposed scheme.

Although text information has aided existing models to achieve promising results in open vocabulary object detection (OVD), the lack of semantic information has led to the difficulty in small objects detection (SOD). Moreover, such semantic gap also causes failure when matching texts and image features, resulting in false negative instances being detected. To address these issues, we propose a Cascade Open Vocabulary Detector (Cas-OVD), which builds upon existing multi-stage detection pipelines but specializes in text-vision alignment for small objects. In particular, we adapt a multi-refined region proposal network, guided by a non-sampled anchor strategy, to reduce the missing and false detections of small objects. Meanwhile, a deformable convolution network based feature conversion module is proposed to enhance the semantic information of small objects even the potential ones with low confidence. Unlike existing methods that rely on coarse-grained image-based features for image-text matching, Cas-OVD refines these features through a cascade alignment process, allowing each stage to build on the results of the previous one. This can progressively enhance the feature correlation between the image regions and the textual descriptions through successive error correction. On the joint BDD100K-SODA-D dataset, Cas-OVD achieved 17.95% AP<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{\mathrm{all}}$</tex-math></inline-formula> and 14.6% AP<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{\mathrm{s}}$</tex-math></inline-formula>, outperforming RegionCLIP by 3.5% AP<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{\mathrm{all}}$</tex-math></inline-formula> and 3.0% AP<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{\mathrm{s}}$</tex-math></inline-formula>, respectively. On the OV_COCO dataset, Cas-OVD has the 32.71% AP<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{\mathrm{all}}$</tex-math></inline-formula> and 17.26% AP<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{\mathrm{s}}$</tex-math></inline-formula>, surpassing the RegionCLIP by 6.6% AP<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{\mathrm{all}}$</tex-math></inline-formula> and 6.1% AP<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{\mathrm{s}}$</tex-math></inline-formula>, respectively.

In many multimedia scenarios, large volumes of unlabeled data are continuously generated, with new categories emerging over time. Traditional supervised methods fall short in such open-world settings. Generalized Category Discovery (GCD) addresses this challenge by identifying both known and novel categories from unlabeled data. Although existing methods perform well on generic datasets, they struggle in fine-grained scenarios. We attribute this difficulty to their reliance on contrastive learning over global image features to automatically capture discriminative cues, which fails to capture the subtle local differences essential for distinguishing fine-grained categories. Therefore, in this paper, we propose incorporating part knowledge to address fine-grained GCD, which introduces two key challenges: the absence of annotations for novel classes complicates the extraction of the part features, and global contrastive learning prioritizes holistic feature invariance, inadvertently suppressing discriminative local part patterns. To address these challenges, we propose <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PartGCD</b>, including 1) Adaptive Part Decomposition, which automatically extracts class-specific semantic parts via Gaussian Mixture Models, and 2) Part Discrepancy Regularization, enforcing explicit separation between part features to amplify fine-grained local part distinctions. Experiments demonstrate state-of-the-art performance across multiple fine-grained benchmarks while maintaining competitiveness on generic datasets, validating the effectiveness and robustness of our approach.