Face sketch synthesis is a technique aimed at converting face photos into sketches. Existing face sketch synthesis research mainly relies on training with numerous photo–sketch sample pairs from existing datasets. However, these large-scale discriminative learning methods will have to face problems, such as data scarcity and high human labor costs. Once the training data become scarce, their generative performance significantly degrades. In this article, we propose a one-shot face sketch synthesis method based on diffusion models. We optimize text instructions on a diffusion model using face photo–sketch image pairs. Then, the instructions derived through gradient-based optimization are used for inference. To simulate real-world scenarios more accurately and evaluate method effectiveness more comprehensively, we introduce a new benchmark named One-shot Face Sketch Dataset (OS-Sketch). The benchmark consists of 400 pairs of face photo–sketch images, including sketches with different styles and photos with different backgrounds, ages, sexes, expressions, illumination, and so on. For a solid out-of-distribution evaluation, we select only one pair of images for training at each time, with the rest used for inference. Extensive experiments demonstrate that the proposed method can convert various photos into realistic and highly consistent sketches in a one-shot context. Compared to other methods, our approach offers greater convenience and broader applicability. The dataset will be available at: https://github.com/HanWu3125/OS-Sketch .

Video-based alarm notification systems play a critical role in safety-critical applications such as fire detection and pedestrian monitoring, because they allow for prompt intervention in the event of dangerous situations. However, existing approaches often struggle to balance high sensitivity and specificity with real-time performance, particularly in complex scenes. To address these limitations, we propose a novel method enabling Fast Alarm Notification with Task-Aware Spatio-Temporal Image Classification (FAN-TAST-IC). It is an innovative and efficient framework that combines a lightweight task-specific object detector with a pre-trained Vision-Language Model (VLM) encoder and a binary classifier. Unlike end-to-end multimodal systems, FAN-TAST-IC leverages the VLM solely as a frozen visual feature extractor, preserving its rich semantic knowledge while ensuring computational efficiency. The object detector performs a rough but real-time filtering of the temporal frames and spatial positions where objects can be. This filtering is then refined in two distinct ways, depending on the time constraints of the specific application, either to discard temporally incoherent detections or to confirm those with high confidence. The selection of such candidates drastically reduces the input space for the VLM and classifier to dubious detections only. Thus, the latter is trained on detector-guided positive and negative samples, enabling precise alarm validation, improving specificity, and preserving sensitivity without requiring extensive labeled datasets or fine-tuning the VLM. Experiments on fire and pedestrian detection tasks demonstrate the effectiveness of our method since FAN-TAST-IC consistently outperforms all the compared approaches, achieving superior F-scores and precision on challenging benchmarks while maintaining real-time capabilities.

Offline signature verification remains vulnerable to real-world degradations such as low-resolution scans, transmission artifacts, and environmental noise. We introduce a variant robustness mechanism that foregrounds stroke-level evidence, and instantiate it in a Diffusion-based Robust Signature Verification (DRSV) framework. DRSV integrates two components: (i) a diffusion-driven noise-signature relation module that synthesizes plausible variations to enhance structural resilience, and (ii) a transformer-based verifier that fuses the predicted noise with the signature image via bi-directional channel attention to decide authenticity. Experiments on five public datasets (CEDAR, BHSig-Bengali, BHSig-Hindi, GPDS, UTSig) show consistent accuracy gains and improved robustness across diverse perturbations, indicating the effectiveness of the proposed framework.

Lip reading via event cameras could capture subtle and similar lip motions with large dynamic range and microsecond latency, offering high temporal resolution over traditional frame-based cameras. However, existing approaches often overlook the exploitation of distinctive lip motion patterns, instead opting to adapt existing video recognition architectures. In this paper, we propose an event-specified framework named the Triplane Fusion Network (TF-Net) to harness lip movements through analysis from three distinct but complementary views. Specifically, while adhering to the standard XYT view, we further incorporate two additional perspectives: XT and YT, aiming to capitalize on the unique streaming nature of events over time. Given the three views, TF-Net contains multiple expert blocks for every view and mutual information exchange blocks that facilitate the multi-directional exchange of motion information among the different views. We observe that these three views complement each other, further enhancing the learning of the event-specified distribution. Extensive experiments validate the effectiveness of the proposed method, surpassing +1.6% and +2.3% accuracy over other competitive methods on the real-world dataset DVS-Lip and the synthetic dataset Modality, respectively.

Composed Video Retrieval (CVR) is a novel video retrieval paradigm. Unlike traditional single-modal video retrieval paradigms ( e.g. , text to video or video to video), CVR employs multi-modal queries (including both a reference video and a natural language modification) to retrieve the target video that best matches the modified reference video. Existing CVR methods primarily rely on generalized knowledge from vision-language pre-trained models or utilize caption expansions to enhance video comprehension. However, these approaches overlook the benefits offered by the shareability and variability of videos for multi-modal query understanding. To overcome this limitation, we introduce a novel CVR framework named shaREd and diFferential semantIcs eNhancement nEtwork ( REFINE ). REFINE is the first framework to exploit the shareability and variability of videos to improve multi-modal query comprehension. Specifically, REFINE leverages learnable tokens to achieve enhanced shared feature representation. Moreover, it introduces a carefully designed Differential Block to disentangle differential semantics between frames and employs modification associations to guide multi-modal query feature fusion. Additionally, REFINE has been extended to the Composed Image Retrieval task, making it effectively generalize across existing composed multi-modal retrieval scenarios and outperform existing methods. Extensive qualitative and quantitative evaluations on four benchmark datasets validate the superiority of the proposed REFINE framework.

Visible-infrared person re-identification (VI-ReID) aims to match pedestrian images captured under visible and infrared modalities, which suffer from significant domain discrepancies. Existing approaches either synthesize cross-modal images or learn modality-invariant representations, yet often encounter semantic degradation or limited alignment capacity. Recent vision-language models leverage textual semantics for modality bridging; however, CLIP-based frameworks typically rely on learnable token proxies with limited expressiveness. In this paper, we propose a novel semantic-driven framework that explicitly generates rich, modality-agnostic textual descriptions from images as alignment cues. Specifically, we design a dual-branch Text Semantic Generation Module that includes: (1) an Attribute-Aware text descriptions Generation module using prompt-based templates to capture modality-invariant identity cues, and (2) a Contour-Aware text prompt Module that provides complementary structural information often missing in textual descriptions. To reconcile semantic heterogeneity, a Text Re-definition Module(TRM) fuses instance-level and class-level semantics into unified representations, enabling fine-grained alignment with image features. Furthermore, we construct category-level textual prototypes as global semantic anchors to enhance cross-modal consistency. Extensive experiments on two standard VI-ReID benchmarks demonstrate that our method achieves superior performance, validating its effectiveness in semantic-guided modality alignment.

Clinical depression diagnosis relies heavily on both verbal and non-verbal cues in patient interviews, yet existing automated methods often operate as black-box models and fail to provide trustworthy explanations, limiting their clinical applicability. Moreover, depression datasets commonly impose strict privacy constraints that prohibit access to raw audio–visual data, and the deployment of large LLMs in real-world medical environments is often constrained by computational cost limitations. To address these challenges, we propose MLlm-DR, a multimodal large language model for explainable depression recognition. We first employ knowledge distillation to transfer high-quality diagnostic rationales from a powerful LLMs to a smaller, deployable LLMs, thereby equipping it with clinically aligned reasoning abilities. We further introduce a lightweight query module (LQ-former) that extracts salient depression-related cues from pre-extracted audio and visual features and maps them into an LLM-compatible representation. MLlm-DR is trained in two stages: multimodal alignment via LQ-former pretraining, followed by multi-task optimization that jointly learns rationale generation and score regression, encouraging consistency and improving interpretability. Experimental results show that MLlm-DR achieves state-of-the-art performance on two interview-based benchmark datasets, CMDC and E-DAIC-WOZ, while simultaneously generating clinically meaningful and readable explanations.

With advances in generation techniques, malicious users can easily generate deepfake videos, which can cause severe social problems and trust issues. Therefore, deepfake video detection has received increasing attention in recent years. Given that forgery clues are often subtle and imperceptible, effective detection relies heavily on multi-grained learning. However, existing approaches fail to systematically incorporate multi-grained learning across the key components of network training—namely, the training data, network structure, and supervision strategy—thus limiting their performance. In this paper, we propose a multi-grained parallel spatio-temporal deepfake video detection architecture, which introduces a novel framework to mine more discriminative deepfake cues throughout the training pipeline. Firstly, we design a parallel spatio-temporal network combined with a cross-guided mechanism to concurrently extract frame-level spatial features and patch-level temporal features, while leveraging the relationship between spatial artifacts and temporal inconsistencies to enable multi-grained spatio-temporal synchronous learning. Secondly, we propose segment-level data augmentation strategies, including frame-random consistent self-blending and spatio-temporal data augmentation, which improve training data diversity at both frame and patch levels, thereby improving the model's ability to learn comprehensive deepfake representations. Finally, we construct a multi-grained supervision, comprising a patch-level temporal loss, a distance-based frame-level spatial loss, and a standard segment-level loss, for subtle deepfake feature learning. Extensive experiments demonstrate that our method possesses strong robustness and the generalization ability outperforms the current state-of-the-art methods across a series of deepfake datasets, including FaceForensics++, CelebDF, DFDC, DeeperForensics, and Faceshifter, on average.

Recent technological advancements have led to the widespread presence of video in our daily lives, particularly through social media, where it has become the dominant medium of communication. As a result, the ability to analyze and understand videos has become essential for managing such a massive volume of content. Within this context, video captioning is one of the most effective ways to achieve this understanding. Recent advances in Natural Language Generation (NLG), combined with breakthroughs in Computer Vision, has facilitated the development of robust Video Captioning (VC) models. However, despite these advancements, selecting the right model for the right application remains challenging due to limited evaluation methods, and significant limitations still hinder the widespread adoption of these models. This survey aims to address these issues by proposing a comprehensive overview of the VC models landscape, along with a comparative experimental study, carried out on two benchmark datasets and involving eight representative, high quality models.

Motion customization plays a pivotal role in video generation by preserving the original appearance and context while adhering to specific motion patterns. In contrast, video generation techniques often lack coherence and realism due to difficulties in capturing and transferring motion patterns. Building upon the Video Motion Customization (VMC) framework, we proposed a few-shot learning approach using our unified Multi-Head Temporal Attention (MHTA) module for motion customization in text-to-video diffusion models. This significantly reduces computational requirements while maintaining and improving motion quality. Our model provides a streamlined mechanism for motion distillation while maintaining separate self-, cross-, and Temporal Attention. Moreover, the temporal attention layer is adapted through a simplified mechanism with efficient Q/K/V projections, while maintaining fixed spatial self- and cross-attention. The model distills a ground-truth motion vector from consecutive frames to align the predicted and ground-truth motion. Our proposed MHTA model outperforms the baseline in video generation using motion customization while being significantly more resource-efficient. Moreover, our approach can easily be applied to generate conditional prompt-based videos in the gaming industry.