Vision Transformers have achieved remarkable success in visual recognition tasks due to their strong capability to model long-range dependencies through global self-attention. However, the inherent quadratic complexity of global attention severely limits their scalability when processing high-resolution inputs. To address this issue, window-based attention mechanisms have been proposed to reduce computational overhead, but they often struggle with limited receptive fields and insufficient global context modeling. In this work, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GTR-ViT</i>, a novel and efficient Vision Transformer framework that integrates a Global Token Retrieval (GTR) mechanism with an enhanced feedforward network in a synergistic manner. The GTR mechanism dynamically retrieves globally relevant tokens for each token, enriching the local features with comprehensive global context. These globally enhanced features are then fed into a gated feedforward network equipped with depthwise convolution, which further refines local representations through adaptive feature modulation. In turn, the improved local features serve as more representative queries for the GTR module, enabling more accurate global token retrieval in subsequent layers. This mutual reinforcement forms a virtuous cycle of local-global interaction, significantly boosting the models capability in both context understanding and feature discrimination. Extensive experiments on image classification and downstream tasks across multiple datasets demonstrate that GTR-ViT consistently surpasses prior state-of-the-art methods, highlighting the effectiveness of this collaborative design.

Metro trains often operate in highly complex environments, characterized by illumination variations, high-speed motion, and adverse weather conditions. These factors pose significant challenges for visual perception systems, especially those relying solely on conventional RGB cameras. To tackle these difficulties, we explore the integration of event cameras into the perception system, leveraging their advantages in low-light conditions, high-speed scenarios, and low power consumption. Specifically, we focus on Kilometer Marker Recognition (KMR), a critical task for autonomous metro localization under GNSS-denied conditions. In this context, we propose a robust baseline method based on a pre-trained RGB OCR foundation model, enhanced through multi-modal adaptation. Furthermore, we construct the first large-scale RGB-Event dataset, EvMetro5K, containing 5,599 pairs of synchronized RGB-Event samples, split into 4,479 training and 1,120 testing samples. Extensive experiments on EvMetro5K and other widely used benchmarks demonstrate the effectiveness of our approach for KMR. Both the dataset and source code will be released on <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Event-AHU/EvMetro5K_benchmark</uri>

Evolutionary neural architecture search (ENAS) automates the design of neural network architectures but incurs significant computational costs during the evaluation phase of candidate neural networks in the evolutionary algorithm. To address this issue, surrogate models are widely adopted to estimate the performance of candidate architectures, thereby reducing resource consumption. However, existing surrogate models are primarily based on regression or classification. They are often hindered by interference from low-quality architectures, which limits search efficiency and final performance. To overcome these challenges, we propose surrogate-assisted neural architecture search with performance clustering (PCNAS). PCNAS introduces a contrastive learning-based performance clustering encoder (PCE). The PCE maps candidate architectures into an embedding space to form high-performance clusters while maximizing separation from low-performance architectures. This design mitigates the negative influence of poor-quality candidates. To enhance the discriminative power of the PCE, we incorporate a staged sample generation strategy to strengthen the hierarchical representation capacity of the PCE. Additionally, we introduce a training-free simple classifier to efficiently select high-performance architectures from the embedding space. PCNAS achieves high accuracy with low computational cost on NAS-Bench-101 and NAS-Bench- 201, outperforming previous approaches. Furthermore, PCNAS exhibits strong efficiency in the DARTS search space, discovering a high-performing architecture with a 2.53% error rate using only 0.5 GPU days of search cost, and on MobileNetV3 with ImageNet, achieving 78.12% top-1 accuracy using only 0.3 GPU days.

The recent proliferation of hyper-realistic deepfake videos has drawn attention to the threat of audio and visual forgeries. Most previous studies on detecting artificial intelligence-generated fake videos only utilize visual modality or audio modality. While some methods exploit audio and visual modalities to detect forged videos, they have not been comprehensively evaluated on multimodal datasets of deepfake videos involving acoustic and visual manipulations, and are mostly based on convolutional neural networks with low detection accuracy. Considering that human cognition instinctively integrates multisensory information including audio and visual cues to perceive and interpret content and the success of transformer in various fields, this study introduces the audio-visual transformer-based ensemble network (AVTENet). This innovative framework tackles the complexities of deepfake technology by integrating both acoustic and visual manipulations to enhance the accuracy of video forgery detection. Specifically, the proposed model integrates several purely transformer-based variants that capture video, audio, and audio-visual salient cues to reach a consensus in prediction. For evaluation, we use the recently released benchmark multimodal audio-video FakeAVCeleb dataset. For a detailed analysis, we evaluate AVTENet, its variants, and several existing methods on multiple test sets of the FakeAVCeleb dataset. Experimental results show that the proposed model outperforms all existing methods and achieves state-of-the-art performance on Testset-I and Testset-II of the FakeAVCeleb dataset. We also compare AVTENet against humans in detecting video forgery. The results show that AVTENet significantly outperforms humans.

Individual fluctuations and temporal variability of electroencephalogram (EEG) data pose challenges in precisely identifying emotions. Although a model may perform well with data specific to a certain subject or session, the fluctuations in EEG data can significantly impair the model's performance on a different subject or session. To tackle this problem, current approaches synchronize the original and new subject or session feature distributions. Directly matching EEG data across individuals or sessions may undermine the inherent distinguishability due to the heterogeneity in data distribution. Instead of direct alignment, this work utilizes multisource structural deep clustering to identify the inherent structural knowledge of the target itself and regularize it through the distribution of source labels. Furthermore, the method was implemented on the intermediate output utilizing high-confidence features to improve pattern identification in the latent feature space. This led to more distinct differentiations across subdomains with varying labels. Comparative analyses were performed with state of-the-art (SOTA) models on SEED and SEED-IV datasets. The model proposed outperformed other baseline models, reaching an average accuracy of 90.69%/95.05% in a cross-subject/cross session experiment on SEED and 74.35%/78.56% in SEED-IV. This research provides a novel approach to align EEG features without the need for direct distance calculation.

In this article, human–robot sharing operation in cotransporting for a nonholonomic mobile robot is studied. Sharing operation weights of human and nonholonomic mobile robot are proposed to avoid dynamical obstacles in an unknown environment, which assigns operation weights to human and robot based on sensory information of environment, the balance of task between “following the human” and “autonomous obstacle avoidance” is further realized. A local multimodal obstacle avoidance method based on 2-D Lidar is proposed, which deals with the interference of unrelated obstacles. In addition, we designed a mechanism to estimate human motion intention and integrated it into the nonholonomic mobile robotic platform for following the human. The experimental results show that the proposed method can effectively deal with the problem of human–robot cotransporting for a nonholonomic mobile robot in an unknown environment, can avoid obstacle and ensure human's operation comfort in cotransporting.

In this study, we propose a multitask reinforcement learning (RL) algorithm for foundational policy acquisition to generate novel motor skills. Learning the rich representation of the multitask policy is a challenge in dynamic movement generation tasks because the policy needs to cope with changes in goals or environments with different reward functions or physical parameters. Inspired by human sensorimotor adaptation mechanisms, we developed the learning pipeline to construct the encoder–decoder networks and network selection to facilitate foundational policy acquisition under multiple situations. First, we compared the proposed method with previous multitask RL methods in the standard multilocomotion tasks. The results showed that the proposed approach outperformed the baseline methods. Then, we applied the proposed method to the ball heading task using a monopod robot model to evaluate skill generation performance. The results showed that the proposed method was able to adapt to novel target positions or inexperienced ball restitution coefficients but to acquire a foundational policy network, originally learned for heading motion, which can generate an entirely new overhead kicking skill.

Recent advancements in spiking neural networks (SNNs) have highlighted their advantages, including energy efficiency, real-time processing, and compatibility with neuromorphic hardware. These features make SNNs particularly well-suited for human-robot interaction (HRI) in rehabilitative wearable robotics, where real-time adaptability and low power consumption are essential. However, there is still a lack of comprehensive reviews on SNNs’ application to HRI. This paper addresses this gap by providing a detailed overview of the latest advancements in SNNs from the perspective of embodied intelligence in rehabilitative wearable robots. We systematically examine recent progress in SNNs, including spiking neuron models, encoding methods, and learning mechanisms. These advancements are then analyzed with a focus on HRI, addressing specific challenges in rehabilitative wearable robots from three key perspectives: human motion decoding, robotic control, and neuromorphic implementation for embedded systems. By reviewing current research, this paper highlights the potential benefits and limitations of SNNs in achieving embodied intelligence and identifies crucial areas for further investigation, offering new insights and directions for their future applications in rehabilitative wearable robotics.