Learning from demonstration is a powerful method for robotic skill acquisition. Nevertheless, a critical limitation lies in the substantial costs associated with gathering demonstration datasets, typically action-labeled robot data, which creates a fundamental constraint in the field. Video data offer a compelling solution as an alternative rich data source, containing diverse behavioral and physical knowledge. This study introduces G3M, an innovative framework that exploits video data via <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</u>raph-to-<underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</u>raphs <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</u>enerative <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</u>odeling, which pre-trains models to generate future graphs conditioned on the graph within a video frame. The proposed G3M abstracts video frame into graph representations by identifying object and visual action vertices for capturing state information. It then effectively models internal structures and spatial relationships present in these graph constructions, with the objective of predicting forthcoming graphs. The generated graphs function as conditional inputs that guide the control policy in determining robotic behaviors. This concise method effectively encodes critical spatial relationships while facilitating accurate prediction of subsequent graph sequences, thus allowing the development of resilient control policy despite constraints in action-annotated training samples. Furthermore, these transferable graph representations enable the effective extraction of manipulation knowledge through human videos as well as recordings from robots with different embodiments. The experimental results demonstrate that G3M attains superior performance using merely 20% action-labeled data relative to comparable approaches. Moreover, our method outperforms the state-of-the-art method, showing performance gains exceeding 19% in simulated environments and 23% in real-world experiments, while delivering improvements of over 35% in cross-embodiment transfer experiments and exhibiting strong performance on long-horizon tasks. Our project page is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://g3m-project.github.io/</uri>.

Semantic segmentation based on RGB-Thermal (RGB-T) data fusion has made great progress in the field of autonomous driving. However, we find that most existing RGB-T semantic segmentation methods exhibit inferior performance in image central regions, in which segmentation performance is critical for driving safety. We refer to this phenomenon as spatial bias. To discover the reason for spatial bias, we design a series of experiments. The results challenge the common knowledge that more training data lead to better segmentation performance, and reveal a close causal relationship between segmentation performance and object complexity as well as image quality. We also provide a theoretical interpretation for the causal relationship using information theory and feature space analysis. Based on the findings, we propose a Gaussian-guided regional balancing masking method to balance segmentation performance across different image regions. Moreover, we introduce a spatial-weighted loss to further enhance the overall segmentation performance. Experimental results on two public datasets demonstrate the effectiveness of our method in mitigating spatial bias and improving balanced performance.

LiDAR-based Global Localization (LGL) is an essential ingredient for autonomous robots. However, existing LGL methods typically consider only partial information (e.g., geometric features) from LiDAR observations or are designed for homogeneous LiDAR sensors, overlooking the uniformity in LGL. In this work, a uniform LGL method is proposed, termed UniLGL, which simultaneously achieves spatial and material uniformity, as well as sensor-type uniformity. The key idea of the proposed method is to encode the complete point cloud, which contains both geometric and material information, into a pair of Bird's Eye View (BEV) images (i.e., a spatial BEV image and an intensity BEV image), thereby transforming the LGL problem into a cascaded LiDAR Place Recognition (LPR) and pose estimation problem from the perspective of image fusion. An end-to-end multi-BEV fusion network is designed to extract uniform features, equipping UniLGL with spatial and material uniformity. To ensure robust LGL across heterogeneous LiDAR sensors, a viewpoint invariance hypothesis is introduced, which replaces the conventional translation equivariance assumption commonly used in existing LPR networks and supervises UniLGL to achieve sensor type uniformity in both global descriptors and local feature representations. Moreover, UniLGL introduces a pipeline that leverages a pre-trained single-image Vision Foundation Model (VFM) for feature extraction to enhance the multi-BEV fusion LPR network, enabling strong generalization with only a few LiDAR data for fine-tuning. Finally, based on the mapping between local features on the 2D BEV image and the point cloud, a robust global pose estimator is derived that determines the global minimum of the global pose on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{SE}(3)$</tex-math></inline-formula> without requiring additional registration. To validate the effectiveness of the proposed uniform LGL, extensive benchmarks are conducted in real-world environments, and the results show that the proposed UniLGL is demonstratively competitive compared to other State-of-the-Art (SOTA) LGL methods. Furthermore, UniLGL has been deployed on diverse platforms, including full-size trucks and agile Micro Aerial Vehicles (MAVs), to enable high-precision localization and mapping as well as multi-MAV collaborative exploration in port and forest environments, demonstrating the applicability of UniLGL in industrial and field scenarios. The code will be released at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/shenhm516/UniLGL</uri>.

Avoiding the usage of joint accelerations during robot online identification and control is significant in improving modeling and tracking accuracy. Regressor filtering is feasible to achieve acceleration-free online identification, where the robot dynamics is linearly parameterized and filtered to obtain an acceleration-free filtered regressor. Nevertheless, existing calculation methods of the filtered regressor are either applicable only to robots with low degrees of freedom (DoFs) or restricted by robot modeling techniques. We propose an acceleration-free analytical regressor filtering (AF-ARF) method to obtain the filtered regressor without restrictions on the number of DoFs or robot modeling techniques, where joint accelerations are eliminated by integration by parts, and the filtered regressor is derived by using the skew-symmetric property of the inertia matrix and some matrix operations. An acceleration-free composite learning robot control strategy based on AF-ARF is developed for exact online identification and control, where closed-loop exponential stability with parameter convergence is established under a weakened condition of interval excitation. Simulative and experimental comparisons based on a seven-DoF industrial robot have validated the superiority of our method over state-of-the-art methods in online identification, model prediction, and tracking control under reduced computing burden.

Flexible sensors are increasingly employed in soft robotics and wearable devices to provide proprioception of freeform deformations. Although supervised learning can train shape predictors from sensor signals, prediction accuracy strongly depends on sensor layout, which is typically determined heuristically or through trial-and-error. This work introduces a model-free, data-driven computational pipeline that jointly optimizes the number, length, and placement of flexible length-measurement sensors together with the parameters of a shape prediction network for large freeform deformations. Unlike model-based approaches, the proposed method relies solely on datasets of deformed shapes, without requiring physical simulation models, and is therefore broadly applicable to diverse robotic sensing tasks. The pipeline incorporates differentiable loss functions that account for both prediction accuracy and manufacturability constraints. By co-optimizing sensor layouts and network parameters, the method significantly improves deformation prediction accuracy over unoptimized layouts while ensuring practical feasibility. The effectiveness and generality of the approach are validated through numerical and physical experiments on multiple soft robotic and wearable systems.

In agile quadrotor flight, accurately modeling the varying aerodynamic drag forces encountered at different speeds is critical. These drag forces significantly impact the performance and maneuverability of the UAV, especially during high-speed maneuvers. Traditional control models based on first principles struggle to capture these dynamics due to the complexity and variability of aerodynamic effects, which are challenging to model accurately. To address these challenges, this study proposes a meta-learning-based control strategy for accurately modeling quadrotor dynamics under varying speeds, treating each velocity condition as an independent learning task with a specifically trained neural network to ensure precise dynamic predictions. The meta-learning framework rapidly generates task-specific parameters adapted to speed variations by solving an optimization problem and employs an online incremental learning strategy to integrate real-time data for continuous model updates, enhancing system robustness. Regularization is introduced to prevent overfitting and improve generalizability. The integration of the meta-learned model into Model Predictive Contouring Control (MPCC) allows the system to achieve optimal control across different velocity levels, ensuring efficient and accurate flight control even during sharp turns and high-speed maneuvers. Extensive simulations and real-world experiments confirm that the proposed algorithm maintains a high level of control precision despite the nonlinear effects of rapid speed changes, complex flight trajectories and wind disturbances. The results highlight the advantages of combining meta-learning with adaptive control strategies, providing a robust framework for quadrotors operating in diverse and dynamic environments.

Aerial swarm systems possess immense potential in various aspects, such as cooperative exploration, target tracking, search and rescue. Efficient, accurate self and mutual state estimation are the critical preconditions for completing these swarm tasks, which remain challenging research topics. This paper proposes Swarm-LIO2: a fully decentralized, plug-andplay, computationally efficient, and bandwidth-efficient LiDARinertial odometry for aerial swarm systems. Swarm-LIO2 uses a decentralized, plug-and-play network as the communication infrastructure. Only bandwidth-efficient and low-dimensional information is exchanged, including identity, ego-state, mutual observation measurements, and global extrinsic transformations. To support the plug-and-play of new teammate participants, Swarm-LIO2 detects potential teammate UAVs and initializes the temporal offset and global extrinsic transformation all automatically. To enhance the initialization efficiency, novel reflectivitybased UAV detection, trajectory matching, and factor graph optimization methods are proposed. For state estimation, Swarm-LIO2 fuses LiDAR, IMU, and mutual observation measurements within an efficient ESIKF framework, with careful compensation of temporal delay and modeling of measurements to enhance the accuracy and consistency. Moreover, the proposed ESIKF framework leverages the global extrinsic for ego-state estimation in case of LiDAR degeneration or refines the global extrinsic along with the ego-state estimation otherwise. To enhance the scalability, Swarm-LIO2 introduces a novel marginalization method in the ESIKF, which prevents the growth of computational time with swarm size. Extensive simulation and realworld experiments demonstrate the broad adaptability to largescale aerial swarm systems and complicated scenarios, including GPS-denied scenes, degenerated scenes for cameras or LiDARs. The experimental results showcase the centimeter-level localization accuracy which outperforms other state-of-the-art LiDARinertial odometry for a single UAV system. Furthermore, diverse applications demonstrate the potential of Swarm-LIO2 to serve as reliable infrastructure for various aerial swarm missions. In addition, we open-source all the system designs on GitHub to benefit society: github.com/hku-mars/Swarm-LIO2.

The recent development of connected and automated vehicle (CAV) technologies has spurred investigations to optimize dense urban traffic, maximizing vehicle speed and throughput. This paper explores advisory autonomy, in which real-time driving advisories are issued to human drivers, thus achieving near-term performance of automated vehicles. Due to the complexity of traffic systems, recent studies of coordinating CAVs have leveraged deep reinforcement learning (RL). Coarse-grained advisory is formalized as zero-order holds, and we consider a range of hold durations from 0.1 to 40 seconds. However, despite the similarity of the higher-frequency tasks for CAVs, a direct application of deep RL fails to generalize to advisory autonomy tasks. To overcome this, we employ zero-shot transfer, training policies on a set of source tasks-specific traffic scenarios with designated hold durations-and then evaluating the efficacy of these policies on different target tasks. We introduce Temporal Transfer Learning (TTL) algorithms to select source tasks for zero-shot transfer, systematically leveraging the temporal structure to solve the full range of tasks. TTL selects the most suitable source tasks to maximize the performance of the range of tasks. We validate our algorithms on diverse mixed-traffic scenarios, demonstrating that TTL more reliably solves the tasks than baselines. This paper underscores the potential of coarse-grained advisory autonomy with TTL in traffic flow optimization.

Teams of automated battery-powered electric vehicles have the potential to execute complex mission tasks in off-road environments for agriculture, military, and other applications. Limited on-board energy reserves hinder their adoption in large scale, resource constrained environments, where recharging is a necessity. It may be infeasible to install a network of static charging stations in off-road environments. For this reason, dedicated mobile host vehicles with charging capabilities are proposed as a means to increase range and capabilities of the multi-vehicle team. Here, we consider an ad hoc planning framework where results from a high-confidence trajectory planner are leveraged to plan charging rendezvous between a host and other worker vehicles in a receding horizon fashion to provide high-confidence that energy reserves will not be prematurely exhausted. The core problem is posed so as to minimize the impact of recharging on the mission in terms of task delays, overall energy utilization and costs of fast charging. Through extensive Monte-Carlo simulations of an off-road mission, we show a decrease in task delays without substantial increases in energy needs by updating the charging rendezvous plan during the mission. However, if updates are made too often, model mismatch may cause unnecessary cycling and mission failure.