Assistive aerial teleoperation enables novice pilots to safely navigate vehicles in unknown, cluttered environments by planning collision-free trajectories. However, existing methods often neglect pilots' reaction time and proficiency level in human-computer interaction. As a result, novice pilots experience significant cognitive and operational burdens in complex scenarios. To address these challenges, this paper proposes a human-guided motion planner with environmental adaptation (EA). Specifically, we introduce a fast visibility-checking algorithm based on convex sensing hulls, integrated with deceleration constraints, to extend flight time at intersections and facilitate human decision-making. Furthermore, we develop a personalized environment-adaptive algorithm that dynamically adjusts flight aggressiveness by leveraging environmental perception and human feedback through a human-in-the-loop reinforcement learning framework. Finally, we benchmark our proposed approach against state-of-the-art assistive aerial teleoperation methods, demonstrating its superior performance in both simulation and real-world experiments.

We present a novel framework for human-robot interaction designers to analyze and explore expectations of their robot designs. It consists of a model of how people form expectations of robots, and a taxonomy for classifying them. A known challenge in human-robot interactions is expectation discrepancy, in which the expectations people form when interacting with a social robot are not aligned with its actual capabilities. This can disappoint users and hinder interaction. Research has proposed ways to mitigate expectation discrepancy, but designers lack a systematic approach to analyzing and describing expectations. We developed a rigorous theoretical framework by drawing from theories and models from psychology and sociology on expectations between people, and by conducting a field review of expectations in human-robot interactions. We further propose methods for designers to leverage the framework in systematic analysis of how and why people form expectations of a given robot and what those expectations may be. This can empower designers with greater control over people’s expectations, enabling them to combat expectation discrepancy.

Rendering haptic feedback on the wrist is an effective solution for freeing users' hands during virtual interactions while still providing haptic feedback, mostly focusing on Virtual Reality (VR) environments. Unfortunately, whether these solutions can be extended to Augmented Reality (AR) interactions has not been investigated before. In this paper, we investigate the perceptual differences between VR and AR environments with a user study experiment based on a stiffness discrimination task. Our findings revealed similar task accuracy and sensitivity in VR and AR but different exploration behaviors (i.e. higher interaction time in AR) and user experience (i.e. higher enjoyment and less mental fatigue in AR). These differences may be explained by the increased cognitive load and the environmental immersion in AR, where users navigate both virtual and real elements simultaneously. This suggests that visuo-haptic psychophysical findings obtained in VR can be extended to AR, even though user experience and exploration behavior might differ.

Head-tilting motions are notable social cues that have a significant impact on human-robot interaction, since prior studies suggest they can enhance the feeling of kawaii given by a robot. Kawaii, a Japanese term referring to cuteness or endearment, is an emotional response that has proven critical for designing socially acceptable robots. However, the influence of head-tilting characteristics remains underexplored. Therefore, we investigate how variations in both the speed and direction of head-tilting motion affect the feeling of kawaii, as well as investigate the effectiveness of hand gestures during the motion. Moreover, we explore whether the kawaii characteristics of an object can be transferred to its surrounding objects, i.e. the kawaii spillover effect, by using the robot and its outfits. We conduct web surveys for three different purposes. First, the feeling of kawaii toward the robot is enhanced when it tilts its head to its right (from participant’s viewpoint) within 0.5–1 s. Second, the hand gesture (hands-on-cheeks) along with the head-tilting motion also enhances the feeling of kawaii toward the robot. Finally, this head-tilting motion with the hands-on-cheeks gesture increases the feeling of kawaii toward the robot and its outfit, as evidence of the kawaii spillover effect.

In this work, we address the problem of predicting the future success of open-vocabulary object manipulation tasks. Conventional approaches typically determine success or failure after the action has been carried out. However, they make it difficult to prevent potential hazards and rely on failures to trigger replanning, thereby reducing the efficiency of object manipulation sequences. To overcome these challenges, we propose a model, which predicts the alignment between a pre-manipulation egocentric image with the planned trajectory and a given natural language instruction. We introduce a Multi-Level Trajectory Fusion module, which employs a state-of-the-art deep state-space model and a transformer encoder in parallel to capture multi-level time-series self-correlation within the end effector trajectory. Our experimental results indicate that the proposed method outperformed existing methods, including foundation models.

Robotic grasping is a fundamental yet crucial component of robotic applications, as effective grasping often serves as the starting point for various tasks. With the rapid advancement of neural networks, data-driven approaches for robotic grasping have become mainstream. However, efficiently generating grasp datasets for training remains a bottleneck. This is compounded by the diverse structures of robotic hands, making the design of generalizable grasp generation methods even more complex. In this work, we propose a teleoperation-based framework to collect a small set of grasp pose demonstrations, which are augmented using FSG–a Fingertip-contact-aware Sampling-based Grasp generator. Based on the demonstrated grasp poses, we propose AutoWS, which automatically generates structured workspace clouds of robotic fingertips, embedding the hand structure information directly into the clouds to eliminate the need for inverse kinematics calculations. Experiments on grasping the YCB objects show that our method significantly outperforms existing approaches in both speed and valid pose generation rate. Our framework enables real-time grasp generation for hands with arbitrary structures and produces human-like grasps when combined with demonstrations, providing an efficient and robust data augmentation tool for data-driven grasp training.

Planning smooth orientation trajectories is a complex challenge in robotic spray painting, especially due to the need to ensure continuity in velocity and acceleration while avoiding singularities associated with Euler angles. This paper introduces a method for planning the orientation trajectories of spray-painting robots. The proposed approach leverages cubic B-splines with cumulative basis functions and unit quaternions to achieve a global interpolation of input orientations, guiding the robot tool along the required path. The approach is first compared with a standard strategy in which orientations are expressed as Euler angles, to demonstrate that the use of unit quaternions for orientation representation helps circumvent potential singularity issues associated with Euler angles. Then, the experimental validation on an industrial spray-painting robot with six degrees of freedom, following the trajectory needed to paint a three-dimensional model of a car bumper, illustrates the effectiveness of the proposed strategy in approximating input orientations. The approach also allows maintaining continuity in velocity and acceleration, crucial for successful industrial spray painting, while meeting joint velocities and accelerations limits. Furthermore, the maximum variation of the tangential velocity of the robot end-effector, relative to the imposed value, reaches 4% near the car bumper headlights, where the orientation changes rapidly.

While general-purpose models for robot learning have been extensively studied in the field of navigation, their translational movement capabilities are often limited to quasi-static motion. In contrast, quadrotors have demonstrated agile obstacle avoidance through reinforcement learning (RL) in drone racing. However, such policies are specifically designed for agile quadrotors with particular dynamic models, limiting their applicability to other types of aerial vehicles. To address these two limitations, we present a general-purpose reinforcement learning framework that enables different velocity configurations for the quadrotor. The key point of our framework is the randomization of three key transitional parameters of the vehicle: ‘Acc-Property’ (maximum acceleration, response time of acceleration, and quadrotor radius for collision detection). We demonstrate that randomizing dynamics and controllers is necessary to generalize policies to different types of quadrotors in smooth flight. Furthermore, we evaluate our learned policy on existing quadrotor systems ranging in size from 0.5 meters to 1.0 meter on real hardware, achieving 100% static obstacle avoidance in 33 trials and 78.6% dynamic obstacle avoidance in 42 trials. To the best of our knowledge, this study represents the first exploration of a generalized policy for multirotor collision avoidance in non-static flight conditions.

In this study, we aim to develop a domestic service robot (DSR) that, guided by open-vocabulary instructions, can carry everyday objects to the specified pieces of furniture. Few existing methods handle mobile manipulation tasks with open-vocabulary instructions in the image retrieval setting, and most do not identify both the target objects and the receptacles. We propose the Dual-Mode Multimodal Ranking model (DM 2 RM), which enables images of both the target objects and receptacles to be retrieved using a single model based on multimodal foundation models. We introduce a switching mechanism that leverages a mode token and phrase identification via a large language model to switch the embedding space based on the prediction target. To evaluate the DM 2 RM, we construct a novel dataset including real-world images collected from hundreds of building-scale environments and crowd-sourced instructions with referring expressions. The evaluation results show that the proposed DM 2 RM outperforms previous approaches in terms of standard metrics in image retrieval settings. Furthermore, we demonstrate the application of the DM 2 RM on a standardized real-world DSR platform including fetch-and-carry actions, where it achieves a task success rate of 82% despite the zero-shot transfer setting. Demonstration videos, code, and more materials are available at https://kkrr10.github.io/dm2rm/.

In this study, we propose the use of the phase-amplitude reduction method to construct an imitation learning framework. Imitating human movement trajectories is recognized as a promising strategy for generating a range of human-like robot movements. Unlike previous dynamical system-based imitation learning approaches, our proposed method allows the robot not only to imitate a limit cycle trajectory but also to replicate the transient movement from the initial or disturbed state to the limit cycle. Consequently, our method offers a safer imitation learning approach that avoids generating unpredictable motions immediately after disturbances or from a specified initial state. We first validated our proposed method by reconstructing a simple limit-cycle attractor. We then compared the proposed approach with a conventional method on a lemniscate trajectory tracking task with a simulated robot arm. Our findings confirm that our proposed method can more accurately generate transient movements to converge on a target periodic attractor compared to the previous standard approach. Subsequently, we applied our method to a real robot arm to imitate periodic human movements.