Ontology based autonomous robot task processing framework.

When a robot manipulator performs tasks in a dynamic unstructured environment, it not only should account for where and how it should move, but also ensure that the robot can effectively avoid collisions with static and moving obstacles. Planning in absolutely unstructured and dynamic environment is much more challenging, up-to-date there is no universal solution to this problem. In this paper, we believed the concept that for motion planning in real dynamic environments, usually there is only a part of a path invalidated by obstacles, and only this part needs replanning. Thus, in this paper a practical strategy was presented for motion planning in dynamic environments. First, a variant dynamic Rapidly-exploring Random Tree (RRT) planner was designed to combine global planning with local replanning. Then, the proposed dynamic RRT algorithm was implemented under Robotic Operation System (ROS) framework and applied to a 7-degree-of-freedom manipulator. Experimental results have shown that the proposed dynamic RRT planner can plan and execute movement of robot manipulator between a starting and target position while preserving task constraints and reacting to environment changes in real time.

Advanced robots are desired to improve their reasoning ability in increasingly practical situations to quickly respond to urgent challenges such as search and rescue, public security, and autonomous driving. Advances in sensor technology and intelligent algorithms are accelerating the development of autonomous systems, allowing them to interact with dynamic environments more independently and intelligently, completing relatively complicated tasks. Nevertheless, the safety and security problems are yet to be comprehensively addressed. Currently, many efforts have been made to solve this problem by improving the unimodal perception ability of intelligent systems. Methods using surveillance cameras to collect large amounts of visual data and process them using deep networks. Various types of range scanner can sense and reconstruct the surrounding environment through precise distance measurements. However, in the above cases, there may be a lack of training data, fixed cameras cannot adequately and dynamically model the surroundings, unimodal analysis is hard to capture both appearance and depth information. Therefore, higher levels of perception and reasoning abilities are required to handle unexpected conditions and scenarios efficiently. In such cases, mobile robots should be fully aware of the dynamic environment, prioritize, and continuously analyze critical situations, share information, and collaboration under limited computing power and resources. Furthermore, considering the robots are operating alongside humans, reliable and interpretable robot behavior is highly preferred for more natural interactions.
\n
\nThis study aims to develop reasoning strategies for autonomous robots to improve robotic intelligence under different assumptions and conditions. The framework allows one or several robots to address safety or security problems in a targeted manner in partially known environments, where time, dynamics, and interactions play a significant role. The objective is to identify the unique individual whose activity, in space and time, appear different from the other people in the scene. The activities considered not only the motion pattern, but also the object associated with this person. Unlike traditional abnormal detection, the uniqueness reasoning in this work does not require a predefined expected pattern. A modular analysis is also applied to decompose the entire reasoning process into submodules such as perception, analysis, and inference, to achieve algorithmic flexibility and interpretability. Probabilistic modeling is used to associate spatial relationships between semantic labels and their temporal changes. The relationships are described in either instant spatial spaces (semantic-interaction feature) or spatiotemporal spaces (spatiotemporal feature), and it provides essential information for robot navigation, situation awareness, and task planning. Unlike existing research, this is the first time that such a mission is implemented on mobile robot platforms by using the probabilistic fusion of multimodal information.
\n
\nConsidering how evidence information is obtained and linked in the process of reasoning, we study the problem of uniqueness reasoning at three levels. The first level is called knowledge-based single robot uniqueness reasoning and runs on a single robot platform. At this level, we focus on a knowledge-based approach, where knowledge is provided in terms of how motion and an object associated with a person can be used to infer his/her uniqueness, as determined by human expertise. Because such a method has advantages in dealing with structured missions and with limited training data. Qualitative rules are applied to extract human activities from observations, and then the uniqueness of each person is judged by associating the prior knowledge with the performed activities. Experiments were conducted in indoor environments with various degrees of occlusion, and the experimental results showed the rationality and accuracy of the reasoning.
\n
\nHowever, though the knowledge-based method can successfully address specific tasks, it lacks flexibility and generality to adapt to different situations. The assumption of prior knowledge should be replaced by general analytical modeling. Thus, the second level is analytics-based single robot uniqueness reasoning. For one thing, it applies quantitative analysis to model human activities from observations rather than using qualitative rules. For another, instead of leveraging prior knowledge, the distribution of the features characterizing a human activity is constructed and the activity of an individual is analyzed within the distribution to determine if it is unique. Experiments were conducted in both indoor and urban outdoor environments. The experimental results demonstrated the rationality, accuracy, and efficiency of the reasoning.
\n
\nMoreover, since many applications require to work in an unstructured and large-scale environment as well as a long-term operation, the sensing and reasoning capabilities of a single robot are limited by occlusion, sensor failure, and narrow field of view. Robots are desired to share information and perform collaborative reasoning. Therefore, if say the first two levels are mainly using sensory data to implement the algorithm, the third level is more focused on applying the algorithm to the robot platform. The third level is analytics-based multi-robot uniqueness reasoning, in which a multi-robot system is adopted as the platform. By obtaining the relative pose between robots, local observations can be transformed and transmitted among the multi-robot system. Since it is a distributed system, each robot still first performs its local uniqueness reasoning in the case of long-distance and poor communication. Meanwhile, the system performs a re-judgment based on the shared global observation and then shares the judgment results back to each robot. Experiments were carried out in various unstructured environments and contained day and night situations. The experimental results showed the rationality, accuracy, and robustness of the reasoning. This is the first time that reasoning is categorized by considering the way of obtaining and linking evidence information. This study has potential applications in a variety of situations, and it can serve as a guide for further robotic motion planning and human-robot interaction.

Automated task planning for service robots faces great challenges in handling dynamic domestic environments. Classical methods in the Artificial Intelligence (AI) area mostly focus on relatively structured environments with fewer uncertainties. This work proposes a method to combine semantic knowledge representation with classical approaches in AI to build a flexible framework that can assist service robots in task planning at the high symbolic level. A semantic knowledge ontology is constructed for representing two main types of information: environmental description and robot primitive actions. Environmental knowledge is used to handle spatial uncertainties of particular objects. Primitive actions, which the robot can execute, are constructed based on a STRIPS-style structure, allowing a feasible solution (an action sequence) for a particular task to be created. With the Care-O-Bot (CoB) robot as the platform, we explain this work with a simple, but still challenging, scenario named “get a milk box”. A recursive back-trace search algorithm is introduced for task planning, where three main components are involved, namely primitive actions, world states, and mental actions. The feasibility of the work is demonstrated with the CoB in a simulated environment.

Robot grasping and manipulation planning in unstructured and dynamic environments is heavily dependent on the attributes of manipulated objects. Although deep learning approaches have delivered exceptional performance in robot perception, human perception and reasoning are still superior in processing novel object classes. Moreover, training such models requires large datasets that are generally expensive to obtain. This work combines crowdsourcing and gamification to leverage human intelligence, enhancing the object recognition and attribute estimation aspects of robot perception. The framework employs an attribute matching system that encodes visual information into an online puzzle game, utilizing the collective intelligence of players to expand an initial attribute database and react to real-time perception conflicts. The framework is deployed and evaluated in a proof-of-concept application for enhancing object recognition in autonomous robot grasping and a model for estimating the response time is proposed. The obtained results demonstrate that given enough players, the framework can offer near real-time labeling of novel objects, based purely on visual information and human experience.

Purpose– The purpose of this paper is to design intelligent robots operating in such dynamic environments like the RoboCup Middle-Size League (MSL). In the RoboCup MSL, two teams of five autonomous robots play on an 18- × 12-m field. Equipped with sensors and on-board computers, each robot should be able to perceive the environment, make decision and control itself to play the soccer game autonomously.Design/methodology/approach– This paper presents the design of our soccer robots, participating in RoboCup MSL. The mechanical platform, electrical architecture and software framework are discussed separately. The mechanical platform is designed modularly, so easy maintainability is achieved; the electronic architecture is built on industrial standards using PC-based control technique, which results in high robustness and reliability during the intensive and fierce MSL games; the software is developed upon the open-source Robot Operating System (ROS); thus, the advantages of ROS such as modularity, portability and expansibility are inherited.Findings– Based on this paper and the open-source hardware and software, the MSL robots can be re-developed easily to participate in the RoboCup MSL. The robots can also be used in other research and education fields, especially for multi-robot systems and distributed artificial intelligence. Furthermore, the main designing ideas proposed in the paper, i.e. using a modular mechanical structure, an industrial electronic system and ROS-based software, provide a common solution for designing general intelligent robots.Originality/value– The methodology of the intelligent robot design for highly competitive and dynamic RoboCup MSL environments is proposed.

Autonomous service robots are gaining popularity as they find application in different sectors such as logistics, hospitality, medical, space exploration and military. It is widely seen that autonomous service robots help increase efficiency, reduce cost and ensure safer operations. Automating material handling operations in highly dynamic indoor environments is risky and involves use of complex algorithms. However, continuously advancing technologies and machine programming tend to overcome these challenges. The three essential functions of an autonomous robot are to identify its instantaneous location in three-dimensional space, generate the environment map and decide an optimal path for locomotion. Simultaneous Localization and Mapping (SLAM) is a widely used algorithm for navigation in unfamiliar environments. This chapter presents a cost-effective autonomous ground robot for automating material handling, transportation and other repetitive tasks in dynamically changing surroundings. The robot is programmed to map the indoor environment and localize itself in the workspace with the help of SLAM and Adaptive Monte Carlo Localization (AMCL). In addition, the robot can manoeuvre through the indoor workspace by avoiding static and dynamic obstacles. This work employs the GMapping open-source package in the Robot Operating System (ROS) for performing SLAM with the aid of a laser range finder, wheel odometry and inertial data. The GMapping algorithm uses a probabilistic approach to construct a map and estimate the robot’s location in two-dimensional space. In addition, this chapter presents an autonomous navigation feature with the help of the AMCL method for localizing the robot within an uncertain environment and making the robot move to its goal using move base actions in ROS. The path planning of the mobile robot is classified into global and local plans. The global plan is responsible for planning the path from the start point to the endpoint. The local plan is responsible for planning the actual movement of the robot by avoiding any dynamic obstacles not mapped earlier. This work uses the Dijistra algorithm for the global and the dynamic window approach (DWA) algorithm for the local path planning. The optimization algorithm minimizes the error between the global and local paths. The simulation study of the proposed algorithm is performed in a virtual environment called Gazebo in ROS, which is nearly similar to a real-time indoor workspace. Further, a robot prototype is fabricated, and its performance is evaluated in an unknown environment. The robot prototype is equipped with brushed DC motors with optical encoder feedback for wheel odometry (ODOM), Light Detection and Ranging (LiDAR), and Inertial Measurement Unit (IMU) sensors controlled using a Jetson Nano. The simulation and experimental results showcase the robot’s successful two-dimensional localization and navigation in the factory environment. The current system has the potential for indoor assistance during material transfer and logistic applications. Further upgrades to the system can make it usable for Automated Storage and Retrieval System (ASRS) applications in warehouses/factories.

Multi-Sensor Fusion (MSF) based perception systems have become the foundation supporting many industrial applications and domains, such as self-driving cars, robotic arms, and unmanned aerial vehicles. With the rapid development of data-driven artificial intelligence (AI), the perception capabilities of MSF have been comprehensively enhanced, especially in understanding complex, dynamic external environments. Similar to traditional software, AI-enabled MSF systems also require rigorous testing. However, existing testing methods are still limited to evaluating the frame-level perception capabilities (e.g., object detection in static scenes) of singlesensor systems (e.g., image-based and point cloud-based systems). Given that many safety-critical intelligent systems, such as selfdriving cars, are operated in dynamic environments where perception systems play an important role, there comes an urgent need to assess their dynamic perception capabilities in understanding and responding to external environmental variations in real-time. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">To bridge this gap, we design and implement DRIVENCE, an automated metamorphic testing tool for testing the dynamic perception capabilities of MSF-based systems. DRIVENCE accounts for various real-world physical constraints to generate realistic multi-modal test sequences by inserting multiple dynamic traffic participants into the background image and point cloud driving sequences. To diversify testing sequences, we incorporate six driving patterns derived from real-world common driving behaviors into the testing process. We conduct experiments with five SOTA MSF-based tracking systems to evaluate DRIVENCE from the perspectives of (1) generated test cases’ realism, (2) fault detection capabilities, and (3) test efficiency. The results show that DRIVENCE can generate realistic and modality-consistent test driving sequences and effectively detect various dynamic perception errors within MSF systems.

The Robot Operating System (ROS) is a set of software libraries and tools used to build robotic systems. ROS is known for a distributed and modular design. Given a model of the environment, task planning is concerned with the assembly of actions into a structure that is predicted to achieve goals. This can be done in a way that minimises costs, such as time or energy. Task planning is vital in directing the actions of a robotic agent in domains where a causal chain could lock the agent into a dead-end state. Moreover, planning can be used in less constrained domains to provide more intelligent behaviour. This paper describes the ROSP LAN framework, an architecture for embedding task planning into ROS systems. We provide a description of the architecture and a case study in autonomous robotics. Our case study involves autonomous underwater vehicles in scenarios that demonstrate the flexibility and robustness of our approach.

Common challenges in the area of robotics include issues such as sensor modeling, dynamic operating environments, and limited on-broad computational resources. To improve decision making, robots need a dependable framework to facilitate communication between different modules and the optimal action for real-world applications. The Robotics Operating System (ROS) and Reinforcement Learning (RL) are two promising approaches that help accomplish precise control, seamless integration of sensors-actuators, and exhibit learned behavior. The ROS enables seamless communication between heterogeneous components, while RL focuses on learning optimal behaviors through trial-and-error scenarios. Combining the ROS and RL offers superior decision making, improved perception, enhanced automation, and reliability. This work focuses on investigating ROS-based RL applications across various domains, aiming to enhance understanding through comprehensive discussion, analysis, and summarization. We base our evaluation on the application area, type of RL algorithm used, and degree of ROS–RL integration. Additionally, we provide summary of seminal works that define the current state of the art, along with GitHub repositories and resources for research purposes. Based on the review of successfully implemented projects, we make recommendations highlighting the advantages and limitations of RL techniques for specific applications and environments. The ultimate goal of this work is to advance the robotics field by providing a comprehensive overview of the recent important works that incorporate both the ROS and RL, thereby improving the adaptability of these emerging techniques.

This paper presents a method to potentate the multi-segmented inspection robot Kairo-II to navigate in unstructured and dynamic environment. Previous methods for motion planning for such robots come from driving scenarios in highly structured areas. The virtual tube algorithm is introduced which enables a multi-segmented robot to range in such complex environment. Precise force feedback is required. Therefore, we present a sensor system which is based on strain-gauges technology. Information extracted by this sensor enables the trajectory planning algorithm to adapt its curve. Thus, the proposed system provides and evaluates key functions for compliant motion of a multi-segmented robot within unstructured environment.

This paper presents a learning-based model predictive trajectory planning controller for automated driving in unstructured, dynamic environments with obstacle avoidance. We first address the problem of lacking prior knowledge in unstructured environments by introducing a risk map that maps the density and motion of obstacles and the road to an occupancy risk. Model predictive control is then used to integrate trajectory planning and tracking control into one framework to bridge the gap between planning and control. Meanwhile, we use Gaussian Process (GP) regression to learn the residual model uncertainty for improving the model accuracy. An objective function considering both risks within the feasible region and vehicle dynamics is carefully formulated to obtain collision-free and kinematically-feasible local trajectories. Field experiments are performed on real unstructured environments with our automated vehicle. Experimental results demonstrate the effectiveness of the proposed algorithm for successful obstacle avoidance in various complex unstructured scenarios.

With the rapid advancement of robotics and artificial intelligence, embodied intelligent agents operating in dynamic and unstructured environments have emerged as a key technological direction within digital twin systems. Nevertheless, their exploration capabilities in indoor digital-twin scenes remain limited. Dense semantic maps incur substantial maintenance costs and exhibit poor adaptability to dynamic environments. Furthermore, current navigation processes lack mechanisms for assessing and revisiting semantic relevance, leading to redundant exploration and inefficient resource utilization. Finally, environmental changes often result in semantic anchor degradation, with existing methods lacking effective incremental update and self-evolution mechanisms to maintain long-term consistency. To address these challenges, this paper proposes a multimodal, lightweight mapping, and revisitable self-evolving exploration agent (LRSA) for indoor digital-twin scenarios. The framework consists of three key modules: a multi-scale structured environment modeling process based on sparse semantic mapping, a large-model-driven memory-intelligence system that maintains an “area-object-feature” relational network, and a perceptual-memory self-evolution mechanism that marks inactive objects upon navigation failure and uses a memory large language model (MemoLLM) to generate and iteratively refine candidate search strategies for dynamic memory updating. Experiments conducted on the HM3D indoor dataset and the TurtleBot3 platform demonstrate that LRSA achieves a Success Rate (SR) of 55.2% and a Success weighted by Path Length (SPL) of 30.5%, surpassing state-of-the-art baselines such as ZSON, PSL, and PixNav. The proposed framework eliminates reliance on dense maps, enabling lightweight sparse modeling with reduced computational cost, synergistic memory-intelligence semantic reasoning, and self-adaptive evolution in dynamic environments. Its contributions enhance the robustness and efficiency of embodied agents’ open-world navigation and provide real-time, long-term semantic updating for urban digital twin systems, supporting autonomous “self-perception, self-evolution, and self-maintenance.”

With the expansion of human activities into mountainous regions, jungles, deserts, rural roads, and off-road terrains, there is an increasing demand for reliable navigation and location services in wild unstructured environments. Unlike urban environments, where established methodologies exist for constructing navigation maps, roadless unstructured environments lack comprehensive frameworks for navigation map construction. The conventional waypoint-based structures, which are well-suited to urban environments, are often ill-suited to the expansive and unstructured nature of field regions. Moreover, the absence of predefined user paths necessitates a fundamentally different approach to map construction. To address this challenge, we propose a novel method for generating navigation maps in unstructured roadless environment. New methodology leverages remote sensing as input for environmental perception, translating user traversability into geographical parameters and constructing a navigation mesh as a computational map. Compared with existing methods, the new method can adapt to large-scale unstructured environments. To validate the proposed method, reliability sampling tests and operational experiments were conducted on traversable area delineation and the navigation map generation. The experimental results indicate an 84% accuracy in traversable area analysis, with the constructed navigation map effectively supporting positioning and path planning while significantly reducing computational complexity in large-scale path planning tasks. The navigation mesh generated through this method effectively enhances the implementation of navigation and positioning services in off-road and roadless environments. The proposed method facilitates the construction of navigation maps capable of delivering navigation and localization services without requiring human presence in the target area.

To ensure a robot capable of robust task execution in unstructured environments, task planners need to have a high-level understanding of the nature of the world, reasoning for deliberate actions, and reacting to environment changes. Proposed is a practical task planning approach that seamlessly integrating deeper domain knowledge, real time perception and symbolic planning for robot operation. A higher degree of autonomy under unstructured environment will be endowed to the robot with the proposed approach.

SRS is a European research project for building robust personal assistant robots using ROS (Robotic Operating System) and Care-O-bot (COB) 3 as the initial demonstration platform. In this paper, experience gained while building the SRS system is presented. A main contribution of the paper is the SRS autonomous control framework. The framework is divided into two parts. First, it has an automatic task planner, which initialises actions on the symbolic level. The planner produces proactive robotic behaviours based on updated semantic knowledge. Second, it has an action executive for coordination actions at the level of sensing and actuation. The executive produces reactive behaviours in well-defined domains. The two parts are integrated by fuzzy logic based symbolic grounding. As a whole, they represent the framework for autonomous control. Based on the framework, several new components and user interfaces are integrated on top of COB's existing capabilities to enable robust fetch and carry in unstructured environments. The implementation strategy and results are discussed at the end of the paper.