Compliant motion of a multi-segmented inspection robot

Advanced robots are desired to improve their reasoning ability in increasingly practical situations to quickly respond to urgent challenges such as search and rescue, public security, and autonomous driving. Advances in sensor technology and intelligent algorithms are accelerating the development of autonomous systems, allowing them to interact with dynamic environments more independently and intelligently, completing relatively complicated tasks. Nevertheless, the safety and security problems are yet to be comprehensively addressed. Currently, many efforts have been made to solve this problem by improving the unimodal perception ability of intelligent systems. Methods using surveillance cameras to collect large amounts of visual data and process them using deep networks. Various types of range scanner can sense and reconstruct the surrounding environment through precise distance measurements. However, in the above cases, there may be a lack of training data, fixed cameras cannot adequately and dynamically model the surroundings, unimodal analysis is hard to capture both appearance and depth information. Therefore, higher levels of perception and reasoning abilities are required to handle unexpected conditions and scenarios efficiently. In such cases, mobile robots should be fully aware of the dynamic environment, prioritize, and continuously analyze critical situations, share information, and collaboration under limited computing power and resources. Furthermore, considering the robots are operating alongside humans, reliable and interpretable robot behavior is highly preferred for more natural interactions.
\n
\nThis study aims to develop reasoning strategies for autonomous robots to improve robotic intelligence under different assumptions and conditions. The framework allows one or several robots to address safety or security problems in a targeted manner in partially known environments, where time, dynamics, and interactions play a significant role. The objective is to identify the unique individual whose activity, in space and time, appear different from the other people in the scene. The activities considered not only the motion pattern, but also the object associated with this person. Unlike traditional abnormal detection, the uniqueness reasoning in this work does not require a predefined expected pattern. A modular analysis is also applied to decompose the entire reasoning process into submodules such as perception, analysis, and inference, to achieve algorithmic flexibility and interpretability. Probabilistic modeling is used to associate spatial relationships between semantic labels and their temporal changes. The relationships are described in either instant spatial spaces (semantic-interaction feature) or spatiotemporal spaces (spatiotemporal feature), and it provides essential information for robot navigation, situation awareness, and task planning. Unlike existing research, this is the first time that such a mission is implemented on mobile robot platforms by using the probabilistic fusion of multimodal information.
\n
\nConsidering how evidence information is obtained and linked in the process of reasoning, we study the problem of uniqueness reasoning at three levels. The first level is called knowledge-based single robot uniqueness reasoning and runs on a single robot platform. At this level, we focus on a knowledge-based approach, where knowledge is provided in terms of how motion and an object associated with a person can be used to infer his/her uniqueness, as determined by human expertise. Because such a method has advantages in dealing with structured missions and with limited training data. Qualitative rules are applied to extract human activities from observations, and then the uniqueness of each person is judged by associating the prior knowledge with the performed activities. Experiments were conducted in indoor environments with various degrees of occlusion, and the experimental results showed the rationality and accuracy of the reasoning.
\n
\nHowever, though the knowledge-based method can successfully address specific tasks, it lacks flexibility and generality to adapt to different situations. The assumption of prior knowledge should be replaced by general analytical modeling. Thus, the second level is analytics-based single robot uniqueness reasoning. For one thing, it applies quantitative analysis to model human activities from observations rather than using qualitative rules. For another, instead of leveraging prior knowledge, the distribution of the features characterizing a human activity is constructed and the activity of an individual is analyzed within the distribution to determine if it is unique. Experiments were conducted in both indoor and urban outdoor environments. The experimental results demonstrated the rationality, accuracy, and efficiency of the reasoning.
\n
\nMoreover, since many applications require to work in an unstructured and large-scale environment as well as a long-term operation, the sensing and reasoning capabilities of a single robot are limited by occlusion, sensor failure, and narrow field of view. Robots are desired to share information and perform collaborative reasoning. Therefore, if say the first two levels are mainly using sensory data to implement the algorithm, the third level is more focused on applying the algorithm to the robot platform. The third level is analytics-based multi-robot uniqueness reasoning, in which a multi-robot system is adopted as the platform. By obtaining the relative pose between robots, local observations can be transformed and transmitted among the multi-robot system. Since it is a distributed system, each robot still first performs its local uniqueness reasoning in the case of long-distance and poor communication. Meanwhile, the system performs a re-judgment based on the shared global observation and then shares the judgment results back to each robot. Experiments were carried out in various unstructured environments and contained day and night situations. The experimental results showed the rationality, accuracy, and robustness of the reasoning. This is the first time that reasoning is categorized by considering the way of obtaining and linking evidence information. This study has potential applications in a variety of situations, and it can serve as a guide for further robotic motion planning and human-robot interaction.

The autonomous vehicle is an automated system equipped with features like environment perception, decision-making, motion planning, and control and execution technology. Navigating in an unstructured and complex environment is a huge challenge for autonomous vehicles, due to the irregular shape of road, the requirement of real-time planning, and the nonholonomic constraints of vehicle. This paper presents a motion planning method, based on the Radial Basis Function (RBF) neural network, to guide the autonomous vehicle in unstructured environments. The proposed algorithm extracts the drivable region from the perception grid map based on the global path, which is available in the road network. The sample points are randomly selected in the drivable region, and a gradient descent method is used to train the RBF network. The parameters of the motion-planning algorithm are verified through the simulation and experiment. It is observed that the proposed approach produces a flexible, smooth, and safe path that can fit any road shape. The method is implemented on autonomous vehicle and verified against many outdoor scenes; furthermore, a comparison of proposed method with the existing well-known Rapidly-exploring Random Tree (RRT) method is presented. The experimental results show that the proposed method is highly effective in planning the vehicle path and offers better motion quality.

Motion planning of robots based on the Markov decision process (MDP) is now one of the most important topics of robotics research. In motion planning of robots by the MDP, an efficient way to divide robot motion is essential because discrete expressions of states and transitions are used in the calculation. In preparing the discrete expressions of robot motion, it is important that the motion follows the MDP, second, that the time and the amount of memory for the planning calculation are reduced, and that the maneuverability and smoothness of the robot motion are not lost. This paper proposes a new discrete expression of robot motion that can meet these requirements considering the effect of robot dynamics to robot kinematics. The method uses a continuous-time continuous-variable neuron model, which has the stimulation and adaptation properties. The performance of the proposed method was examined by simulation of the motion planning of an under actuated underwater robot and under actuated manipulator with a free joint. The results show the high reliability and flexibility of the method.

We present a novel algorithm for motion planning of multiple robots amongst dynamic obstacles. Our approach is based on a new roadmap representation that uses deformable links and dynamically retracts to capture the connectivity of the free space. We use Newtonian physics and Hooke's Law to update the position of the milestones and deform the links in response to the motion of other robots and the obstacles. Based on this roadmap representation, we describe our planning algorithms that can compute collision-free paths for tens of robots in complex dynamic environments.

Dynamics and motion planning of a wheel-legged robot with two manipulator arms is discussed in this paper. This robot can move on flat surfaces fast, while it has the capability of moving over rough terrains. The two locomotion subsystems, i.e. wheeled mechanism and legged mechanism, will be discussed separately. Wheeled locomotion mechanism includes three wheels, two of them are in the back as active wheels with a differential driver. Legged locomotion mechanism includes two legs each with 4 revolute joints, i.e. 4 degrees of freedom (DOF). Two manipulators are installed on the main body to perform variant tasks, each of them is a PRR 3 DOF arm. Using direct path method (DPM) the robot kinematics is developed, and using Lagrangian approach the equations of motion are derived. A new method, called return path method (RPM), is proposed to add the constraints of the system to the kinetics equations without introducing additional variables. To eliminate the terms corresponding to the constraints, natural orthogonal complement method is used. Finally, motion planning of the robot is investigated to develop a stable gait planning procedure.

This paper presents a learning-based model predictive trajectory planning controller for automated driving in unstructured, dynamic environments with obstacle avoidance. We first address the problem of lacking prior knowledge in unstructured environments by introducing a risk map that maps the density and motion of obstacles and the road to an occupancy risk. Model predictive control is then used to integrate trajectory planning and tracking control into one framework to bridge the gap between planning and control. Meanwhile, we use Gaussian Process (GP) regression to learn the residual model uncertainty for improving the model accuracy. An objective function considering both risks within the feasible region and vehicle dynamics is carefully formulated to obtain collision-free and kinematically-feasible local trajectories. Field experiments are performed on real unstructured environments with our automated vehicle. Experimental results demonstrate the effectiveness of the proposed algorithm for successful obstacle avoidance in various complex unstructured scenarios.

There are many sources of uncertainty facing the fuzzy logic controller (FLC) for autonomous mobile robots navigating in changing and dynamic unstructured environments. The traditional type-1 FLC using precise type-1 fuzzy sets cannot fully handle such uncertainties. A type-2 fuzzy logic controller (FLC) using type-2 fuzzy sets can handle such uncertainties to produce a better performance. In this paper, we present the type-2 FLC and its novel application for the real time control of mobile robots. We used the type-2 FLC to implement different robotic behaviours on different robotic platforms in indoor and outdoor unstructured and challenging environments. The type-2 FLCs dealt with the uncertainties facing mobile robots in unstructured environments and resulted in a very good performance that outperformed the type-1 FLCs whilst using smaller rule bases.

Recently, stereovision has been used in robotics for obtaining information for real-time mapping. Stereovision makes use of two cameras to solve the problems of a lack of discriminative image features and repetitive patterns and to extend the field of view. Stereovision algorithms and techniques are working to create a map to know its surrounding environment. Despite advances in the field of autonomous robotics, several challenges still exist. The majority of these challenges exist owing to the dynamic nature of the navigational environment and the lack of complete knowledge of new, unknown, and complex environments. In this paper, we create a novel, intelligent, and dynamic mapping system for autonomous navigation using stereovision. The proposed system is called intelligent multi-sensor multi-baseline mapping system (I3MS). The main contribution of this paper is the dynamic tuning of the stereovision parameters to address the dynamics of the environment (various light conditions, movement obstacles, and various times of day). The I3MS works in unstructured, unknown, dynamic, and indoor environments. It is based on an in-depth study of the effects of light and distance from obstacles on the accuracy of the 3-D map. It uses a fuzzy-logic system to tune the stereovision parameters to handle dynamic environments. The obtained experimental results illustrate the robustness and accuracy of the proposed system.

In this paper, we present an automatic motion planner for agricultural robots that allows us to set up a robot to follow rows without having to explicitly enter waypoints. In most cases, when robots are used to cover large agricultural areas, they will need waypoints as inputs, either as premeasured coordinates in an outdoor environment, or as positions in a map in an indoor environment. This can be a tedious process as several hundreds or even thousands of waypoints will be needed for large farms. In particular, we find that in unstructured environments such as the ones found on farms, the need for waypoints increases. In this paper, we present an approach that enables robots to safely traverse not only between straight rows but also through curved rows without the need for any predetermined waypoints. Most types of infrastructure found in agriculture, such as polytunnels, are built on uneven terrain, thus containing a mix of straight and curved plant rows, for which traditional methods of row following will fail. Different from traditional approaches of row following that only consider straight‐line‐of‐sight rows, our approach identifies the rows on each side with the goal of staying in the middle of the rows, even if the rows are not straight. Waypoints are only needed on the very extreme of the rows, and these will be automatically generated by the system. With our approach, the robot can just be placed in the corner of the field and will then determine the trajectory without further input from the user. We thus obtain an approach that can reduce the installation time from potentially hours to just a matter of minutes. The final autonomous system is low cost and efficient for various tasks that requires moving between plant rows inside a polytunnel. Several experiments were performed and the robot demonstrates 1.4% position drift over 21 m of navigation path.

To address the low efficiency and poor robustness of robot plug-in charging in unstructured environments, this paper proposes a method for plug-in autonomous charging via a wheeled mobile manipulator in unstructured environments. First, a two-stage paradigm target recognition and positioning algorithm named You Only Look Once version 8 (YOLOv8) and Point Pair Features (PPF) pose estimation is constructed based on YOLOv8 and PPF. By integrating 2D image detection and 3D point cloud processing, it can quickly and accurately recognize and locate charging guns in complex environments. Then, according to the application characteristics of unstructured scenarios, an integrated system of the wheeled mobile manipulator is designed and verified through simulation. Finally, based on the custom-developed wheeled mobile manipulator platform, plug-in autonomous charging experiments are carried out in unstructured scenarios. The results show that in three typical experimental scenarios, namely normal illumination without occlusion, dim light with slight occlusion, and dark with large-area occlusion, the method proposed in this paper can achieve rapid recognition and positioning of the charging gun, meeting the precision requirements that the angle error of the charging gun pose estimation does not exceed $$10^{\circ }$$ and that the distance error does not exceed 15mm; this demonstrates the effectiveness and robustness of the designed plug-in autonomous charging method for the wheeled mobile manipulator.

When a robot manipulator performs tasks in a dynamic unstructured environment, it not only should account for where and how it should move, but also ensure that the robot can effectively avoid collisions with static and moving obstacles. Planning in absolutely unstructured and dynamic environment is much more challenging, up-to-date there is no universal solution to this problem. In this paper, we believed the concept that for motion planning in real dynamic environments, usually there is only a part of a path invalidated by obstacles, and only this part needs replanning. Thus, in this paper a practical strategy was presented for motion planning in dynamic environments. First, a variant dynamic Rapidly-exploring Random Tree (RRT) planner was designed to combine global planning with local replanning. Then, the proposed dynamic RRT algorithm was implemented under Robotic Operation System (ROS) framework and applied to a 7-degree-of-freedom manipulator. Experimental results have shown that the proposed dynamic RRT planner can plan and execute movement of robot manipulator between a starting and target position while preserving task constraints and reacting to environment changes in real time.

In order to improve the operation efficiency and impact resistance of the robotic arm in unstructured environments such as orchards, this paper proposes an online trajectory planning method for multi-index comprehensive optimization. First, the kinematics and path planning model of the robotic arm is established; then, the joint motion trajectory is constructed using seven polynomials, and a multi-objective comprehensive trajectory optimization model is established using the nonlinear planning method based on kinematic constraints. Secondly, after obtaining the mathematical relationship of each optimization target through analysis, reconstruct the model, simplify the kinematic constraints by using the extreme value principle, and then define the target function by the weighting coefficient method to achieve the comprehensive optimization of the working time, energy consumption and acceleration of the robotic arm motion trajectory. Finally, use the algorithm to optimize the target function to obtain the comprehensive optimal motion trajectory, and take the citrus picking operation as an example for simulation and experimental testing. The simulation results show that the average calculation time of the algorithm is 61 ms, the time index of the obtained comprehensive optimal trajectory is 15.78 s, the energy consumption index is 188.86 °/s², the acceleration index is 405.08 °/s³, and the speed, acceleration and acceleration of each joint are all satisfied. Kinematic constraints. The experimental results show that the citrus picking robot arm runs smoothly and can meet the picking requirements. The average picking time for a single fruit is 16.4 s. This research provides a new method for the online trajectory planning of picking operations with high computational efficiency and can achieve multi-objective comprehensive optimization.

In recent years, the perceptual capabilities of robots have been significantly enhanced. However, the task execution of the robots still lacks adaptive capabilities in unstructured and dynamic environments. In this paper, we propose an ontology based autonomous robot task processing framework (ARTProF), to improve the robot's adaptability within unstructured and dynamic environments. ARTProF unifies ontological knowledge representation, reasoning, and autonomous task planning and execution into a single framework. The interface between the knowledge base and neural network-based object detection is first introduced in ARTProF to improve the robot's perception capabilities. A knowledge-driven manipulation operator based on Robot Operating System (ROS) is then designed to facilitate the interaction between the knowledge base and the robot's primitive actions. Additionally, an operation similarity model is proposed to endow the robot with the ability to generalize to novel objects. Finally, a dynamic task planning algorithm, leveraging ontological knowledge, equips the robot with adaptability to execute tasks in unstructured and dynamic environments. Experimental results on real-world scenarios and simulations demonstrate the effectiveness and efficiency of the proposed ARTProF framework. In future work, we will focus on refining the ARTProF framework by integrating neurosymbolic inference.

The potential field method is widely used for autonomous mobile robot path planning due to its elegant mathematical analysis and simplicity. However, most researches have been focused on solving the motion planning problem in a stationary environment where both targets and obstacles are stationary. This paper proposes a new potential field method for motion planning of mobile robots in a dynamic environment where the target and the obstacles are moving. Firstly, the new potential function and the corresponding virtual force are defined. Then, the problem of local minima is discussed. Finally, extensive computer simulations and hardware experiments are carried out to demonstrate the effectiveness of the dynamic motion planning schemes based on the new potential field method.

To ensure the safe production of mines, the intelligent trend of underground mining operations is gradually advancing. However, the operational environment of subterranean mining is intricate, making the conventional path-planning algorithm used by mining inspection robots frequently inadequate for real requirements. To safeguard the mining inspection robot, targeting the problem of low search efficiency and path redundancy in the path planning of the existing rapidly exploring random tree (RRT) algorithm in the narrow and complex unstructured environment, a path-planning algorithm combining improved RRT and a probabilistic road map (PRM) is proposed. Initially, the target area is efficiently searched according to the fan-shaped goal orientation strategy and the adaptive step size expansion strategy. Subsequently, the PRM algorithm and the improved RRT algorithm are combined to reduce the redundant points of the planning path. Ultimately, considering the kinematics of the vehicle, the path is optimized by the third-order Bessel curve. The experimental simulation results show that the proposed path-planning algorithm has a higher success rate, smoother path, and shorter path length than other algorithms in complex underground mining environments, which proves the effectiveness of the proposed algorithm.