Navigation Path Based Universal Mobile Manipulator Integrated Controller (NUMMIC)

Mobile manipulators have attracted a lot of interest recently. This paper discusses the development of mobile manipulators in recent years in the aspect of motion planning and coordinated control between mobile platform and manipulator. An intelligent controller based on RBF neural network is introduced for the coordinating control of the mobile manipulator, which mimics human behavior. Simulation results was given out and the method has been applied on the Hebut-II mobile robot platform. Experimental results verified the validity of the method.

The construction of a smart autonomous mobile robot system that can operate in an anonymous environment can be difficult. Building an autonomous mobile robot requires collecting data from sensors and cameras in order to recognize the surrounding environment, get knowledge of existing landmarks, and figure out its position and orientation. Many research studies have been carried out in recent years providing algorithms for achieving autonomous motion of mobile robot systems and object detection. The most common platform utilized for robot motion design is ROS (Robot Operating System) and for object detection YOLO (You Only Look Once) is a common choice. Additionally, the autonomous system needs motion planning in a systematic way to avoid obstacles by using LIDAR or other sensing methods generating the feedback signals required for control of motion around occupied spaces. One of the most effective and versatile feedback systems for mobile robot control can be constructed using visual feedback, which relies on a high-resolution camera that processes the image in real-time and the result is fed back to the control system for decision making. Visual feedback can be employed alone without sensors for feedback control of autonomous mobile robots, and it can be used for multiple tasks like path planning, obstacle avoidance, object detection, object recognition in addition to motion control. In this research, results of design and construction of an autonomous mobile rescue robot for finding victims through video data are presented. It is aimed to find victims via applying video as a data collection and feedback generation tool in the autonomous mobile robot control system. The visual feedback and YOLO v3 are used not only for finding victims but also for motion control. Our autonomous mobile robot is designed to move smoothly to the victim's direction and avoids obstacles by using ROS Global Planner. The robot is tested in multiple scenarios in different locations, various obstacle and victim numbers, and satisfactory results with good performances are obtained.

Object slip perception is essential for mobile manipulation robots to perform manipulation tasks reliably in the dynamic real-world. Traditional approaches to robot arms’ slip perception use tactile or vision sensors. However, mobile robots still have to deal with noise in their sensor signals caused by the robot’s movement in a changing environment. To solve this problem, we present an anomaly detection method that utilizes multisensory data based on a deep autoencoder model. The proposed framework integrates heterogeneous data streams collected from various robot sensors, including RGB and depth cameras, a microphone, and a force-torque sensor. The integrated data is used to train a deep autoencoder to construct latent representations of the multisensory data that indicate the normal status. Anomalies can then be identified by error scores measured by the difference between the trained encoder’s latent values and the latent values of reconstructed input data. In order to evaluate the proposed framework, we conducted an experiment that mimics an object slip by a mobile service robot operating in a real-world environment with diverse household objects and different moving patterns. The experimental results verified that the proposed framework reliably detects anomalies in object slip situations despite various object types and robot behaviors, and visual and auditory noise in the environment.

JL-II is a reconfigurable multi-robot system, which consists of three modules with mobile manipulator. To reveal the kinematic cooperation among modules, this paper analyzes the mechanism of each part, including the mobile platform and docking manipulator. The kinematic models are established to present the 3DOF docking manipulator and mobile body in unconnected and connected states. Finally, the paper obtains a suitable and definite inverse solution through presenting a motion planning of a grasping task.

Omni-directional Wheeled Mobile Robots (Omni-WMR) are holonomic system able to move sideways. They have locomotive advantage and are widely deployed in a larger range of application fields, especially in narrowly constrained service-styled manufacturing industries. To ensure smooth motion and full development of locomotive advantages, motion planning draws great attention. Thanks to the mechanism of powered caster wheel (PCW), the motion of Omni-WMR can be decoupled. The collective motion can be decomposed into dimensions of x, y and θ, then planned independently and superposed. In this way trajectory can be planned in the Cartesian space, and later on transformed to the joint space through inverse instantaneous kinematics model for actuation of the motor at each joint. Quintic-polynomial interpolation is used for the case of start to goal with only initial and final configurations known, and piecewise-cubic-spline interpolation is applied for the case of a sequence of intermediate way-points known in the configuration space as input. Here the geometric points for the robot to go through are assumed to be known as a priori without loss of generality. The approaches are implemented in python on ROS (Robot Operating System), and these modular ROS packages are tested on a physical Omni-WMR developed at our lab. Meanwhile, these ROS packages are also tested equivalently in a virtual simulation environment modelled using ROS stage simulator. The experimental results have proved that the motion planning algorithms are able to generate smooth velocity profile for the robot to achieve the goal, which have validated the effectiveness of these algorithms.

PurposeThe purpose of this study is to address the welding demands within large steel structures by presenting a global spatial motion planning algorithm for a mobile manipulator. This algorithm is based on an independently developed wall-climbing robot, which comprises a four-wheeled climbing mobile platform and a six-degree-of-freedom robotic manipulator, ensuring high mobility and operational flexibility.Design/methodology/approachA convex hull feasible domain constraint is developed for motion planning in the mobile manipulator. For extensive spatial movements, connected sequences of convex polyhedra are established between the composite robot’s initial and target states. The composite robot’s path and obstacle avoidance optimization problem are solved by constraining the control points on B-spline curves. A dynamic spatial constraint rapidlye-xploring random trees-connect (RRTC) motion planning algorithm is proposed for the manipulator, which quickly generates reference paths using spherical spatial constraints at the manipulator’s end, eliminating the need for complex nonconvex constraint modeling.FindingsExperimental results show that the proposed motion planning algorithm achieves optimal paths that meet task constraints, significantly reducing computation times in task conditions and shortening operation times in non-task conditions.Originality/valueThe algorithm proposed in this paper holds certain application value for the realization of automated welding operations within large steel structures using mobile manipulator.

In this study, we present a socially aware navigation framework for mobile service robots by integrating the systems proposed in our previous studies into a completed mobile robot navigation system, including the human detection and tracking system that is used to detect and track humans; the encoder and laser-based localization and mapping system that is used to determine the position of the robot in the environment; the approaching pose estimation system that is used to estimate the approaching pose of the robot to the human; and the social timed elastic band-based motion planning system that is used to socially navigate the mobile robots to approach the human. In addition, in the paper we describe in detail the motor control system and the design of our mobile robot platform, which is then utilized to conduct experiments in the real-world environments. We verify the feasibility and usefulness of the proposed socially aware robot navigation framework through a series of experiments in a corridor-like environment. The experimental results show that our proposed framework is able to drive the mobile robots to both avoid and approach the humans, providing the safety and comfort for the humans and socially acceptable behaviors for the mobile service robots in the dynamic social environments.

This work focuses on the operation of picking an object on a table with a mobile manipulator. We use deep reinforcement learning (DRL) to learn a positioning policy for the robot’s base by considering the reachability constraints of the arm. This work extends our first proof-of-concept with the ultimate goal of validating the method on a real robot. Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm is used to model the base controller, and is optimised using the feedback from the MoveIt! based arm planner. The idea is to encourage the base controller to position itself in areas where the arm reaches the object. Following a simulation-to-reality approach, first we create a realistic simulation of the robotic environment in Unity, and integrate it in Robot Operating System (ROS). The drivers for both the base and the arm are also implemented. The DRL-based agent is trained in simulation and, both the robot and target poses are randomised to make the learnt base controller robust to uncertainties. We propose a task-specific setup for TD3, which includes state/action spaces, reward function and neural architectures. We compare the proposed method with the baseline work and show that the combination of TD3 and the proposed setup leads to a 11% higher success rate than with the baseline, with an overall success rate of 97%. Finally, the learnt agent is deployed and validated in the real robotic system where we obtain a promising success rate of 75%.

The document proposes the development of a 3D virtual environment, oriented to the common work activities between a unicycle type mobile manipulator robot and a human operator, in collaborative tasks. This strategy is focuses on the incorporation of virtual reality (VR), in which the operator will have access to visualize in an immersive way the behavior of the mobile manipulator robot in common tasks where the human being and the mobile manipulator robot interact. For the interaction between the human operator and the mobile manipulator robot, the graphic engine Unity 3D is used, which exchanges information with the mathematical software Matlab, in order to execute the control algorithm through the use of shared memories. The Novint Falcon haptic device allows human-robot interaction, which provides the operator with force feedback on what is happening in the virtual environment generated by the Unity 3D software and the interaction it has with the mobile manipulator. The HTC Vive immersion device allows the operator to visualize the virtual environment created for the execution of the task. In this work, the design and simulation of the locomotion system of a mobile manipulator robot is carried out for manipulation and object transfer tasks together with the human operator. Finally, the simulation results that validate the proposed control strategy are presented and discussed.

The legged mobile manipulator consists of a multi-legged robot mounted with a robotic arm. The advantage of this robotic system is that it has more mobility and manipulability compared to wheeled mobile manipulator. In this work, a legged mobile manipulator is developed for education and research purpose at low cost, which is named as SiMMER (Six-legged Mobile Manipulator for Education and Research). Then, a novel control strategy is proposed for the coordinated movement of all the joints to execute a given task in Cartesian space. Robot Operating System, a Raspberry Pi and two PCA9685 16-Channel 12-bit PWM Servo Motor Driver I2C Modules are used to control the robot. Finally, the control strategy is implemented on a physical prototype. The experimental results show that all the joints are moving as desired, so that the end-effector is able to follow the given trajectory.

Mecanum wheels can apply a flexible mobility to a mobile robot, in this paper, a mobile manipulation robot is designed with combination of a Mecanum wheeled mobile platform and a 7-degrees of freedom (DoF) robot manipulator. Due to the mechanism of Omni-directional mobility, the Mecanum wheeled robot suffers with slip errors on the free rollers, these errors may be reduced with additional sensors and appropriate feedback control. However, while the mobile platform and the manipulator operate simultaneously, the displacement of the center of gravity of the whole robot increases the complexity to the motion control. To solve this problem, we analyze and model the effect of inconsistency of slippage errors caused by the unbalance of normal force on the wheels, and propose an on-line adaption slippage model for the Mecanum vehicle to generate a better control system to reduce these errors. We proof the effectiveness by using our proposed adaption model and on-line compensation control with experiments.

This paper reports on a data-driven motion planning approach for interaction-aware, socially-compliant robot navigation among human agents. Autonomous mobile robots navigating in workspaces shared with human agents require motion planning techniques providing seamless integration and smooth navigation in such. Smooth integration in mixed scenarios calls for two abilities of the robot: predicting actions of others and acting predictably for them. The former requirement requests trainable models of agent behaviors in order to accurately forecast their actions in the future, taking into account their reaction on the robot's decisions. A human-like navigation style of the robot facilitates other agents—most likely not aware of the underlying planning technique applied—to predict the robot motion vice versa, resulting in smoother joint navigation. The approach presented in this paper is based on a feature-based maximum entropy model and is able to guide a robot in an unstructured, real-world environment. The model is trained to predict joint behavior of heterogeneous groups of agents from onboard data of a mobile platform. We evaluate the benefit of interaction-aware motion planning in a realistic public setting with a total distance traveled of over 4 km. Interestingly the motion models learned from human-human interaction did not hold for robot-human interaction, due to the high attention and interest of pedestrians in testing basic braking functionality of the robot.

Mobile manipulator combines the mobility of mobile platform and the dexterity of manipulator, which has the great application potential in industry scenarios. In practical applications, efficiently planning a path in a given environment is the main objective, but the high degrees of redundancy, which is caused by the coupling of platform and manipulator, will make the planning problem more challenging. Therefore, the motion planning based on the RRT-GoalBias algorithm and path smoothing is proposed in this paper, where the kinematics of mobile manipulator is modeled, and the constraints of configuration and obstacle avoidance are analyzed. Subsequently, the RRT-GoalBias algorithm is applied and modified to generate an initial path for mobile manipulator in a given site, and a novel post-processing algorithm is proposed to smooth and optimize the initial path. Comparative Simulations are carried out with MATLAB and V-REP to verify the effectiveness of the proposed path planning algorithm for mobile manipulator under the given environment.

A mobile manipulator is a mobile platform with a mounted serial manipulator. A Mobile Manipulator is a system subject to its kinematic restrictions and the degrees of freedom of the manipulator arm mounted on it. These systems combine the advantages of mobile platforms and robotic arms, and reduce their disadvantages. For example, a mobile manipulator has larger working space when it has a mobile platform, as it offers more functionality during operation. For the previously mentioned in this work will be shown the implementation in the robotic simulation software Webots, a mobile manipulator that allows to determine its operation in a virtual environment

This paper introduces a novel and general real-time adaptive motion planning (RAMP) approach suitable for planning trajectories of high-DOF or redundant robots, such as mobile manipulators, in dynamic environments with moving obstacles of unknown trajectories. The RAMP approach enables simultaneous path and trajectory planning and simultaneous planning and execution of motion in real time. It facilitates real-time optimization of trajectories under various optimization criteria, such as minimizing energy and time and maximizing manipulability. It also accommodates partially specified task goals of robots easily. The approach exploits redundancy in redundant robots (such as locomotion versus manipulation in a mobile manipulator) through loose coupling of robot configuration variables to best achieve obstacle avoidance and optimization objectives. The RAMP approach has been implemented and tested in simulation over a diverse set of task environments, including environments with multiple mobile manipulators. The results (and also the accompanying video) show that the RAMP planner, with its high efficiency and flexibility, not only handles a single mobile manipulator well in dynamic environments with various obstacles of unknown motions in addition to static obstacles, but can also readily and effectively plan motions for each mobile manipulator in an environment shared by multiple mobile manipulators and other moving obstacles.