Supervised deep actor network for imitation learning in a ground-air UAV-UGVs coordination task

The problem of cooperative obstacle avoidance by a group of unmanned ground vehicles (UGVs) and unmanned air vehicles (UAVs) during a typical search mission is addressed in this paper. The group of UAVs and UGVs are performing a search operation in the designated area. All the UAVs and UGVs are equipped with a vision sensor/LIDAR to identify the possible obstacles in the search space. Due to their operation on the ground, UGVs are more likely to encounter obstacles. The obstacle avoidance for UGV under the event of sensor failure is done with environment information from the nearest UAV. The UAV plans its trajectory according to the UGV’s expected future trajectory, leading towards the base station. The UGV replans its trajectory to avoid obstacles after obtaining the information from its nearest UAV. A simulation study is performed with 10 UAVs and five UGVs performing a search mission in 1[Formula: see text]km[Formula: see text][Formula: see text][Formula: see text]1[Formula: see text]km area. The proposed obstacle avoidance method is experimentally validated in the outdoor environment with an autonomous UAV equipped with a camera and an autonomous UGV navigating based on GPS localization and environment information from the UAV.

The following topics are dealt with: wireless sensor networks; learning (artificial intelligence); probability; autonomous aerial vehicles; radio networks; telecommunication control; radiowave propagation; wireless channels; ultra wideband communication; protocols.

This research presents a cooperation strategy for a heterogeneous group of robots that comprises two Unmanned Aerial Vehicles (UAVs) and one Unmanned Ground Vehicles (UGVs) to perform tasks in dynamic scenarios. This paper defines specific roles for the UAVs and UGV within the framework to address challenges like partially known terrains and dynamic obstacles. The UAVs are focused on aerial inspections and mapping, while UGV conducts ground-level inspections. In addition, the UAVs can return and land at the UGV base, in case of a low battery level, to perform hot swapping so as not to interrupt the inspection process. This research mainly emphasizes developing a robust Coverage Path Planning (CPP) algorithm that dynamically adapts paths to avoid collisions and ensure efficient coverage. The Wavefront algorithm was selected for the two-dimensional offline CPP. All robots must follow a predefined path generated by the offline CPP. The study also integrates advanced technologies like Neural Networks (NN) and Deep Reinforcement Learning (DRL) for adaptive path planning for both robots to enable real-time responses to dynamic obstacles. Extensive simulations using a Robot Operating System (ROS) and Gazebo platforms were conducted to validate the approach considering specific real-world situations, that is, an electrical substation, in order to demonstrate its functionality in addressing challenges in dynamic environments and advancing the field of autonomous robots.

In the research reported in this paper, we propose to overcome the unavailability of Global Positioning System (GPS) using combined information obtained from a scanning LADAR rangefinder on an Unmanned Ground Vehicle (UGV) and a LADAR mounted on an Unmanned Aerial Vehicle (UAV) that flies over the terrain being traversed. The approach to estimate and update the position of the UGV involves registering range data from the two LADARs using a combination of a feature-based registration method and a modified version of the well-known Iterative Closest Point (ICP) algorithm. Registration of range data thus guarantees an estimate of the vehicle's position even when only one of the vehicles has GPS information. Additionally, such registration over time (i.e., from sample to sample), enables position information to be maintained even when both vehicles can no longer maintain GPS contact. The approach has been validated by conducting systematic experiments on complex real-world data.

We study the problem of covering an environment using an Unmanned Aerial Vehicle (UAV) with limited battery capacity. We consider a scenario where the UAV can land on an Unmanned Ground Vehicle (UGV) and recharge the onboard battery. The UGV can also recharge the UAV while transporting the UAV to the next take-off site. We present an algorithm to solve a new variant of the area coverage problem that takes into account this symbiotic UAV and UGV system. The input consists of a set of boustrophedon cells -- rectangular strips whose width is equal to the field-of-view of the sensor on the UAV. The goal is to find a coordinated strategy for the UAV and UGV that visits and covers all cells in minimum time, while optimally finding how much to recharge, where to recharge, and when to recharge the battery. This includes flight time for visiting and covering all cells, recharging time, as well as the take-off and landing times. We show how to reduce this problem to a known NP-hard problem, Generalized Traveling Salesperson Problem (GTSP). Given an optimal GTSP solver, our approach finds the optimal coverage paths for the UAV and UGV. Our formulation models multi-rotor UAVs as well as hybrid UAVs that can operate in fixed-wing and Vertical Take-off and Landing modes. We evaluate our algorithm through simulations and proof-of-concept experiments.

This paper focuses on the real time implementation of cooperative navigation aiding for a small unmanned aerial vehicle (UAV) based on vision systems on board unmanned ground vehicles (UGV). In urban environments the signal quality of global satellite navigation systems (GNSS) often is bad or signals are even lost, so that the navigation solution of a UAV is aected. Especially in such situations the UAV’s geo-referenced position has to be known to ensure a safe guidance. A team of UGVs can overcome the problem of no GNSS position solution by the detection and tracking of the UAV in its on board images and the subsequent geo-localization. The determination of the UAV’s geo-referenced position can be achieved with several algorithms, depending on the number of UGVs. To integrate these position measurements in the UAV’s on board navigation computer the quality of the detection and geo-localization has to be estimated. All parts are implemented on embedded PC platforms. To allow real time usage on board the UGVs special algorithms have to be used for the most tasks, as image processing needs high computational power. The particular parts of this setup as well as the whole system are tested with experimental data.

ABSTRACTThe aim of this paper is to present a cooperative simultaneous localization and mapping (CSLAM) solution based on a laser telemeter. The proposed solution gives the opportunity to a group of unmanned ground vehicles (UGVs) to construct a large map and localize themselves without any human intervention. Many solutions proposed to solve this problem, most of them are based on the sequential probabilistic approach, based around Extended Kalman Filter (EKF) or the Rao-Blackwellized particle filter. In our work, we propose a new alternative to avoid these limitations, a novel alternative solution based on the smooth variable structure filter (SVSF) to solve the UGV SLAM problem is proposed. This version of SVSF-SLAM algorithm uses a boundary layer width vector and does not require covariance derivation. The new algorithm has been developed to implement the SVSF filter for CSLAM. Our contribution deals with adapting the SVSF to solve the CSLAM problem for multiple UGVs. The algorithms developed in this work were implemented using a swarm of mobile robots Pioneer 3–AT. Two mapping approaches, point-based and line-based, are implemented and validated experimentally using 2D laser telemeter sensors. Good results are obtained by the Cooperative SVSF-SLAM algorithm compared with the Cooperative EKF-SLAM.

The use of unmanned ground vehicles (mobile robots) and unmanned aerial vehicles (drones) in the civil infrastructure asset management sector: Applications, robotic platforms, sensors, and algorithms

This paper addresses the control of a formation composed of an unmanned ground vehicle (UGV) and an unmanned aerial vehicle (UAV). The formation navigates in a coordinated way, with the UAV following the UGV while keeping a relative pose with respect to the UGV as long as the UGV follows a specified path, so that the UAV can land on the UGV at any desired instant. This application is conceived in a context in which the UAV has finished a package delivery task and comes back to land over a platform used as a base for the package delivery operation. Experimental results are presented, which validate the approach adopted to accomplish such a task.

Selection of sugar beet (Beta vulgaris L.) cultivars that are resistant to Cercospora Leaf Spot (CLS) disease is critical to increase yield. Such selection requires an automatic, fast, and objective method to assess CLS severity on thousands of cultivars in the field. For this purpose, we compare the use of submillimeter scale RGB imagery acquired from an Unmanned Ground Vehicle (UGV) under active illumination and centimeter scale multispectral imagery acquired from an Unmanned Aerial Vehicle (UAV) under passive illumination. Several variables are extracted from the images (spot density and spot size for UGV, green fraction for UGV and UAV) and related to visual scores assessed by an expert. Results show that spot density and green fraction are critical variables to assess low and high CLS severities, respectively, which emphasizes the importance of having submillimeter images to early detect CLS in field conditions. Genotype sensitivity to CLS can then be accurately retrieved based on time integrals of UGV- and UAV-derived scores. While UGV shows the best estimation performance, UAV can show accurate estimates of cultivar sensitivity if the data are properly acquired. Advantages and limitations of UGV, UAV, and visual scoring methods are finally discussed in the perspective of high-throughput phenotyping.

This paper addresses a multiobjective Unmanned Aerial Vehicle (UAV) and Unmanned Ground Vehicle (UGVs) collaboration scheduling problem in which UAVs must be recharged periodically during a persistent task, such as wireless communication and making aerial panoramic VR video by UAV. The proposal is to introduce a UGV/UAV cooperative system in which UAVs should be recharged by UGVs periodically, and also, the UGVs have to visit their task points expect for the recharge points. The objective of the problem is to schedule and plan paths with the tradeoff of path length and makespan for UAVs and UGVs. The mathematical model which can be considered a combinatorial multiobjective optimization problem is presented firstly, and the solution of the problem is composed of presubsolution and postsubsolution. The multiobjective adaptive large neighborhood is hybrid with the Pareto local search (MOALP) to resolve the problem. The experimental result shows that the proposed algorithm outperforms the compared algorithms on the rendezvous planning problem for UAVs and UGVs working collaboratively in intelligent tasks.

Unmanned aerial vehicles (UAVs) have become increasingly popular in the civil field, and building inspection is one of the most promising applications. In a rural area, the UAVs are assigned to inspect the surface of buildings, and an unmanned ground vehicle (UGV) is introduced to carry the UAVs to reach the rural area and also serve as a charging station. In this paper, the mission planning problem for UAVs and UGV systems is focused on, and the goal is to realize an efficient inspection of buildings in a specific rural area. Firstly, the mission planning problem (MPP) involving UGVs and UAVs is described, and an optimization model is established with the objective of minimizing the total UAV operation time, fully considering the impact of UAV operation time and its cruising capability. Subsequently, the locations of parking points are determined based on the information about task points. Finally, a hybrid ant colony optimization-genetic algorithm (ACO-GA) is designed to solve the problem. The update mechanism of ACO is incorporated into the selection operation of GA. At the same time, the GA is improved and the defects that make GA easy to fall into local optimal and ACO have insufficient searching ability are solved. Simulation results demonstrate that the ACO-GA algorithm can obtain reasonable solutions for MPP, and the search capability of the algorithm is enhanced, presenting significant advantages over the original GA and ACO.

We consider persistent (long-horizon) surveillance over an environment by using energy-constrained unmanned aerial vehicles (UAVs), which are supported by unmanned ground vehicles (UGVs) serving as mobile charging stations. The goal is to periodically visit a set of monitoring points by the UAVs while minimizing the maximum time between the consecutive visits to any of those points. In general, the optimal planning of UAVs and UGVs in such a persistent surveillance scenario is an NP-hard combinatorial optimization problem. Furthermore, the problem also demands a solution strategy that can successfully handle obstacles, especially on the ground, that are unknown a priori in many real-life scenarios. We present a scalable and robust approximate algorithm that is based on 1) forming uniform UAV-UGV teams, 2) decomposing the environment into maximal partitions that can be covered by the UAVs in a single fuel cycle as long as the UAVs are released sufficiently close to the centers of the partitions, 3) maintaining the teams uniformly distributed over a cyclic path traversing those partitions, and 4) having the UAVs in each team cover their current partition and be transported to the next partition while being recharged by the UGV. We support our proposed algorithm with some theoretical results and simulations.

The autonomous landing of fixed-wing unmanned aerial vehicles (UAVs) on moving unmanned ground vehicles (UGVs) demands precise spatial synchronization of both vehicle systems. To achieve this goal, using individual GNSS receivers and additional sensors for both vehicles and having the vehicles exchange their respective 6D pose via radio communication is a straightforward approach for mutual localization. However, failure of the radio communication or individual sensors can critically affect the safety of the landing maneuver without suitable additional sensors or fallback systems. In this work, we present and compare two methods for estimating the relative 6D pose between a UGV and a UAV using camera systems attached to the UGV. The first approach uses a stereo grayscale camera setup with fiducial markers on the underside of the wings of the UAV. The second approach uses a single RGB camera and a convolutional neural network (CNN) trained on synthetic data, which is combined with a pose tracking filter to improve the update rate. Requirements for the pose estimation system are an average position error below 0.5 m and an orientation angle error below 10 deg, while the update rate should be at least 20 Hz. Different experiments with a demonstrator vehicle setup are presented, showing the performance and advantages of the respective approaches. During flight experiments with eleven flyovers, the CNN-based system was consistently able to detect the UAV pose, while the marker-based system had detections in seven cases. The CNN-based system had RMS position errors between 0.19 m and 1.36 m, and RMS yaw angle errors between 1.2 deg and 7.7 deg, which is an average improvement of 0.33 m and 5.6 deg, respectively, compared to the marker-based system. In future work, the pose estimation will be tested in the control loop with an automated UAV and UGV.

In this paper, we present a novel integrated vehicular system using collaborative unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) for autonomous exploration, mapping, and navigation in GPS-denied 3-dimensional (3-D) unknown environments. The system implements a novel two-layered exploration strategy to decompose the perception task into a coarse exploration layer and a fine mapping layer. The coarse exploration makes use of a UGV to carry out fast autonomous exploration and active 2.5D simultaneous localization and mapping (SLAM) to generate a coarse environment model, which serves as a navigation reference for subsequent complementary 3-D fine mapping conducted by a UAV. The two layers share a novel optimized exploration path planning and navigation framework, which provides optimal exploration paths and integrates the collaborative exploration and mapping efforts through an OctoMap-based volumetric motion planning interface. The proposed system provides an efficient pipeline of fast environment perception taking advantages of the agility of the UAVs as well as the powerful computation resource aboard UGVs, also allowing assistive local perception with augmented object information when necessary. The effectiveness of our system is verified by both simulations and experiments, which demonstrate its capability of implementing heterogeneous UAV and UGV collaborative exploration and structural reconstruction of the environments through active SLAM, providing optimized perception for navigation tasks.