Pursuit and evasion in a recursive nested behavioral control structure for unmanned aerial vehicles

The proliferation of unmanned aerial vehicles (UAVs) has driven the emergence of urban air mobility (UAM) as a solution to alleviate the increasing demand for UAVs applications in various domains. The concept of UAM envisions a future where UAVs play a pivotal role in short-distance transportation, delivery services, surveillance, emergency rescue, and more. However, the rapid growth of UAVs poses significant challenges arising from the need to ensure the safe coexistence of UAVs with other manned/unmanned aerial vehicles. To ensure the safe integration of UAVs into low-altitude urban airspace and support high-density on-demand mobility, this research focuses on handling the safety-centric challenges related to UAV collision risk assessment, obstacle avoidance, and conflict resolution in complex urban environments.
\nOne of the main challenges is the intrusion of UAVs into airport runways. Airports serve as critical hubs for civil aircraft takeoff and landing operations. However, the increasing presence of UAVs has raised concerns about intruding into airports, leading to flight delays and temporary runway closures. To quantify the potential interference of UAVs on civil aircraft, a collision-course based trajectory planning model is proposed to evaluate the collision risk between manned civil aircraft and intruding UAVs. In this risk assessment model, the trajectory of the intruding UAV is formulated using a collision-course concept, taking into account the limited trajectory information available about intruding UAVs. The planned trajectory of the UAVs is then used to predict its relative position concerning civil aircraft. To account for the possible disturbances of the relative distance, a stochastic position prediction model based on Brownian motion is developed. The collision risk is assessed based on this stochastic kinematic model, providing a quantitative measure of the potential collisions. The proposed model is validated through extensive simulations involving different UAV initial positions, position updates, and different sizes of collision zone tests, as well as comparative tests with the Monte-Carlo algorithm, demonstrating its robustness in assessing the collision risk of the airport restricted area. To facilitate the application, the model is finally used to develop a platform to generate runway heatmaps, accounting for airport runway risk distributions. 
\nAnother safety-centric challenge is ensuring autonomous obstacle avoidance and conflict resolution of UAVs. To enable UAV autonomous safe operations in uncontrolled urban low-altitude airspace, a comprehensive collision-free path planning framework is developed to address the avoidance of both static obstacles and dynamic threats. Firstly, to handle static obstacles, a novel algorithm of 3D voxel jump point research is introduced to generate a global reference path for UAV navigation to avoid static obstacles, particularly for a free flight at different altitudes. Additionally, to optimize the reference path for specific mission requirements, techniques such as trajectory de-diagonalization, reconstruction, and smoothness are proposed to reduce the risk of static collisions, shorten the fly range, and enable UAVs to negotiate turns more efficiently. Secondly, real-time conflict resolution actions based on the Markov decision process are developed to avoid local dynamic threats encountered along the optimized desired path. This allows the host UAV to dynamically adjust its course to a resolution trajectory while maintaining its planned speed without hovering to avoid the potential collision with the approaching intruders. By avoiding near mid-air collisions (NMAC) and preventing the UAV from hovering, the proposed method ensures the timely completion of UAV missions. Simulation results demonstrate that the developed method successfully achieves UAV autonomous global path planning and local real-time conflict resolution in complex urban low-altitude environments, thereby avoiding both static obstacles and dynamic threats. 
\nMoreover, to bridge the research gap of UAV hard conflict resolution in mitigating NMAC risks arising from local dynamic threats, a novel situational NMAC risk assessment approach is developed by integrating the conditional random field (CRF) algorithm with safety metrics and conflict resolution performance to facilitate UAV tiered aerial passage operations. To improve UAV operating environments in urban airspace, a concept of tiered aerial passages is first designed to provide a practical urban airspace planning scheme that addresses UAV heterogeneity and optimizes the utilization of urban airspace. For UAV tiered aerial passage operations, a situational NMAC risk assessment module coupled with a conflict resolution decision-making module is formulated, enabling the host UAV to evaluate and respond to real-time conflicts, ensuring avoidance of potential collisions. To assess NMAC risks, a general risk assessment model is proposed based on the CRF algorithm, considering UAV safety metrics and resolution performance. Then the overtaking, approaching head-on, and converging UAV encounters are analyzed, and situational risk potential scores are created using the Gaussian kernel function in the CRF algorithm, considering advisory, caution, and warning risk alert levels. To resolve conflict, acceptable, tolerable, and unacceptable scenarios are proposed to enhance the flexibility and personalization of the resolution decision-making. The proposed scheme is validated by simulating geometric encounter situations and comparative tests with the previous hard resolution technique. The results indicate that the proposed method outperforms UAV NMAC risk assessment and conflict resolution. Even in the most dangerous unacceptable resolution decision-making scenarios across three conflict encounters, the proposed algorithm achieves NMAC rates of 0.036, 0.070, and 0.043, which are lower than the corresponding NMAC rates associated with the hard resolution approach. Together with the acceptable and tolerable scenarios, the proposed negotiation resolution effectively tailors different risk requirements to accommodate diverse UAV missions. 
\nWith a primary focus on ensuring the safety of the UAM architecture, this research endeavors to assess UAV NMAC risks and efficiently manage potential conflicts. The proposed work are thoroughly validated to demonstrate their effectiveness in real-world scenarios. As a result, this study significantly contributes to the establishment of a robust UAM architecture, facilitating the realization of safe and reliable UAM.

U NMANNED Aerial Vehicles (UAVs) hold good promise for autonomously carrying out complex civilian and military operations. However, many of these missions require them to fly at low altitudes, making them vulnerable to collision with both stationary as well as moving obstacles. Hence, it is vital that UAVs are equipped with autonomous capability to sense and avoid collisions, especially for the pop-up threats. When such a threat is sensed and a collision is predictedwithin a short time ahead, theUAV should be able to react and maneuver away quickly so that the collision is avoided. An algorithmwhich can assure such amaneuver is called a “reactive collision avoidance algorithm.” Since the available reaction time in such a scenario is usually is small and UAVs are usually limited by computational resources, such an algorithm should also be computationally very efficient (it should preferably be noniterative). It is also required that whilemaneuvering away, it should notmaneuver toomuch away from the obstacle either. This is both to avoid collision from other nearby obstacles and not to compromise on the overall mission objective. There are various attempts in the literature to develop algorithms for collision avoidance purpose, many of which are inspired from global path planning algorithms. The artificial potential field method is such an approach where the motion of the vehicle is guided under the influence of a potential field. The potential field (which is essentially a cost function) is designed in such a way that obstacles have repulsive fields while the destination has an attractive field. The safe path of the UAV is then found by optimizing the carefully selected cost function. To tune this basic philosophy for reactive collision avoidance, a model predictive control-based algorithm has been proposed in the literature. This algorithm essentially assures path following under safe conditions (i.e. if no collision is predicted in the near future) and invokes the potential field function when new collisions are sensed. However, in potential field based techniques the associated optimization process is typically done in an iterative manner. Because of this they are usually computationally intensive and hence are not suitable for reactive collision avoidance of airborne UAVs in general. A promising algorithm in collision avoidance and global path planning is the philosophy of rapidly-exploring random tree (RRT) [1], which has also been used for reactive collision avoidance. However, there are many concerns about the RRT approach, which can largely be attributed to the random nature of the algorithm. For example, the path predicted by RRT is usually a sting of connected straight lines that does not reflect the path followed by a vehicle with nonholonomic constraints. More important, it is a probabilistic approach and hence there is no guarantee of finding a feasible path within a limited finite time. Other graph search algorithms such as best-first search are also implemented for reactive collision avoidance [2]. However, this is not systematic approach and could result in the algorithm searching far too many nodes under some conditions. Moreover, precomputing motion primitives and saving them in a lookup table is infeasible for UAVs, which are usually resource-limited. An interesting perspective to collision avoidance problem is the minimum effort guidance [3], where an optimal control-based approach has been proposed after applying the collision cone philosophy to detect collisions. This method is computationally nonintensive as a closed form solution has been proposed. Even though this is an interesting idea, by minimizing the lateral acceleration, perhaps it imposes unwanted extra constraint on the problem formulation as reactive collision avoidance problems do not necessarily have to be carried out with minimum lateral acceleration. More important, one can observe that this formulation only assures position guarantee and no constraint is imposed on the velocity vector. Hence, even though it guides the vehicle to a carefully selected target point on the safety boundary (we call it the “aiming point”), it causes the vehicle to maneuver until this point. This can be risky as the vehicle may enter the safety ball before reaching the aiming point. Even though the collision cone based aiming point philosophy is a very good idea, the authors of this Note strongly believe that instead of only position guarantee, rather the velocity vector should be aligned towards the aiming point as soon as a collision is detected (which will automatically lead to position guarantee as well). Towards this objective, two new nonlinear guidance laws are proposed in this Note, which are named as nonlinear geometric guidance (NGG) and differential geometric guidance (DGG). These guidance laws are inspired by the philosophy of “aiming point guidance” (APG) [4], which has been proposed in missile guidance literature. It turns out that the APG is a simplified case of the NGG where the associated since function is replaced by its linear approximation (hence, for a systematic discussion, it is renamed as linear geometric guidance (LGG) in this Note). Both of the guidance algorithms proposed in this Note quickly align the velocity vector of the UAValong the aiming point within a part of the available time-togo, which ensures quick reaction and hence safety of the vehicle. The main feature of this philosophy is that they effect high maneuvering at the beginning, causing the velocity vector of the UAV to align with the aiming point direction quickly and then settling along it. Therefore there is no need to maneuver all the way until the aiming point is reached and hence the chance of the UAV entering into the safety ball is minimized. Using the point of closest approach (PCA) [5], the proposed NGG andDGGalgorithms have also been extended for collision avoidance with moving obstacles in both cooperative as well as ignorant scenarios. Mathematical correlations between the guidance laws have also been established, which show that the NGG and DGG are exactly correlated to each other with appropriate gain selections, while the LGG is an approximation of DGG. A “sphere-tracking algorithm” is also proposed in this Note where the UAV is guided to track the surface of the safety spherewhenever a brief violation of the safety boundary occurs after reaching the aiming point because of the location of the next aiming point (which may include the target in Presented as Paper 2010-8315 at the AIAA Guidance, Navigation and Control, Toronto, 2–5August 2010; received 26May 2010; revision received 4 October 2010; accepted for publication 6 October 2010. Copyright © 2010 by Radhakant Padhi. Published by the American Institute of Aeronautics and Astronautics, Inc., with permission. Copies of this paper may be made for personal or internal use, on condition that the copier pay the $10.00 per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923; include the code 0731-5090/11 and $10.00 in correspondence with the CCC. ∗Former Project Assistant, Department of Aerospace Engineering; anushamujumdar87@gmail.com. Associate Professor, Department of Aerospace Engineering; padhi@ aero.iisc.ernet.in. JOURNAL OF GUIDANCE, CONTROL, AND DYNAMICS Vol. 34, No. 1, January–February 2011

Autonomous Landing of Fixed Wing Unmanned Aerial Vehicle with Reactive Collision Avoidance

Unmanned Aerial Vehicle (UAV) has received wide interest in recent years because of their promising potentials for several applications that include border surveillance, intrusion, detection, emergent item delivery, wildlife monitoring, rescue operations etc. The growing number of entrances of drones into airspace has emerged a significant interest in the collision and obstacle avoidance in autonomous operation. One of major challenges for safer deployment of UAVs is collision avoidance in order to make UAVs autonomous and self-sustainable. In today’s technological era, it is possible for UAVs to avoid obstacles and prevent collisions while navigation by implementing cutting edge computer vision algorithms and visual tracking systems. But it is challenging due to the limited payload in UAV that only permits vision based sensors such as camera, inertial measurement unit such for control and obstacle detection. This paper aims to summarize existing solutions of collision avoidance strategies used in the navigation of UAV. Collision avoidance strategies based on geometry, optimization, sense and avoid, vision-based and force field based methods are investigated in this study.

The number of Unmanned Aerial Vehicles (UAVs) flying around is increasing year after year due to their popularity. This new scenario considerably increases the risk of collision between UAVs, which makes the development of protocols to avoid this issue an emerging topic. This work focuses on the development of a collision avoidance protocol between UAVs performing planned missions. Our proposal is based on the periodic submission of state information, and future position estimates of the UAV, allowing the detection of collision risks. Collision avoidance is achieved by forcing UAVs to stop when they are flying critically near each other, and establishing the priority of the UAV passing by the critical area first; this decision is based on a per-UAV unique identifier. We prove that our solution is robust enough to avoid collisions in all the situations tested; in addition, the overhead introduced by our protocol in terms of additional UAV flight time is low.

Autonomous formation flight has been a fast growing research area during the last decade. Unmanned aerial vehicles (UAVs) can achieve a wide range of benefits from formation flight, such as fuel saving and aerial refueling. In this paper, the development of a small UAV system, called Swallow UAV system, as testbed of formation flight, is presented. Swallow UAV system includes Swallow UAV itself and a Hardware-In-the-Loop Simulation Environment (HILSE). Swallow UAV is a fixed wing, pusher-type, electric motor driven aircraft with 1.4 meter wing span. Its avionics system includes an onboard computer and a sensor system with GPS, IMU, air data sensor, and motor monitoring unit to provide position, velocity, inertial data, and system health. Communications between UAVs and the ground control station (GCS) are conducted by wireless modem. A fuzzy navigation and control system (FNCS) is developed for the waypoint navigation and attitude control of the aircraft. GPS velocity based pseudo attitude and fuzzy logic control are applied to FNCS for attitude estimation and fast implementation of controllers. HILSE is built to verify FNCS with onboard computer on the ground. X-Plane flight simulator is applied to simulate aircraft dynamics and output sensor data. GCS software, built with data distributor, transmits these data to the onboard computer and receive control signal from it so the onboard computer can control the aircraft in X-Plane. HILSE can perform pre-flight test for software debug, which improves safety and efficiency of flight test. The design of Swallow UAV is verified by flight test and the result shows that it has sufficient flight performance with waypoint navigation and attitude control capability to be used as a formation flight testbed.

This paper is focused on dynamic modeling and control system design as well as vision based collision avoidance for multi-rotor unmanned aerial vehicles (UAVs). Multi-rotor UAVs are defined as rotary-winged UAVs with multiple rotors. These multi-rotor UAVs can be utilized in various military situations such as surveillance and reconnaissance. They can also be used for obtaining visual information from steep terrains or disaster sites. In this paper, a quad-rotor model is introduced as well as its control system, which is designed based on a proportional-integral-derivative controller and vision-based collision avoidance control system. Additionally, in order for a UAV to navigate safely in areas such as buildings and offices with a number of obstacles, there must be a collision avoidance algorithm installed in the UAV's hardware, which should include the detection of obstacles, avoidance maneuvering, etc. In this paper, the optical flow method, one of the vision-based collision avoidance techniques, is introduced, and multi-rotor UAV's collision avoidance simulations are described in various virtual environments in order to demonstrate its avoidance performance.

Unmanned aerial vehicles (UAV) have been widely used in many fields. UAV formation flight can complete more assignments than single UAV. Collision avoidance is an important part of UAV formation flight. In this paper, we put forward a three-dimensional method of dynamic collision avoidance for UAV formation flight based on the theory of velocity obstacle. The collection of all possible accelerations of the UAV can be obtained by the input of UAV. Considering that the relative velocity between the UAV and obstacle must be out of the collision cone, we can obtain the collection of feasible accelerations next moment. Then we choose the acceleration which is able to let the UAV closest to the target next moment in the collection as the real acceleration. The maximum and minimum velocity should also be set up. Each UAV can determine the best acceleration next time for itself. In addition to formation keeping, this paper also presents how to split the formation on the basis of the direction of each UAV. Numerical simulation is used to test the performance of this method at the end of this paper.

Multi-agent pursuit-evasion games are a common way to describe problems in topics like police missions, surveillance, or warfare. Since many agents are involved in such problems, a game-theoretical solution seems reasonable. Game-theoretical solution approaches are generally very complex, especially in games with more than two agents, as it is the case in this paper. More and more unmanned aerial vehicles (UAV), in particular multi-rotors, are used in such applications because of their high maneuverability in all three dimensions. Especially in dangerous missions and applications UAVs excel, as they do not risk the life of the pilot. Tele-operated UAVs still require a pilot which has to be trained to do a task and a constant radio link between the pilot and the vehicle has to be maintained. An autonomous agent, able of performing the task of cooperative pursue and evasion can substitute the pilot. In this work, a realization of cooperative pursuit-evasion games with UAVs is presented. A real-time implementation is proposed, enabling a game-theoretical solution of the regarded problem. The applicability was proven during several real outdoors flight experiments with a hex-rotor UAV. Furthermore, the results obtained by the flight experiments were compared to simulations of the same games. Although, the most experimental results met the reference, disturbances like strong wind, wind gusts and sensor uncertainties affected the results significantly.

Unmanned Aerial Vehicles (UAVs), or drones, have recently become a favorable solution for fast parcel delivery due to their maneuverability and advances in navigation technologies. With the limitation of battery capacity and payload of drones, it is crucial to consider both efficiency and cost while conducting the tasks. Meanwhile, UAVs should not collide with each other while traveling to customers. In this article, we propose a UAV parcel delivery system involving deep reinforcement learning (DRL) approach for collision avoidance and a genetic algorithm for route optimization. Specifically, a delivery center generates near-optimal routes, loading UAV with parcels according to demands. Each UAV takes charge of delivering packages in compliance with the assigned route while avoiding collision with each other. We utilize DRL to achieve collision avoidance without having prior knowledge about the trajectories of other UAVs. Additionally, we adopt a genetic algorithm to obtain the lowest energy cost path for each UAV. To find such an optimized path, we solve a capacitated vehicle routing problem (CVRP) with a modified cost function and extra constraints. Realistic simulations using a physics engine and software-in-the-loop (SITL) are conducted to evaluate the feasibility of the proposed methods.

Unmanned Aerial Vehicles (UAV) have become more popular for usage due to the low cost of deployment and maintenance. Single UAV employment allows remote area monitoring and transferring different payloads to inaccessible or dangerous zones for human. In order to deal with flight tasks that are more complex, UAV swarms are applied. The main challenge of UAV swarm formation and flight control is to avoid vehicle collisions. In this case, artificial intelligence is responsible for flight performance in the airspace in such way that collision is avoided. The main requirements to the method, which will provide conflict-free maneuvers, are safety (collision avoidance), liveness (decentralized control, destination area reachability) and flyability (UAV flight performance constraints are satisfied). Artificial force field method fulfills all of these demands. It allows to detect a potential conflict between multiple UAVs in a swarm and other static or moving obstacles found in airspace, to provide collision resolution by changing UAVs flight parameters through maintaining minimum separation distance, including cases when manned vehicles are found in the same airspace. There can be distinguished by a wide range of obstacles: static (buildings, restricted areas and bed weather conditions) and dynamic ones (other UAVs, manned aircraft). Method allows keeping UAV swarm shape on the flight path, taking into account ground speed and turn bank angle values restrictions according to UAV’s flight performance characteristics.

The flexibility and maneuverability of unmanned aerial vehicles (UAVs) lend themselves to tracking users and operating as an aerial base station carrying out communication enhancement functionality. A core challenge neglected by most existing works is that the true-but-unknown obstacles can jeopardize UAV flight security and shadow its communication links with users, resulting in poor achievable rate and high collision risks. In this paper, a deep-reinforcement-learning (DRL)-based UAV target tracking and digital-twin (DT)-assisted collision avoidance method is proposed to optimize UAV’s communication performance while tracking moving users. Toward this end, Twin Delayed Deep Deterministic policy gradient (TD3) as a novel and policy-based DRL algorithm is used to construct an agent responsible for adaptive deciding UAV flying control actions. To efficiently detect unknown obstacles in a flight environment, an orthogonal frequency division multiple (OFDM)-based integrated sensing and communication (ISAC) system is investigated, endowing UAV’s agent with real-time obstacle distance. Finally, we present a DT obstacle model construction mechanism and integrate it with TD3 agent training. The extensive simulations demonstrate the reward convergence of the TD3 algorithm and the communication improvement with stable user tracking and reliable collision avoidance, compared with conventional approaches.

Trajectory planning in airborne networks is a primary application of Unmanned Aerial Vehicle (UAV) communication. The literature contains algorithms focusing on collision avoidance, intelligent task assignment and path planning among UAVs. Cooperative schemes exist in which the UAVs communicate with each other to reach a consensus on how each vehicle will take up one trajectory at a time efficiently based on its capability. In the last decade, much research is being done to use UAVs to track contamination in the environment, detect fires and even volcanic eruptions. As the plumes are constantly varying in space and time, there have to be efficient guidance algorithms for the UAVs in order to keep track of the shape of the plume. In this paper, we survey the literature to explore the currently available trajectory planning algorithms and also the few available plume tracking algorithms. We then formulate the Plume wrapping problem and propose a cooperative trajectory planning approach for a group of UAVs, through which the UAVs communicate their position estimates and planned trajectories to update each others trajectory to efficiently wrap around a plume region so that the contamination in the region can be studied.

In this paper, we propose a reachable set based collision avoidance algorithm for unmanned aerial vehicles (UAVs). UAVs have been deployed for agriculture research and management, surveillance and sensor coverage for threat detection and disaster search and rescue operations. It is essential for the aircraft to have on-board collision avoidance capability to guarantee safety. Instead of the traditional approach of collision avoidance between trajectories, we propose a collision avoidance scheme based on reachable sets and tubes. We then formulate the problem as a convex optimization problem seeking time varying control sets for the ego aircraft given the predicted intruder reachable tube. We have applied the approach on a case study of two quadrotors collision avoidance scenario.

In recent years, the development of small unmanned aerial vehicle (UAV) has greatly promoted the development of UAV swarms. As multi-UAV are widely applied in more scenarios, the collision avoidance problems have attracted more attention. This paper discussed the basic problems of collision avoiding about multi-UAV, and sorted out the collision avoidance algorithms of multiple UAVs. We divide the UAV collision avoidance algorithm into two parts, one is cooperative algorithm and another is non-cooperative algorithm. Then, according to the current algorithm development, the development trend of multi-UAV collision avoidance research in the further are discussed.