Obstacle Recognition and Avoidance for UAVs Under Resource-Constrained Environments

A new consensus theory-based method for formation control and obstacle avoidance of UAVs

The proliferation of unmanned aerial vehicles (UAVs) has driven the emergence of urban air mobility (UAM) as a solution to alleviate the increasing demand for UAVs applications in various domains. The concept of UAM envisions a future where UAVs play a pivotal role in short-distance transportation, delivery services, surveillance, emergency rescue, and more. However, the rapid growth of UAVs poses significant challenges arising from the need to ensure the safe coexistence of UAVs with other manned/unmanned aerial vehicles. To ensure the safe integration of UAVs into low-altitude urban airspace and support high-density on-demand mobility, this research focuses on handling the safety-centric challenges related to UAV collision risk assessment, obstacle avoidance, and conflict resolution in complex urban environments.
\nOne of the main challenges is the intrusion of UAVs into airport runways. Airports serve as critical hubs for civil aircraft takeoff and landing operations. However, the increasing presence of UAVs has raised concerns about intruding into airports, leading to flight delays and temporary runway closures. To quantify the potential interference of UAVs on civil aircraft, a collision-course based trajectory planning model is proposed to evaluate the collision risk between manned civil aircraft and intruding UAVs. In this risk assessment model, the trajectory of the intruding UAV is formulated using a collision-course concept, taking into account the limited trajectory information available about intruding UAVs. The planned trajectory of the UAVs is then used to predict its relative position concerning civil aircraft. To account for the possible disturbances of the relative distance, a stochastic position prediction model based on Brownian motion is developed. The collision risk is assessed based on this stochastic kinematic model, providing a quantitative measure of the potential collisions. The proposed model is validated through extensive simulations involving different UAV initial positions, position updates, and different sizes of collision zone tests, as well as comparative tests with the Monte-Carlo algorithm, demonstrating its robustness in assessing the collision risk of the airport restricted area. To facilitate the application, the model is finally used to develop a platform to generate runway heatmaps, accounting for airport runway risk distributions. 
\nAnother safety-centric challenge is ensuring autonomous obstacle avoidance and conflict resolution of UAVs. To enable UAV autonomous safe operations in uncontrolled urban low-altitude airspace, a comprehensive collision-free path planning framework is developed to address the avoidance of both static obstacles and dynamic threats. Firstly, to handle static obstacles, a novel algorithm of 3D voxel jump point research is introduced to generate a global reference path for UAV navigation to avoid static obstacles, particularly for a free flight at different altitudes. Additionally, to optimize the reference path for specific mission requirements, techniques such as trajectory de-diagonalization, reconstruction, and smoothness are proposed to reduce the risk of static collisions, shorten the fly range, and enable UAVs to negotiate turns more efficiently. Secondly, real-time conflict resolution actions based on the Markov decision process are developed to avoid local dynamic threats encountered along the optimized desired path. This allows the host UAV to dynamically adjust its course to a resolution trajectory while maintaining its planned speed without hovering to avoid the potential collision with the approaching intruders. By avoiding near mid-air collisions (NMAC) and preventing the UAV from hovering, the proposed method ensures the timely completion of UAV missions. Simulation results demonstrate that the developed method successfully achieves UAV autonomous global path planning and local real-time conflict resolution in complex urban low-altitude environments, thereby avoiding both static obstacles and dynamic threats. 
\nMoreover, to bridge the research gap of UAV hard conflict resolution in mitigating NMAC risks arising from local dynamic threats, a novel situational NMAC risk assessment approach is developed by integrating the conditional random field (CRF) algorithm with safety metrics and conflict resolution performance to facilitate UAV tiered aerial passage operations. To improve UAV operating environments in urban airspace, a concept of tiered aerial passages is first designed to provide a practical urban airspace planning scheme that addresses UAV heterogeneity and optimizes the utilization of urban airspace. For UAV tiered aerial passage operations, a situational NMAC risk assessment module coupled with a conflict resolution decision-making module is formulated, enabling the host UAV to evaluate and respond to real-time conflicts, ensuring avoidance of potential collisions. To assess NMAC risks, a general risk assessment model is proposed based on the CRF algorithm, considering UAV safety metrics and resolution performance. Then the overtaking, approaching head-on, and converging UAV encounters are analyzed, and situational risk potential scores are created using the Gaussian kernel function in the CRF algorithm, considering advisory, caution, and warning risk alert levels. To resolve conflict, acceptable, tolerable, and unacceptable scenarios are proposed to enhance the flexibility and personalization of the resolution decision-making. The proposed scheme is validated by simulating geometric encounter situations and comparative tests with the previous hard resolution technique. The results indicate that the proposed method outperforms UAV NMAC risk assessment and conflict resolution. Even in the most dangerous unacceptable resolution decision-making scenarios across three conflict encounters, the proposed algorithm achieves NMAC rates of 0.036, 0.070, and 0.043, which are lower than the corresponding NMAC rates associated with the hard resolution approach. Together with the acceptable and tolerable scenarios, the proposed negotiation resolution effectively tailors different risk requirements to accommodate diverse UAV missions. 
\nWith a primary focus on ensuring the safety of the UAM architecture, this research endeavors to assess UAV NMAC risks and efficiently manage potential conflicts. The proposed work are thoroughly validated to demonstrate their effectiveness in real-world scenarios. As a result, this study significantly contributes to the establishment of a robust UAM architecture, facilitating the realization of safe and reliable UAM.

Unmanned Aerial Vehicles (UAVs) are playing an increasingly vital role in modern battlefields. Accordingly, considerable research has been devoted to Manned–Unmanned Teaming (MUM-T) systems, with formation flight recognized as a key enabling technology for coordinating multiple UAVs. In MUM-T operations, leader–follower formations are commonly employed, while distributed formation methods have gained increasing attention owing to their stability and scalability. Among these, bearing-based control provides unique advantages for managing dynamic formations involving scaling and rotation. However, conventional bearing-based approaches typically require multiple leaders and encounter inherent limitations in flexibly handling obstacle avoidance. To address these challenges, this study proposes a hierarchical bearing-based leader–follower formation system comprising a single leader and multiple follower UAVs. By introducing the concept of virtual leaders, the proposed method enables the construction of formations with only one leader, thereby simplifying the system architecture while preserving scalability. In addition, a novel obstacle-avoidance strategy is developed, allowing followers to avoid collisions efficiently while maintaining formation integrity. The effectiveness of the proposed framework is demonstrated through numerical simulations of representative formation patterns, including V-shaped and hexagonal configurations, in obstacle-rich environments. The results confirm that follower UAVs successfully tracked the leader, preserved the designated formation, and achieved effective obstacle avoidance, thereby validating the stability and robustness of the proposed approach.

With the development and widely use of unmanned aerial vehicles (UAVs) in recent years, the development of efficient path planning methods for automation has become crucial. Obstacle avoidance and path planning are the key components of UAV path planning. This article provides an overview of obstacle avoidance and path planning techniques for UAVs based on the artificial potential field method (APF method). This article begins with the explaining the principles of artificial potential field on this basis discusses its advantages and limitations. The article then summarizes the improvement strategies proposed by previous researchers to address issues like local minimum values and unreachable targets, such as introducing a new repulsive potential energy function, combining APF with other planning methods, and utilizing flow functions. Furthermore, it presents examples of the application and the performance of usage of these techniques in both static and dynamic environments. Based on this, the prospects and developing trend of UAV obstacle avoidance methods based on artificial potential field are foresee, such as combined with DRL and deep learning.

<div>In this dissertation, methods for real-time trajectory generation and autonomous obstacle avoidance for fixed-wing and quad-rotor unmanned aerial vehicles (UAV) are studied. A key challenge for such trajectory generation is the high computation time required to plan a new path to safely maneuver around obstacles instantaneously. Therefore, methods for rapid generation of obstacle avoidance trajectory are explored. The high computation time is a result of the computationally intensive algorithms used to generate trajectories for real-time object avoidance. Recent studies have shown that custom solvers have been developed that are able to solve the problem with a lower computation time however these designs are limited to small sized problems or are proprietary. Additionally, for a swarm problem, which is an area of high interest, as the number of agents increases the problem size increases and in turn creates further computational challenges. A solution to these challenges will allow for UAVs to be used in autonomous missions robust to environmental uncertainties.</div><div><br></div><div>In this study, a trajectory generation problem posed as an optimal control problem is solved using a sequential convex programming approach; a nonlinear programming algorithm, for which custom solver is used. First, a method for feasible trajectory generation for fast-paced obstacle-rich environments is presented for the case of fixed-wing UAVs. Next, a problem of trajectory generation for fixed-wing and quad-rotor UAVs is defined such that starting from an initial state a UAV moves forward along the direction of flight while avoiding obstacles and remaining close to a reference path. The problem is solved within the framework of finite-horizon model predictive control. Finally, the problem of trajectory generation is extended to a swarm of quad-rotors where each UAV in a swarm has a reference path to fly along. Utilizing a centralized approach, a swarm scenario with moving targets is studied in two different cases in an attempt to lower the solution time; the first, solve the entire swarm problem at once, and the second, solve iteratively for a UAV in the swarm while considering trajectories of other UAVs as fixed.</div><div><br></div><div>Results show that a feasible trajectory for a fixed-wing UAV can be obtained within tens of milliseconds. Moreover, the obtained feasible trajectories can be used as initial guesses to the optimal solvers to speed up the solution of optimal trajectories. The methods explored demonstrated the ability for rapid feasible trajectory generation allowing for safe obstacle avoidance, which may be used in the case an optimal trajectory solution is not available. A comparative study between a dynamic and a kinematic model shows that the dynamic model provides better trajectories including aggressive trajectories around obstacles compared to the kinematic counterpart for fixed-wing UAVs, despite having approximately the same computational demands. Whereas, for the case of quad-rotor UAVs, the kinematic model takes almost half the solution time than with a reduced dynamic model, despite having approximately the similar range of values for the cost function. When extended to a swarm, solving the problem for each UAV is four to seven times computationally cheaper than solving the swarm as a whole. With the improved computation time for trajectory generation for a swarm of quad-rotors using centralized approach, the problem is now reasonably scalable, which opens up the possibility to increase the number of agents in a swarm using high-end computing machines for real-time applications. Overall, a custom solver jointly with a sequential convex programming approach solves an optimization problem in a low computation time.</div>

The use of unmanned aerial vehicle (UAV) has been recognized by the majority of people, and is regarded as a reliable tool to complete some tasks. As for the obstacle avoidance of coaxial rotor UAV (CR-UAV), a flight pattern map (Flight Situation Awareness Map: FSAM) that can sense the flight environment and an integrated control method (Improved Artificial Potential Field: IAPF) based on the common artificial potential field method with appropriate improvements to achieve obstacle avoidance (IAPF). Firstly, we construct the FSAM, which can map the environment information around the UAV on the FSAM. Then, based on the FSAM, the IAPF functions are established to achieve the obstacles avoidance. Cause the artificial potential field (APF) has a characteristic that cannot avoid: the problem of local minima, a rotating potential field is put forward to ensure that the CR-UAV has only one potential equilibrium point at the target point in the environment, and it will improve the ability of CR-UAV to avoid complex obstacles. At the end of this study, through data analysis, the performance of obstacle avoidance and the attitude stability of CR-UAV are good, the simulation results confirm that the approaches proposed in this paper can address the obstacles avoidance successfully.KeywordsCR-UAVImproved Artificial Potential Field (IAPF)Flight Situation Awareness Map (FSAM)Obstacles avoidance

Unmanned aerial vehicles (UAVs), also known as drones, have gained considerable interest among academics in recent years, which significantly increases the demand for autonomous navigation systems for UAVs. In this study, autonomous navigation is addressed as a two-stage problem. The first stage is the obstacle detection phase, during which the UAVs leverage their onboard sensors to perceive the surroundings and detect obstacles. The second stage is the obstacle avoidance phase, where the UAVs make decisions to continue operations without colliding with anything while approaching the target. This study proposes a self-supervised learning-based depth prediction framework as the criterion for obstacle proximity detection. The framework consists of two subnetworks: the depth and pose estimation modules. The former predicts depth values, while the latter evaluates the object’s translation and rotation matrix. The framework leverages the subnetworks’ outputs to reconstruct the projection relationships of pixels between adjacent frames to optimize the training process. To address the challenge of UAVs operating in large-scale environments, a modified loss function that combines photometric loss and edge-aware smoothness is presented to increase the depth prediction module’s accuracy. This study also presents a deep reinforcement learning-based obstacle avoidance system that takes depth maps as inputs to generate evasive maneuvers. The system works independently from priori maps and models while it learns the collision-free trajectory from interacting with the environment. This study deploys value-based algorithms to learn the nonlinear mapping between the continuous inputs and discrete output motion commands. Furthermore, the proposed system is validated on different simulation scenarios. It is capable of completing the obstacle avoidance task with relatively optimal trajectories in varied environments, demonstrating its superior effectiveness and transferability. Keywords: unmanned aerial vehicles, urban airspace, obstacle detection and avoidance, self-supervised learning, deep reinforcement learning.

Path planning and obstacle avoidance are pivotal for intelligent unmanned aerial vehicle (UAV) systems in various domains, such as postdisaster rescue, target detection, and wildlife conservation. Currently, reinforcement learning (RL) has become increasingly popular in UAV decision-making. However, the RL approaches confront the challenges of partial observation and large state space when searching for random targets through continuous actions. This paper proposes a representation enhancement-based proximal policy optimization (RE-PPO) framework to address these issues. The representation enhancement (RE) module consists of observation memory improvement (OMI) and dynamic relative position-attitude reshaping (DRPAR). OMI reduces collision under partially observable conditions by separately extracting perception features and state features through an embedding network and feeding the extracted features to a gated recurrent unit (GRU) to enhance observation memory. DRPAR compresses the state space when modeling continuous actions by transforming movement trajectories of different episodes from an absolute coordinate system into different local coordinate systems to utilize similarity. In addition, three step-wise reward functions are formulated to avoid sparsity and facilitate model convergence. We evaluate the proposed method in three 3D scenarios to demonstrate its effectiveness. Compared to other methods, our method achieves a faster convergence during training and demonstrates a higher success rate and a lower rate of timeout and collision during inference. Our method can significantly enhance the autonomy and intelligence of UAV systems under partially observable conditions and provide a reasonable solution for UAV decision-making under uncertainties.

Electric Unmanned Aerial Vehicles (UAVs) experience problems and risks associated with battery aging and abuse effects. Therefore, a Battery Health Management (BHM) system is necessary to make the battery a safe, reliable, and cost-efficient solution. BHM systems are essential to ensure that the mission goal(s) can be achieved and to aid in online decision-making activities such as fault mitigation and mission replanning. To accomplish these tasks, we have adopted a model-based prognostics architecture for battery-powered UAVs where a battery model is used as the basis of two sequential task, (i) the State of Charge (SOC) estimation, and (ii) the End of Discharge (EOD) prediction. Small-size aircraft usually have weight, size and cost constraints. Therefore, there is a need to accurately (i) estimate the SOC, and (ii) predict the EOD time of Li-Po batteries in small-size UAVs that can operate in constrained environments. This work proposes a modification to an electrochemistry-based battery model that allows reducing computational resources without losing accuracy in prognostic results. The resulting hybrid battery model is validated and applied to prognostic of the EOD time in discharge cycles of a Li-Po battery of a small size quadcopter that performs delivery missions. Prediction results using the proposed hybrid battery model are shown to be very accurate while its estimation and prediction processing times are significantly lower than processing times using the electrochemistry-based battery model.

In order to solve multiple unmanned aerial vehicle (UAV) dynamic collision avoidance, a cooperative obstacle avoidance algorithm considering UAV’s kinematic constraints has been developed. In the proposed algorithm, the useful information of UAVs is screened out by a Heartbeat information filtering mechanism and fused by the user datagram protocol (UDP) communication method, which improves communication performance among UAVs. In addition, the velocity obstacle (VO) method combined with cubic uniform B-spline curve is used to avoid obstacles and generate smooth paths, which can be applied to practical scenes. Finally, dynamic and static obstacle avoidance simulations are carried out to verify the effectiveness of the proposed algorithm.

In this paper, the development and demonstration of a modified Voronoi algorithm for unmanned aerial vehicle (UAV) path planning and obstacle avoidance is presented. This algorithm is intended to produce a flyable collision-free path through a series of obstacles/threats represented by cylindrical risk zones. A model of the West Virginia University (WVU) YF-22 research aircraft implemented within the WVU UAV simulation environment is used to demonstrate the functionality of the proposed algorithm and its effectiveness in the presence of several risk zones configurations. As compared to traditional methods, the approach is more general, coping with obstacles and risk zones of variable intensity and providing additional flexibility and better coverage of the possible solution space.

In this paper autonomous formation control for Unmanned Aerial Vehicles (UAVs) has been discussed and a real time solution has been put forward by benefiting General Purpose Graphical Processing Units (GPGPU) accelerated potential field approach while ensuring obstacle and collision avoidance in unknown environment by using real-time sensors. GPGPU accelerated real time formation control for UAVs was designed and the basic model of the approach has been explained in our previous work (Cetin and Yilmaz 2014). As the deficiencies of the previous approach, autonomous real time collision and coordinated obstacle avoidance features in unknown environments are also handled while maintaining formation flight conditions in this work. With these features, improved autonomous formation control approach is discussed as a real time solution. The computation is performed by using Graphical Processing Units (GPUs) as parallel computation architectures by benefiting from Single Instruction Multiple Data (SIMD) type parallel algorithms. Classic binary map conversation, connected component labeling and minimum bounding box algorithms which are commonly used for image processing applications, has been evaluated for real time obstacle detection and avoidance features by developing GPGPU suitable parallel algorithms. Real-time solution has been developed by integrated these parallel algorithms with parallel Artificial Potential Field (APF) computation algorithm. Simulation results are proved that this novel autonomous improved formation control approach is successful and it would be used in real time applications like UAV formation flight missions.

This paper addresses the problem of controlling a heterogeneous system composed of multiple Unmanned Aerial Vehicles (UAVs) and Autonomous Underwater Vehicles (AUVs) for formation and containment maintenance. The proposed approach considers actuator time delay and, in addition to formation and containment, considers obstacle avoidance, and offers a robust navigation algorithm and uses a reliable middleware for data transmission and exchange. The methodology followed uses both flocking technique and modified L1 adaptive control to ensure the proper navigation and coordination while avoiding obstacles. The data exchange between all the agents is provided through the data distribution services (DDS) middleware, which solves the interoperability issue when dealing with heterogeneous multiagent systems. The modified L1 controller is a local controller for stabilizing the dynamic model of each UAV and AUV, and the flocking approach is used to coordinate the followers around the leader or within the space delimited by their leaders. Potential Field (PF) allows obstacle avoidance during the agents’ movement. The performance of the proposed approach under the considerations mentioned above are verified and demonstrated using simulations.

The obstacle avoidance problem of unmanned aerial vehicle (UAV) mainly refers to the design of a method that can safely reach the target point from the starting point in an unknown flight environment. In this paper, we mainly propose an obstacle avoidance method composed of three modules: environment perception, algorithm obstacle avoidance and motion control. Our method realizes the function of reasonable and safe obstacle avoidance of UAV in low-altitude complex environments. Firstly, we use the light detection and ranging (LiDAR) sensor to perceive obstacles around the environment. Next, the sensor data is processed by the vector field histogram (VFH) algorithm to output the desired speed of drone flight. Finally, the expected speed value is sent to the quadrotor flight control and realizes autonomous obstacle avoidance flight of the drone. We verify the effectiveness and feasibility of the proposed method in 3D simulation environment.

In the past decade, a lot of researches have been done on the topic of real-time obstacle avoidance of Unmanned Aerial Vehicle (UAV). For example, the use of Light Detection and Ranging (LIDAR), ultrasonic sensors, and now more popular visual sensors can effectively avoid obstacles. Although these methods are simple and effective, there are few of UAV real-time obstacle avoidance for complex environments, and most of them are used to avoid a single obstacle. When facing multiple obstacles, they cannot choose the optimal strategy, which inevitably consumes more power and time. In view of the complex environment that UAV may face multiple obstacles, in order to avoid multiple obstacles with minimum efforts in the process of real obstacle avoidance, this paper proposes a new method to detect multiple obstacles. Firstly, it extracts obstacle feature points, and then uses unsupervised clustering algorithm in machine learning to divide the feature points of different obstacles into different categories, and finally detects the obstacles represented by convex hull of same category feature points. The method is simple and effective. This paper compares the time of several popular feature point detection and unsupervised clustering algorithms through experiments and estimates the approximate relationship between the number of feature points detected and the distance when the UAV approaches the obstacle, which is a priori study for the future practical obstacle avoidance link. It summarizes which algorithm can save more time and is more suitable for real-time obstacle avoidance of multiple obstacles.