Hardware Implementation of Ultra‐Fast Obstacle Avoidance Based on a Single Photonic Spiking Neuron

A new consensus theory-based method for formation control and obstacle avoidance of UAVs

In the past decade, a lot of researches have been done on the topic of real-time obstacle avoidance of Unmanned Aerial Vehicle (UAV). For example, the use of Light Detection and Ranging (LIDAR), ultrasonic sensors, and now more popular visual sensors can effectively avoid obstacles. Although these methods are simple and effective, there are few of UAV real-time obstacle avoidance for complex environments, and most of them are used to avoid a single obstacle. When facing multiple obstacles, they cannot choose the optimal strategy, which inevitably consumes more power and time. In view of the complex environment that UAV may face multiple obstacles, in order to avoid multiple obstacles with minimum efforts in the process of real obstacle avoidance, this paper proposes a new method to detect multiple obstacles. Firstly, it extracts obstacle feature points, and then uses unsupervised clustering algorithm in machine learning to divide the feature points of different obstacles into different categories, and finally detects the obstacles represented by convex hull of same category feature points. The method is simple and effective. This paper compares the time of several popular feature point detection and unsupervised clustering algorithms through experiments and estimates the approximate relationship between the number of feature points detected and the distance when the UAV approaches the obstacle, which is a priori study for the future practical obstacle avoidance link. It summarizes which algorithm can save more time and is more suitable for real-time obstacle avoidance of multiple obstacles.

Large areas across the United States are potentially contaminated with UXO, with some ranges encompassing tens to hundreds of thousands of acres. Technologies are needed which will allow for cost effective wide area scanning with 1) near 100 % coverage and 2) near 100 % detection of subsurface ordnance or features indicative of subsurface ordnance. The current approach to wide area scanning is a multi-level one, in which medium altitude fixed wing optical imaging is used for an initial site assessment. This assessment is followed with low altitude manned helicopter based magnetometry followed by surface investigations using either towed geophysical sensor arrays or man portable sensors. In order to be effective for small UXO detection, the sensing altitude for magnetic site investigations needs to be on the order of 1 – 3 meters. These altitude requirements means that manned helicopter surveys will generally only be feasible in large, open and relatively flat terrains. While such surveys are effective in mapping large areas relatively fast there are substantial mobilization/demobilization, staffing and equipment costs associated with these surveys (resulting in costs of approximately $100-$150/acre). Surface towed arrays provide high resolution maps but have other limitations, e.g. in their ability to navigate rough terrain effectively. Thus, other systems are needed allowing for effective data collection. An UAV (Unmanned Aerial Vehicle) magnetometer platform is an obvious alternative. The motivation behind such a system is that it would be safer for the operators, cheaper in initial and O&M costs, and more effective in terms of site characterization. However, while UAV data acquisition from fixed wing platforms for large (> 200 feet) stand off distances is relatively straight forward, a host of challenges exist for low stand-off distance (~ 6 feet) UAV geophysical data acquisition. The objective of SERDP SEED 1509:2006 was to identify the primary challenges associated with a low stand off distance autonomous UAV magnetometer platform and to investigate whether these challenges can be resolved successfully such that a successful UAV magnetometer platform can be constructed. The primary challenges which were identified and investigated include: 1. The feasibility of assembling a payload package which integrates magnetometers, accurate positioning systems (DGPS, height above ground measurement), obstacle avoidance systems, power infrastructure, communications and data storage as well as auxiliary flight controls 2. The availability of commercial UAV platforms with autonomous flight capability which can accommodate this payload package 3. The feasibility of integrating obstacle avoidance controls in UAV platform control 4. The feasibility of collecting high quality magnetic data in the vicinity of an UAV.

Unmanned aerial vehicles (UAVs), also known as drones, have gained considerable interest among academics in recent years, which significantly increases the demand for autonomous navigation systems for UAVs. In this study, autonomous navigation is addressed as a two-stage problem. The first stage is the obstacle detection phase, during which the UAVs leverage their onboard sensors to perceive the surroundings and detect obstacles. The second stage is the obstacle avoidance phase, where the UAVs make decisions to continue operations without colliding with anything while approaching the target. This study proposes a self-supervised learning-based depth prediction framework as the criterion for obstacle proximity detection. The framework consists of two subnetworks: the depth and pose estimation modules. The former predicts depth values, while the latter evaluates the object’s translation and rotation matrix. The framework leverages the subnetworks’ outputs to reconstruct the projection relationships of pixels between adjacent frames to optimize the training process. To address the challenge of UAVs operating in large-scale environments, a modified loss function that combines photometric loss and edge-aware smoothness is presented to increase the depth prediction module’s accuracy. This study also presents a deep reinforcement learning-based obstacle avoidance system that takes depth maps as inputs to generate evasive maneuvers. The system works independently from priori maps and models while it learns the collision-free trajectory from interacting with the environment. This study deploys value-based algorithms to learn the nonlinear mapping between the continuous inputs and discrete output motion commands. Furthermore, the proposed system is validated on different simulation scenarios. It is capable of completing the obstacle avoidance task with relatively optimal trajectories in varied environments, demonstrating its superior effectiveness and transferability. Keywords: unmanned aerial vehicles, urban airspace, obstacle detection and avoidance, self-supervised learning, deep reinforcement learning.

Existing resource-intensive obstacle avoidance techniques are hard to be applied on the small-size Unmanned Aerial Vehicles (UAVs) that have limited sensing and computation capacity. Therefore, it is necessary to develop an obstacle recognition and avoidance scheme which works under resource-constrained environments. To this backdrop, this paper first presents an obstacle recognition model based on monocular vision feature points. Afterwards, an obstacle recognition algorithm is put forward, whose computational complexity is low. Then, an obstacle avoidance method is proposed to regulate the obstacle-avoidance path of a UAV until it arrives its destination. To evaluate the effectiveness of the presented algorithms, we design and implement a simulation platform on Objective Modular Network Testbed in C++ (OMNeT++), and conduct a series of experiments. Experimental results show that the proposed model and algorithms can effectively guide a micro UAV to its destination using only an embedded processor and 0.5kg extra load. Even in poor communication conditions, the UAV can independently avoid obstacles and reach the destination only by acquiring the destination coordinates from the ground station.

The proliferation of unmanned aerial vehicles (UAVs) has driven the emergence of urban air mobility (UAM) as a solution to alleviate the increasing demand for UAVs applications in various domains. The concept of UAM envisions a future where UAVs play a pivotal role in short-distance transportation, delivery services, surveillance, emergency rescue, and more. However, the rapid growth of UAVs poses significant challenges arising from the need to ensure the safe coexistence of UAVs with other manned/unmanned aerial vehicles. To ensure the safe integration of UAVs into low-altitude urban airspace and support high-density on-demand mobility, this research focuses on handling the safety-centric challenges related to UAV collision risk assessment, obstacle avoidance, and conflict resolution in complex urban environments.
\nOne of the main challenges is the intrusion of UAVs into airport runways. Airports serve as critical hubs for civil aircraft takeoff and landing operations. However, the increasing presence of UAVs has raised concerns about intruding into airports, leading to flight delays and temporary runway closures. To quantify the potential interference of UAVs on civil aircraft, a collision-course based trajectory planning model is proposed to evaluate the collision risk between manned civil aircraft and intruding UAVs. In this risk assessment model, the trajectory of the intruding UAV is formulated using a collision-course concept, taking into account the limited trajectory information available about intruding UAVs. The planned trajectory of the UAVs is then used to predict its relative position concerning civil aircraft. To account for the possible disturbances of the relative distance, a stochastic position prediction model based on Brownian motion is developed. The collision risk is assessed based on this stochastic kinematic model, providing a quantitative measure of the potential collisions. The proposed model is validated through extensive simulations involving different UAV initial positions, position updates, and different sizes of collision zone tests, as well as comparative tests with the Monte-Carlo algorithm, demonstrating its robustness in assessing the collision risk of the airport restricted area. To facilitate the application, the model is finally used to develop a platform to generate runway heatmaps, accounting for airport runway risk distributions. 
\nAnother safety-centric challenge is ensuring autonomous obstacle avoidance and conflict resolution of UAVs. To enable UAV autonomous safe operations in uncontrolled urban low-altitude airspace, a comprehensive collision-free path planning framework is developed to address the avoidance of both static obstacles and dynamic threats. Firstly, to handle static obstacles, a novel algorithm of 3D voxel jump point research is introduced to generate a global reference path for UAV navigation to avoid static obstacles, particularly for a free flight at different altitudes. Additionally, to optimize the reference path for specific mission requirements, techniques such as trajectory de-diagonalization, reconstruction, and smoothness are proposed to reduce the risk of static collisions, shorten the fly range, and enable UAVs to negotiate turns more efficiently. Secondly, real-time conflict resolution actions based on the Markov decision process are developed to avoid local dynamic threats encountered along the optimized desired path. This allows the host UAV to dynamically adjust its course to a resolution trajectory while maintaining its planned speed without hovering to avoid the potential collision with the approaching intruders. By avoiding near mid-air collisions (NMAC) and preventing the UAV from hovering, the proposed method ensures the timely completion of UAV missions. Simulation results demonstrate that the developed method successfully achieves UAV autonomous global path planning and local real-time conflict resolution in complex urban low-altitude environments, thereby avoiding both static obstacles and dynamic threats. 
\nMoreover, to bridge the research gap of UAV hard conflict resolution in mitigating NMAC risks arising from local dynamic threats, a novel situational NMAC risk assessment approach is developed by integrating the conditional random field (CRF) algorithm with safety metrics and conflict resolution performance to facilitate UAV tiered aerial passage operations. To improve UAV operating environments in urban airspace, a concept of tiered aerial passages is first designed to provide a practical urban airspace planning scheme that addresses UAV heterogeneity and optimizes the utilization of urban airspace. For UAV tiered aerial passage operations, a situational NMAC risk assessment module coupled with a conflict resolution decision-making module is formulated, enabling the host UAV to evaluate and respond to real-time conflicts, ensuring avoidance of potential collisions. To assess NMAC risks, a general risk assessment model is proposed based on the CRF algorithm, considering UAV safety metrics and resolution performance. Then the overtaking, approaching head-on, and converging UAV encounters are analyzed, and situational risk potential scores are created using the Gaussian kernel function in the CRF algorithm, considering advisory, caution, and warning risk alert levels. To resolve conflict, acceptable, tolerable, and unacceptable scenarios are proposed to enhance the flexibility and personalization of the resolution decision-making. The proposed scheme is validated by simulating geometric encounter situations and comparative tests with the previous hard resolution technique. The results indicate that the proposed method outperforms UAV NMAC risk assessment and conflict resolution. Even in the most dangerous unacceptable resolution decision-making scenarios across three conflict encounters, the proposed algorithm achieves NMAC rates of 0.036, 0.070, and 0.043, which are lower than the corresponding NMAC rates associated with the hard resolution approach. Together with the acceptable and tolerable scenarios, the proposed negotiation resolution effectively tailors different risk requirements to accommodate diverse UAV missions. 
\nWith a primary focus on ensuring the safety of the UAM architecture, this research endeavors to assess UAV NMAC risks and efficiently manage potential conflicts. The proposed work are thoroughly validated to demonstrate their effectiveness in real-world scenarios. As a result, this study significantly contributes to the establishment of a robust UAM architecture, facilitating the realization of safe and reliable UAM.

Unmanned Aerial Vehicles (UAVs), commonly referred to as drones, have witnessed rapid growth in applications such as environmental monitoring, disaster response, surveillance, and logistics. The effectiveness and safety of UAV operations rely heavily on the implementation of advanced control strategies that ensure accurate trajectory tracking, obstacle avoidance, energy efficiency, and autonomous mission execution. This paper presents a comprehensive comparative study of artificial intelligence (AI)-based control methodologies designed to enhance the autonomy, adaptability, and decision-making capabilities of UAVs. The study evaluates several prominent AI-driven approaches, including neural networks, reinforcement learning, fuzzy logic systems, and evolutionary algorithms, with a focus on their applicability to key UAV control tasks. Each technique is critically examined in terms of its control accuracy, computational efficiency, energy management, and suitability for swarm coordination in dynamic environments. Furthermore, the paper addresses real-world deployment challenges such as training data requirements, onboard computational limitations, and real-time responsiveness. By systematically analyzing the strengths, limitations, and practical feasibility of different AI paradigms, this research offers valuable insights for the development of intelligent, robust, and sustainable UAV control systems in environmental science applications.

Path planning and obstacle avoidance problems are now the focus of robotics research. This paper uses the Dyna-Q reinforcement learning algorithm to implement an obstacle avoidance and a path planning algorithm for unmanned ground vehicle(UGV) under urban environment. Using the reinforcement learning algorithm, we calculate the waypoints of the unmanned vehicle and achieve obstacle avoidance tasks and path planning using a vector field. Finally, we use a PID controller on unmanned aerial vehicle (UAV) to realize the air-ground collaboration task. The algorithms and the agents’ modeling in this paper are implemented in the lab’s simulation platform.

In this paper autonomous formation control for Unmanned Aerial Vehicles (UAVs) has been discussed and a real time solution has been put forward by benefiting General Purpose Graphical Processing Units (GPGPU) accelerated potential field approach while ensuring obstacle and collision avoidance in unknown environment by using real-time sensors. GPGPU accelerated real time formation control for UAVs was designed and the basic model of the approach has been explained in our previous work (Cetin and Yilmaz 2014). As the deficiencies of the previous approach, autonomous real time collision and coordinated obstacle avoidance features in unknown environments are also handled while maintaining formation flight conditions in this work. With these features, improved autonomous formation control approach is discussed as a real time solution. The computation is performed by using Graphical Processing Units (GPUs) as parallel computation architectures by benefiting from Single Instruction Multiple Data (SIMD) type parallel algorithms. Classic binary map conversation, connected component labeling and minimum bounding box algorithms which are commonly used for image processing applications, has been evaluated for real time obstacle detection and avoidance features by developing GPGPU suitable parallel algorithms. Real-time solution has been developed by integrated these parallel algorithms with parallel Artificial Potential Field (APF) computation algorithm. Simulation results are proved that this novel autonomous improved formation control approach is successful and it would be used in real time applications like UAV formation flight missions.

This paper addresses the problem of controlling a heterogeneous system composed of multiple Unmanned Aerial Vehicles (UAVs) and Autonomous Underwater Vehicles (AUVs) for formation and containment maintenance. The proposed approach considers actuator time delay and, in addition to formation and containment, considers obstacle avoidance, and offers a robust navigation algorithm and uses a reliable middleware for data transmission and exchange. The methodology followed uses both flocking technique and modified L1 adaptive control to ensure the proper navigation and coordination while avoiding obstacles. The data exchange between all the agents is provided through the data distribution services (DDS) middleware, which solves the interoperability issue when dealing with heterogeneous multiagent systems. The modified L1 controller is a local controller for stabilizing the dynamic model of each UAV and AUV, and the flocking approach is used to coordinate the followers around the leader or within the space delimited by their leaders. Potential Field (PF) allows obstacle avoidance during the agents’ movement. The performance of the proposed approach under the considerations mentioned above are verified and demonstrated using simulations.

<div>In this dissertation, methods for real-time trajectory generation and autonomous obstacle avoidance for fixed-wing and quad-rotor unmanned aerial vehicles (UAV) are studied. A key challenge for such trajectory generation is the high computation time required to plan a new path to safely maneuver around obstacles instantaneously. Therefore, methods for rapid generation of obstacle avoidance trajectory are explored. The high computation time is a result of the computationally intensive algorithms used to generate trajectories for real-time object avoidance. Recent studies have shown that custom solvers have been developed that are able to solve the problem with a lower computation time however these designs are limited to small sized problems or are proprietary. Additionally, for a swarm problem, which is an area of high interest, as the number of agents increases the problem size increases and in turn creates further computational challenges. A solution to these challenges will allow for UAVs to be used in autonomous missions robust to environmental uncertainties.</div><div><br></div><div>In this study, a trajectory generation problem posed as an optimal control problem is solved using a sequential convex programming approach; a nonlinear programming algorithm, for which custom solver is used. First, a method for feasible trajectory generation for fast-paced obstacle-rich environments is presented for the case of fixed-wing UAVs. Next, a problem of trajectory generation for fixed-wing and quad-rotor UAVs is defined such that starting from an initial state a UAV moves forward along the direction of flight while avoiding obstacles and remaining close to a reference path. The problem is solved within the framework of finite-horizon model predictive control. Finally, the problem of trajectory generation is extended to a swarm of quad-rotors where each UAV in a swarm has a reference path to fly along. Utilizing a centralized approach, a swarm scenario with moving targets is studied in two different cases in an attempt to lower the solution time; the first, solve the entire swarm problem at once, and the second, solve iteratively for a UAV in the swarm while considering trajectories of other UAVs as fixed.</div><div><br></div><div>Results show that a feasible trajectory for a fixed-wing UAV can be obtained within tens of milliseconds. Moreover, the obtained feasible trajectories can be used as initial guesses to the optimal solvers to speed up the solution of optimal trajectories. The methods explored demonstrated the ability for rapid feasible trajectory generation allowing for safe obstacle avoidance, which may be used in the case an optimal trajectory solution is not available. A comparative study between a dynamic and a kinematic model shows that the dynamic model provides better trajectories including aggressive trajectories around obstacles compared to the kinematic counterpart for fixed-wing UAVs, despite having approximately the same computational demands. Whereas, for the case of quad-rotor UAVs, the kinematic model takes almost half the solution time than with a reduced dynamic model, despite having approximately the similar range of values for the cost function. When extended to a swarm, solving the problem for each UAV is four to seven times computationally cheaper than solving the swarm as a whole. With the improved computation time for trajectory generation for a swarm of quad-rotors using centralized approach, the problem is now reasonably scalable, which opens up the possibility to increase the number of agents in a swarm using high-end computing machines for real-time applications. Overall, a custom solver jointly with a sequential convex programming approach solves an optimization problem in a low computation time.</div>

Precise timing and precise location information are provided by Global Navigation Satellite Systems (GNSS) and play a crucial role in the positioning, navigation and data acquisition of most Unmanned Aerial Vehicles (UAV). GNSS functions include the following applications in UAVs: time-stamping and geo-referencing of collected data and images, synchronization of swarm flying and follow-me flights, determination of position and attitude in-flight, flight trajectory by following a pre-defined number of waypoints, mission planning, return home automatically without external control, avoidance of obstacles and geo-fencing. 
 Some of these critical operations have implications for the safety of the UAV, the surrounding environment and health and safety of people, for example UAVs threatening to bring down aircrafts at airports, which are no-fly zones for UAVs. The appropriate GNSS based function to avoid this is geo-fencing. Another example is obstacle avoidance to prevent collisions and damages both for the UAV and the obstacle, e.g. anything from a window pane, tree, human being, to a power line.
 In order to ensure health and safety it is thus important to ensure correct function of the navigation and the timing, under a wide variety of circumstances, and in different signal environments. There can be signal disturbances, such as obscurations by buildings or reflected GNSS signals, called multipath. The performance of timing and navigation based on GPS/GNSS can be tested and verified in a controlled and repeatable way in the laboratory with different types of test equipment. We will give an introduction to a wide range of potential threats to GNSS Positioning, navigation and timing and an overview of different test methods. In addition, we are presenting a method for time synchronization of drones to enable safe swarm and follow flights in UAVs.

The use of unmanned aerial vehicle (UAV) has been recognized by the majority of people, and is regarded as a reliable tool to complete some tasks. As for the obstacle avoidance of coaxial rotor UAV (CR-UAV), a flight pattern map (Flight Situation Awareness Map: FSAM) that can sense the flight environment and an integrated control method (Improved Artificial Potential Field: IAPF) based on the common artificial potential field method with appropriate improvements to achieve obstacle avoidance (IAPF). Firstly, we construct the FSAM, which can map the environment information around the UAV on the FSAM. Then, based on the FSAM, the IAPF functions are established to achieve the obstacles avoidance. Cause the artificial potential field (APF) has a characteristic that cannot avoid: the problem of local minima, a rotating potential field is put forward to ensure that the CR-UAV has only one potential equilibrium point at the target point in the environment, and it will improve the ability of CR-UAV to avoid complex obstacles. At the end of this study, through data analysis, the performance of obstacle avoidance and the attitude stability of CR-UAV are good, the simulation results confirm that the approaches proposed in this paper can address the obstacles avoidance successfully.KeywordsCR-UAVImproved Artificial Potential Field (IAPF)Flight Situation Awareness Map (FSAM)Obstacles avoidance

The obstacle avoidance problem of unmanned aerial vehicle (UAV) mainly refers to the design of a method that can safely reach the target point from the starting point in an unknown flight environment. In this paper, we mainly propose an obstacle avoidance method composed of three modules: environment perception, algorithm obstacle avoidance and motion control. Our method realizes the function of reasonable and safe obstacle avoidance of UAV in low-altitude complex environments. Firstly, we use the light detection and ranging (LiDAR) sensor to perceive obstacles around the environment. Next, the sensor data is processed by the vector field histogram (VFH) algorithm to output the desired speed of drone flight. Finally, the expected speed value is sent to the quadrotor flight control and realizes autonomous obstacle avoidance flight of the drone. We verify the effectiveness and feasibility of the proposed method in 3D simulation environment.

This paper utilizes the distributed model predictive control (DMPC) method to investigate the formation control problem of unmanned aerial vehicles (UAVs) in the obstacle environment and establishes cooperative capability evaluation metrics of the swarm. Based on the DMPC approach, the formation cost function is constructed to adjust the relative positions and velocities of UAVs, ensuring the desired formation. Additionally, to address the obstacle avoidance problem in the formation, the obstacle avoidance function is designed to provide safe formation control in the obstacle environment. To evaluate the cooperative capability of UAVs, we design evaluation metrics from multiple dimensions to reflect the swarm’s cooperative capability. Finally, the simulation results show the effectiveness of the formation control method with obstacle avoidance and the applicability of the swarm’s cooperative capability evaluation metrics.