Artificial Intelligence-Based Decision Support System for UAV Control in a Simulated Environment.

Optimization of high-speed fixed-wing UAV penetration strategy based on deep reinforcement learning

Design complexities of trending UAVs and the operational harsh environments necessitates Control Law formulation utilizing intelligent techniques that are both robust, model-free and adaptable. In this research, an intelligent control architecture for an Unmanned Aerial Vehicle (UAV) having an unconventional inverted V-tail design, is presented. Due to unique design of the vehicle strong roll and yaw coupling exists, making the control of vehicle challenging. To handle UAV’s inherent control complexities, while keeping them computationally acceptable, a variant of distinct Deep Reinforcement Learning (DRL) algorithm, namely Deep Deterministic Policy Gradient (DDPG) is proposed. Conventional DDPG algorithm after being modified in its learning architecture becomes capable of intelligently handling the continuous state and control space domains besides controlling the platform in its entire flight regime. This stabilization and tracking controller for the UAV achieves the goal of an optimal flight path and exhibits satisfactory performance. The paper illustrates the application of modified DDPG algorithm towards the design, while the performance of the resulting controller is assessed in simulation using dynamic model of the vehicle. Nonlinear simulations were then performed to analyze UAV performance under different environmental and launch conditions. The effectiveness of the proposed strategy is further demonstrated by comparing the results with the linear controller for the same UAV whose feedback loop gains are optimized by employing technique of optimal control theory and another DRL method named Proximal Policy Optimization (PPO). Results indicate the significance of the proposed control architecture and its inherent capability to adapt dynamically to the changing environment, thereby making it of significance utility to airborne applications.

The increasing usage of Unmanned Aerial Vehicles (UAVs) in diverse applications necessitates the development of effective flight control systems. A major difficulty, nevertheless, is maintaining robust flight control under complex and dynamic environmental conditions, such as obstacle and non-flying zones, wind disturbances, and sensor noise. Traditional control techniques fail to achieve required flight control in these environments. To address this point, an adaptive control strategy is proposed based on a Deep Reinforcement Learning (DRL) model to enhance the flight performance of quadcopters. The learning and adaptation of the DRL-based control strategy are implemented in real-time through continuous interaction with the environment. This is to improve flight control and achieve consistent UAV performance under varying conditions. Proximal policy optimization with a reward function is used to minimize positional errors, ensure collision-free flight paths, and reduce energy consumption. The developed DRL model for quadcopters is trained in a simulated environment and then tested in three complex environmental scenarios, including urban, forest and mountain terrains. Experimental results demonstrate remarkable improvements in UAV flight performance. In the training phase, the reported training reward increased from 10 to 110 and the train loss is dropped from 0.85 to 0.05, which indicated successful model learning. Also, during system verification, rewards increased from 12 to 115 and UAV flight path deviations were decreased from 0.5 to 0.08m. The proposed controller outperforms conventional approaches in urban environments by lowering average trajectory deviations to 0.2m from 0.35m for MPC and 0.6m for PID. Also, the developed DRL-based controller outperformed the PID and MPC controllers, with path deviations of 0.18m in mountains and 0.12m in forests. In addition, fewer collision rates with obstacles are achieved with the model, 3% in forest, 1.8% in urban areas, and 4.5% in mountains. Furthermore, the consumed energy is reduced to 950J as compared to 1200J for PID and 1050J for MPC. The results show the strength of deploying the proposed controller in following the intended UAV flight path with high precision, effectively avoiding detected conflicts and minimizing consumed energy by the UAV.

: Unmanned aerial vehicles (UAVs) are critical components of the future naval forces. UAV control and monitoring with autonomous operation will become an absolute necessity and adaptive cooperation of vehicles is the only practical alternative. The objective of this project is to develop and evaluate new methodologies for cooperative (formation) control of multiple unmanned air vehicles. The goal is to have multiple UAVs working together as a group. Instead of separately assigning distinct tasks to each vehicle, the operator would assign tasks to the UAV group, which then determines the best way to accomplish each task, freeing the operator to maintain surveillance over the entire operation. In this project we investigated Path Tracking and obstacle avoidance of UAVs using fuzzy logic method. Algorithms for close formation control of multi-UAVs are developed and simulated. We also investigated fault-tolerant control of single UAVs by neuro-adaptive method. Detailed description of this method is provided in this document. The project has supported S graduate students with 9 technical papers published.

As a complicated multi-input, multi-output, and time-varying nonlinear system, flight control system of unmanned aerial vehicle (UAV), which determines the whole system’s performance directly, is crucial for the simulation training system design. This chapter mainly focuses on parameter identification of flight control system based on the modeling of UAVs and a specific controller design for the pendulum-like oscillation in micro aerial vehicle (MAV). A predator–prey particle swarm optimization (PSO) algorithm for identifying parameters of UAV flight control system is presented, with the aim of reducing the workload of the designers during the process of designing complicated UAV control system. Besides, a software environment for UAV controller design was developed based on the UAV model and the proposed method. Then a specific kind of controller design involving the pendulum-like oscillation in the hover and stare state for a MAV in the presence of external disturbances is discussed in detail, since the pendulum-like oscillation caused by uncertainty and external disturbances badly jeopardize the performance of hover and stare, resulting in blurred images or even MAV’s overturning. A novel type of PSO-based linear-quadratic regulator (LQR) controller for stabilizing the pendulum-like oscillation is developed, which can enhance the MAV’s performance efficiently.

Today, the use of small unmanned aerial vehicles (UAVs) has increased due to technological advances. There has been an interest in using multiple UAVs instead of a single UAV to accomplish a given mission. This is because there are many scenarios where the capabilities of a single UAV are inadequate due to certain constraints (battery capacity, time in the air). For this reason, swarm UAV studies have increased. A swarm UAV consists of a large number of UAVs cooperating to accomplish a specific mission. This study shows three-dimensional formation control of a swarm UAV system in an obstacle environment. A centralized control architecture is used in this process. All task assignments are made from a centralized system. The Artificial potential fields method creates the formation at the target point by avoiding obstacles. The study was carried out using the Robot Operating System (ROS). The methods were tested in Webots simulation environment. Crazyflie robots were used in the experiments. In the simulation environment, square, star and v formations were first tested in two dimensions. Then, as an example of three-dimensional formation, the cubic and pyramid formations were created and observed.

The unmanned aerial vehicle (UAV) has been applied in unmanned air combat because of its flexibility and practicality. The short-range air combat situation is rapidly changing, and the UAV has to make the autonomous maneuver decision as quickly as possible. In this paper, a type of short-range air combat maneuver decision method based on deep reinforcement learning is proposed. Firstly, the combat environment, including UAV motion model and the position and velocity relationships, is described. On this basic, the combat process is established. Secondly, some improved points based on proximal policy optimization (PPO) are proposed to enhance the maneuver decision-making ability. The gate recurrent unit (GRU) can help PPO make decisions with continuous timestep data. The actor network's input is the observation of UAV, however, the input of the critic network, named state, includes the blood values which cannot be observed directly. In addition, the action space with 15 basic actions and well-designed reward function are proposed to combine the air combat environment and PPO. In particular, the reward function is divided into dense reward, event reward and end-game reward to ensure the training feasibility. The training process is composed of three phases to shorten the training time. Finally, the designed maneuver decision method is verified through the ablation study and confrontment tests. The results show that the UAV with the proposed maneuver decision method can obtain an effective action policy to make a more flexible decision in air combat.

This study is aimed at the utilization of a video repeater for unmanned aerial vehicle (UAV) control, involving a comparative analysis of existing scientific literature, methodologies, and available solutions. Through comparative analysis, the method of using an external repeater emerged as the most promising, offering flexibility in selecting repeaters with superior performance and advanced capabilities. Notably, this approach allows for the utilization of a single repeater across multiple UAVs and facilitates convenient modification or upgrade of the repeater without necessitating alterations to the UAV itself. Following component selection, an experimental prototype was designed to facilitate empirical investigations. Additionally, a frequency transmission scheme was devised for quadcopter control employing the repeater. This research contributes to advancing UAV control systems by introducing a novel approach to video repeater integration, poised to enhance operational efficiency and adaptability in diverse settings. The findings from this study are anticipated to inform further developments in UAV technology, particularly in optimizing video transmission for improved situational awareness and mission effectiveness.

Hybrid modeling and control issues provide a flexible powerful tool for Unmanned Aerial Vehicles (UAVs) as complex control platforms. This paper aims to briefly review recent results in this area. Firstly, existing works on hybrid modeling and control of a single UAV are reviewed. Then multiple UAVs control in a synchronously cooperative task planning, which gives much more maneuverability, has been surveyed in the hybrid framework. Formation control of UAVs is a typical cooperative strategy with considerable applications in multi-agent aerial robotics area and can leverage limited abilities of single UAVs into complicated tasks of group missions. There are a few works in this field which we briefly reviewed their results and explored the possible future directions of these research activities.

This paper investigates a new scenario of spectrum sharing between unmanned\naerial vehicle (UAV) and terrestrial wireless communication, in which a\ncognitive/secondary UAV transmitter communicates with a ground secondary\nreceiver (SR), in the presence of a number of primary terrestrial communication\nlinks that operate over the same frequency band. We exploit the UAV's mobility\nin three-dimensional (3D) space to improve its cognitive communication\nperformance while controlling the co-channel interference at the primary\nreceivers (PRs), such that the received interference power at each PR is below\na prescribed threshold termed as interference temperature (IT). First, we\nconsider the quasi-stationary UAV scenario, where the UAV is placed at a static\nlocation during each communication period of interest. In this case, we jointly\noptimize the UAV's 3D placement and power control to maximize the SR's\nachievable rate, subject to the UAV's altitude and transmit power constraints,\nas well as a set of IT constraints at the PRs to protect their communications.\nNext, we consider the mobile UAV scenario, in which the UAV is dispatched to\nfly from an initial location to a final location within a given task period. We\npropose an efficient algorithm to maximize the SR's average achievable rate\nover this period by jointly optimizing the UAV's 3D trajectory and power\ncontrol, subject to the additional constraints on UAV's maximum flying speed\nand initial/final locations. Finally, numerical results are provided to\nevaluate the performance of the proposed designs for different scenarios, as\ncompared to various benchmark schemes. It is shown that in the quasi-stationary\nscenario the UAV should be placed at its minimum altitude while in the mobile\nscenario the UAV should adjust its altitude along with horizontal trajectory,\nso as to maximize the SR's achievable rate in both scenarios.\n

In this article simultaneous arrival of multiple unmanned aerial vehicle (UAV) at a common point was studied in a fixed network hierarchy is used. By considering the model of fixed wing UAV the decentralized based control algorithm is developed for the formation control of all UAV’s. The tree based networked geometry is developed for the formation control of multiple UAV’s with respect to the leader. Whereas, UAV’s agent or (leader / follower) are autonomously connected with each other. Formation of flight via decentralized control structure which contains two parts, one is to achieve the arrival of expected meet-point for the formation geometry and other is to maintain the synchronization of all UAV’s. Moreover, the leader is supposed and able to regulate the speed or velocity of UAV’s. Simulated results showed that arrival of multiple UAV’s at a common point for the formation is achieved successfully and the above designed control algorithm has good robustness and flexible.

This paper proposes a hybrid Unmanned Aerial Vehicle (UAV) path planning method that combines the Rapidly-exploring Random Tree (RRT) algorithm with Proximal Policy Optimization (PPO). The proposed method aims to enhance the efficiency and adaptability of UAV path planning in complex and dynamic environments. The RRT algorithm excels at quickly generating a feasible path from a start point to a goal point. However, the quality of its paths is often suboptimal, and it lacks adaptability in dynamic settings. In contrast, PPO, a deep reinforcement learning algorithm, optimizes paths through iterative policy updates, enabling the UAV to adapt to environmental changes. Our approach first employs RRT to generate an initial path, which is subsequently refined by PPO to improve smoothness and adaptability. Experimental results demonstrate that the RRT-PPO hybrid method performs favorably in terms of path length, computational time, and obstacle avoidance capability, effectively improving the task completion efficiency of UAVs in complex environments.

The object recognition technology of unmanned aerial vehicles (UAVs) equipped with “You Only Look Once” (YOLO) has been validated in actual flights. However, here, the challenge lies in efficiently utilizing camera gimbal control technology to swiftly capture images of YOLO-identified target objects in aerial search missions. Enhancing the UAV’s energy efficiency and search effectiveness is imperative. This study aims to establish a simulation environment by employing the Unity simulation software for target tracking by controlling the gimbal. This approach involves the development of deep deterministic policy-gradient (DDPG) reinforcement-learning techniques to train the gimbal in executing effective tracking actions. The outcomes of the simulations indicate that when actions are appropriately rewarded or penalized in the form of scores, the reward value can be consistently converged within the range of 19–35. This convergence implies that a successful strategy leads to consistently high rewards. Consequently, a refined set of training procedures is devised, enabling the gimbal to accurately track the target. Moreover, this strategy minimizes unnecessary tracking actions, thus enhancing tracking efficiency. Numerous benefits arise from training in a simulated environment. For instance, the training in this simulated environment is facilitated through a dataset composed of actual flight photographs. Furthermore, offline operations can be conducted at any given time without any constraint of time and space. Thus, this approach effectively enables the training and enhancement of the gimbal’s action strategies. The findings of this study demonstrate that a coherent set of action strategies can be proficiently cultivated by employing DDPG reinforcement learning. Furthermore, these strategies empower the UAV’s gimbal to rapidly and precisely track designated targets. Therefore, this approach provides both convenience and opportunities to gather more flight-scenario training data in the future. This gathering of data will lead to immediate training opportunities and help improve the system’s energy consumption.

Manipulating objects using arms mounted to unmanned aerial vehicles (UAVs) is attractive because UAVs may access many locations that are otherwise inaccessible to traditional mobile manipulation platforms such as ground vehicles. Historically, UAVs have been employed in ways that avoid interaction with the environment at all costs. The recent trend of increasing small UAV lift capacity and the reduction of the weight of manipulator components make the realization of mobile manipulating UAVs imminent. Despite recent work, several major challenges remain to be overcome before it will be common practice to manipulate objects from UAVs. Among these challenges, the constantly moving UAV platform and compliance of manipulator arms make it difficult to position the UAV and end-effector relative to an object of interest precisely enough for reliable manipulation. Solving this challenge will bring UAVs one step closer to being able to perform meaningful tasks such as infrastructure repair, disaster response, law enforcement, and personal assistance. Toward a solution to this challenge, this thesis describes a way forward that uses the UAV as a means to crudely position a manipulator within reach of the end-effector's goal position in the world. The manipulator then performs the fine positioning of the end-effector, rejecting position perturbations caused by UAV motions. An algorithm to coordinate the redundant degrees of freedom of an aerial manipulation system is described that allows the motions of the manipulator to serve as inputs to the UAV's position controller. To demonstrate this algorithm, the manipulator's six degrees of freedom are servoed using visual sensing to drive an eye-in-hand camera to a specified pose relative to a target while treating motions of the host platform as perturbations. Simultaneously, the host platform's degrees of freedom are regulated using kinematic information from the manipulator. This ultimately drives the UAV to a position that allows the manipulator to assume a pose relative to the UAV that maximizes reachability, thus facilitating the arm's ability to compensate for undesired UAV motions. Maintaining this loose kinematic coupling between the redundant degrees of freedom of the host UAV and manipulator allows this type of controller to be applied to a wide variety of platforms, including manned aircraft, rather than a single instance of a purpose-built system. As a result of this loose coupling, careful consideration must be given to the manipulator design so that it can achieve useful poses while minimally influencing the stability of the host UAV. Accordingly, the novel application of a parallel manipulator mechanism is described.

Instruction sender authentication of UAV (Unmanned Aerial Vehicle) control is critical for the secure operation of UAV systems. By exploring user touch action characteristics of the user interaction with the UAV control mobile terminal, this paper proposes a novel instruction sender continuous authentication framework for UAV control systems. In particular, we first obtain the touch screen data in real time from the interaction with the mobile device screen of a user when he controls the UAV. Then we create a polynomial to fit the touch trajectory on the screen of the user, and use the least square method to obtain the optimal estimate of the polynomial coefficients. Finally, we mark the grid area covered by the polynomial to extract the user’s touch screen operation action fingerprint(OAF), and employ the fuzzy extractor method to realize the identity authentication of the sender with certain error tolerance. Extensive experiments are conducted to illustrate the authentication performance of the proposed authentication framework in terms of false acceptance rate, false rejection rate and equal-error rate.