A Quadratic Programming Based Neural Dynamic Controller and Its Application to UAVs for Time-Varying Tasks

: Unmanned aerial vehicles (UAVs) are critical components of the future naval forces. UAV control and monitoring with autonomous operation will become an absolute necessity and adaptive cooperation of vehicles is the only practical alternative. The objective of this project is to develop and evaluate new methodologies for cooperative (formation) control of multiple unmanned air vehicles. The goal is to have multiple UAVs working together as a group. Instead of separately assigning distinct tasks to each vehicle, the operator would assign tasks to the UAV group, which then determines the best way to accomplish each task, freeing the operator to maintain surveillance over the entire operation. In this project we investigated Path Tracking and obstacle avoidance of UAVs using fuzzy logic method. Algorithms for close formation control of multi-UAVs are developed and simulated. We also investigated fault-tolerant control of single UAVs by neuro-adaptive method. Detailed description of this method is provided in this document. The project has supported S graduate students with 9 technical papers published.

As a complicated multi-input, multi-output, and time-varying nonlinear system, flight control system of unmanned aerial vehicle (UAV), which determines the whole system’s performance directly, is crucial for the simulation training system design. This chapter mainly focuses on parameter identification of flight control system based on the modeling of UAVs and a specific controller design for the pendulum-like oscillation in micro aerial vehicle (MAV). A predator–prey particle swarm optimization (PSO) algorithm for identifying parameters of UAV flight control system is presented, with the aim of reducing the workload of the designers during the process of designing complicated UAV control system. Besides, a software environment for UAV controller design was developed based on the UAV model and the proposed method. Then a specific kind of controller design involving the pendulum-like oscillation in the hover and stare state for a MAV in the presence of external disturbances is discussed in detail, since the pendulum-like oscillation caused by uncertainty and external disturbances badly jeopardize the performance of hover and stare, resulting in blurred images or even MAV’s overturning. A novel type of PSO-based linear-quadratic regulator (LQR) controller for stabilizing the pendulum-like oscillation is developed, which can enhance the MAV’s performance efficiently.

Continuous trajectory control of fixed-wing unmanned aerial vehicles (UAVs) is complicated when considering hidden dynamics. Due to UAV multi degrees of freedom, tracking methodologies based on conventional control theory, such as Proportional-Integral-Derivative (PID) has limitations in response time and adjustment robustness, while a model based approach that calculates the force and torques based on UAV's current status is complicated and rigid. We present an actor-critic reinforcement learning framework that controls UAV trajectory through a set of desired waypoints. A deep neural network is constructed to learn the optimal tracking policy and reinforcement learning is developed to optimize the resulting tracking scheme. The experimental results show that our proposed approach can achieve 58.14% less position error, 21.77% less system power consumption and 9.23% faster attainment than the baseline. The actor network consists of only linear operations, hence Field Programmable Gate Arrays (FPGA) based hardware acceleration can easily be designed for energy efficient real-time control.

Unmanned Aerial Vehicles (UAV), such as quadrotors or fixed-wing aircraft, are increasingly common in airspace between 100 and 2000 meters. Extending current aviation infrastructure management operations, such as flight plan approval or aircraft tracking, to include UAV systems is difficult for multiple reasons, such as the technical challenge of tracking objects that frequently travel below the radar floor. This work explores the feasibility of using cellular data modems to achieve low-cost UAV tracking solutions for small consumer UAVs, such as *rotors, which typically have a very low-altitude flight path. We also explore the altitude at which the data modem functionality diminishes. We present a UAV tracking architecture and web service interface, as well as the empirical evidence gathered from testing the real-time capabilities and limitations of such a system.

Recent advancements in visual object tracking have markedly improved the capabilities of unmanned aerial vehicle (UAV) tracking, which is a critical component in real-world robotics applications. While the integration of hierarchical lightweight networks has become a prevalent strategy for enhancing efficiency in UAV tracking, it often results in a significant drop in network capacity, which further exacerbates challenges in UAV scenarios, such as frequent occlusions and extreme changes in viewing angles. To address these issues, we introduce a novel family of UAV trackers, termed CGTrack, which combines explicit and implicit techniques to expand network capacity within a coarse-to-fine framework. Specifically, we first introduce a Hierarchical Feature Cascade (HFC) module that leverages the spirit of feature reuse to increase network capacity by integrating the deep semantic cues with the rich spatial information, incurring minimal computational costs while enhancing feature representation. Based on this, we design a novel Lightweight Gated Center Head (LGCH) that utilizes gating mechanisms to decouple target-oriented coordinates from previously expanded features, which contain dense local discriminative information. Extensive experiments on three challenging UAV tracking benchmarks demonstrate that CGTrack achieves state-of-the-art performance while running fast. Code will be available at https://github.com/Nightwatch-Fox11/CGTrack.

Unmanned aerial vehicle (UAV) tracking is of great significance for a wide range of applications, such as delivery and agriculture. Previous benchmarks in this area mainly focused on small-scale tracking problems while ignoring the amounts of data, types of data modalities, diversities of target categories and scenarios, and evaluation protocols involved, greatly hiding the massive power of deep UAV tracking. In this article, we propose WebUAV-3M, the largest public UAV tracking benchmark to date, to facilitate both the development and evaluation of deep UAV trackers. WebUAV-3M contains over 3.3 million frames across 4,500 videos and offers 223 highly diverse target categories. Each video is densely annotated with bounding boxes by an efficient and scalable semi-automatic target annotation (SATA) pipeline. Importantly, to take advantage of the complementary superiority of language and audio, we enrich WebUAV-3M by innovatively providing both natural language specifications and audio descriptions. We believe that such additions will greatly boost future research in terms of exploring language features and audio cues for multi-modal UAV tracking. In addition, a fine-grained UAV tracking-under-scenario constraint (UTUSC) evaluation protocol and seven challenging scenario subtest sets are constructed to enable the community to develop, adapt and evaluate various types of advanced trackers. We provide extensive evaluations and detailed analyses of 43 representative trackers and envision future research directions in the field of deep UAV tracking and beyond. The dataset, toolkits, and baseline results are available at https://github.com/983632847/WebUAV-3M.

The aim of this article was to discuss the modeling and control method of quadrotor unmanned aerial vehicle (UAV). In the process of modeling, mechanism modeling and experimental testing are combined, especially the motor and propeller are modeled in detail. Through the understanding of the body structure and flight principle of the quadrotor UAV, the Newton–Euler method is used to analyze the dynamics of the quadrotor UAV, and the mathematical model of the UAV is established under the small angle rotation. Process identifier (PID) is used to control it. First, the attitude angle of the model is controlled by PID, and based on this, the speed in each direction is controlled by PID. Then, the PID control of the four rotor aircraft with the center of gravity offset is simulated by MATLAB. The results show that the pitch angle and roll angle can be controlled by 5 degrees together without center of gravity deviation, and the PID can effectively control the control quantity and achieve the desired effect in a short time. Classical BP algorithm, classical GA-BP algorithm, and improved GA-BP algorithm were trained, respectively, with a total of 150 sets of training data, training function uses Levenberg-Marquardt (trainlm), and performance function uses mean squared error (MSE). In the background of the same noise, the improved GA-BP algorithm has the highest detection rate, classical GA-BP algorithm is the second, and classical BP algorithm is the worst. The simulation results show that the PID control law can effectively control the attitude angle and speed of the rotor UAV in the case of center of gravity deviation.

In this paper, a novel control strategy based on model predictive control is proposed for tracking the trajectory of unmanned aerial vehicles (UAV). First, the dynamic model of an unmanned aerial vehicle UAV is established by considering the linear velocities in terms of the analyzed system’s attitude angles. The UAV’s dynamic model is suitable for kinematic control by using two attitude angles as control inputs. The controller design is made up of an output feedback controller formed by the error variables, which are the difference between the measured output and the reference variables. Then, a model predictive control is established by considering an optimization problem with a performance index and the corresponding constraints. Furthermore, the stochastic properties of the UAV dynamic model are taken into account in this research study. The measured outputs provided by the sensors are constrained by the topological characteristics of the unmanned aerial vehicle to be analyzed. Finally, a numerical experiment and its conclusions are presented, demonstrating that the proposed control strategy provides optimal trajectory tracking.

Neural networks (NNs) are now being extensively utilized in various artificial intelligence platforms specifically in the area of image classification and real-time object tracking. We propose a novel design to address the problem of real-time unmanned aerial vehicle (UAV) monitoring and detection using a Zynq UltraScale FPGA-based convolutional neural network (CNN). The biggest challenge while implementing real-time algorithms on FPGAs is the limited DSP hardware resources available on FPGA platforms. Our proposed design overcomes the challenge of autonomous real-time UAV detection and tracking using a Xilinx’s Zynq UltraScale XCZU9EG system on a chip (SoC) platform. Our proposed design explores and provides a solution for overcoming the challenge of limited floating-point resources while maintaining real-time performance. The solution consists of two modules: UAV tracking module and neural network–based UAV detection module. The tracking module uses our novel background-differencing algorithm, while the UAV detection is based on a modified CNN algorithm, designed to give the maximum field-programmable gate array (FPGA) performance. These two modules are designed to complement each other and enabled simultaneously to provide an enhanced real-time UAV detection for any given video input. The proposed system has been tested on real-life flying UAVs, achieving an accuracy of 82%, running at the full frame rate of the input camera for both tracking and neural network (NN) detection, achieving similar performance than an equivalent deep learning processor unit (DPU) with UltraScale FPGA-based HD video and tracking implementation but with lower resource utilization as shown by our results.

With the rapid development in the field of computer vision, the vision-based approach to unmanned aerial vehicle (UAV) tracking and landing technology in weak global positioning system (GPS) or GPS-free environments has become prominent in military and civilian missions. However, this technique still suffers from problems such as interference by similar targets in the environment, low tracking accuracy, slow processing speed, and poor stability. To solve these problems, we propose the designated target anti-interference tracking (DTAT) method, which integrates YOLOv5 and SiamRPN, and built a system to achieve UAV tracking and the landing of a designated target in an environment with multiple interference targets. The system consists of the following parts: first, an image is acquired by a monocular camera to obtain the pixel position information of the designated target. Next, the position of the UAV relative to the target is estimated based on the pixel location information of the target and the known target size information. Finally, the discrete proportion integration differentiation (PID) control law is used to complete the target tracking and landing task of the UAV. To test the system performance, we deployed it on a robot operating system (ROS) platform, conducted many simulation experiments, and observed the real-time trajectories of the UAV and the target through Gazebo software. The results show that the relative distance between the UAV and the target during the tracking process when the target was moving at 0.6 m/s does not exceed 0.8 m, and the landing error of the UAV during the landing process after the target is stationary does not exceed 0.01 m. The results validate the effectiveness and robustness of the system and lay a foundation for subsequent research.

As a significant research direction in remote sensing fields, unmanned aerial vehicles (UAVs) tracking has achieved rapid development in recent years. However, due to limited power and computation resources on aerial platforms, the tracking methods deployed on UAVs usually require high computational efficiency and performance. In addition, various challenges (i.e., similar object, background clutter and occlusion) have inevitably occurred during the UAV tracking phase. Therefore, considering the above issues comprehensively, this paper proposes a dynamic aberrance-repressed temporal regularized correlation filter to achieve stable tracking in UAV remote sensing videos. First, we have introduced the aberrance-repressed temporal regularizations into the discriminative correlation filter (DCF) framework. Second, a novel objective loss function is constructed to adjust the strength of each regularization for training the filter. Then, a new judgment mechanism based on the response variation is exploited to reflect the response fluctuation and applied to tune parameters of both regularizations. Finally, comprehensive experiments are done on three different UAV benchmarks, i.e., UAV123@10fps, UAVDT and VisDrone2018, to verify the performance of our tracker and have demonstrated that our tracker achieves superior performance against other total 25 state-of-the-art trackers while reaching <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 35 FPS on a single CPU.

In order to track the time-varying trajectory efficiently and accurately, a multilayer neural dynamic controller is proposed and exploited for quadrotor unmanned aerial vehicles (UAVs). Previous UAV controllers based on neural dynamic methods used forces and torques as control variables. However, in fact, the real control variables of a UAV are the PWM waves for controlling motors. What is more, the motors are always modeled as one-order systems since the existence of inertia and friction, which means that using forces and torques as control variables is not proper and may lead to low precision and inefficient tracking for UAVs. To solve this problem, a new UAV controller which uses exactly what inputs to motors as control variables is designed, so that the designed controller is more practical for UAVs. In the design process, a multilayer neural dynamic controller is obtained by applying the neural dynamic method to position layer, velocity, torque layer and motor layer successively. Both theoretical analysis and computer simulation results verify the effectiveness, convergence, stability and accuracy of the proposed multilayer neural dynamic controller for tracking time-varying trajectories.

Aiming at the time-varying and nonlinear problems of fixed-wing unmanned aerial vehicle (UAV) system, a UAV flight control algorithm was proposed based on PID control algorithm and the adaptive genetic algorithm (GA). On the basis of the traditional algorithm, the settling time was considered as the influencing factors of the objective function. The optimal index was determined by using the overshoot, rise time and settling time of the system response as variables, and adjusting the corresponding weight according to the actual situation to change the system response result. Moreover, the crossover probability and mutation probability in GA could be adaptively changed, so that the optimal parameters in the PID control process could be quickly and accurately found. Based on the rigid body dynamics model and kinematic model of the fixed-wing UAV, the simulation models of five control loops were designed in MATLAB/Simulink, namely the pitch angle, roll angle, yaw angle, speed and height control loops. Compared with the traditional PID controller, the system responses of the five control loops were significantly improved with faster response speed, and less than 5% overshoot. The proposed algorithm had strong adaptability and anti-interference ability.

Siamese-based trackers have shown excellent performance in unmanned aerial vehicle (UAV) tracking due to their balanced accuracy and speed. However, most existing methods rely heavily on local convolution operations of CNNs for feature extraction, which fails to establish global feature relationships. In addition, the challenge of information imbalance, caused by differences in background and the prevalence of single-object scenarios, remains a significant obstacle in the current domain. This can potentially result in unwanted boundary effects in UAV tracking. To address these challenges, this article introduces a novel tracker called eccentric loss and adaptive search (ELAS), which is specifically designed for UAV videos and aims to overcome these challenges. The proposed ELAS enhances feature representation capabilities by utilizing multi-input and multimapping of information. It also addresses information imbalance through a differentiated loss function. Furthermore, an adaptive search strategy is introduced to accommodate varying search scales across different scenarios. Specifically, an additional multilayer feature enhancement (MLE) network is first introduced, which absorbs the middle and high layer information in the backbone network and directs the features to focus on global connections using self-attention, avoiding the loss of useful semantic information due to falling into local optima. Then, a novel eccentric loss is designed to guide network training, enhancing the saliency of object features on the feature map by accounting for the volatility of negative samples and estimating classification performance. Finally, an adaptive multiscale search strategy is proposed to detect the saliency of objects through between-class segmentation effectiveness and, thus, adjust the proportional scale parameter. Extensive experiments on challenging UAV benchmarks demonstrate that ELAS achieves state-of-the-art (SOTA) performance with real-time speed, confirming its effectiveness and robustness.

Affected by the mutation of runway condition, crosswind, unbalance load and different performance of brake device on both sides of the wheels, there will generally appear yaw angle and lateral offset while the unmanned aerial vehicle (UAV) is taxiing. The UAV would skid off the runway and cause serious accident if the yaw angle and lateral offset are not corrected in time. In this paper, a dynamic allocation algorithm for the gain of UAV nose wheel steering and differential braking based on decomposition control is proposed to correct these departures and make the UAV turning back to the center of runway. By this algorithm, the route following control of UAV is achieved.