Robust unmanned aerial vehicles tracking amid electronic interference utilizing auxiliary particle filtering

Recent advancements in visual object tracking have markedly improved the capabilities of unmanned aerial vehicle (UAV) tracking, which is a critical component in real-world robotics applications. While the integration of hierarchical lightweight networks has become a prevalent strategy for enhancing efficiency in UAV tracking, it often results in a significant drop in network capacity, which further exacerbates challenges in UAV scenarios, such as frequent occlusions and extreme changes in viewing angles. To address these issues, we introduce a novel family of UAV trackers, termed CGTrack, which combines explicit and implicit techniques to expand network capacity within a coarse-to-fine framework. Specifically, we first introduce a Hierarchical Feature Cascade (HFC) module that leverages the spirit of feature reuse to increase network capacity by integrating the deep semantic cues with the rich spatial information, incurring minimal computational costs while enhancing feature representation. Based on this, we design a novel Lightweight Gated Center Head (LGCH) that utilizes gating mechanisms to decouple target-oriented coordinates from previously expanded features, which contain dense local discriminative information. Extensive experiments on three challenging UAV tracking benchmarks demonstrate that CGTrack achieves state-of-the-art performance while running fast. Code will be available at https://github.com/Nightwatch-Fox11/CGTrack.

Unmanned aerial vehicle (UAV) tracking is of great significance for a wide range of applications, such as delivery and agriculture. Previous benchmarks in this area mainly focused on small-scale tracking problems while ignoring the amounts of data, types of data modalities, diversities of target categories and scenarios, and evaluation protocols involved, greatly hiding the massive power of deep UAV tracking. In this article, we propose WebUAV-3M, the largest public UAV tracking benchmark to date, to facilitate both the development and evaluation of deep UAV trackers. WebUAV-3M contains over 3.3 million frames across 4,500 videos and offers 223 highly diverse target categories. Each video is densely annotated with bounding boxes by an efficient and scalable semi-automatic target annotation (SATA) pipeline. Importantly, to take advantage of the complementary superiority of language and audio, we enrich WebUAV-3M by innovatively providing both natural language specifications and audio descriptions. We believe that such additions will greatly boost future research in terms of exploring language features and audio cues for multi-modal UAV tracking. In addition, a fine-grained UAV tracking-under-scenario constraint (UTUSC) evaluation protocol and seven challenging scenario subtest sets are constructed to enable the community to develop, adapt and evaluate various types of advanced trackers. We provide extensive evaluations and detailed analyses of 43 representative trackers and envision future research directions in the field of deep UAV tracking and beyond. The dataset, toolkits, and baseline results are available at https://github.com/983632847/WebUAV-3M.

This paper proposes two new particle filters, namely, the auxiliary extended Kalman particle filter (AEKPF) and the auxiliary unscented Kalman particle filter (AUKPF). The theory governing the newly proposed filtering techniques is developed and the algorithms are described and contrasted. Next, a series of tests is presented in which the new filters are compared against the extended Kalman filter (EKF), the unscented Kalman filter (UKF), and several existing particle filters. The test results are from simulations with synthetic mathematical models that incorporate elements that are nonlinear, non-stationary, and stochastic. Performance results are presented for various degrees of model nonlinearity including first, second, and third order systems. Furthermore, experimental results are also reported comparing the filters performances with different signal to noise ratios and noise models, including Gaussian, Cauchy, and Gamma distributions. Various metrics are used to compare the filters performances and to make conclusions about future work. It is shown to be advantageous to use certain particle filters depending on the noise distribution of the system of interest. In particular, the AUKPF and the AEKPF outperform existing particle filters in many cases.

With the rapid development in the field of computer vision, the vision-based approach to unmanned aerial vehicle (UAV) tracking and landing technology in weak global positioning system (GPS) or GPS-free environments has become prominent in military and civilian missions. However, this technique still suffers from problems such as interference by similar targets in the environment, low tracking accuracy, slow processing speed, and poor stability. To solve these problems, we propose the designated target anti-interference tracking (DTAT) method, which integrates YOLOv5 and SiamRPN, and built a system to achieve UAV tracking and the landing of a designated target in an environment with multiple interference targets. The system consists of the following parts: first, an image is acquired by a monocular camera to obtain the pixel position information of the designated target. Next, the position of the UAV relative to the target is estimated based on the pixel location information of the target and the known target size information. Finally, the discrete proportion integration differentiation (PID) control law is used to complete the target tracking and landing task of the UAV. To test the system performance, we deployed it on a robot operating system (ROS) platform, conducted many simulation experiments, and observed the real-time trajectories of the UAV and the target through Gazebo software. The results show that the relative distance between the UAV and the target during the tracking process when the target was moving at 0.6 m/s does not exceed 0.8 m, and the landing error of the UAV during the landing process after the target is stationary does not exceed 0.01 m. The results validate the effectiveness and robustness of the system and lay a foundation for subsequent research.

As a significant research direction in remote sensing fields, unmanned aerial vehicles (UAVs) tracking has achieved rapid development in recent years. However, due to limited power and computation resources on aerial platforms, the tracking methods deployed on UAVs usually require high computational efficiency and performance. In addition, various challenges (i.e., similar object, background clutter and occlusion) have inevitably occurred during the UAV tracking phase. Therefore, considering the above issues comprehensively, this paper proposes a dynamic aberrance-repressed temporal regularized correlation filter to achieve stable tracking in UAV remote sensing videos. First, we have introduced the aberrance-repressed temporal regularizations into the discriminative correlation filter (DCF) framework. Second, a novel objective loss function is constructed to adjust the strength of each regularization for training the filter. Then, a new judgment mechanism based on the response variation is exploited to reflect the response fluctuation and applied to tune parameters of both regularizations. Finally, comprehensive experiments are done on three different UAV benchmarks, i.e., UAV123@10fps, UAVDT and VisDrone2018, to verify the performance of our tracker and have demonstrated that our tracker achieves superior performance against other total 25 state-of-the-art trackers while reaching <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 35 FPS on a single CPU.

Siamese-based trackers have shown excellent performance in unmanned aerial vehicle (UAV) tracking due to their balanced accuracy and speed. However, most existing methods rely heavily on local convolution operations of CNNs for feature extraction, which fails to establish global feature relationships. In addition, the challenge of information imbalance, caused by differences in background and the prevalence of single-object scenarios, remains a significant obstacle in the current domain. This can potentially result in unwanted boundary effects in UAV tracking. To address these challenges, this article introduces a novel tracker called eccentric loss and adaptive search (ELAS), which is specifically designed for UAV videos and aims to overcome these challenges. The proposed ELAS enhances feature representation capabilities by utilizing multi-input and multimapping of information. It also addresses information imbalance through a differentiated loss function. Furthermore, an adaptive search strategy is introduced to accommodate varying search scales across different scenarios. Specifically, an additional multilayer feature enhancement (MLE) network is first introduced, which absorbs the middle and high layer information in the backbone network and directs the features to focus on global connections using self-attention, avoiding the loss of useful semantic information due to falling into local optima. Then, a novel eccentric loss is designed to guide network training, enhancing the saliency of object features on the feature map by accounting for the volatility of negative samples and estimating classification performance. Finally, an adaptive multiscale search strategy is proposed to detect the saliency of objects through between-class segmentation effectiveness and, thus, adjust the proportional scale parameter. Extensive experiments on challenging UAV benchmarks demonstrate that ELAS achieves state-of-the-art (SOTA) performance with real-time speed, confirming its effectiveness and robustness.

This paper proposes a novel particle filter, namely, the auxiliary iterated extended Kalman particle filter (AIEKPF). To generate the importance density, based on the auxiliary particle filtering (APF) technique the proposed filter uses the iterated extended Kalman filter (IEKF) to integrate the latest measurements into state transition density. This new filter can match the posterior density well, because of the robustness of the APF and the importance density generated by the IEKF. The performance of the presented particle filter is evaluated by two different estimation problems with the noise of Gaussian distribution and Gamma distribution, respectively. The experimental results illustrate that the AIEKPF is superior to the extended Kalman filter and some existing particle filters, such as the standard particle filter (PF), the extended Kalman particle filter, the unscented Kalman particle filter (UKPF) and the auxiliary extended Kalman particle filter, where the number of particles is relatively small, such as 200 and 1,000. However, with an increase of particles, the superiority of the proposed method may decline compared with the PF and APF as showed in the experiments. Also, the AIEKPF has less running time than the UKPF under the same conditions, and from the viewpoint of the average effective sample sizes, it is clear that the AIEKPF has the slightest degeneracy in all filters presented in the experiments.

Recently, correlation filter (CF)-based tracking algorithms have attained extensive interest in the field of unmanned aerial vehicle (UAV) tracking. Nonetheless, existing trackers still struggle with selecting suitable features and alleviating the model drift issue for online UAV tracking. In this paper, a robust CF-based tracker with feature integration and response map enhancement is proposed. Concretely, we develop a novel feature integration method that comprehensively describes the target by leveraging auxiliary gradient information extracted from the binary representation. Subsequently, the integrated features are utilized to learn a background-aware correlation filter (BACF) for generating a response map that implies the target location. To mitigate the risk of model drift, we introduce saliency awareness in the BACF framework and further propose an adaptive response fusion strategy to enhance the discriminating capability of the response map. Moreover, a dynamic model update mechanism is designed to prevent filter contamination and maintain tracking stability. Experiments on three public benchmarks verify that the proposed tracker outperforms several state-of-the-art algorithms and achieves a real-time tracking speed, which can be applied in UAV tracking scenarios efficiently.

Neural networks (NNs) are now being extensively utilized in various artificial intelligence platforms specifically in the area of image classification and real-time object tracking. We propose a novel design to address the problem of real-time unmanned aerial vehicle (UAV) monitoring and detection using a Zynq UltraScale FPGA-based convolutional neural network (CNN). The biggest challenge while implementing real-time algorithms on FPGAs is the limited DSP hardware resources available on FPGA platforms. Our proposed design overcomes the challenge of autonomous real-time UAV detection and tracking using a Xilinx’s Zynq UltraScale XCZU9EG system on a chip (SoC) platform. Our proposed design explores and provides a solution for overcoming the challenge of limited floating-point resources while maintaining real-time performance. The solution consists of two modules: UAV tracking module and neural network–based UAV detection module. The tracking module uses our novel background-differencing algorithm, while the UAV detection is based on a modified CNN algorithm, designed to give the maximum field-programmable gate array (FPGA) performance. These two modules are designed to complement each other and enabled simultaneously to provide an enhanced real-time UAV detection for any given video input. The proposed system has been tested on real-life flying UAVs, achieving an accuracy of 82%, running at the full frame rate of the input camera for both tracking and neural network (NN) detection, achieving similar performance than an equivalent deep learning processor unit (DPU) with UltraScale FPGA-based HD video and tracking implementation but with lower resource utilization as shown by our results.

Real-time object tracking for unmanned aerial vehicles (UAVs) is an essential and challenging research topic for computer vision. However, the scenarios that UAVs deal with are complicated, and the UAV tracking targets are small. Therefore, general trackers often fail to take full advantage of their performance in UAV scenarios. In this paper, we propose a tracking method for UAV scenes, which utilizes background cues and aberrances response suppression mechanism to track in 4 degrees-of-freedom. Firstly, we consider the tracking task as a similarity measurement problem. In this study, we decompose this problem into two subproblems for optimization. Secondly, to alleviate the problem of small targets in UAV scenes, we utilize background cues fully. Also, to reduce interference by background information, we employ an aberrance response suppression mechanism. Then, to obtain accurate target state information, we introduce a logarithmic polar coordinate system. We perform phase correlation calculations in logarithmic polar coordinates to obtain the rotation and scale changes of the target. Finally, target states are obtained through response fusion, which includes displacement, scale, and rotation angle. Our approach is carried out in a large number of experiments on various UAV datasets, such as UAV123, DBT70, and UAVDT2019. Compared with the current advanced trackers, our method is superior on UAV small target tracking.

Although the performance of unmanned aerial vehicle (UAV) tracking has benefited from the successful application of discriminative correlation filters (DCF) and convolutional neural networks (CNNs), UAV tracking under occlusion and deformation remains a challenge. The main dilemma is that challenging scenes, such as occlusion or deformation, are very complex and changeable, making it difficult to obtain training data covering all situations, resulting in trained networks that may be confused by new contexts that differ from historical information. Data-driven strategies are the main direction of current solutions, but gathering large-scale datasets with object instances under various occlusion and deformation conditions is difficult and lacks diversity. This paper proposes an attention-based mask generation network (AMGN) for UAV-specific tracking, which combines the attention mechanism and adversarial learning to improve the tracker’s ability to handle occlusion and deformation. After the base CNN extracts the deep features of the candidate region, a series of masks are determined by the spatial attention module and sent to the generator, and the generator discards some features according to these masks to simulate the occlusion and deformation of the object, producing more hard positive samples. The discriminator seeks to distinguish these hard positive samples while guiding mask generation. Such adversarial learning can effectively complement occluded and deformable positive samples in the feature space, allowing to capture more robust features to distinguish objects from backgrounds. Comparative experiments show that our AMGN-based tracker achieves the highest area under curve (AUC) of 0.490 and 0.349, and the highest precision scores of 0.742 and 0.662, on the UAV123 tracking benchmark with partial and full occlusion attributes, respectively. It also achieves the highest AUC of 0.555 and the highest precision score of 0.797 on the DTB70 tracking benchmark with the deformation attribute. On the UAVDT tracking benchmark with the large occlusion attribute, it achieves the highest AUC of 0.407 and the highest precision score of 0.582.

In this work, we propose a new ground moving target indicator (GMTI) radar based ground vehicle tracking method which exploits domain knowledge. Multiple state models are considered and a Monte-Carlo sampling based algorithm is preferred due to the manoeuvring of the ground vehicle and the non-linearity of the GMTI measurement model. Unlike the commonly used algorithms such as the interacting multiple model particle filter (IMMPF) and bootstrap multiple model particle filter (BS-MMPF), we propose a new algorithm integrating the more efficient auxiliary particle filter (APF) into a Bayesian framework. Moreover, since the movement of the ground vehicle is likely to be constrained by the road, this information is taken as the domain knowledge and applied together with the tracking algorithm for improving the tracking performance. Simulations are presented to show the advantages of both the new algorithm and incorporation of the road information by evaluating the root mean square error (RMSE).

Continuous trajectory control of fixed-wing unmanned aerial vehicles (UAVs) is complicated when considering hidden dynamics. Due to UAV multi degrees of freedom, tracking methodologies based on conventional control theory, such as Proportional-Integral-Derivative (PID) has limitations in response time and adjustment robustness, while a model based approach that calculates the force and torques based on UAV's current status is complicated and rigid. We present an actor-critic reinforcement learning framework that controls UAV trajectory through a set of desired waypoints. A deep neural network is constructed to learn the optimal tracking policy and reinforcement learning is developed to optimize the resulting tracking scheme. The experimental results show that our proposed approach can achieve 58.14% less position error, 21.77% less system power consumption and 9.23% faster attainment than the baseline. The actor network consists of only linear operations, hence Field Programmable Gate Arrays (FPGA) based hardware acceleration can easily be designed for energy efficient real-time control.

A quadratic programming based neural dynamic (QPND) controller design method is proposed for time-varying trajectory tracking of unmanned aerial vehicles (UAVs). Different from the traditional controller frameworks which only use pitch and roll angles to achieve the horizontal movement of the UAV, all the three attitude angles, i.e., roll, pitch and yaw angles, are applied for horizontal moving in this paper. There are at least three great superiorities of the proposed QPND controller. Firstly, three attitude angles can be coordinated control since the quadratic programming framework incorporates three attitudes. Secondly, the new framework enables the UAV to achieve the additional performance criteria or to perform subtasks. Third, the new controller can track time-varying trajectories due to neural dynamic approach. Specifically, the design process of the controller is divided into position controller design, attitude controller design and quadratic programming formulation. Finally, simulation experiments verify the feasibility and high efficiency of the proposed controller and its superiority compared with some state-of-the-art UAV controllers.

Unmanned Aerial Vehicles (UAV), such as quadrotors or fixed-wing aircraft, are increasingly common in airspace between 100 and 2000 meters. Extending current aviation infrastructure management operations, such as flight plan approval or aircraft tracking, to include UAV systems is difficult for multiple reasons, such as the technical challenge of tracking objects that frequently travel below the radar floor. This work explores the feasibility of using cellular data modems to achieve low-cost UAV tracking solutions for small consumer UAVs, such as *rotors, which typically have a very low-altitude flight path. We also explore the altitude at which the data modem functionality diminishes. We present a UAV tracking architecture and web service interface, as well as the empirical evidence gathered from testing the real-time capabilities and limitations of such a system.