Development of a YOLO-KCF Coupling Algorithm for Miniature Fixed-Wing UAVs in Target Detection and Tracking

Unmanned aerial vehicles (UAVs) become increasingly available devices with extensive usage as environmental monitoring systems. With the benefit of higher mobility, UAVs are applied to fuel various significant uses in computer vision (CV), providing more effectiveness and accessibility than surveillance cameras with permanent camera view, angle, and scale. Nevertheless, owing to camera motion and composite environments, it is problematic to identify the UAV; conventional models frequently miss UAV detection and make false alarms. Drone-equipped cameras monitor objects at changing altitudes, leading to essential scale variants. The model increases targeted accuracy and decreases false positives using real-time data and machine learning (ML) methods. Its enormous applications range from military operations to urban planning and wildlife monitoring. Therefore, this study develops a novel long-term target detection and tracking model for unmanned aerial vehicles using a deep fusion-based convolutional transform learning (LTTDT–UAVDFCTL) model. The LTTDT–UAVDFCTL model presents a new model to improve the robustness and accuracy of target tracking and detection in scale-variant environments. At first, the presented LTTDT–UAVDFCTL technique performs image pre-processing by utilizing the median median-enhanced wiener filter (MEWF) technique to improve clarity and reduce noise. For object detection (OD), the highly accurate YOLOv8 technique is utilized, followed by feature extraction through a backbone deep fusion-based convolutional transform learning of VGG16, CapsNet, and EfficientNetB7 to capture both spatial and hierarchical features across varying scales. Moreover, the graph convolutional neural network (GCN) technique is employed for long-term target detection and tracking models. Finally, the hybrid nonlinear whale optimization algorithm with sine cosine (SCWOA) is implemented for the optimum choice of the hyperparameters involved in the GCN technique. The experimental study of the LTTDT–UAVDFCTL approach is performed under the VisDrone dataset. The performance validation of the LTTDT–UAVDFCTL approach portrayed a superior mAP value of 80.13% over existing models.

<div>In this dissertation, methods for real-time trajectory generation and autonomous obstacle avoidance for fixed-wing and quad-rotor unmanned aerial vehicles (UAV) are studied. A key challenge for such trajectory generation is the high computation time required to plan a new path to safely maneuver around obstacles instantaneously. Therefore, methods for rapid generation of obstacle avoidance trajectory are explored. The high computation time is a result of the computationally intensive algorithms used to generate trajectories for real-time object avoidance. Recent studies have shown that custom solvers have been developed that are able to solve the problem with a lower computation time however these designs are limited to small sized problems or are proprietary. Additionally, for a swarm problem, which is an area of high interest, as the number of agents increases the problem size increases and in turn creates further computational challenges. A solution to these challenges will allow for UAVs to be used in autonomous missions robust to environmental uncertainties.</div><div><br></div><div>In this study, a trajectory generation problem posed as an optimal control problem is solved using a sequential convex programming approach; a nonlinear programming algorithm, for which custom solver is used. First, a method for feasible trajectory generation for fast-paced obstacle-rich environments is presented for the case of fixed-wing UAVs. Next, a problem of trajectory generation for fixed-wing and quad-rotor UAVs is defined such that starting from an initial state a UAV moves forward along the direction of flight while avoiding obstacles and remaining close to a reference path. The problem is solved within the framework of finite-horizon model predictive control. Finally, the problem of trajectory generation is extended to a swarm of quad-rotors where each UAV in a swarm has a reference path to fly along. Utilizing a centralized approach, a swarm scenario with moving targets is studied in two different cases in an attempt to lower the solution time; the first, solve the entire swarm problem at once, and the second, solve iteratively for a UAV in the swarm while considering trajectories of other UAVs as fixed.</div><div><br></div><div>Results show that a feasible trajectory for a fixed-wing UAV can be obtained within tens of milliseconds. Moreover, the obtained feasible trajectories can be used as initial guesses to the optimal solvers to speed up the solution of optimal trajectories. The methods explored demonstrated the ability for rapid feasible trajectory generation allowing for safe obstacle avoidance, which may be used in the case an optimal trajectory solution is not available. A comparative study between a dynamic and a kinematic model shows that the dynamic model provides better trajectories including aggressive trajectories around obstacles compared to the kinematic counterpart for fixed-wing UAVs, despite having approximately the same computational demands. Whereas, for the case of quad-rotor UAVs, the kinematic model takes almost half the solution time than with a reduced dynamic model, despite having approximately the similar range of values for the cost function. When extended to a swarm, solving the problem for each UAV is four to seven times computationally cheaper than solving the swarm as a whole. With the improved computation time for trajectory generation for a swarm of quad-rotors using centralized approach, the problem is now reasonably scalable, which opens up the possibility to increase the number of agents in a swarm using high-end computing machines for real-time applications. Overall, a custom solver jointly with a sequential convex programming approach solves an optimization problem in a low computation time.</div>

In this work we address the problem of autonomously tracking a group of ground static targets in an urban area using fixed wing unmanned aerial vehicles (UAVs). In this scenario, UAVs are equipped with sensors that are subject to the constraints of urban area occlusions. Information regarding the line of sight (LOS) occlusions in the urban area is assumed available. Each UAV, considering the dynamic constrains arising from the fixed wing, is modeled as a Dubins car. The motion planning of searching the shortest path for UAVs, namely Dubins path, is a NP hard problem. We propose an algorithm, which can provide sub-optimal Dubins path, hence can bring a closed-loop path for UAVs to continuously track the group of static targets and minimize the time of loss track. The proposed algorithm is validated through simulations on a visual database that is based on the CBD in Singapore.

The use of small and miniature unmanned aerial vehicles (UAVs) for remote sensing and surveillance applications has become increasingly popular in the last two decades. The intermittent connectivity in a sparse UAV network makes it challenging to efficiently gather sensed image data. This paper investigates the communication of sensed images from a set of mobile survey UAVs to a static base station through the assistance of a relay UAV. Given the planned routes of survey UAVs, a set of relay waypoints are found for the relay UAV to meet the survey UAVs and receive the sensed images. An Online Message Relaying technique (OMR) is proposed to schedule the relay UAV to collect images. Without any global collaboration between the relay UAV and the survey UAVs, OMR utilizes a markov decision process (MDP) that determines the best schedules for the relay UAV such that the image acquisition rate could be maximized. Evaluation results show that the proposed relaying technique outperforms traditional relaying techniques, such as the traveling salesman problem (TSP) and the random walk, in terms of end-to-end delay and frame delivery ratio.

As a key technology of detection and tracking system, moving target detection and tracking has attracted a great deal of attention. To achieve the goal of tracking target, conventional methods have to complete the detection of targets firstly, and then track the target. However, such kind of approaches require using the method of constant false-alarm rate to detect the whole frame image received at that time, which not only reduces the detection efficiency but also degrades the performance of target tracking. In order to implement real-time monitoring divers and other small moving targets under the conditions of strong underwater noise and clutter interference, an interactive algorithm for moving target detection and tracking is proposed based on acoustic image, which jointly analyzes the target detection system and the target tracking system. In the target tracking unit, the interactive algorithm takes advantage of data interconnection to remove the false targets produced by target detection; extrapolates the target's track, and predicts the location that the target may appear at the next moment by employing the Kalman filter. These messages will be used as priori information in the detection unit. In the detection unit, first of all, a gate is established whose center is the position that the target tracking unit predicted. For the image received at the next moment, detection in the predicted gate in implemented by using the method of constant false-alarm rate, followed by feeding back the detection results to the tracking unit. Compared with existed methods, the proposed algorithm can effectively improve the target detection efficiency and tracking performance, providing a better approach to monitor underwater small targets.

When Unmanned Aerial Vehicle (UAV) follows a dynamic object as a target, it sometimes loses. In this paper, we propose a tracker switcher module to determine which algorithm is utilized for tracking. The author uses Kernelized Correlation Filter (KCF) to judge if the information from vision system is still reliable enough to leading UAV or lost. When the target is lost, we use Kalman Filter (KF) to predict the moving trajectory of the target, control the UAV movement and let UAV follow the virtual trajectory, until the target reappears and the system is in the correct tracking state again. Finally, we use OTB dataset to verify the feasibility of this method. Our tracker obtains Precision and FPS plot generator, which validate the effectiveness of this method. This method can make up for the problem under the premise of low delay, and has a wide range of application scenarios.

Target detection and tracking is a key technology in intelligent monitoring system. However, it is still difficult to design a robust, accurate and real-time target detection and tracking algorithm. Taking target Tracking as the research focus, this paper focuses on one of the current mainstream algorithms, namely, TLD (tracking-learning-detection) target Tracking algorithm. Based on the theory of TLD target tracking and corner detection, a new target tracking method was proposed to solve the existing problems. A classifier is built online to track and learn the target in real time, and a target contour detector is added in the system to complement the random forest detector to solve the problem of tracking failure. Experimental results show that the proposed algorithm can effectively overcome the influence of fast change or depth occlusion, and improve the tracking efficiency of moving targets.

The on‐board target tracking of unmanned aerial vehicle (UAV) relies on on‐board computer to process the collected ground image information, and then use the target tracking algorithm to achieve the target tracking function, which provides the user with target position, trajectory and other information. UAVs are limited by load and power consumption. Due to the high resolution and frame rate of UAV's image collection system, traditional target tracking platforms cannot meet the requirements. As a result, the real‐time nature of the tracking platform urgently needs to be resolved. This study designs and implements the airborne ground target tracking accelerator for UAV. The kernelised correlation filters (KCFs) algorithm with low computational complexity and precision is selected to achieve the airborne ground target tracking function and its performance bottleneck is analysed. In order to improve the real‐time performance of the tracking task processing, the ZYNQ platform based on ARM and field programmable gate arrays is chosen to achieve the hardware acceleration of the KCF algorithm. The KCF algorithm is implemented in hardware using the high‐level synthesis technology, and a high‐speed data transmission link is established. Experiments show that the design can achieve a tracking rate of more than 30 FPS under 960 × 540 resolution, which meets the real‐time requirements of UAV target tracking task.

针对图像序列中运动目标检测、跟踪的难点问题，提出了一种实时运动目标检测与跟踪算法。该算法基于自适应背景建模，获取运动目标背景模型和前景图像，从而实现运动目标检测；通过建立运动目标的位置、大小、形状以及颜色分布模型，构造运动目标全局匹配函数，结合目标活力特征，实现多运动目标连续匹配和跟踪。实验结果表明，相对于传统的运动目标跟踪方法，本文方法明显减少了运算时间，增强了环境适应性，实现了复杂场景下运动目标的准确检测和稳定跟踪，对非刚性目标的形变、旋转具有较强的鲁棒性。In this paper, we propose a real-time mobile target detection and tracking algorithm for challenges of mobile target detection and tracking in sequential images. This algorithm based on the adaptive background modeling obtains background model and front-view images of mobile targets, which is the way to achieve target detection. Continuous matching and tracking of multiple mobile targets are realized through constructing position, size, shape and color distribution of the mobile targets, defining a global matching function for those targets, and associating their vitality characteristics. It is demonstrated by experiments that the algorithm presented in this paper, compared to the traditional methods of mobile target tracking, significantly reduces the computation time, improves adaptive feature to environments, achieves accurate detection and robust tracking of mobile targets in complex environments, and shows strong robustness to deformation and rotation of non-rigid targets.

This brief is concerned with integrated autonomous takeoff, target search, task assignment, and tracking using multiple fixed-wing unmanned aerial vehicles (UAVs) in urban environments. The problem is to design flight autonomy that is embedded onboard each UAV to enable autonomous flight coordination and distributed tasking. Control logic design based on a finite state automaton model, integrating four modes of operations, namely, the takeoff mode, the fly-to-area of operation mode, the search mode, and the tracking mode, is presented. Different from the state-of-the-art of recent research, this brief provides a preliminary research on the autonomous cooperative takeoff for miniature fixed-wing UAVs, by considering collision avoidance, communication failure, etc. To make UAVs autonomously and cooperatively search roads in the urban environments, an efficient improved search algorithm is proposed based on recent research on the coverage search in the literature. For the target tracking, using geometric relations (relative position, orientations, speed ratio, and minimal turning radius), a systematic algorithm is developed to generate an optimal online path. All the algorithms in this work are developed based on realistic miniature fixed-wing UAV dynamic models. The main focus of the brief is to test the developed control logic and also the algorithms. The proposed framework is evaluated by our 3-D multi-UAV test bed.

Object detection from UAV thermal infrared images and videos using YOLO models

In this paper, a scale adaptive target tracking algorithm based on deep residual network and kernel correlation filter is proposed. Although kernel correlation filter is a fast and effective target tracking algorithm, its tracking effect is not ideal in the case of the occlusion, blur and scale change caused by the fast moving target in the real environment. To deal with these problems, corresponding measures are integrated in the proposed new algorithm. Firstly, the structure of the ResNet50 network was adjusted and trained. The deep residual network was used to extract target features and integrate the kernel correlation filter algorithm to carry out adaptive response graph fusion to find the target location. Then, we add the scale estimation module, and use the HOG feature to replace the original deep feature training scale filter. Based on the structure of binary sort tree, binary search is carried out on the scale, and the scale discriminant index is used to judge whether the target has reached the appropriate scale. Experimental results show that our proposed algorithm can achieve scale adaptation of target tracking, and the tracking performance is further improved.

The use of an appropriate sensor on an unmanned aerial vehicle (UAV) is vital to assess specific environmental conditions successfully. In addition, technicians and scientists also appreciate a platform to carry the sensors with some advantages such as the low costs or easy pilot management. However, extra requirements like a low-altitude flight are necessary for special applications such as plant density or rice yield. A rotary UAV matches this requirement, but the flight endurance is too short for large areas. Therefore, in this article, a fixed-wing UAV is used, which is more appropriate because of its longer flight endurance. It is necessary to develop an own controller system to use special sensors such as Lidar or Radar on the platform as a multifunctionality system. Thereby, these sensors are used to generate a digital elevation model and also as a collision avoidance sensor at the same time. To achieve this goal, a small UAV was equipped with a hardware platform including a microcontroller and sensors. After testing the system and simulation, the controller was converted into program code to implement it on the microcontroller. After that, several real flights were performed to validate the controller and sensors. We demonstrated that the system is able to work and match the high requirements for future research.

With the development of low-cost, durable unmanned aerial vehicles (UAVs), it is now practical to perform persistent sensing and target tracking autonomously over broad surveillance areas. These vehicles can sense the environment directly through onboard active sensors, or indirectly when aimed toward ground targets in a mission environment by ground-based passive sensors operating wirelessly as an ad hoc network in the environment. The combination of the swarm intelligence of the airborne infrastructure comprised of UAVs with the ant-like collaborative behavior of the unattended ground sensors creates a system capable of both persistent and pervasive sensing of mission environment, such that, the continuous collection, analysis and tracking of targets from sensor data received from the ground can be achieved. Mobile software agents are used to implement intelligent algorithms for the communications, formation control and sensor data processing in this composite configuration. The enabling mobile agents are organized in a hierarchy for the three stages of processing in the distributed system: target detection, location and recognition from the collaborative data processing among active ground-sensor nodes; transfer of the target information processed on the ground to the UAV swarm overhead; and formation control and sensor activation of the UAV swarm for sustained ground-target surveillance and tracking. Intelligent algorithms are presented that can adapt to the operation of the composite system to target dynamics and system resources. Established routines, appropriate to the processing needs of each stage, are selected as preferred based on their published use in similar scenarios, ability to be distributively implemented over the set of processors at system nodes, and ability to conserve the limited resources at the ground nodes to extend the lifetime of the pervasive network. In this paper, the performance of this distributed, collaborative system concept for persistent-pervasive sensing of a ground environment is assessed via simulation of the selected adaptive algorithms using parameter values planned for ground sensors and UAVs and mission scenarios found in published studies.

In this paper, a real-time image detection system for unmanned aerial vehicle (UAV) is presented. First, the You Only Look Once (YOLO) detector is retrained to detect and recognize objects in UAV images fast and accurately. The retrained YOLO detector possesses a tradeoff between quickly objects identification and precisely objects localization, accounting for 4 general moving objects (i.e., car, bus, truck and pedestrian) in UAV images. Second, extra 4,000 UAV images shot by the embedded UAV camera are fed into the YOLO which is used to predict bounding-box with label probabilities from a full image. The YOLO is compared with the Faster Region Convolutional Neural Network, as well as other deep-learning frameworks for object detection. Field experiments on UAV videos shot in various backgrounds are performed to testify that the proposed system can detect objects in UAV images effectively and robustly, achieving a real-time detection speed at 15 frames per second and a satisfying accuracy on the wild test set of 400 UAV images with a Graphic Processing Unit (GPU) acceleration.