UAV based cost-effective real-time abnormal event detection using edge computing

In this paper, a real-time image detection system for unmanned aerial vehicle (UAV) is presented. First, the You Only Look Once (YOLO) detector is retrained to detect and recognize objects in UAV images fast and accurately. The retrained YOLO detector possesses a tradeoff between quickly objects identification and precisely objects localization, accounting for 4 general moving objects (i.e., car, bus, truck and pedestrian) in UAV images. Second, extra 4,000 UAV images shot by the embedded UAV camera are fed into the YOLO which is used to predict bounding-box with label probabilities from a full image. The YOLO is compared with the Faster Region Convolutional Neural Network, as well as other deep-learning frameworks for object detection. Field experiments on UAV videos shot in various backgrounds are performed to testify that the proposed system can detect objects in UAV images effectively and robustly, achieving a real-time detection speed at 15 frames per second and a satisfying accuracy on the wild test set of 400 UAV images with a Graphic Processing Unit (GPU) acceleration.

Object detection from UAV thermal infrared images and videos using YOLO models

In response to the challenges faced by unmanned aerial vehicles (UAV) in cluttered environments such as forests, ruins, and pipelines, this study introduces a ground–air amphibious UAV specifically designed for personnel search and rescue in complex environments. By innovatively designing and applying a separation cage structure, the UAV’s capabilities for ground movement and aerial flight have been enhanced, effectively overcoming the limitations of traditional single-mode robots operating in narrow or obstacle-dense areas. This design addresses the occlusion issue of sensing components in traditional caged UAVs while maintaining protection for both the UAV itself and the surrounding environment. Additionally, through the innovative design of an H-shaped quadcopter frame skeleton structure, the UAV has gained the ability to perform steady-state aerial flight while also better adapting to the separation cage structure, achieving a reduced energy consumption and significantly improving its operational capabilities in complex environments. The experimental results demonstrate that the UAV prototype, weighing 1.2 kg with a 1 kg payload capacity, achieves a 40 min maximum endurance under full payload conditions at the endurance speed of 10 m/s while performing real-time object detection. The system reliably executes multimodal operations, including stable takeoff, landing, aerial hovering, directional maneuvering, and terrestrial locomotion with coordinated steering control.

Real-time detection of cracks in tiled sidewalks using YOLO-based method applied to unmanned aerial vehicle (UAV) images

The unmanned aerial vehicle (UAV) is one of the mobile robots which has grown rapidly in popularity in recent years. UAVs have been used in various activities to convey important information on the aerial transportation system. However, an application to get real-time information from UAVs remains a challenging task. Therefore, this paper presents an integration system to optimise the use of UAVs for aerial surveillance. The proposed system explains the design and implementation a UAV system which consists of on-board and ground stations. The on-board system has the function to send UAV flight data, send aerial images and carry items using a gripper which equipped with sensors to detect obstacles. The ground station has the function to control UAV flights, receive and display UAV flight data and aerial images, display the attitude of the UAV and send commands to move the camera and gripper. Flight controllers in this system use non-linear complementary filter on SO(3) to estimate the attitude of the UAV and proportional-integral (PI) controller to stabilise the UAV flight. By providing the real-time acquisition data of the UAV, we believe this system can be a useful tool for many implementations using UAV.

Abnormal event detection, also known as anomaly detection, is one challenging task in security video surveillance. It is important to develop effective and robust movement representation models for global and local abnormal event detection to fight against factors such as occlusion and illumination change. In this paper, a new algorithm is proposed. It can locate the abnormal events on one frame, and detect the global abnormal frame. The proposed algorithm employs a sparse measurement matrix designed to represent the movement feature based on optical flow efficiently. Then, the abnormal detection mission is constructed as a one-class classification task via merely learning from the training normal samples. Experiments demonstrate that our algorithm performs well on the benchmark abnormal detection datasets against state-of-the-art methods.

With the booming development of tourism, travel security problems are becoming more and more prominent. Congestion, stampedes, fights and other tourism emergency events occurred frequently, which should be a wake-up call for tourism security. Therefore, it is of great research value and application prospect to real-time monitor tourists and detect abnormal events in tourism surveillance video by using computer vision and video intelligent processing technology, which can realize the timely forecast and early warning of tourism emergencies. At present, although most of the video-based abnormal event detection methods work well in simple scenes, there are often problems such as low detection rate and high false positive rate in complex motion scenarios, and the detection of abnormal events can’t be processed in real time. To tackle these issues, we propose an abnormal event detection model in tourism video based on salient spatio-temporal features and sparse combination learning, which has good robustness and timeliness in complex motion scenarios and can be adapted to real-time anomaly detection in practical applications. Specifically, spatio-temporal gradient model is combined with foreground detection to extract 3D gradient features on the foreground target of video sequence as the salient spatio-temporal features, which can eliminate the interference of the background. Sparse combination learning algorithm is used to establish the abnormal event detection model, which can realize the real-time detection of abnormal events. In addition, we construct a new ScenicSpot dataset with 18 video clips (5964 frames) containing both normal and abnormal events. The experimental results on ScenicSpot dataset and two standard benchmark datasets show that our method can realize the automatic detection and recognition of tourists’ abnormal behavior, and has better performance compared with the classical methods.

In recent years, unmanned aerial vehicles (UAV) have been increasingly used in power line inspections. Birds often nest on transmission line towers, which threatens safe power line operation. The existing research on bird’s nest inspection using UAVs mainly stays at the level of image postprocessing detection, which has poor real-time performance and cannot obtain timely bird’s nest detection results. Considering the above shortcomings, we designed a power inspection UAV system based on deep learning technology for autonomous flight, positioning and photography, real-time bird nest detection, and result export. In this research, 2000 bird’s nest images in the actual power inspection environment were shot and collected to create the dataset. The parameter optimization and test comparison for bird’s nest detection are based on the three target detection models of YOLOv3, YOLOv5-s, and YOLOX-s. A YOLOv5-s bird’s nest detection model optimized for bird’s nest real-time detection is proposed, and it is deployed to the onboard computer for real-time detection and verification during flight. The DJI M300 RTK UAV was used to conduct a test flight in a natural power inspection environment. The test results show that the mAP of the UAV system designed in this paper for bird’s nest detection is 92.1%, and the real-time detection frame rate is 33.9 FPS. Compared with the previous research results, this paper proposes a new practice of using drones for bird’s nest detection, dramatically improving the real-time accuracy of bird’s nest detection. The UAV system can efficiently complete the task of bird’s nest detection in the process of electric power inspection, which can significantly reduce manpower consumption in the power inspection process.

Integrating advanced technologies such as artificial intelligence (AI) with traditional agricultural practices has changed how activities are developed in agriculture, with the aim of automating manual processes and improving the efficiency and quality of farming decisions. With the advent of deep learning models such as convolutional neural network (CNN) and You Only Look Once (YOLO), many studies have emerged given the need to develop solutions to problems and take advantage of all the potential that this technology has to offer. This systematic literature review aims to present an in-depth investigation of the application of AI in supporting the management of weeds, plant nutrition, water, pests, and diseases. This systematic review was conducted using the PRISMA methodology and guidelines. Data from different papers indicated that the main research interests comprise five groups: (a) type of agronomic problems; (b) type of sensor; (c) dataset treatment; (d) evaluation metrics and quantification; and (e) AI technique. The inclusion (I) and exclusion (E) criteria adopted in this study included: (I1) articles that obtained AI techniques for agricultural analysis; (I2) complete articles written in English; (I3) articles from specialized scientific journals; (E1) articles that did not describe the type of agrarian analysis used; (E2) articles that did not specify the AI technique used and that were incomplete or abstract; (E3) articles that did not present substantial experimental results. The articles were searched on the official pages of the main scientific bases: ACM, IEEE, ScienceDirect, MDPI, and Web of Science. The papers were categorized and grouped to show the main contributions of the literature to support agricultural decisions using AI. This study found that AI methods perform better in supporting weed detection, classification of plant diseases, and estimation of agricultural yield in crops when using images captured by Unmanned Aerial Vehicles (UAVs). Furthermore, CNN and YOLO, as well as their variations, present the best results for all groups presented. This review also points out the limitations and potential challenges when working with deep machine learning models, aiming to contribute to knowledge systematization and to benefit researchers and professionals regarding AI applications in mitigating agronomic problems.

Bird activities around power lines and transmission towers pose significant risks to the stability and safety of power infrastructure, often leading to equipment damage, power outages, and even wildfires. Accurate and real-time detection of birds near critical components is essential for implementing proactive mitigation strategies. However, traditional monitoring methods such as manual inspection and fixed surveillance systems suffer from limited coverage, high labor costs, and insufficient detection accuracy under varying environmental conditions. In this paper, we propose a robust bird detection framework based on YOLOv10, the latest advancement in the You Only Look Once (YOLO) family of real-time object detection algorithms. Our approach leverages YOLOv10’s lightweight architecture and enhanced feature fusion capabilities to achieve fast and precise detection of birds in aerial imagery collected by unmanned aerial vehicles (UAVs). The model is trained and evaluated on a custom dataset composed of annotated bird images captured in diverse transmission line environments, including varying altitudes, backgrounds, and lighting conditions. Experimental results show that our YOLOv10-based detector outperforms previous state-of-the-art models in both detection accuracy and inference speed.

This study introduced a novel driver safety reward system (DSRS) leveraging You Only Look Once (YOLO)-based traffic violation detection and an incentive mechanism to promote safer driving behaviors at roundabouts. Unmanned aerial vehicle (UAV)-captured aerial videos at unsignalized roundabouts were used for detecting vehicles and extracting trajectory data to analyze various traffic violations such as wrong-way driving, anti-clockwise driving, and improper U-turns. The YOLOv8m model, coupled with ByteTrack, facilitated robust vehicle tracking using high-quality aerial video data. Driver behavior was scored based on detected violations, overspeed events, and sharp acceleration and deceleration instances. Additionally, a MERN stack-based reward system was developed to assign and visualize reward points for drivers. The system provided a clear and accessible way for drivers to understand their driving performance. Furthermore, a multinomial logistic regression (MLR) model was developed to predict violation types, demonstrating good prediction accuracy. This study highlighted the potential of cutting-edge image processing technologies to enhance road safety by identifying problematic traffic scenarios and developing a proactive driver reward disbursement system. The system provided valuable insights into violation patterns predominantly occurring at roundabouts under mixed traffic conditions for different vehicle types. The research, uniquely combining aerial surveillance, machine learning, and a user-friendly reward platform, aimed to reduce traffic violations and promote safer driving practices significantly. This innovative approach contributed to the broader goal of enhancing road safety through technological innovation.

To enhance the intelligent obstacle avoidance capabilities of unmanned aerial vehicles (UAVs), we introduce a refined version of the You Only Look Once (YOLO) algorithm, termed Improved YOLO (I-YOLO). This novel algorithm not only maintains the swift detection speed inherent in the original YOLO but also augments its feature enhancement network. Specifically, we propose a three-branch parallel feature pyramid network (TBPFP) to capture richer semantic information pertaining to the context of small obstacles. Furthermore, we incorporate a generative adversarial network (GAN) in a cascaded manner to synthesize more realistic super-resolution images, thereby bolstering detection accuracy. Experimental evaluations demonstrate that, in comparison to the baseline YOLO algorithm, our I-YOLO variant exhibits superior detection precision and feature extraction capabilities in the perception of intricate environmental targets. While there is a modest decrement in detection speed, it remains adequate to fulfill real-time requirements, particularly in the context of video stream processing.

Unmanned Aerial Vehicles (UAVs) have attracted great interest in the last few years owing to their ability to cover large areas and access difficult and hazardous target zones, which is not the case of traditional systems relying on direct observations obtained from fixed cameras and sensors. Furthermore, thanks to the advancements in computer vision and machine learning, UAVs are being adopted for a broad range of solutions and applications. However, Deep Neural Networks (DNNs) are progressing toward deeper and complex models that prevent them from being executed on-board. In this paper, we propose a DNN distribution methodology within UAVs to enable data classification in resource-constrained devices and avoid extra delays introduced by the server-based solutions due to data communication over air-to-ground links. The proposed method is formulated as an optimization problem that aims to minimize the latency between data collection and decision-making while considering the mobility model and the resource constraints of the UAVs as part of the air-to-air communication. We also introduce the mobility prediction to adapt our system to the dynamics of UAVs and the network variation. The simulation conducted to evaluate the performance and benchmark the proposed methods, namely Optimal UAV-based Layer Distribution (OULD) and OULD with Mobility Prediction (OULD-MP), were run in an HPC cluster. The obtained results show that our optimization solution outperforms the existing and heuristic-based approaches.

It is important for an Unmanned Aerial Vehicle (UAV) to detect any object in its view. This enables the UAV to locate the object with respect to itself and is required for locking and tracking the object. Object detection and location estimation makes the UAV capable of manipulating the environment as well as to follow the target. In the present work, object detection has been carried out using You Only Look Once (YOLO) to detect the object in the image stream of the Robot Operating System (ROS) bag file. The GPS information of the UAV is used to further calculate the GPS coordinates of the object. The images are acquired using a single monocular camera.

Unmanned Aerial Vehicles (UAVs) have been widely used in the fields of military and civilian due to their small size, low cost, and high flexibility. However, various security threats gradually increase by continuously expanding the scopes of UAVs’ applications. Among all the security threats, the false spoofing attack is a typical network threat faced by UAVs. In real scenes, there is inevitably a large amount of random noise in the collected signals of UAVs, which leads to interference of original signals of UAVs, thus affecting the detection accuracy of abnormal signals for UAVs. This paper proposes an abnormal signals detection method for UAVs based on bimodal fusion. First, the collected signals of UAVs are modally transformed to generate waveform images. Second, the Double Shortcuts Zero-Bias ResNet is used to extract bimodal features from the waveform image modal data and the frequency-domain signal modal data of original signals for UAVs. Then, the bimodal fusion classifier is used to fuse the classification results of different modal data to alleviate the interference of random noises to the original signal of UAVs and improve the detection accuracy of abnormal signals on UAVs. Simulation results indicate that based on the original ADS-B signal data set of UAVs with random noises, the proposed method can improve the detection accuracy compared with the existing abnormal signal detection methods.