Towards Robust Physical Adversarial Attacks on UAV Object Detection: A Multi-Dimensional Feature Optimization Approach

In recent years, deep learning has been extensively deployed on unmanned aerial vehicles (UAVs), particularly for object detection. As the cornerstone of UAV-based object detection, deep neural networks are susceptible to adversarial attacks, with adversarial patches being a relatively straightforward method to implement. However, current research on adversarial patches, especially those targeting UAV object detection, is limited. This scarcity is notable given the complex and dynamically changing environment inherent in UAV image acquisition, which necessitates the development of more robust adversarial patches to achieve effective attacks. To address the challenge of adversarial attacks in UAV high-altitude reconnaissance, this paper presents a robust adversarial patch generation framework. Firstly, the dataset is reconstructed by considering various environmental factors that UAVs may encounter during image collection, and the influences of reflections and shadows during photography are integrated into patch training. Additionally, a nested optimization method is employed to enhance the continuity of attacks across different altitudes. Experimental results demonstrate that the adversarial patches generated by the proposed method exhibit greater robustness in complex environments and have better transferability among similar models.

Misleading attention and classification: An adversarial attack to fool object detection models in the real world

With the rapid development of adversarial example technologies, the concept of adversarial patches has been proposed, which can successfully transfer adversarial attacks to the real world and fool intelligent object detection systems. However, the real-world environment is complex and changeable, and the adversarial patch attack technology is susceptible to real-world factors, resulting in a decrease in the success rate of attack. Existing adversarial-patch-generation algorithms have a single direction of patch initialization and do not fully consider the impact of initial diversification on its upper limit of adversarial patch attack. Therefore, this paper proposes an initial diversified adversarial patch generation technology to improve the effect of adversarial patch attacks on the underlying algorithms in the real world. The method uses YOLOv4 as the attack model, and the experimental results show that the attack effect of the adversarial-patch-attack method proposed in this paper is higher than the baseline 8.46%, and it also has a stronger attack effect and fewer training rounds.

Deep neural networks (DNNs) can improve the image analysis and interpretation of remote sensing technology by extracting valuable information from images, and has extensive applications such as military affairs, agriculture, environment, transportation, and urban division. The DNNs for object detection can identify and analyze objects in remote sensing images through fruitful features of images, which improves the efficiency of image processing and enables the recognition of large-scale remote sensing images. However, many studies have shown that deep neural networks are vulnerable to adversarial attack. After adding small perturbations, the generated adversarial examples will cause deep neural network to output undesired results, which will threaten the normal recognition and detection of remote sensing systems. According to the application scenarios, attacks can be divided into the digital domain and the physical domain, the digital domain attack is directly modified on the original image, which is mainly used to simulate the attack effect, while the physical domain attack adds perturbation to the actual objects and captures them with device, which is closer to the real situation. Attacks in the physical domain are more threatening, however, existing attack methods generally generate the patch with bright style and a large attack range, which is easy to be observed by human vision. Our goal is to generate a natural patch with a small perturbation area, which can help some remote sensing images used in the military to avoid detection by object detectors and im-perceptible to human eyes. To address the above issues, we generate a rust-style adversarial patch generation framework based on style transfer. The framework takes a heat map-based interpretability method to obtain key areas of target recognition and generate irregular-shaped natural-looking patches to reduce the disturbance area and alleviates suspicion from humans. To make the generated adversarial examples have a higher attack success rate in the physical domain, we further improve the robustness of the adversarial patch through data augmentation methods such as rotation, scaling, and brightness, and finally, make it impossible for the object detector to detect the camouflage patch. We have attacked the YOLOV3 detection network on multiple datasets. The experimental results show that our model has achieved a success rate of 95.7% in the digital domain. We also conduct physical attacks in indoor and outdoor environments and achieve an attack success rate of 70.6% and 65.3%, respectively. The structural similarity index metric shows that the adversarial patches generated are more natural than existing methods.

The adversarial attack is a cutting-edge technique used to study the vulnerability of deep neural networks (DNNs). However, most existing studies focus on the additive perturbation-based attack, which cannot represent real-world corruption and limit their applications. In particular, haze is a common natural phenomenon that significantly corrupts an image, which inevitably poses a potential threat to deep models. In this work, for the first attempt, we study the effects of haze on DNNs from the perspective of adversarial attacks and propose two adversarial haze attack methods. We first propose the optimization-based adversarial haze attack (OAdvHaze) that optimizes the parameters of the atmospheric scattering model with the guidance of a DNN to synthesize a hazy image, which leads to a high attack success rate. To achieve a more efficient attack, we further propose a predictive adversarial haze attack (PAdvHaze) employing a DNN to predict the hazing parameters through a one-step way. To validate the effectiveness of both methods, we conducted extensive experiments on two publicly available datasets, i.e., ILSVRC 2012 and NIPS 2017. OAdvHaze and PAdvHaze achieve comparable attack success rates and transferability to state-of-the-art attacks. This work would contribute to the evaluation and enhancement of the robustness of DNNs against haze perturbation that may happen in the real world.

Deep Neural Networks (DNNs) are very vulnerable to adversarial attacks because of the instability and unreliability under the training process. Recently, many studies about adversarial patches have been conducted that aims to misclassify the image classifier model by attaching patches to images. However, most of the previous research employs adversarial patches that are visible to human vision, making them easy to be identified and responded to. In this paper, we propose a new method entitled Spatially Localized Perturbation GAN (SLP-GAN) that can generate visually natural patches while maintaining a high attack success rate. SLP-GAN utilizes a spatially localized perturbation taken from the most representative area of target images (i.e., attention map) as the adversarial patches. The patch region is extracted using the Grad-CAM algorithm to improve the attacking ability against the target model. Our experiment, tested on GTSRB and CIFAR-10 datasets, shows that SLP-GAN outperforms the state-of-the-art adversarial patch attack methods in terms of visual fidelity.

Object detection plays a key role in many security-critical systems. Adversarial patch attacks, which are easy to implement in the physical world, pose a serious threat to state-of-the-art object detectors. Developing reliable defenses for object detectors against patch attacks is critical but severely understudied. In this paper, we propose Segment and Complete defense (SAC), a general framework for defending object detectors against patch attacks through detection and removal of adversarial patches. We first train a patch segmenter that outputs patch masks which provide pixel-level localization of adversarial patches. We then propose a self adversarial training algorithm to robustify the patch segmenter. In addition, we design a robust shape completion algorithm, which is guaranteed to remove the entire patch from the images if the outputs of the patch segmenter are within a certain Hamming distance of the ground-truth patch masks. Our experiments on COCO and xView datasets demonstrate that SAC achieves superior robustness even under strong adaptive attacks with no reduction in performance on clean images, and generalizes well to unseen patch shapes, attack budgets, and unseen attack methods. Furthermore, we present the APRICOT-Mask dataset, which augments the APRICOT dataset with pixel-level annotations of adversarial patches. We show SAC can significantly reduce the targeted attack success rate of physical patch attacks. Our code is available at https://github.com/joellliu/SegmentAndComplete.

Adversarial attacks against pedestrian detection systems face challenges due to high intra-class variety. Existing research primarily focuses on low intra-class variety targets or optimizes patch content while neglecting spatial placement. This paper proposes a genetic algorithm-based method for optimizing adversarial patch placement to enhance attack success rates against the YOLOv2 and Faster R-CNN detectors. First, base adversarial patches are generated via multi-loss joint optimization. Then, a genetic algorithm framework searches for optimal positions within detection bounding boxes, utilizing the sum of pedestrian confidence scores as the fitness function and incorporating tournament selection, linear crossover, and adaptive mutation. Experiments demonstrate significant improvements in attack success rates on both the Inria and PennFudan datasets. For YOLOv2, the improvements over fixed-position attacks are 10.38% on Inria and 9.25% on PennFudan. For Faster R-CNN, the corresponding improvements are 2.48% and 8.68% respectively. This work highlights the critical role of spatial optimization, providing valuable insights and guidance for future research on robustness, particularly in the context of security-critical systems potentially vulnerable to physical-world attacks.

Abstract. Beach litter destroys marine ecosystems and creates aesthetic discomfort that lowers the value of the beach. In order to solve this beach litter problem, it is necessary to study the generation and distribution pattern of waste and the cause of the inflow. However, the data for the study are only sample data collected in some areas of the beach. Also, most of the data covers only the total amount of beach litter. UAV(Unmanned Aerial Vehicle) and Deep Neural Network can be effectively used to detect and monitor beach litter. Using UAV, it is possible to easily survey the entire beach. The Deep Neural Network can also identify the type of coastal litter. Therefore, using UAV and Deep Neural Network, it is possible to acquire spatial information by type of beach litter. This paper proposes a Beach litter detection algorithm based on UAV and Deep Neural Network and a Beach litter monitoring process using it. It also offers optimal shooting altitude and film duplication to detect small beach litter such as plastic bottles and styrofoam pieces found on the beach. In this study, DJI Mavic 2 Pro was used. The camera on the UAV is a 1-inch CMOS with a resolution of 20MP. The images obtained through UAV are produced as orthoimages and input into a pre-trained neural network algorithm. The Deep Neural Network used for Beach litter detection removed the Fully Connected Layer from the Convolutional Neural Network for semantic segmentation.

In this paper a multiple object detection, recognition, and tracking system for unmanned aerial vehicles (UAVs) has been studied. The system can be implemented on any UAVs platform, with the main requirement being that the UAV has a suitable onboard computational unit and a camera. It is intended to be used in a maritime object tracking system framework for UAVs, which enables a UAV to perform multiobject tracking and situational awareness of the sea surface, in real time, during a UAV operation. Using machine vision to automatically detect objects in the camera's image stream combined with the UAV's navigation data, the onboard computer is able to georeference each object detection to measure the location of the detected objects in a local North‐East (NE) coordinate frame. A tracking algorithm which uses a Kalman filter and a constant velocity motion model utilizes an object's position measurements, automatically found using the object detection algorithm, to track and estimate an object's position and velocity. Furthermore, a global‐nearest‐neighbor algorithm is applied for data association. This is achieved using a measure of distance that is based not only on the physical distance between an object's estimated position and the measured position, but also how similar the objects appear in the camera image. Four field tests were conducted at sea to verify the object detection and tracking system. One of the flight tests was a two‐object tracking scenario, which is also used in three scenarios with an additional two simulated objects. The tracking results demonstrate the effectiveness of using visual recognition for data association to avoid interchanging the two estimated object trajectories. Furthermore, real‐time computations performed on the gathered data show that the system is able to automatically detect and track the position and velocity of a boat. Given that the system had at least 100 georeferenced measurements of the boat's position, the position was estimated and tracked with an accuracy of 5–15 m from 400 m altitude while the boat was in the camera's field of view (FOV). The estimated speed and course would also converge to the object's true trajectories (measured by Global Positioning System, GPS) for the tested scenarios. This enables the system to track boats while they are outside the FOV of the camera for extended periods of time, with tracking results showing a drift in the boat's position estimate down to 1–5 m/min outside of the FOV of the camera.

Unmanned aerial vehicle (UAV) communication is one of the enabling technology in the 5G/B5G era, which has received extensive research attention recently. In this paper, we consider a UAV employing amplify-and-forward protocol to facilitate the secure transmissions for a legitimate user pair, where the direct link is not available. In order to protect the legitimate transmissions and prevent the eavesdropper from intercepting the source message, the legitimate receiver launches a jamming signal to confuse the eavesdropper. In this regard, the optimal jamming power and UAV placement are jointly investigated towards the highest secrecy rate. For this problem, we first propose an efficient power control scheme based on bi-section search with fixed UAV placement. Then, a deep neural network (DNN) is constructed and trained based on the data from exhaustive search in order to determine the best UAV placement. Through numerical simulations, we verify that the trained DNN effectively approximates the optimal UAV placement, also demonstrated is the evident superiority of our proposed scheme as compared to the baselines in terms of security performance.

Deep Learning (DL) system has been widely used in many critical applications, such as autonomous vehicles and unmanned aerial vehicles. However, their security is threatened by backdoor attack, which is achieved by adding artificial patterns on specific training data. Existing attack methods normally poison the data using a patch, and they can be easily detected by existing detection methods. In this work, we propose the Adversarial Backdoor, which utilizes the Targeted Universal Adversarial Perturbation (TUAP) to hide the anomalies in DL models and confuse existing powerful detection methods. With extensive experiments, it is demonstrated that Adversarial Backdoor can be injected stably with an attack success rate around 98%. Moreover, Adversarial Backdoor can bypass state-of-the-art backdoor detection methods. More specifically, only around 37% of the poisoned models can be caught, and less than 29% of the poisoned data cannot bypass the detection. In contrast, for the patch backdoor, all the poisoned models and more than 80% of the poisoned data will be detected. This work intends to alarm the researchers and developers of this potential threat and to inspire the designing of effective detection methods.

Ultra-reliable intelligent link scheduling based on DRL for manned/unmanned aerial vehicle cooperative scenarios

Adversarial attacks represent a serious evolving threat to the operation of deep neural networks. Recently, adversarial algorithms were developed to facilitate hallucination of deep neural networks for ordinary attackers. State-of-the-arts algorithms could generate offline printable adversarial patches that can be interspersed within fields of view of the capturing cameras in an innocently unnoticeable action. In this paper, we propose an algorithm to ravage the operation of these adversarial patches. The proposed algorithm uses intrinsic information contents of the input image to extract a set of ally patches. The extracted patches break the salience of the attacking adversarial patch to the network. To our knowledge, this is the first time to address the defense problem against such kinds of adversarial attacks by counter-processing the input image in order to ravage the effect of any possible adversarial patches. The classification decision is taken according to a late-fusion strategy applied to the independent classifications generated by the extracted patch alliance. Evaluation experiments were conducted on the 1000 classes of the ILSVRC benchmark. Different convolutional neural network models and varying-scale adversarial patches were used in the experimentation. Evaluation results showed the effectiveness of the proposed ally patches in reducing the success rates of adversarial patches.

Key Smart City applications such as traffic management and public security rely heavily on the intelligent processing of video and image data, often in the form of visual retrieval tasks, such as person Re-IDentification (ReID) and vehicle re-identification. For these tasks, Deep Neural Networks (DNNs) have been the dominant solution for the past decade, for their remarkable ability in learning discriminative features from images to boost retrieval performance. However, it is been discovered that DNNs are broadly vulnerable to maliciously constructed adversarial examples. By adding small perturbations to a query image, the returned retrieval results will be completely dissimilar from the query image. This poses serious challenges to vital systems in Smart City applications that depend on the DNN-based visual retrieval technology, as in the physical world, simple camouflage can be added on the subject (a few patches on the body or car), and turn the subject completely untrackable by person or vehicle Re-ID systems. To demonstrate the potential of such threats, this paper proposes a novel adversarial patch generative adversarial network (AP-GAN) to generate adversarial patches instead of modifying the entire image, which also causes the DNNs-based image retrieval models to return incorrect results. AP-GAN is trained in an unsupervised way that requires only a small amount of unlabeled data for training. Once trained, it produces query-specific perturbations for query images to form adversarial queries. Extensive experiments show that the AP-GAN achieves excellent attacking performance with various application scenarios that are based on deep features, including image retrieval, person ReID and vehicle ReID. The results of this study provide a warning that when deploying a DNNs-based image retrieval system, its security and robustness needs to be thoroughly considered.